Deep learning reasoning implementation system and method based on embedded operating system
By employing a layered design and dynamic thread pool technology, the problems of high computational load and high hardware cost of deep learning networks on embedded devices are solved, enabling efficient and low-cost deep learning inference and improving system independence and scalability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI AEROSPACE CONTROL TECH INST
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-17
AI Technical Summary
When deploying deep learning networks on embedded devices with limited computing resources, existing technologies suffer from problems such as high computational load, high hardware cost, and dependence on network connectivity.
The deep learning inference system adopts a layered design, including a model layer, a module layer, a computation layer, and an execution layer. It utilizes matrix block computation and dynamic thread pool technology to break down the inference task into multiple subtasks, execute them in parallel, dynamically adjust the number of worker threads, and make full use of multi-core CPU resources.
It improves the speed of deep learning inference, reduces hardware costs and power consumption, enhances system independence and flexibility, simplifies the model upgrade process, and improves maintenance efficiency and scalability.
Smart Images
Figure CN121882259A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a deep learning inference implementation system and method based on an embedded operating system, belonging to the field of deep learning deployment technology. Background Technology
[0002] Deep learning algorithms have achieved great success in various fields such as computer graphics, speech, and text processing. However, as the number of network layers increases, the number of parameters also increases, and the computational load becomes increasingly large. Deploying deep learning networks on devices with limited computing resources, such as embedded ARM or DSP devices, has become a challenge. Currently, there are two main solutions. One is to adopt a "local + cloud" approach, which requires the embedded device to be constantly connected to the server and also has certain requirements for network and bandwidth. The other solution is to use a heterogeneous multi-core approach, using GPU or FPGA acceleration modules to accelerate the deep learning inference process, but this increases the hardware cost of the device. Summary of the Invention
[0003] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a deep learning inference implementation system and method based on an embedded operating system, thereby improving the speed of deep learning inference.
[0004] The technical solution of this invention is: This invention discloses a deep learning inference implementation system based on an embedded operating system, comprising: Model layer: Reads model files from the embedded operating system's file system, parses the network layer structure and parameters in the model files, and generates a node sequence in a linked storage structure; forms each network layer based on the node sequence in the linked storage structure; and generates the corresponding inference task based on the type of the network layer and the input data. Module layer: Receives inference tasks from model layer nodes; breaks down the inference tasks into multiple parallelizable subtasks using matrix block computation and sends them to the computation layer; Computation layer: Adds subtasks sent by the module layer to the task queue; competes to obtain subtasks from the task queue and performs calculations; dynamically adjusts the number of worker threads based on the length of the task queue and the idle status of the worker threads. Execution layer: Based on the task queue length and the current CPU core load, a suitable scheduling strategy is dynamically selected, and worker threads are bound to different cores of a multi-core CPU to achieve task parallelization.
[0005] Furthermore, in the above system, the computing layer includes a dynamic thread pool module, which comprises a task manager, a task queue, worker thread groups, and a manager thread; wherein, The task manager receives subtasks from the module layer and adds the subtasks to the task queue; The worker thread group competes to obtain subtasks from the task queue and performs calculations; The manager thread dynamically adjusts the number of worker threads based on the length of the task queue and the idle status of the worker threads.
[0006] Furthermore, in the above system, the method for dynamically adjusting the number of worker threads is specifically as follows: When the number of tasks in the task queue exceeds the current number of worker threads, new worker threads are added according to the step size of the new threads until the maximum number of threads is reached. When the idle time of a worker thread exceeds a preset threshold and the current number of threads is greater than the minimum number of threads, the idle thread is destroyed according to the step size of the destroyed thread.
[0007] Furthermore, in the above system, the maximum number of threads is 2 to 3 times the number of cores in a multi-core CPU, and the minimum number of threads is 1 / 3 to 1 / 2 of the number of cores in a multi-core CPU.
[0008] Furthermore, in the above system, the module layer includes sequentially connected fully connected modules, convolutional modules, pooling modules, normalization modules, and activation function modules.
[0009] Furthermore, in the above system, the matrix block calculation method is specifically as follows: The input data is divided into multiple sub-regions, and each sub-region is used to generate an independent sub-task. The computation results of subtasks are aggregated through memory sharing or message queues.
[0010] Furthermore, in the above system, in the linked storage structure of the model layer, each node includes: network layer type, input / output dimension, weight pointer, bias pointer, doubly linked list pointers pointing to the previous and next nodes, and module type identifier; the module type identifier is used to associate the corresponding computation module of the module layer.
[0011] This invention discloses a deep learning inference implementation method based on an embedded operating system, comprising: Model loading: Read the model file from the embedded operating system's file system, parse the network layer structure and parameters in the model file, and generate a node sequence in a linked storage structure; form each network layer according to the node sequence in the linked storage structure; generate the corresponding inference task according to the type of the network layer and the input data; Task splitting: Based on the type of network layer, the inference task is split into multiple subtasks using a matrix partitioning method; Task scheduling: Subtasks are added to the task queue through a dynamic thread pool, and the number of worker threads is dynamically adjusted according to the task load; Parallel computing: Utilizes the scheduler of the embedded operating system to bind worker threads to different cores of a multi-core CPU, enabling the parallel execution of subtasks; Result aggregation: The computation results of subtasks are merged according to block rules and used as input data for the next network layer.
[0012] Furthermore, in the above method, the method for dynamically adjusting the number of worker threads specifically includes: When the number of tasks in the task queue exceeds the current number of worker threads, new worker threads are added according to the step size of the new threads until the maximum number of threads is reached. When the idle time of a worker thread exceeds a preset threshold and the current number of threads is greater than the minimum number of threads, the idle thread is destroyed according to the step size of the destroyed thread.
[0013] Furthermore, in the above method, the maximum number of threads is 2 to 3 times the number of cores in a multi-core CPU, and the minimum number of threads is 1 / 3 to 1 / 2 of the number of cores in a multi-core CPU.
[0014] The advantages of this invention over the prior art are as follows: (1) The present invention adopts a layered design, which makes model deployment convenient. When upgrading the model, only the model files in the file system need to be replaced. (2) The present invention uses matrix block computing technology, dynamic thread pool technology and embedded operating system load balancing technology to split the deep learning inference computing process into several sub-tasks to be executed in parallel, making full use of multi-core CPU computing resources and improving the speed of deep learning inference. (3) This invention does not use cloud servers and does not rely on the network; it does not rely on cloud computing, thus improving the system's independence and reliability.
[0015] (4) This invention does not use acceleration modules such as GPU or FGPA, which reduces the hardware cost of deploying deep learning algorithms in embedded systems and lowers hardware cost and power consumption.
[0016] (5) Supports multiple embedded operating systems and CPU architectures, enhancing system flexibility.
[0017] (6) This invention simplifies the model upgrade process, improves maintenance efficiency, and makes model upgrades convenient.
[0018] (7) The present invention can maintain high performance even when the workload fluctuates greatly, and the dynamic thread pool optimizes resource utilization.
[0019] (8) The present invention improves scalability through layered design, thereby enhancing the scalability and maintainability of the system. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the deep learning inference implementation method based on an embedded operating system according to the present invention; Figure 2 This is a schematic diagram of the deep learning model structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the deep learning inference computation process according to an embodiment of the present invention. Detailed Implementation
[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0022] like Figure 1 As shown, this invention discloses a deep learning inference implementation system based on an embedded operating system, comprising: Model layer: Reads model files from the embedded operating system's file system, parses the network layer structure and parameters in the model files, and generates a node sequence in a linked storage structure; forms each network layer based on the node sequence in the linked storage structure; and generates the corresponding inference task based on the type of the network layer and the input data. Module layer: Receives inference tasks from model layer nodes; breaks down the inference tasks into multiple parallelizable subtasks using matrix block computation and sends them to the computation layer; Computation layer: Adds subtasks sent by the module layer to the task queue; competes to obtain subtasks from the task queue and performs calculations; dynamically adjusts the number of worker threads based on the length of the task queue and the idle status of the worker threads. Execution layer: Based on the task queue length and the current CPU core load, a suitable scheduling strategy is dynamically selected, and worker threads are bound to different cores of a multi-core CPU to achieve task parallelization.
[0023] Preferably, the computing layer includes a dynamic thread pool module, which comprises a task manager, a task queue, worker thread groups, and a manager thread; wherein, The task manager receives subtasks from the module layer and adds the subtasks to the task queue; The worker thread group competes to obtain subtasks from the task queue and performs calculations; The manager thread dynamically adjusts the number of worker threads based on the length of the task queue and the idle status of the worker threads.
[0024] Preferably, the method for dynamically adjusting the number of worker threads specifically includes: When the number of tasks in the task queue exceeds the current number of worker threads, new worker threads are added according to the step size of the new threads until the maximum number of threads is reached. When the idle time of a worker thread exceeds a preset threshold and the current number of threads is greater than the minimum number of threads, the idle thread is destroyed according to the step size of the destroyed thread.
[0025] Preferably, the maximum number of threads is 2 to 3 times the number of cores in a multi-core CPU, and the minimum number of threads is 1 / 3 to 1 / 2 of the number of cores in a multi-core CPU.
[0026] Preferably, the module layer includes a sequentially connected fully connected module, a convolutional module, a pooling module, a normalization module, and an activation function module.
[0027] Preferably, the matrix block calculation method is as follows: The input data is divided into multiple sub-regions, and each sub-region is used to generate an independent sub-task. The computation results of subtasks are aggregated through memory sharing or message queues.
[0028] Preferably, in the linked storage structure of the model layer, each node includes: network layer type, input / output dimension, weight pointer, bias pointer, doubly linked list pointers pointing to the previous and next nodes, and module type identifier; the module type identifier is used to associate the corresponding computation module of the module layer.
[0029] This invention discloses a deep learning inference implementation method based on an embedded operating system, comprising: Model loading: Read the model file from the embedded operating system's file system, parse the network layer structure and parameters in the model file, and generate a node sequence in a linked storage structure; form each network layer according to the node sequence in the linked storage structure; generate the corresponding inference task according to the type of the network layer and the input data; Task splitting: Based on the type of network layer, the inference task is split into multiple subtasks using a matrix partitioning method; Task scheduling: Subtasks are added to the task queue through a dynamic thread pool, and the number of worker threads is dynamically adjusted according to the task load; Parallel computing: Utilizes the scheduler of the embedded operating system to bind worker threads to different cores of a multi-core CPU, enabling the parallel execution of subtasks; Result aggregation: The computation results of subtasks are merged according to block rules and used as input data for the next network layer.
[0030] Preferably, the method for dynamically adjusting the number of worker threads specifically includes: When the number of tasks in the task queue exceeds the current number of worker threads, new worker threads are added according to the step size of the new threads until the maximum number of threads is reached. When the idle time of a worker thread exceeds a preset threshold and the current number of threads is greater than the minimum number of threads, the idle thread is destroyed according to the step size of the destroyed thread.
[0031] Preferably, the maximum number of threads is 2 to 3 times the number of cores in a multi-core CPU, and the minimum number of threads is 1 / 3 to 1 / 2 of the number of cores in a multi-core CPU.
[0032] Example This embodiment provides a deep learning inference implementation method based on an embedded operating system. The aim is to improve the computational speed of the deep learning inference process by fully utilizing the computing resources of embedded CPUs such as ARM and DSP, without needing to connect to a cloud server or add hardware acceleration modules such as GPUs or FPGAs. This is particularly effective when using multi-core CPUs, resulting in better real-time inference. This method supports various DSP and ARM chips and is compatible with multiple embedded operating systems such as VxWorks and embedded Linux.
[0033] Specifically, the present invention is implemented using the following technical solution.
[0034] This invention provides a deep learning inference implementation method based on an embedded operating system, which designs the deep learning inference process in layers, including a model layer, a module layer, a computation layer, and an execution layer.
[0035] Furthermore, the model layer's function during initialization is to read the model file configured in the embedded operating system's file system to obtain the network structure of each layer of the deep learning network, as well as parameters such as weights and biases. Each layer of the network adopts a linked storage structure, forming a linked structure of the entire deep learning network. During inference, the first node of the linked list receives system input, submits an inference task to the module layer, and uses the output as the input for the next node, until all nodes have completed their work and the system output is obtained.
[0036] Furthermore, the module layer includes modules commonly found in deep learning network structures, such as fully connected modules, convolutional modules, pooling modules, normalization modules like LN and BN, and activation function modules like ReLU, Sigmoid, and Softmax. The module layer's function is to receive inference tasks submitted by the model layer, use matrix block computation to break down the inference process of each module into parallelizable sub-processes, and submit computation tasks to the computation layer. This breaks down large matrix calculations in deep learning inference into several smaller sub-tasks, reducing the computation time of each sub-task.
[0037] Furthermore, the computing layer is implemented using a dynamic thread pool based on the embedded operating system, including modules such as a task manager, task queue, worker thread groups, and a manager thread. The task manager receives computing tasks submitted by the module layer, adds them to the task queue, and wakes up worker threads that are blocked and waiting for tasks. The task queue stores received computing tasks, waiting for worker threads to retrieve them. Worker thread groups compete to acquire computing tasks from the task queue. The manager thread periodically checks the status of the worker thread groups and dynamically creates or destroys threads based on the number of tasks and the idle status of the worker threads. The dynamic thread pool technology ensures that at least a minimum number of worker threads are always running, reducing the time consumed by frequently creating threads. When the number of tasks increases suddenly, the number of worker threads is appropriately increased to complete the computing tasks as quickly as possible; when the number of tasks is small, the number of worker threads is appropriately reduced to free up system computing resources for other functional modules of the embedded system.
[0038] Furthermore, the execution layer consists of the individual cores of a multi-core CPU. Leveraging the load balancing capabilities of the embedded operating system, the operating system automatically or programmatically schedules worker threads to run in parallel on different CPU cores, thereby fully utilizing the embedded CPU's computing resources and improving the speed of deep learning inference.
[0039] The invention will be further described in detail using the deployment of Lenet-5 networks in embedded systems as an example. A schematic diagram of the invention is shown below. Figure 1 The deep learning model structure for the embodiment is shown in the appendix. Figure 2 The embedded CPU used is the FT 6678DSP, which contains 8 cores; the operating system used is the SylixOS embedded real-time operating system.
[0040] The steps for implementing this example are as follows.
[0041] 1. The model layer reads the model file trained by deep learning frameworks such as PyTorch and TensorFlow, obtains the model structure of each layer of the network and parameter information such as weights and biases, and automatically generates a deep learning model chain structure. Each node contains information such as node type and parameters, which are used to describe the network structure and parameters corresponding to the current node.
[0042] The parameters for each network structure are different. For example, the parameters for a convolutional layer should include the number of output channels, the number of input channels, the kernel size, the stride, and the kernel data, while the parameters for a pooling layer should include compensation and filter size.
[0043] 2. Each node in the model layer receives input sequentially, submits inference tasks to the module layer, and obtains the output as the input for the next node. The input and output structure of each network node includes information such as dimension, size of each dimension, and data parameters.
[0044] 3. The module layer provides a unified interface to receive inference tasks submitted by the model layer and complete the forward propagation. The interface input consists of standard module input, module type, and module parameters, and the output is the standard module output.
[0045] 4. After receiving the inference task, the module layer, based on the module type, breaks down the computation task into several sub-tasks using a matrix-based block computation method, and submits them to the computation layer for calculation. Taking the convolution module as an example, as shown in the figure, computation tasks are submitted to the computation layer sequentially in the direction of convolution kernel movement, with each sub-task completing one convolution operation. See Appendix. Figure 3 .
[0046] 5. The dynamic thread pool in the computation layer specifies a minimum of 4 worker threads and a maximum of 10 worker threads. The thread pool provides a method for adding tasks; the task manager receives computation tasks submitted by the module layer and places them in the task queue; worker threads within a worker thread group compete to obtain tasks for execution. See appendix. Figure 3 .
[0047] 6. The execution layer schedules worker threads to different CPU cores according to the operating system, enabling parallel computation of multiple subtasks and reducing network inference time.
[0048] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the scope of the present invention should be included within the protection scope of the present invention.
[0049] 1. Model loading phase Function Description: The model layer reads the deep learning model file from the embedded operating system's file system, parses the network layer structure and parameters, and generates a node sequence with a linked storage structure.
[0050] Implementation steps: Model files trained using deep learning frameworks such as PyTorch or TensorFlow (e.g., .pt or .pb format).
[0051] Analyze the network layer structure, weights, and bias parameters in the model file.
[0052] Each network layer is organized into a linked storage structure in the form of a doubly linked list, and each node contains the following information: Network layer type (such as convolutional layer, fully connected layer, etc.).
[0053] Input and output dimensions.
[0054] Weight pointers and bias pointers.
[0055] A doubly linked list pointer that points to the previous and next nodes.
[0056] Module type identifier, used to associate the corresponding computing module in the module layer.
[0057] Case study: Taking LeNet-5 network as an example: Input layer: 28x28 grayscale image.
[0058] Convolutional layer 1: Number of input channels = 1, number of output channels = 6, kernel size = 5x5, stride = 1.
[0059] Pooling layer 1: Filter size = 2x2, step size = 2.
[0060] Convolutional layer 2: Number of input channels = 6, number of output channels = 16, kernel size = 5x5, stride = 1.
[0061] Pooling layer 2: Filter size = 2x2, step size = 2.
[0062] Fully connected layer 1: Input dimension = 400, output dimension = 120.
[0063] Fully connected layer 2: Input dimension = 120, output dimension = 84.
[0064] Output layer: Input dimension = 84, output dimension = 10 (classification task).
[0065] Data example: Model file size: approximately 5MB.
[0066] Model loading time: Approximately 200ms (tested on FT 6678DSP).
[0067] 2. Task Breakdown Phase Function Description: Based on the current network layer type, the input data is split into multiple subtasks using a matrix partitioning method.
[0068] Implementation steps: Choose the appropriate partitioning strategy based on the module type.
[0069] The input data is divided into multiple sub-regions, and the computation of each sub-region generates an independent sub-task.
[0070] The computation results of subtasks are aggregated through memory sharing or message queues.
[0071] Example: Taking convolutional layers as an example: Input data size: 28x28x1.
[0072] Kernel size: 5x5x1.
[0073] Step size: 1.
[0074] Blocking strategy: Divide the input data into several sub-regions by row or column, and perform a convolution operation on each sub-region.
[0075] Data example: Original convolution task: computationally intensive and time-consuming.
[0076] Number of subtasks after splitting: 10.
[0077] The computation time for each subtask is approximately 10ms.
[0078] 3. Task Scheduling Phase Function Description: Subtasks are added to the task queue through a dynamic thread pool, and the number of worker threads is dynamically adjusted according to the task load.
[0079] Implementation steps: Dynamic thread pool initialization: Minimum number of worker threads: 4.
[0080] Maximum number of worker threads: 10.
[0081] The Task Manager receives subtasks split from the module layer and adds them to the task queue.
[0082] The worker thread group competes for subtasks from the task queue and performs computations.
[0083] The manager thread dynamically adjusts the number of threads based on the task queue length and the idle status of worker threads.
[0084] rule: When the number of tasks in the task queue significantly exceeds the current number of worker threads, new worker threads are added, but the number of new threads shall not exceed a preset proportion (e.g., 50%) of the number of multi-core CPU cores.
[0085] When the idle time of a worker thread exceeds a preset threshold and the current number of threads is greater than the minimum number of threads, the idle thread is destroyed in increments.
[0086] Case Study: Initial number of threads: 4.
[0087] Task queue length: 15.
[0088] Number of new threads added: 3 (reaching the maximum number of threads of 10).
[0089] Adjusted number of threads: 7 (idle threads are destroyed after some tasks are completed).
[0090] 4. Parallel Computing Stage Function Description: Utilizes the scheduler of the embedded operating system to bind worker threads to different cores of a multi-core CPU, enabling the parallel execution of subtasks.
[0091] Implementation steps: The execution layer dynamically selects a suitable scheduling strategy based on the task queue length and the current CPU core load.
[0092] By binding worker threads to different cores of a multi-core CPU, task parallelization is achieved.
[0093] Case Study: Embedded CPU: FT 6678DSP (8 cores).
[0094] Operating system: SylixOS.
[0095] Thread binding strategy: Use APIs provided by the operating system (such as pthread_setaffinity_np) to bind threads to different cores.
[0096] Parallel computing effect: Inference time is reduced by about 60% compared to single-core operation.
[0097] 5. Results Aggregation Stage Function Description: Merges the computation results of subtasks according to block rules and uses them as input data for the next network layer.
[0098] Implementation steps: The results of subtask computations are aggregated through memory sharing or message queues.
[0099] According to the block division rules, the results of all subtasks are merged into complete output data.
[0100] Example: Taking convolutional layers as an example: Number of subtasks: 10.
[0101] Output size for each subtask: 5x5x1.
[0102] The merged output size is 24x24x6.
[0103] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.
[0104] The contents not described in detail in this specification are common knowledge to those skilled in the art.
Claims
1. An embedded operating system-based deep learning inference implementation system, characterized in that, include: Model layer: Reads model files from the embedded operating system's file system, parses the network layer structure and parameters in the model files, and generates a node sequence with a linked storage structure; Based on the node sequence of the linked storage structure, each network layer is formed; based on the type of the network layer and the input data, the corresponding inference task is generated and sent to the module layer; Module layer: Receives inference tasks from model layer nodes; breaks down the inference tasks into multiple parallelizable subtasks using matrix block computation and sends them to the computation layer; Computation layer: Adds subtasks sent by the module layer to the task queue; competes to obtain subtasks from the task queue and performs calculations; dynamically adjusts the number of worker threads based on the length of the task queue and the idle status of the worker threads. Execution layer: Based on the task queue length and the current CPU core load, a suitable scheduling strategy is dynamically selected, and worker threads are bound to different cores of a multi-core CPU to achieve task parallelization.
2. The system of claim 1, wherein, The computing layer includes a dynamic thread pool module, which comprises a task manager, a task queue, worker thread groups, and a manager thread; wherein, The task manager receives subtasks from the module layer and adds the subtasks to the task queue; The worker thread group competes to obtain subtasks from the task queue and performs calculations; The manager thread dynamically adjusts the number of worker threads based on the length of the task queue and the idle status of the worker threads.
3. The system of claim 2, wherein, The method for dynamically adjusting the number of worker threads is as follows: When the number of tasks in the task queue exceeds the current number of worker threads, new worker threads are added according to the step size of the new threads until the maximum number of threads is reached. When the idle time of a worker thread exceeds a preset threshold and the current number of threads is greater than the minimum number of threads, the idle thread is destroyed according to the step size of the destroyed thread.
4. The system of claim 3, wherein, The maximum number of threads is 2 to 3 times the number of cores in a multi-core CPU, and the minimum number of threads is 1 / 3 to 1 / 2 of the number of cores in a multi-core CPU.
5. The system according to claim 1, characterized in that, The module layer includes sequentially connected fully connected modules, convolutional modules, pooling modules, normalization modules, and activation function modules.
6. The system of claim 1, wherein, The matrix block calculation method is as follows: The input data is divided into multiple sub-regions, and each sub-region is used to calculate and generate an independent sub-task. The computation results of subtasks are aggregated through memory sharing or message queues.
7. The system of claim 1, wherein, In the linked storage structure of the model layer, each node contains: network layer type, input / output dimension, weight pointer, bias pointer, doubly linked list pointers to the previous and next nodes, and module type identifier; the module type identifier is used to associate the corresponding computation module of the module layer.
8. An embedded operating system-based deep learning inference implementation method, characterized in that, include: Model loading: Read the model file from the embedded operating system's file system, parse the network layer structure and parameters in the model file, and generate a node sequence in a linked storage structure; Based on the node sequence of the linked storage structure, each network layer is formed; based on the type of the network layer and the input data, the corresponding inference task is generated. Task splitting: Based on the type of network layer, the inference task is split into multiple subtasks using a matrix partitioning method; Task scheduling: Subtasks are added to the task queue through a dynamic thread pool, and the number of worker threads is dynamically adjusted according to the task load; Parallel computing: Utilizes the scheduler of the embedded operating system to bind worker threads to different cores of a multi-core CPU, enabling the parallel execution of subtasks; Result aggregation: The computation results of subtasks are merged according to block rules and used as input data for the next network layer.
9. The method of claim 8, wherein, The method for dynamically adjusting the number of worker threads is as follows: When the number of tasks in the task queue exceeds the current number of worker threads, new worker threads are added according to the step size of the new threads until the maximum number of threads is reached. When the idle time of a worker thread exceeds a preset threshold and the current number of threads is greater than the minimum number of threads, the idle thread is destroyed according to the step size of the destroyed thread.
10. The method of claim 9, wherein, The maximum number of threads is 2 to 3 times the number of cores in a multi-core CPU, and the minimum number of threads is 1 / 3 to 1 / 2 of the number of cores in a multi-core CPU.