Hardware resource allocation method and device for edge devices
By acquiring task information in real time and dynamically configuring hardware resource modules on edge devices, reconfigurable operators are realized, solving the problem of limited computing power on edge devices, improving resource utilization and execution efficiency, and supporting the deep integration of edge computing and artificial intelligence.
Patent Information
- Application Number
- CN202611235309.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-17
AI Technical Summary
Edge computing devices have limited computing power, scarce memory resources, and strict power consumption constraints. Traditional computing operators cannot dynamically adapt to hardware resources, resulting in low matching degree between AI model algorithms and hardware platforms and insufficient resource utilization efficiency.
By acquiring task information in real time, the hardware resources of edge devices are virtually divided into multiple modules, generating M control commands, reconfiguring the calculation, storage, and control array parameters, realizing reconfigurable operators, and adapting to the allocation of hardware resources for different algorithms.
It improves the compatibility between operators and edge hardware, enhances the execution efficiency and resource utilization of edge devices, and supports the deep integration of edge computing and artificial intelligence.
Smart Images

Figure CN123114733A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of edge computing technology, and more specifically to a hardware resource allocation method for edge devices, a hardware resource allocation device for edge devices, a machine-readable storage medium, and an electronic device. Background Technology
[0002] With the deep integration of edge computing and artificial intelligence, edge application scenarios such as smart terminals, industrial IoT, and autonomous driving place stringent demands on the real-time performance and efficiency of AI tasks. The end-to-end latency issue of AI inference has become a key bottleneck restricting the large-scale deployment of these scenarios. Edge devices are typically characterized by limited computing power, scarce memory resources, and strict power consumption constraints, which pose significant challenges to the lightweight, energy-efficient, and adaptable computing architecture.
[0003] When faced with the application requirements of edge computing integration, traditional computing operators have poor algorithm adaptability and rigid structure. They cannot dynamically adjust the operator structure according to the hardware resource characteristics such as computing power, memory bandwidth and other resources available in real time on edge devices, resulting in low matching degree between AI model algorithms and heterogeneous hardware platforms and insufficient resource utilization efficiency. Summary of the Invention
[0004] The purpose of this invention is to provide a hardware resource allocation method for edge devices, a hardware resource allocation device for edge devices, a machine-readable storage medium, and an electronic device. This method achieves adaptive reconstruction of operator structures, enables flexible hardware adaptation for different algorithms, improves the matching degree between operators and edge hardware, and enhances the execution efficiency and resource utilization of edge devices.
[0005] To achieve the above objectives, the first aspect of this application provides a hardware resource allocation method for an edge device, comprising: Get task information in real time; Based on the task information, a task module is determined from a set of preset modules. The preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules. Based on the task information and the task module, generate M control instructions; The task module re-determines the array configuration parameters of the task module based on the M control command to obtain a reconfigurable operator, and executes the task based on the reconfigurable operator.
[0006] In this embodiment of the application, the step of the task module re-determining the array configuration parameters of the task module based on the M control instruction to obtain the reconfigurable operator includes: The task module determines the computing array configuration parameters, storage array configuration parameters, control array configuration parameters, and in-memory computing configuration parameters based on the M control instructions. Based on the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters, a reconfigurable operator is obtained; The sum of the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters is equal to the total configuration parameters of the task module.
[0007] In this embodiment of the application, determining the task module from a set of preset modules based on the task information includes: Based on the task information, determine whether the current task is a convolution task; If the current task is confirmed to be a convolution task, the original data of the convolution task is structured into a convolutional layer matrix to obtain the task matrix. Select a module that meets the preset conditions from a plurality of preset modules as the task module. The preset conditions include: the in-memory computing matrix unit of the module matches the task matrix.
[0008] In this embodiment of the application, selecting a module that meets preset conditions from a plurality of preset modules as a task module includes: Based on the computing array, storage array, and task matrix in each module, the number of convolutional layer input channel split groups and the number of convolutional layer output channel split groups corresponding to each module are determined respectively. Determine whether there is a module among the preset multiple modules that meets the preset conditions, wherein the preset conditions are that the number of groups split into the input channel of the convolutional layer and the number of groups split into the output channel of the convolutional layer corresponding to the module are both equal to the actual number of groups split into by the module; If a module that meets the preset conditions is found among the preset multiple modules, then the module that meets the preset conditions is designated as the task module.
[0009] In this embodiment of the application, it also includes: If it is determined that there is no module that meets the preset conditions among the preset multiple modules, the convolution task is split into multiple convolution subtasks; Based on each convolutional subtask, multiple subtask modules are determined from a set of pre-defined modules. Based on the multiple sub-task modules, a task module is obtained.
[0010] In this embodiment of the application, generating M control instructions based on the task information and the task module includes: Based on the task information and the task module, multiple splitting strategies are determined, each of which includes task splitting information and array allocation information. The overall optimization index of each splitting strategy was calculated separately. Based on the full-process optimization index of each splitting strategy, the optimal strategy is determined among the multiple splitting strategies. Based on the optimal strategy, M control instructions are generated.
[0011] In this embodiment of the application, the overall optimization index of the splitting strategy is calculated, including: Obtain the target task accuracy requirements, splitting rationality indicators, and edge hardware constraint indicators for the splitting strategy; The target task accuracy requirement index, splitting rationality index, and edge hardware constraint index of the splitting strategy are weighted and summed to obtain the full-process optimization index of the splitting strategy.
[0012] In this embodiment of the application, obtaining the target task accuracy requirement index of the splitting strategy includes: Obtain the original number of samples; Based on the original sample size and the task matrix, the target task accuracy requirement index of the splitting strategy is calculated.
[0013] In this embodiment of the application, obtaining the splitting rationality index of the splitting strategy includes: Based on the task splitting information, array allocation information, and the actual number of module splitting groups of the task module corresponding to the splitting strategy, the splitting rationality index of the splitting strategy is calculated.
[0014] In this embodiment of the application, obtaining the edge hardware constraint metrics of the splitting strategy includes: Obtain the percentage of in-memory computing resources in the task module; Based on the task matrix and the array allocation information corresponding to the splitting strategy, the model inference delay parameters are determined. Based on the proportion of in-memory computing resources in the task module and the model inference latency parameters, the edge hardware constraint index of the splitting strategy is obtained.
[0015] A second aspect of this application provides a hardware resource allocation device for an edge device, comprising: The acquisition module is used to acquire task information in real time; The determination module is used to determine the task module from a set of preset modules based on the task information. The set of preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules. The instruction module is used to generate M control instructions based on the task information and the task module; The reconfiguration module is used by the task module to redetermine the array configuration parameters of the task module based on the M control instructions, obtain a reconfigurable operator, and execute the task based on the reconfigurable operator.
[0016] In this embodiment of the application, the reconstruction module includes: The parameter determination unit is used to determine the computing array configuration parameters, storage array configuration parameters, control array configuration parameters, and in-memory computing configuration parameters by the task module based on the M control command; wherein the sum of the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters is equal to the total configuration parameters of the task module; The reconfiguration unit is used to obtain a reconfigurable operator based on the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters.
[0017] In this embodiment of the application, the determining module includes: The judgment unit is used to determine whether the current task is a convolution task based on the task information. The structuring unit is used to structure the original data of the convolution task into a convolutional layer matrix form when the current task is confirmed to be a convolution task, so as to obtain a task matrix. The selection unit is used to select a module that meets preset conditions from a plurality of preset modules as a task module. The preset conditions include: the in-memory computing matrix unit of the module matches the task matrix.
[0018] A third aspect of this application provides an electronic device, the electronic device comprising: At least one processor; A memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the hardware resource allocation method for the edge device described above by executing the instructions stored in the memory.
[0019] A fourth aspect of this application provides a machine-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the aforementioned hardware resource allocation method for an edge device.
[0020] The above technical solution involves acquiring task information in real time; determining a task module from a set of preset modules based on this information (the preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules); generating M control instructions based on the task information and the task modules; and re-determining the array configuration parameters of the task module based on the M control instructions to obtain a reconfigurable operator, which is then used to execute the task. By virtually dividing the hardware resources of the edge device into multiple reconfigurable modules, acquiring task information in real time during runtime and intelligently matching task modules accordingly, and then generating M control instructions to dynamically configure the computation, storage, control, and storage-computing integrated array parameters within the modules, a reconfigurable operator precisely adapted to the current task can be reconstructed in real time on the hardware. This achieves adaptive reconfiguration of the operator structure, enabling flexible hardware adaptation for different algorithms, improving the matching degree between the operator and the edge hardware, significantly improving the execution efficiency and resource utilization of the edge device, and providing key technical support for the deep integration of edge computing and artificial intelligence.
[0021] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description
[0022] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 The illustration shows a schematic flowchart of a hardware resource allocation method for an edge device according to an embodiment of this application; Figure 2 A schematic diagram of a reconfigurable operator architecture according to an embodiment of this application is shown. Figure 3 This illustration shows a schematic diagram of an in-memory computing configuration according to an embodiment of this application; Figure 4 This illustration schematically shows a second in-memory computing configuration according to an embodiment of this application; Figure 5 This illustration schematically shows a memory computing diagram according to an embodiment of the present application; Figure 6 This illustration schematically shows a structural diagram of a hardware resource allocation device for an edge device according to an embodiment of this application; Figure 7 The diagram illustrates the internal structure of a computer device according to an embodiment of this application.
[0023] Explanation of reference numerals in the attached figures 210 - Acquisition module; 220 - Determining module; 230 - Instruction module; 240 - Reconfiguration module; A01 - Processor; A02 - Network interface; A03 - Internal memory; A04 - Display screen; A05 - Input device; A06 - Non-volatile storage medium; B01 - Operating system; B02 - Computer program. Detailed Implementation
[0024] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.
[0025] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.
[0026] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.
[0027] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0028] Please refer to Figure 1 , Figure 1 This illustration schematically shows a flowchart of a hardware resource allocation method for an edge device according to an embodiment of this application. This embodiment provides a hardware resource allocation method for an edge device, including the following steps: Step 110: Obtain task information in real time; In this embodiment, the aforementioned task information can be task information determined when the inference engine executes to a certain layer of the model. Specifically, the task information can be obtained during compilation by parsing the model file. The task information can be a structured data packet that includes information such as operator type, tensor shape, data address, precision, and constraints.
[0029] For example, for a CNN model, the input is a feature map containing 64 channels, each with a spatial dimension of 56 pixels high and 56 pixels wide. The output is a new feature map with 128 channels, with the spatial dimensions remaining unchanged at 56 pixels high and 56 pixels wide. The batch size is 1. The corresponding task information can be represented as: convolution, [1, 64, 56, 56] → [1, 128, 56, 56], INT8.
[0030] Step 120: Based on the task information, determine the task module from a set of preset modules. The preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules. In this embodiment, the hardware resources of the edge device can be pre-split into three independent and flexibly combinable arrays: a computing array, a storage array, and a control array. Specifically, all multiply-accumulate units and arithmetic logic units on the edge device can be virtualized into a computing array; all on-chip caches and registers on the edge device can be virtualized into a storage array; and the programmable state machine, microcontroller, and instruction decoding logic can be virtualized into a control array.
[0031] When dividing the system into modules, each module includes a reconstruction of the three arrays mentioned above. Each module is a complete subsystem, integrating a computing array, a storage array, and a control array. Some modules may also optionally integrate an in-memory computing unit. The arrays are connected within the module via a standard bus interface, and inter-module array resource allocation is supported.
[0032] Please refer to Figure 2 , Figure 2A schematic diagram of a reconfigurable operator architecture according to an embodiment of this application is shown. Module i includes a computing array, a storage array, a control array, and a reconfigurable integrated operator. Array resources can be adjusted between modules. For any module, the computing array is responsible for computing functions. It can interconnect data between the computing array and the cache array, and can also adjust the data of the matrix, such as vector transpose. The operation units can be configured based on operators, supporting various operations such as tensor operations and convolution operations. The number and precision of the operation units can be dynamically adjusted to adapt to the computing power characteristics of edge hardware. The storage array is responsible for storing data and also serves as a cache for intermediate data. It supports dynamic allocation and scheduling of data with the computing array, reducing data movement and memory access latency. The control array provides control signals for the above two arrays, including read / write, computation, and timing functions. The built-in reconfigurable operator logic receives and compiles M control instructions to realize dynamic scheduling and collaborative control of the computing module and storage module, supports real-time reconfiguration of the operator structure, and adapts to different algorithms and hardware resource changes.
[0033] In practical implementation, the number and type of modules can be predetermined based on the actual application scenario and hardware resources. For example, the statistical characteristics of various operators in the target algorithm (such as CNN, Transformer) can be analyzed. For instance, computationally intensive operators (such as convolution, matrix multiplication) require large computing arrays and in-memory computing support; memory- or control-intensive operators (such as activation functions, normalization) are more suitable for small computing arrays, large caches, and flexible control logic. Within the chip's total area and power consumption budget, hardware resources are divided into multiple modules. Each module is allocated a fixed total resource limit. The generated modules can have various types; for example, please refer to [link / reference]. Figure 3 and Figure 4 , Figure 3 This illustration shows a schematic diagram of an in-memory computing configuration according to an embodiment of this application; Figure 4 The diagram illustrates a second in-memory computing configuration according to an embodiment of this application. Both types of modules have a higher proportion of in-memory computing units, which can be used to handle core workloads such as convolution and fully connected operations. Modules with a higher proportion of storage and control arrays can also be generated to handle element-wise operations, data rearrangement, etc.
[0034] When determining the task modules, the current operator and its computational requirements can be determined first based on the task information generated after the model is compiled. Then, the real-time status of each module can be evaluated. The real-time status includes load, idle status, resident data, and whether the module integrates a memory computing unit. Based on the real-time status information, the module that can process the current operator and meet the computational requirements can be selected from multiple modules. This can be one or more modules. These modules are the task modules, which are used to execute the current operator.
[0035] In some embodiments, determining the task module from a preset set of modules based on the task information includes: First, based on the task information, determine whether the current task is a convolution task; In this embodiment, the task information includes operator type, and the current task can be determined as a convolution task based on the operator type.
[0036] Then, if the current task is confirmed to be a convolution task, the original data of the convolution task is structured into a convolutional layer matrix to obtain the task matrix. In this embodiment, if the current task is a convolution task, the original data of the convolution task is structured into a convolutional layer matrix; if the current task is not a convolution task, the task module is determined according to the above steps. The above-mentioned structured data of the convolution task into a convolutional layer matrix refers to extracting convolution parameters from the task information, including the number of input channels, the height and width of the convolutional kernel, the spatial dimensions of the input feature map, etc., and rearranging the original four-dimensional input tensor into a two-dimensional matrix based on these parameters. The above-mentioned structuring process includes: expanding each local window participating in the convolution operation in the original data into a column vector, the length of which is equal to the number of input channels multiplied by the height of the convolutional kernel multiplied by the width of the convolutional kernel; arranging the column vectors corresponding to all windows in column order to form a matrix with X rows. kh The number of columns is X kw A two-dimensional task matrix. Where X kh It equals the number of input channels multiplied by the height of the convolution kernel, then multiplied by the width of the convolution kernel, X. kw It equals the height of the output feature map multiplied by its width, where k is the task matrix input sequence.
[0037] Finally, a module that meets the preset conditions is selected from a number of preset modules as the task module. The preset conditions include: the in-memory computing matrix unit of the module matches the task matrix.
[0038] In this embodiment, candidate modules with in-memory computing matrix units can be selected from all modules first, and then the number of rows X of the task matrix can be determined. kh Sum of columns X kw The dimensions of the task matrix are compared with the width and height of the in-memory array of the candidate modules respectively. If the two dimensions of the task matrix do not exceed the corresponding dimensions of the array, it is considered a match. Finally, the module with the lowest load or the shortest estimated execution time is selected from the successfully matched modules as the task module for this convolution task.
[0039] By first structuring the original data of the convolution task into a convolutional layer matrix and then confirming the task modules, complex multidimensional convolution tasks can be uniformly transformed into a standardized two-dimensional matrix scale. This allows for quick and accurate determination of whether a task can be directly and completely executed by a single in-memory computing unit, thus improving the accuracy of task module determination.
[0040] In some embodiments, selecting a module that meets preset conditions from a plurality of preset modules as a task module includes: First, based on the computing array, storage array, and task matrix in each module, determine the number of convolutional layer input channel split groups and the number of convolutional layer output channel split groups corresponding to each module. In this embodiment, for each module, the height and width of its storage array can be obtained, denoted as H respectively. Mis and W Mis Let the height and width of the array be H. Mic and W Mic Based on the computing array, storage array, and task matrix, the number of input channel subdivisions and output channel subdivisions for each module are determined. Specifically, the number of subdivisions required for the input and output channels of the convolutional layer can be represented as G. in and G out ,in:
[0041] Second, determine whether there is a module among the preset multiple modules that meets the preset conditions, wherein the preset conditions are that the number of groups split into the input channel of the convolutional layer and the number of groups split into the output channel of the convolutional layer corresponding to the module are both equal to the actual number of groups split into the module. In this embodiment, after calculating the number of split groups (G) corresponding to each module... in G out After that, it can be determined whether each module meets the preset conditions. The preset conditions refer to: the number of convolutional layer input channel split groups G calculated by the module. in and the number of output channels split into groups G of the convolutional layer out All fall within the module's preset capability range G. i Within. The aforementioned capability range G i This capability range G can be pre-calibrated based on the size of the in-memory computing array, the cache capacity of the storage array, and the cyclic management capability of the control array. i This can represent the set of split groups that the module can process in a storage-computing mode.
[0042] When making the above judgment, G can be checked. in G outDoes it fall within the scope of this module's capabilities? i An element in the set. If it belongs to the set, then it satisfies G. in ∈G i And G out ∈G i If the condition is met, the module is deemed to meet the preset conditions, indicating that the in-memory computing unit of the module is capable of covering the computational scale of the current convolution task, and the convolution operation can be efficiently completed by configuring its internal resources to in-memory computing mode; if the condition is not met, the module is deemed to not meet the preset conditions, indicating that the computational scale of the current convolution task exceeds the individual carrying capacity of the module's in-memory computing unit.
[0043] Third, determine that there is a module among the preset multiple modules that meets the preset conditions, and use the module that meets the preset conditions as the task module.
[0044] In this embodiment, based on the above judgment, if one or more modules among the preset modules meet the preset conditions, the module with the smallest number of split groups can be selected first to reduce loop control overhead; or the module with the lowest current load and highest idle degree can be selected first to achieve load balancing; or the module in the in-memory computing array that already has the weight data required by the current task residing there can be selected first to avoid duplicate loading of weight data. Specific settings can be made according to actual conditions, and this embodiment does not impose limitations.
[0045] Once the task module is identified, for that task module, its corresponding G in G out This represents the actual number of groups split in this convolution task. Based on this, corresponding M control instructions are generated to reconstruct the task module into an in-memory computing mode. The in-memory computing array of this module, while the weight data remains in place, utilizes G... in Group input channel splitting and G out The output channels of the group are split into time-sharing or loop-based calculations to complete the matrix multiplication operations of the entire convolutional layer. This reduces redundant calculations and data movement, lowers memory access latency, and enables in-situ multiplication and accumulation. Weight data does not need to be moved; multiplication and accumulation operations are performed directly in the storage unit, reducing power consumption and latency.
[0046] In some embodiments, it also includes: First, determine that there is no module that meets the preset conditions among the preset multiple modules, and then split the convolution task to obtain multiple convolution sub-tasks; In this embodiment, the convolution task can be divided into multiple convolution subtasks based on the task matrix size of the current convolution task and the overall resource distribution of available modules. This division can be performed along the input channel dimension and / or the output channel dimension. For example, when the input channel dimension is too large, causing G... inG beyond all modules i At the upper limit, the task matrix can be moved along X. kh The direction is divided into multiple segments, each segment corresponding to a continuous subset of the input channels, forming a convolutional subtask; when the output channel dimension is too large, it causes G... out G beyond all modules i At the upper limit, the task matrix can be moved along X. kw The direction is divided into multiple segments, each segment corresponding to a continuous subset of output spatial locations or output channels, forming a convolutional subtask; when both dimensions exceed the upper limit, two-dimensional segmentation is performed simultaneously along both dimensions.
[0047] Second, based on each convolutional subtask, multiple subtask modules are determined from a set of pre-defined modules. In this embodiment, each convolutional subtask can be determined according to the aforementioned process for determining the task module. Specifically, for each convolutional subtask, the number of rows and columns of its subtask matrix can be extracted. All modules are traversed, prioritizing those modules that possess in-memory computing units and whose subtask splitting group number falls within their capability range G. i If a module with in-memory computing capability can be found, the subtask will be assigned to such a module, reconfiguring it into an in-memory computing mode, where multiplication and accumulation operations will be performed by an in-memory computing array. If, for a convolution subtask, no module with in-memory computing capability can be found, the subtask can be assigned to a module with sufficient computing and storage arrays. In this case, the subtask module is configured to use its computing array to perform matrix multiplication and accumulation operations, and its storage array as a buffer for input data, intermediate data, and output data.
[0048] Third, based on the multiple sub-task modules, a task module is obtained.
[0049] In this embodiment, the set of all selected subtask modules is collectively defined as the task module for the current convolutional task. The task module is used to complete the computation of the entire convolutional layer.
[0050] Please refer to Figure 5 , Figure 5 This diagram illustrates an in-memory computing schematic according to an embodiment of this application. Each of the k data points corresponds to a pathway, which sequentially runs from convolutional layer 1 to convolutional layer Gi, culminating in array activation output. The outputs from each pathway are fused and fed into the next network. This achieves parallel and serial data processing for efficient execution.
[0051] Through the above-mentioned serial-parallel collaborative mechanism, when the G of the convolution task... in and G outWhen the data exceeds the matching range of a single in-memory computing unit, the system can flexibly allocate task data to computing arrays, storage arrays, and in-memory computing arrays of different modules for convolution calculations, which helps to achieve efficient execution of large-scale convolution tasks.
[0052] Step 130: Based on the task information and the task module, generate M control instructions; In this embodiment, M control instructions are used to define the function and structure of a task module at runtime. The generation process of the M control instructions can be as follows: based on task information and hardware capability parameters such as the computing array, in-memory computing unit, and cache array within the selected task module, first, the number of partition groups and cache partitioning scheme adapted to the current hardware resources are calculated; then, configuration parameters defining the computing array arrangement, in-memory computing unit operating mode, and the number of loops of the control array state machine are selected from a preset operator microcode component library; finally, the configuration parameters, along with task address, precision, and synchronization signals, are packaged and encoded into M control instructions to guide the task module in internal hardware reconfiguration. The aforementioned operator microcode component library is a pre-compiled set of parameterized hardware configuration templates that the global scheduler can directly retrieve and call based on operator type and module hardware characteristics when generating M control instructions.
[0053] In some embodiments, generating M control instructions based on the task information and the task module includes: First, based on the task information and the task module, multiple splitting strategies are determined, each of which includes task splitting information and array allocation information; In this embodiment, based on the task information and the computational scale and available hardware resources of the task module, multiple splitting strategies are generated. Each splitting strategy is a complete execution scheme description, including task splitting information and array allocation information. The task splitting information describes how to divide the task into multiple subtasks. For example, whether to split along the input channel dimension and the number of groups G split along that dimension. in Whether to split along the output channel dimension and the number of groups G to be split along that dimension. out The information includes the size of each subtask matrix, its starting offset address in the original task matrix, and the data dependencies between subtasks. Array allocation information describes which array resources from which task modules will execute each subtask. For example, which subtasks are executed by task modules with in-memory computing units in in-memory computing mode, and which subtasks are executed by the task module's compute array in conjunction with its storage array; the array configuration size of the compute array within each module, the cache partitioning scheme of the storage array; and the data flow transmission paths and synchronization timing between modules. Multiple splitting strategies can be generated by enumerating different combinations of splitting groups and different task module allocation schemes.
[0054] Then, the overall optimization index of each splitting strategy is calculated. In this embodiment, for each of the above splitting strategies, its corresponding full-process optimization index R can be calculated to quantitatively evaluate the overall advantages and disadvantages of the strategy.
[0055] The overall optimization index of the splitting strategy is calculated, including: First, obtain the target task accuracy requirements, splitting rationality indicators, and edge hardware constraint indicators for the splitting strategy; Second, the target task accuracy requirement index, splitting rationality index, and edge hardware constraint index of the splitting strategy are weighted and summed to obtain the full-process optimization index of the splitting strategy.
[0056] In this embodiment, the overall optimization exponent R of the splitting strategy can be expressed as R = α R task +β R split +γ R hardware , Among them, R task The target task accuracy requirement index for the splitting strategy is used to reflect whether the splitting strategy meets the accuracy requirements of the target task. It is a fundamental constraint for fusion optimization, ensuring that the optimization process does not deviate from the core task requirements; R split This is a metric for the rationality of a splitting strategy, used to reflect the structural rationality of splitting convolutions within the splitting strategy. It ensures that the split convolutional units (array splitting / serial-parallel computation) are adapted to task attributes and hardware resources, improving the computational speed and hardware utilization efficiency of the splitting; R hardware The metric represents the edge hardware constraints of the splitting strategy, reflecting edge hardware constraints (computing power, cache, bandwidth). This ensures that the optimized splitting strategy can be directly deployed on the target edge hardware, avoiding exceeding hardware resource limits after quantization and splitting. α, β, and γ are weighting coefficients.
[0057] The aforementioned weighting coefficients can be used to balance the priority of various loss terms. In practice, they can be dynamically adjusted based on task attributes and edge hardware characteristics to meet the optimization needs of different scenarios. For example, the value range is [0.1, 0.5], and the specific value selection rules are as follows: Accuracy-first scenarios (such as power protection control, high-precision detection): α = 0.5, β = 0.3, γ = 0.2, focusing on suppressing quantization errors; Efficiency-first scenarios (such as lightweight terminals, low-power edge nodes): α = 0.2, β = 0.3, γ = 0.5, focusing on meeting hardware constraints; Balanced scenarios (power acquisition services, general edge computing): α = 0.3, β = 0.4, γ = 0.3, balancing accuracy and hardware adaptability.
[0058] The process of obtaining the target task accuracy requirement index of the splitting strategy includes: obtaining the number of original samples; and calculating the target task accuracy requirement index of the splitting strategy based on the number of original samples and the task matrix.
[0059] In this embodiment, a set of original samples can be pre-stored. These original samples are typical test data collected and labeled from actual business scenarios, and the number of original samples is denoted as X. The target task accuracy requirement index R of the above splitting strategy is... task The calculation formula is: , Where i is the number of task modules, X kh Let be the row number of the task matrix. Let k be the number of columns in the task matrix, and k be the input sequence of the task matrix.
[0060] The process of obtaining the rationality index of the splitting strategy includes: calculating the rationality index of the splitting strategy based on the task splitting information, array allocation information and the actual number of module splitting groups of the task module corresponding to the splitting strategy.
[0061] In this embodiment, the splitting rationality index R of the splitting strategy split The calculation formula is: , Where Gi represents the capability range of task module i, i.e., the actual number of split groups (groups) of the convolution of task module i, which can be the number of split groups calculated based on task attributes (feature map size, number of channels) and hardware. ||. || is the Frobenius norm, used to measure the resource consistency between the computing kernel and the storage kernel; This is a penalty coefficient used to constrain the deviation between the input and output split numbers, ensuring that the split structure fits the task and hardware requirements. It can be set empirically. H Mis and W Mis H represents the height and width of the storage array for task module i, respectively. Mic and W Mic These represent the height and width of the computation array for task module i, respectively.
[0062] The process of obtaining edge hardware constraint metrics for the splitting strategy includes: obtaining the proportion of in-memory computing resources of the task modules; determining model inference latency parameters based on the array allocation information corresponding to the task matrix and the splitting strategy; and obtaining edge hardware constraint metrics for the splitting strategy based on the proportion of in-memory computing resources of the task modules and the model inference latency parameters.
[0063] In this embodiment, the edge hardware constraint index R of the above-mentioned splitting strategy hardwareThe calculation formula is: , in: Configure parameters for the in-memory computing module i. The total configuration parameters for task module i. / The proportion of in-memory computing resources for task module i. / Used to represent inference latency, this ratio comprehensively reflects the computational density and time cost required to complete the convolution task under the current splitting strategy. The larger the ratio, the heavier the computational workload relative to the available in-memory computing resources, and the higher the inference latency; conversely, the smaller the ratio, the more abundant the available resources, and the lower the latency.
[0064] It should be noted that R can also be used. hardware The evaluation is conducted jointly with the real-time requirement threshold for edge scenarios. The real-time requirement threshold can be pre-set according to the application scenario; for example, in real-time control scenarios of the Industrial Internet of Things (IIoT), the threshold can be set to 10 milliseconds, and in data analysis scenarios, it can be set to 10 seconds. / When the real-time requirement threshold for the corresponding scenario is met, R hardware A positive value indicates that the splitting strategy is feasible and efficient under hardware constraints; when / When the threshold is exceeded, R hardware Take a negative value or a very negative value to eliminate the infeasible strategy in the optimization comparison.
[0065] Then, based on the full-process optimization index of each splitting strategy, the optimal strategy is determined among the multiple splitting strategies; In this embodiment, after determining the overall optimization index of each splitting strategy, the strategy with the largest overall optimization index can be selected as the optimal strategy from among the multiple splitting strategies.
[0066] Finally, based on the optimal strategy, M control instructions are generated.
[0067] In this embodiment, after determining the optimal strategy, M control instructions for each task module can be generated based on the task splitting information and array allocation information in the optimal strategy. The specific process of generating M control instructions has been discussed above and will not be repeated here.
[0068] By constructing a full-process optimization index encompassing three indicators—target task accuracy requirements, splitting rationality, and edge hardware constraints—for the fusion splitting strategy, model inference accuracy, execution efficiency, and hardware adaptability are incorporated into the decision-making framework of the splitting strategy. This allows the selection of the optimal solution from multiple splitting strategies that balances accuracy preservation, splitting balance, and real-time constraints. Based on this, M-control instructions are generated to further improve execution efficiency and resource utilization. This achieves the fusion of model inference accuracy, execution efficiency, and hardware adaptability through collaborative optimization of task accuracy requirements, splitting convolutional structures, and edge hardware constraints.
[0069] Step 140: The task module re-determines the array configuration parameters of the task module based on the M control instruction to obtain a reconfigurable operator, and executes the task based on the reconfigurable operator.
[0070] In this embodiment, after receiving the M control command, each task module can parse the command and extract the configuration parameters of the computing array, storage array, in-memory computing unit, and loop control parameters. Based on these parameters, the module can then reconfigure the arrays within each task module. Specifically, the computing array is configured as a pulsating array or vector processing unit of a specified size with a set computational precision; the storage array is divided into input and output buffer areas with specified capacity and partitioning schemes; the in-memory computing unit is set to weighted latching mode or data pass-through mode with a specified address for the weighted data; and the control array... The column itself is loaded into an execution state machine that matches the current operator type. After the above configuration is completed, the task module dynamically generates a hardware execution body that is precisely adapted to the current task in terms of computational scale, data flow mode and accuracy requirements, which is the reconfigurable operator. Finally, according to the reconfigurable operator, under the scheduling of the control array, according to the number of split groups and the number of loops specified in the M control instruction, the in-memory computing unit completes the multiplication and accumulation operations in place, the computing array executes the part and the accumulation and post-processing, and the storage array realizes double buffer data transfer. The arrays work together to complete the calculation of the entire task in a pipeline manner and write the result back to the target memory address.
[0071] In some embodiments, the step of the task module re-determining its array configuration parameters based on the M control instructions to obtain a reconfigurable operator includes: First, the task module determines the computing array configuration parameters, storage array configuration parameters, control array configuration parameters, and in-memory computing configuration parameters based on the M control command; wherein, the sum of the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters is equal to the total configuration parameters of the task module.
[0072] In this embodiment, after receiving the control command M, task module i parses the control command M using its internal control array and extracts the configuration parameters, which include: calculation array configuration parameters M. ic Storage array configuration parameters M is Control array configuration parameters M ik And the in-memory computing configuration parameter M ih .
[0073] Wherein, the computing array configuration parameter M ic The hardware configuration used to define the internal computing array of this task module includes at least: the number of active computing units in this task, and the array height H of the computing units. Mic and width W Mic The storage array configuration parameter M is This is used to define the partitioning and usage of the internal storage array of this module, and includes at least: the cache capacity allocated to this task, the cache area partitioning scheme, and the height H of the storage array. Mis and width W Mis The control array configuration parameter M ik The parameters used to define the operating mode of the internal control array of this module include at least: the type of execution state machine to be loaded for this task, the outer and inner loop counters corresponding to the number of split groups Gin and Gout, the address step size and jump conditions of each loop level, and the synchronization signal identifiers and interrupt response strategies required for synchronization with other modules. The in-memory computing configuration parameter M... ih This is used to define the operating mode of the in-memory computing unit within the task module, and includes at least the logical height H of the in-memory computing array. Mih and width W Mih The starting address of the latch for weighted data and the data mapping method. When M ih If the value is zero or not configured, it means that the in-memory computing unit will not be used in this task.
[0074] The above four sets of configuration parameters satisfy the total resource constraint condition, i.e., M ic M is M ik M ih The sum of the hardware resources occupied by each module is equal to the total configuration parameter M allocated to this task module in this task. i总 The overall configuration parameter is determined by the scheduler when generating the M control command, based on the scale of the subtasks undertaken by the module in the optimal strategy, and its value does not exceed the physical resource limit of the module.
[0075] Then, based on the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters, a reconfigurable operator is obtained.
[0076] In this embodiment, after the control array completes the parsing and extraction of each set of configuration parameters, the control array sends configuration signals to the computing array, storage array, in-memory computing unit and itself through the bus interface inside the module based on these parameters, and performs physical reconstruction of each array.
[0077] For computing arrays, control arrays follow M ic The specified height H Mic and width W Mic The computing units are re-interconnected to form a pulse array or vector pipeline structure of a specified size, and the direction and operation type of its data path are set.
[0078] For storage arrays, the control array follows M is The specified capacity and partitioning scheme physically divides the on-chip cache into multiple independent buffers, and configures the base address, length and read / write permissions of each buffer, establishing a data transfer channel between the computing array and the external bus.
[0079] For in-memory computing units, the control array follows M ih The specified working mode is used to switch the storage unit within the specified address range to the calculation mode, fix the weight data latched inside it to a read-only state, and configure the input and output ports of the data stream so that the input data stream can directly complete the multiplication and accumulation operation when passing through the unit without retrieving the weight data.
[0080] For the control array itself, it follows M ik The system loads the corresponding program, sets the initial value of the loop counter, address jump logic, and synchronization signal trigger conditions, and completes the refactoring of its own logic by specifying the state machine type and loop parameters.
[0081] Once all the aforementioned arrays are configured, a complete hardware execution entity is formed within the task module, comprising a computing array, a storage array, an in-memory computing unit, and a control array. This execution entity is the reconfigurable operator. Under the scheduling of its control array, the reconfigurable operator drives each array to complete all computational work for the current task in a pipelined manner, according to the execution flow defined by the M control instructions.
[0082] In the above implementation process, task information is acquired in real time; based on the task information, a task module is determined from a set of preset modules, which are obtained by virtually dividing the hardware resources of the edge device into multiple modules; based on the task information and the task module, an M control instruction is generated; the task module, based on the M control instruction, redetermines its array configuration parameters to obtain a reconfigurable operator, and executes the task based on the reconfigurable operator. By virtually splitting the hardware resources of the edge device into multiple reconfigurable modules, acquiring task information in real time during runtime and intelligently matching task modules accordingly, and then generating M control instructions to dynamically configure the computation, storage, control, and storage-computing integrated array parameters within the module, a reconfigurable operator precisely adapted to the current task is reconstructed in real time on the hardware. This achieves adaptive reconfiguration of the operator structure, enabling flexible hardware adaptation for different algorithms, improving the matching degree between the operator and the edge hardware, significantly improving the execution efficiency and resource utilization of the edge device, and providing key technical support for the deep integration of edge computing and artificial intelligence.
[0083] Please refer to Figure 6 , Figure 6 This schematically illustrates a structural diagram of a hardware resource allocation device for an edge device according to an embodiment of the present application. This embodiment provides a hardware resource allocation device for an edge device, including an acquisition module 210, a determination module 220, an instruction module 230, and a reconstruction module 240, wherein: Module 210 is used to acquire task information in real time; The determining module 220 is used to determine the task module from a set of preset modules based on the task information. The set of preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules. Instruction module 230 is used to generate M control instructions based on the task information and the task module; The reconfiguration module 240 is used to redetermine the array configuration parameters of the task module based on the M control instructions, obtain a reconfigurable operator, and execute the task based on the reconfigurable operator.
[0084] The hardware resource allocation device of the edge device includes a processor and a memory. The aforementioned acquisition module 210, determination module 220, instruction module 230 and reconstruction module 240 are all stored in the memory as program units. The processor executes the aforementioned program units stored in the memory to realize the corresponding functions.
[0085] The reconstruction module 240 includes: The parameter determination unit is used to determine the computing array configuration parameters, storage array configuration parameters, control array configuration parameters, and in-memory computing configuration parameters by the task module based on the M control command; wherein the sum of the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters is equal to the total configuration parameters of the task module; The reconfiguration unit is used to obtain a reconfigurable operator based on the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters.
[0086] The determining module 220 includes: The judgment unit is used to determine whether the current task is a convolution task based on the task information. The structuring unit is used to structure the original data of the convolution task into a convolutional layer matrix form when the current task is confirmed to be a convolution task, so as to obtain a task matrix. The selection unit is used to select a module that meets preset conditions from a plurality of preset modules as a task module. The preset conditions include: the in-memory computing matrix unit of the module matches the task matrix.
[0087] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and hardware resource allocation for edge devices can be achieved by adjusting kernel parameters.
[0088] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0089] This invention provides a storage medium storing a program that, when executed by a processor, implements a hardware resource allocation method for the edge device.
[0090] This invention provides a processor for running a program, wherein the program executes a hardware resource allocation method for the edge device during runtime.
[0091] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05, and a memory (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A06. The network interface A02 is used for communication with external terminals via a network connection. When the computer program is executed by the processor A01, it implements a hardware resource allocation method for an edge device. The display screen A04 can be a liquid crystal display (LCD) or an e-ink display. The input device A05 can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0092] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0093] In one embodiment, the hardware resource allocation device for the edge device provided in this application can be implemented as a computer program, which can be implemented as follows: Figure 7 It runs on the computer device shown. The computer device's memory can store the various program modules that make up the hardware resource allocation device of the edge device, for example, Figure 6 The diagram shows an acquisition module 210, a determination module 220, an instruction module 230, and a reconstruction module 240. The computer program comprised of these modules causes the processor to execute the steps of the hardware resource allocation methods for edge devices in the various embodiments of this application described in this specification.
[0094] Figure 7 The computer device shown can be used as follows Figure 6 The acquisition module 210 in the hardware resource allocation device of the shown edge device executes step 110. The computer device can execute step 120 through the determination module 220. The computer device can execute step 130 through the instruction module 230. The computer device can execute step 140 through the reconfiguration module 240.
[0095] This application provides an electronic device comprising: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the aforementioned hardware resource allocation method for an edge device by executing the instructions stored in the memory. When the processor executes the instructions, it performs the following steps: Get task information in real time; Based on the task information, a task module is determined from a set of preset modules. The preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules. Based on the task information and the task module, generate M control instructions; The task module re-determines the array configuration parameters of the task module based on the M control command to obtain a reconfigurable operator, and executes the task based on the reconfigurable operator.
[0096] In one embodiment, the step of the task module re-determining its array configuration parameters based on the M control instructions to obtain a reconfigurable operator includes: The task module determines the computing array configuration parameters, storage array configuration parameters, control array configuration parameters, and in-memory computing configuration parameters based on the M control instructions. Based on the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters, a reconfigurable operator is obtained; The sum of the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters is equal to the total configuration parameters of the task module.
[0097] In one embodiment, determining the task module from a preset set of modules based on the task information includes: Based on the task information, determine whether the current task is a convolution task; If the current task is confirmed to be a convolution task, the original data of the convolution task is structured into a convolutional layer matrix to obtain the task matrix. Select a module that meets the preset conditions from a plurality of preset modules as the task module. The preset conditions include: the in-memory computing matrix unit of the module matches the task matrix.
[0098] In one embodiment, selecting a module that meets preset conditions from a plurality of preset modules as a task module includes: Based on the computing array, storage array, and task matrix in each module, the number of convolutional layer input channel split groups and the number of convolutional layer output channel split groups corresponding to each module are determined respectively. Determine whether there is a module among the preset multiple modules that meets the preset conditions, wherein the preset conditions are that the number of groups split into the input channel of the convolutional layer and the number of groups split into the output channel of the convolutional layer corresponding to the module are both equal to the actual number of groups split into by the module; If a module that meets the preset conditions is found among the preset multiple modules, then the module that meets the preset conditions is designated as the task module.
[0099] In one embodiment, it also includes: If it is determined that there is no module that meets the preset conditions among the preset multiple modules, the convolution task is split into multiple convolution subtasks; Based on each convolutional subtask, multiple subtask modules are determined from a set of pre-defined modules. Based on the multiple sub-task modules, a task module is obtained.
[0100] In one embodiment, generating M control instructions based on the task information and the task module includes: Based on the task information and the task module, multiple splitting strategies are determined, each of which includes task splitting information and array allocation information. The overall optimization index of each splitting strategy was calculated separately. Based on the full-process optimization index of each splitting strategy, the optimal strategy is determined among the multiple splitting strategies. Based on the optimal strategy, M control instructions are generated.
[0101] In one embodiment, the overall optimization index of the splitting strategy is calculated, including: Obtain the target task accuracy requirements, splitting rationality indicators, and edge hardware constraint indicators for the splitting strategy; The target task accuracy requirement index, splitting rationality index, and edge hardware constraint index of the splitting strategy are weighted and summed to obtain the full-process optimization index of the splitting strategy.
[0102] In one embodiment, obtaining the target task accuracy requirement metric for the splitting strategy includes: Obtain the original number of samples; Based on the original sample size and the task matrix, the target task accuracy requirement index of the splitting strategy is calculated.
[0103] In one embodiment, obtaining the rationality index of the splitting strategy includes: Based on the task splitting information, array allocation information, and the actual number of module splitting groups of the task module corresponding to the splitting strategy, the splitting rationality index of the splitting strategy is calculated.
[0104] In one embodiment, obtaining the edge hardware constraint metrics of the splitting strategy includes: Obtain the percentage of in-memory computing resources in the task module; Based on the task matrix and the array allocation information corresponding to the splitting strategy, the model inference delay parameters are determined. Based on the proportion of in-memory computing resources in the task module and the model inference latency parameters, the edge hardware constraint index of the splitting strategy is obtained.
[0105] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0106] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0107] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0108] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0109] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0110] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0111] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0112] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0113] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method for allocating hardware resources for an edge device, characterized in that, include: Get task information in real time; Based on the task information, a task module is determined from a set of preset modules. The preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules. Based on the task information and the task module, generate M control instructions; The task module re-determines the array configuration parameters of the task module based on the M control command to obtain a reconfigurable operator, and executes the task based on the reconfigurable operator.
2. The hardware resource allocation method for edge devices according to claim 1, characterized in that, The step of the task module re-determining its array configuration parameters based on the M control command to obtain a reconfigurable operator includes: The task module determines the computing array configuration parameters, storage array configuration parameters, control array configuration parameters, and in-memory computing configuration parameters based on the M control instructions. Based on the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters, a reconfigurable operator is obtained; The sum of the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters is equal to the total configuration parameters of the task module.
3. The hardware resource allocation method for edge devices according to claim 1, characterized in that, The step of determining a task module from a set of preset modules based on the task information includes: Based on the task information, determine whether the current task is a convolution task; If the current task is confirmed to be a convolution task, the original data of the convolution task is structured into a convolutional layer matrix to obtain the task matrix. Select a module that meets the preset conditions from a plurality of preset modules as the task module. The preset conditions include: the in-memory computing matrix unit of the module matches the task matrix.
4. The hardware resource allocation method for edge devices according to claim 3, characterized in that, The step of selecting a module that meets preset conditions from a plurality of preset modules as a task module includes: Based on the computing array, storage array, and task matrix in each module, the number of convolutional layer input channel split groups and the number of convolutional layer output channel split groups corresponding to each module are determined respectively. Determine whether there is a module among the preset multiple modules that meets the preset conditions, wherein the preset conditions are that the number of groups split into the input channel of the convolutional layer and the number of groups split into the output channel of the convolutional layer corresponding to the module are both equal to the actual number of groups split into by the module; If a module that meets the preset conditions is found among the preset multiple modules, then the module that meets the preset conditions is designated as the task module.
5. The hardware resource allocation method for edge devices according to claim 4, characterized in that, Also includes: If it is determined that there is no module that meets the preset conditions among the preset multiple modules, the convolution task is split into multiple convolution subtasks; Based on each convolutional subtask, multiple subtask modules are determined from a set of pre-defined modules. Based on the multiple sub-task modules, a task module is obtained.
6. The hardware resource allocation method for edge devices according to claim 4, characterized in that, The generation of M control instructions based on the task information and the task module includes: Based on the task information and the task module, multiple splitting strategies are determined, each of which includes task splitting information and array allocation information. The overall optimization index of each splitting strategy was calculated separately. Based on the full-process optimization index of each splitting strategy, the optimal strategy is determined among the multiple splitting strategies. Based on the optimal strategy, M control instructions are generated.
7. The hardware resource allocation method for edge devices according to claim 6, characterized in that, The overall optimization index of the splitting strategy is calculated, including: Obtain the target task accuracy requirements, splitting rationality indicators, and edge hardware constraint indicators for the splitting strategy; The target task accuracy requirement index, splitting rationality index, and edge hardware constraint index of the splitting strategy are weighted and summed to obtain the full-process optimization index of the splitting strategy.
8. The hardware resource allocation method for edge devices according to claim 7, characterized in that, Obtain the target task accuracy requirements for the splitting strategy, including: Obtain the original number of samples; Based on the original sample size and the task matrix, the target task accuracy requirement index of the splitting strategy is calculated.
9. The hardware resource allocation method for edge devices according to claim 7, characterized in that, To obtain the rationality indicators of the splitting strategy, including: Based on the task splitting information, array allocation information, and the actual number of module splitting groups of the task module corresponding to the splitting strategy, the splitting rationality index of the splitting strategy is calculated.
10. The hardware resource allocation method for edge devices according to claim 7, characterized in that, Obtain edge hardware constraint metrics for the splitting strategy, including: Obtain the percentage of in-memory computing resources in the task module; Based on the task matrix and the array allocation information corresponding to the splitting strategy, the model inference delay parameters are determined. Based on the proportion of in-memory computing resources in the task module and the model inference latency parameters, the edge hardware constraint index of the splitting strategy is obtained.
11. A hardware resource allocation device for an edge device, characterized in that, include: The acquisition module is used to acquire task information in real time; The determination module is used to determine the task module from a set of preset modules based on the task information. The set of preset modules are obtained by virtually dividing the hardware resources of the edge device into multiple modules. The instruction module is used to generate M control instructions based on the task information and the task module; The reconfiguration module is used by the task module to redetermine the array configuration parameters of the task module based on the M control instructions, obtain a reconfigurable operator, and execute the task based on the reconfigurable operator.
12. The hardware resource allocation device for an edge device according to claim 11, characterized in that, The reconstruction module includes: The parameter determination unit is used to determine the computing array configuration parameters, storage array configuration parameters, control array configuration parameters, and in-memory computing configuration parameters by the task module based on the M control command; wherein the sum of the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters is equal to the total configuration parameters of the task module; The reconfiguration unit is used to obtain a reconfigurable operator based on the computing array configuration parameters, the storage array configuration parameters, the control array configuration parameters, and the in-memory computing configuration parameters.
13. The hardware resource allocation device for the edge device according to claim 11, characterized in that, The determining module includes: The judgment unit is used to determine whether the current task is a convolution task based on the task information. The structuring unit is used to structure the original data of the convolution task into a convolutional layer matrix form when the current task is confirmed to be a convolution task, so as to obtain a task matrix. The selection unit is used to select a module that meets preset conditions from a plurality of preset modules as a task module. The preset conditions include: the in-memory computing matrix unit of the module matches the task matrix.
14. An electronic device, characterized in that, The electronic device includes: At least one processor; A memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the hardware resource allocation method for the edge device according to any one of claims 1 to 10 by executing the instructions stored in the memory.
15. A machine-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, this instruction causes the processor to be configured to perform the hardware resource allocation method for the edge device according to any one of claims 1 to 10.
