Lightweight deployment method and system of large model on edge computing device

Abstract

The invention provides a lightweight deployment method and system of a large model on edge computing equipment. According to the method, the acceleration capability portrait is constructed by extracting the hardware instruction set architecture type of the target edge device and the number of parallel computing units. Based on the instruction type, the large model weight is grouped, divided and pre-calculated through a unified lookup table vectorization engine, and a pre-calculation vector matched with the target instruction set is generated; and according to the number of the parallel units and the instruction-level parallel capability, compiling the pre-calculation vector to generate an adaptive parallel table look-up instruction block, distributing execution threads with the same number as the parallel units, and eliminating data dependence conflicts. And finally, loading the instruction block to a shared memory area, configuring topological logic of the photoconductive switch matrix based on an instruction type, and dynamically switching a data transmission path in a hardware instruction period. According to the method, the efficient deployment of the large model in the edge equipment and the low-delay reasoning in the resource-constrained environment are realized.

Description

Technical Field

[0001] The present application relates to the technical field of lightweight AI deployment driven by edge computing, and in particular to a lightweight deployment method and system for large models on edge computing devices. Background Art

[0002] In edge computing scenarios such as the Industrial Internet of Things, autonomous driving, and medical devices, large models with tens of billions of parameters must be deployed to resource-constrained edge devices to meet real-time, high-precision, and device-universal requirements. This requires a deployment solution that maintains model accuracy while maintaining low latency, while also being flexible and compatible with different edge hardware platforms, addressing the conflict between edge computing fragmentation and the computational intensity of large models.

[0003] At present, the current mainstream solution adopts a CPU-FPGA heterogeneous collaborative framework based on a double-buffer mechanism. The model weights are compressed into INT8 format and solidified in the FPGA memory through a fixed-precision quantization strategy. The processor schedules the computing task process, and the programmable gate array performs core matrix operations. Double buffer technology is used to preload data to reduce communication delays.

[0004] However, this framework has significant flaws. The transmission path relies on a traditional bus, causing communication to consume over 40% of the inference cycle, and the double-buffering mechanism cannot overcome the physical latency limits of hardware transmission. Static compilation of quantized models ignores instruction set differences, resulting in several-fold differences in inference efficiency for the same model on devices with different architectures. Summary of the Invention

[0005] This application provides a lightweight deployment method and system for large models on edge computing devices to solve the problem of low computing efficiency of edge devices in the prior art.

[0006] In a first aspect, the present application provides a lightweight deployment method for a large model on an edge computing device, including: Extract the hardware instruction set architecture type and the number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and construct an acceleration capability profile that includes the instruction type and the number of parallel computing units; Based on the instruction type in the acceleration capability profile, the large model weights are divided into multiple weight subsets by bit grouping through a unified lookup table vectorization engine, and a pre-calculation operation is performed on each weight subset to generate a pre-calculated vector that matches the target instruction set architecture; Based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, compile the pre-calculated vectors to generate parallel table lookup instruction blocks adapted to the target instruction set architecture, assign execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganize the pipeline operation sequence according to the instruction-level parallelism, and eliminate data dependency conflicts; The parallel table lookup instruction block is loaded into the shared memory area of ​​the processing unit and the field programmable gate array of the target edge device. Based on the instruction type in the acceleration capability portrait, the topological connection logic of the optical switch matrix is ​​configured. Through the optical switch matrix, the data transmission path between the processing unit and the field programmable gate array is switched during the hardware instruction execution cycle to complete the lightweight deployment and inference execution of the large model.

[0007] Optionally, based on the instruction type in the acceleration capability profile, the large model weights are divided into multiple weight subsets by bit grouping using a unified lookup table vectorization engine, and a pre-calculation operation is performed on each weight subset to generate a pre-calculation vector that matches the target instruction set architecture, including: Extracting a maximum parallel operation bit width from the instruction type of the acceleration capability profile, obtaining a preset fixed length of a bit grouping unit, dividing the maximum parallel operation bit width by the fixed length of the bit grouping unit, and calculating the number of bit grouping units included in a single weight subset; Cutting the large model weight sequence into a continuous unit sequence according to the fixed length of the bit grouping unit, and intercepting an equal number of continuous units from the continuous unit sequence according to the number of the bit grouping units to divide the sequence into multiple weight subsets; For each weight subset, enumerate the complete value range of the input value in the unified lookup table vectorization engine, and for each input value of the weight subset within the complete value range, calculate the set of simulated output values ​​of the input value on all bit grouping units in the weight subset; The analog output value sets are sequentially spliced ​​into vectorized results with a bit width equal to the maximum parallel operation bit width, and the vectorized results are arranged in ascending order of input values ​​to form a pre-calculated vector in a continuous storage area.

[0008] Optionally, for each weight subset, a complete value range of the input value is enumerated in a unified lookup table vectorization engine, and for each input value of the weight subset within the complete value range, a set of simulated output values ​​of the input value on all bit grouping units in the weight subset is calculated, including: Determining a complete range of input values ​​for the weight subset, where the complete range is defined by the boundaries of the binary value space corresponding to the fixed length of the bit grouping unit; Traverse each input value in the complete value range in increasing integer order, and use the current input value as the input item to enter the unified lookup table vectorization engine; Synchronously calculating independent output values ​​of the input item on all bit grouping units in the weight subset within the complete value range, and generating an output value sequence equal to the number of bit grouping units; The output value sequence is combined according to the physical storage order of the bit grouping units to form a set of analog output values ​​corresponding to the current input value.

[0009] Optionally, the parallel table lookup instruction block is loaded into a shared memory area between a processing unit and a field programmable gate array of a target edge device. Based on the instruction type in the acceleration capability profile, the topological connection logic of a light-conducting switch matrix is ​​configured. Through the light-conducting switch matrix, the data transmission path between the processing unit and the field programmable gate array is switched during the hardware instruction execution cycle, thereby completing the lightweight deployment and inference execution of the large model, including: Loading the reorganized parallel table lookup instruction block into a shared memory area between a processing unit and a field programmable gate array of a target edge device, extracting all vector opcode types from the instruction type of the acceleration capability profile, and generating a vector operation type set; Based on the vector operation type set, configuring the topological connection logic of the first data channel in the optical switch matrix, the channel connecting the processing unit vector register and the field programmable gate array high-speed interface, and setting the topological connection logic of the second data channel, the channel connecting the processing unit scalar register and the shared memory area; Monitor the opcode type in the hardware instruction execution cycle. When a vector opcode is detected, trigger the optical switch matrix to switch to the first data channel. When a non-vector opcode is detected, trigger the optical switch matrix to switch to the second data channel, completing the lightweight deployment and inference execution of the large model.

[0010] Optionally, monitoring the opcode type in the hardware instruction execution cycle, triggering the optical switch matrix to switch to the first data channel when a vector opcode is detected, and triggering the optical switch matrix to switch to the second data channel when a non-vector opcode is detected, thereby completing the lightweight deployment and inference execution of the large model, including: An opcode type monitoring point is established after the instruction decoding stage, the opcode field of the current instruction is intercepted by a hardware logic circuit, a preset identification bit sequence of the opcode field is extracted, and the identification bit sequence is matched with a predefined vector opcode identifier set; When the identification bit sequence matches the vector opcode identifier set, determining that the current instruction type is a vector opcode, and generating a first path switching control signal; When the identification bit sequence does not match the vector opcode identifier set, determining that the current instruction type is a non-vector opcode, and generating a second path switching control signal; The path switching control signal is input into the physical channel selector of the optical switch matrix, wherein the first path switching control signal triggers the physical connection of the first transmission path, and the second path switching control signal triggers the physical connection of the second transmission path, thereby completing the lightweight deployment and inference execution of the large model.

[0011] Optionally, extract the hardware instruction set architecture type and the number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and construct an acceleration capability profile including the instruction type and the number of parallel computing units, including: Reading an identification code of a hardware instruction set architecture type from a system configuration storage area of ​​a target edge device, and querying a predefined architecture capability mapping table based on the identification code to obtain all opcodes and bit width parameters supported by the hardware instruction set architecture type; The number of independent computing cores in the processing unit is collected through the computing unit management interface of the target edge device, and the number of configurable computing blocks is collected through the configuration status interface of the field programmable gate array; Based on the opcode and bit width parameters, identify the instruction level parallelism capability characteristics, and add the number of the independent computing cores and the number of the configurable computing blocks to obtain the total number of parallel computing units; All the opcodes and bit width parameters, instruction-level parallel capability characteristics, and the overall number of parallel computing units are integrated to construct an acceleration capability profile that includes instruction types and the number of parallel computing units.

[0012] Optionally, based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, compiling the pre-calculated vectors, generating parallel table lookup instruction blocks adapted to the target instruction set architecture, allocating execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganizing the pipeline operation sequence according to the instruction-level parallelism, and eliminating data dependency conflicts, including: Analyze the continuous memory distribution characteristics of the pre-calculated vectors based on the instruction-level parallelism capability of the acceleration capability profile, compile the continuously stored fragments of the pre-calculated vectors into the same instruction block based on the continuity of the memory addresses, and generate an equal number of independent instruction blocks based on the number of parallel computing units; Filling the table lookup opcode of the target instruction set architecture and the corresponding pre-calculated vector starting address offset in each of the independent instruction blocks to obtain a parallel table lookup instruction block adapted to the target instruction set architecture, creating an execution thread identifier equal to the number of parallel computing units, and assigning each of the execution thread identifiers to the parallel table lookup instruction block; Scan the output and input relationships between all the parallel table lookup instruction blocks, identify the instruction block pairs with data dependencies, adjust the execution timing sequence of the identified instruction block pairs, insert synchronization control operation codes to eliminate data dependency conflicts, and rearrange the instruction block sequence according to the adjusted timing sequence.

[0013] In a second aspect, the present application provides a lightweight deployment system for large models on edge computing devices, including: A construction module is used to extract the hardware instruction set architecture type and the number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and construct an acceleration capability profile including the instruction type and the number of parallel computing units; a matching module, configured to divide the large model weights into a plurality of weight subsets by bit grouping using a unified lookup table vectorization engine according to the instruction type in the acceleration capability profile, perform a pre-computation operation on each of the weight subsets, and generate a pre-computation vector that matches the target instruction set architecture; a generation module, configured to compile the pre-calculated vectors based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, generate parallel table lookup instruction blocks adapted to the target instruction set architecture, assign execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganize the pipeline operation sequence according to the instruction-level parallelism, and eliminate data dependency conflicts; A switching module is used to load the parallel table lookup instruction block into the shared memory area of ​​the processing unit and the field programmable gate array of the target edge device, configure the topological connection logic of the optical switch matrix based on the instruction type in the acceleration capability profile, and switch the data transmission path between the processing unit and the field programmable gate array through the optical switch matrix during the hardware instruction execution cycle to complete the lightweight deployment and inference execution of the large model.

[0014] In a third aspect, the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a lightweight deployment method of a large model on an edge computing device as described in the first aspect above.

[0015] In a fourth aspect, the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements a lightweight deployment method of a large model on an edge computing device as described in the first aspect.

[0016] This application extracts the target edge device's hardware instruction set architecture type and number of parallel computing units, identifies instruction-level parallelism capabilities based on the architecture type, and constructs an acceleration capability profile. This approach deeply quantifies the device's hardware computing power characteristics, accurately identifies the potential for instruction set parallelism, and provides a basis for hardware customization and optimization for model deployment. Based on the instruction type in the acceleration capability profile, a unified lookup table vectorization engine performs bit grouping on large model weights. Pre-computation operations are then performed on each weight subset to generate pre-computed vectors matching the target instruction set. This step significantly reduces the real-time computational burden, transforming complex calculations into efficient table lookup operations through pre-computation strategies. Based on the number of parallel computing units and instruction-level parallelism capabilities in the acceleration capability profile, the pre-computed vectors are compiled to generate parallel table lookup instruction blocks adapted to the instruction set architecture, assigning equal execution threads and reorganizing pipeline operations. This process significantly improves instruction parallel execution efficiency, completely eliminating data dependency conflicts through dynamic thread allocation and pipeline reorganization. The parallel table lookup instruction blocks are loaded into a shared memory area, and the topological connection logic of the optical switch matrix is ​​configured based on the instruction type, dynamically switching data transmission paths during the hardware instruction execution cycle. Ultimately, ultra-low latency communication at the hardware level is achieved, overcoming the bottleneck of traditional bus transmission and completing lightweight deployment and real-time inference.

[0017] Furthermore, by extracting the maximum parallel operation bit width from the instruction type in the acceleration capability profile and combining it with the preset fixed length of the bit grouping unit, the number of bit grouping units contained in a single weight subset is calculated. This ensures that the weight grouping strictly matches the maximum parallel operation bit width of the hardware instructions, laying the foundation for single-cycle multi-data parallel processing. The large model weight sequence is split into a continuous unit sequence of fixed length, and an equal number of continuous units are cut according to the number of bit grouping units and divided into weight subsets. This operation forms a continuous and regular memory storage structure, significantly improving cache hit efficiency and reducing memory fragmentation access overhead. For each weight subset, the complete range of input values ​​is enumerated, and for each input value, a set of simulated outputs is calculated for all bit grouping units. This pre-calculation mechanism completely eliminates the need for multiply-add operations in real-time inference, converting dynamic computation tasks into static table lookups. The simulated output sets are sequentially concatenated into vectorized results with a width equal to the maximum parallel operation bit width, and then arranged in the order of the input values ​​to form a contiguously stored pre-calculated vector. Ultimately, the memory access pattern is deeply adapted to the continuous addressing characteristics of the hardware, significantly reducing memory access latency and improving computational throughput.

[0018] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of a lightweight deployment method of a large model on an edge computing device provided by the present application is shown; Figure 2 A scenario diagram showing a lightweight deployment method of a large model on an edge computing device provided by this application is shown; Figure 3 A schematic diagram of the structure of a lightweight deployment system for a large model on an edge computing device provided by the present application is shown; Figure 4 A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0022] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0023] Researchers have discovered that existing edge computing devices face significant bottlenecks when deploying large models. Transmission paths rely on traditional bus communication, which consumes a significant number of inference cycles. Double-buffering mechanisms struggle to overcome the physical latency constraints of hardware transmission. Furthermore, statically compiled quantized models fail to adapt to device instruction set differences, resulting in significant disparities in inference efficiency for the same model on heterogeneous devices. Therefore, a lightweight deployment method is urgently needed that can dynamically adapt to edge hardware characteristics and collaboratively optimize communication and computational loads.

[0024] In response to the above problems, the present invention proposes an instruction-level collaborative deployment method based on hardware adaptation, the core of which is to drive the dynamic coupling of hardware resources and computing tasks through acceleration capability profiling. Specifically, the hardware instruction set architecture characteristics and parallel unit scale of the edge device are first extracted to construct an acceleration capability profile that includes instruction type and parallel computing capability; the model weights are vectorized by bit grouping using this profile to generate pre-calculated vectors that match the target architecture; then, according to the actual parallel capability, they are compiled into parallel lookup instruction blocks, execution threads are dynamically allocated, and pipeline operations are reorganized according to the instruction-level parallel characteristics; finally, the data transmission path between the CPU and FPGA is intelligently switched through the optical switch matrix during the hardware execution cycle. This method eliminates the communication delay bottleneck through hardware-aware instruction-level optimization, solves the efficiency gap caused by instruction set differences through an architecture-adaptive pre-compilation mechanism, and reconstructs the pipeline based on real-time parallel capability to eliminate data dependency conflicts, achieving an end-to-end lightweight deployment efficiency leap.

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0026] Figure 1 The present invention provides a flowchart of a lightweight deployment method of a large model on an edge computing device, such as Figure 1 As shown, the method includes: 101. Extract the hardware instruction set architecture type and the number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and construct an acceleration capability profile including the instruction type and the number of parallel computing units; Optionally, step 101 may specifically include the following steps: 1011. Read an identification code of a hardware instruction set architecture type from a system configuration storage area of ​​a target edge device, and query a predefined architecture capability mapping table based on the identification code to obtain all opcodes and bit width parameters supported by the hardware instruction set architecture type. 1012. Collect the number of independent computing cores in the processing unit through the computing unit management interface of the target edge device, and collect the number of configurable computing blocks through the configuration status interface of the field programmable gate array; 1013. Identify instruction-level parallelism characteristics based on the opcode and bit width parameters, and add the number of independent computing cores to the number of configurable computing blocks to obtain the total number of parallel computing units. 1014. Integrate all the opcodes and bit width parameters, instruction-level parallel capability characteristics, and the overall number of parallel computing units to construct an acceleration capability profile that includes instruction types and the number of parallel computing units.

[0027] In the above steps, the target edge device refers to the computing hardware deployed at the edge computing location, such as a smart camera or router; the hardware instruction set architecture type indicates the instruction set format supported by the device processor, such as ARM or x86, which is used to define the basic operating capabilities of the device; the number of parallel computing units refers to the total number of units that the device can simultaneously perform independent calculations; instruction-level parallelism describes how the hardware supports the simultaneous execution of multiple instructions; building an acceleration capability profile is to create a data structure to record the computing performance characteristics of the device; the opcode is the type of processor instruction, such as the code corresponding to addition or subtraction operations; the bit width parameter is the number of data bits that the processing unit operates on at one time, for example, 32 bits means processing 4 bytes of data; the independent computing core is an independent physical processing unit in the processor that can run tasks independently; the configurable computing block is a user-programmable functional block in the field programmable gate array for customized computing tasks.

[0028] In an embodiment of the present application, first, through step 1011, the identification code of the hardware instruction set architecture type is read and the operation code and bit width parameters are obtained. For example, the target edge device is an intelligent industrial sensor, and its system configuration storage area stores the identification code "ARMv8". The predefined architecture capability mapping table is an electronic file that stores data of different architecture types, and the query process uses a string matching algorithm. In a specific implementation, the device startup script automatically accesses the system configuration storage area and extracts the identification code string "ARMv8". Then, the preloaded mapping table file is retrieved in the local file system, and the corresponding row is accurately matched according to the identification code. All operation codes supported by the architecture, such as ADD / SUB / MUL and bit width parameters (64 bits), are read. The acquired data will be output as an operation code list and bit width values ​​for use in subsequent steps.

[0029] Next, in step 1012, the number of independent computing cores in the processing unit and the number of configurable computing blocks in the field-programmable gate array are collected. For example, the processing unit management interface calls the operating system's "getCoreCount" function, which returns a value such as 4, indicating that the device has four processing cores. The field-programmable gate array configuration status interface calls the hardware driver's "fpgaConfigStatus" function to obtain the number of configurable computing blocks, such as 8, indicating that eight computing blocks are available. This collection process is automatically executed by the background service program upon device startup. Finally, the output is the number of independent cores (4) and the number of computing blocks (8). These two values ​​are added together in the next step to calculate the total number of parallel units.

[0030] Next, step 1013 performs two operations: identifying the instruction-level parallelism capability based on the opcode and bitwidth parameters; and adding the number of independent computing cores to the number of configurable compute blocks. For example, when the opcode includes the vector instruction "VADD" and the bitwidth parameter is 64 bits, the identification process uses a bitwidth comparison and opcode scanning algorithm. First, the opcode set is scanned to detect the presence of vector instruction codes, such as VADD; then, the bitwidth is checked to see if it is ≥ 64 bits. If both conditions are met, the instruction-level parallelism capability is marked as supporting single instruction multiple data (SIMD) operations. Before executing the numerical addition operation, the collected values ​​are verified: if both are positive integers, such as the number of cores 4 and the number of compute blocks 8, then 4 + 8 = 12 is calculated; if not, they are reset to 0. Finally, the instruction-level parallelism capability characteristic SIMD and the total number of parallel units 12 are output.

[0031] Finally, the three parts of data are fused in step 1014 to construct an acceleration capability profile. For example, the supported operation codes (ADD / SUB / VADD), bit width parameters (64 bits), instruction-level parallel capability characteristics (SIMD), and the number of parallel units (12) are combined into structured data. The fusion process calls the encapsulation function to create a JSON object with the key-value pair structure: {"operationcodes":["ADD","SUB","VADD"],"bit_width":64,"paralleltype":"SIMD","unitcount":12}. The constructed profile is stored as a local device configuration file, such as devicecapability.json, for direct call by the subsequent task scheduling module.

[0032] In a practical application, in a certain edge computing scenario, when target edge device A is used for real-time analysis tasks in an intelligent security system, it reads the hardware instruction set architecture type identification code, such as architecture type B, from the device system configuration storage area, queries the predefined architecture capability mapping table, and obtains all opcodes supported by the type, such as integer multiplication, floating-point addition, vector load, and bit manipulation, as well as the corresponding bit width parameters, such as 32-bit and 64-bit. Next, the number of independent computing cores in the processing unit is collected through the operation unit management interface, which is 4, and the number of configurable computing blocks is collected through the configuration status interface of the field programmable gate array, which is 6. Subsequently, based on the obtained opcodes and bit width parameters, the instruction-level parallelism capability feature is identified, such as support for data-level parallelism. The number of independent computing cores and the number of configurable computing blocks are added to obtain the total number of parallel computing units, which is 10. Finally, all opcodes and bit width parameters, instruction-level parallelism capability features, and the total number of parallel computing units are integrated to construct an acceleration capability profile that includes instruction types and the number of parallel computing units, thereby enhancing the device's adaptability to computing tasks.

[0033] In the overall solution of the above step 101, the hardware instruction set architecture identification code stored in the target edge device is read and the predefined mapping table is queried to obtain the supported operation codes and bit width parameters. At the same time, the number of independent operation cores of the processor is collected using the operation unit management interface and the number of configurable computing blocks of the field programmable gate array is collected through the configuration status interface. The instruction-level parallel capability characteristics are identified based on the operation code and bit width parameters, and the number of the two types of operation units are comprehensively counted to obtain the total number of overall parallel computing units. Finally, the instruction type characteristics and the number of parallel computing units are integrated to form a complete hardware acceleration capability portrait, thereby realizing a multi-dimensional quantitative characterization of the computing architecture characteristics of the edge device.

[0034] 102. Based on the instruction type in the acceleration capability profile, divide the large model weights into multiple weight subsets by bit grouping using a unified lookup table vectorization engine, perform a pre-computation operation on each weight subset, and generate a pre-computation vector that matches the target instruction set architecture; Optionally, step 102 may specifically include the following steps: 1021. Extract a maximum parallel operation bit width from the instruction type of the acceleration capability profile, obtain a preset fixed length of a bit grouping unit, and calculate the number of bit grouping units included in a single weight subset by dividing the maximum parallel operation bit width by the fixed length of the bit grouping unit. 1022. Cut the large model weight sequence into a continuous unit sequence according to the fixed length of the bit grouping unit, and intercept an equal number of continuous units from the continuous unit sequence according to the number of the bit grouping units to divide the sequence into multiple weight subsets; 1023. For each weight subset, enumerate the complete value range of the input value in the unified lookup table vectorization engine, and for each input value of the weight subset within the complete value range, calculate a set of simulated output values ​​of the input value on all bit grouping units in the weight subset; Among them, step 1023 may specifically include the following processes: determining the complete value range of the input value of the weight subset, the complete value range is defined by the binary numerical space boundary corresponding to the fixed length of the bit grouping unit; traversing each input value in the complete value range in ascending order of integers, taking the current input value as the input item, and inputting it into the unified lookup table vectorization engine; synchronously calculating the independent output values ​​of the input item on all bit grouping units in the weight subset within the complete value range, and generating an output value sequence equal to the number of bit grouping units; combining the output value sequence according to the physical storage order of the bit grouping units to form a set of analog output values ​​corresponding to the current input value.

[0035] 1024. Sequentially concatenate the analog output value sets into vectorized results with a bit width equal to the maximum parallel operation bit width, arrange the vectorized results in ascending order of input values, and form a pre-calculated vector in a continuous storage area.

[0036] In the above steps, the acceleration capability profile is a performance data structure extracted from the target edge device, which contains the instruction type information and computing capability data supported by the device; the large model weight refers to the parameter value sequence in the large machine learning model, which is used for model inference calculation; the unified lookup table vectorization engine is a computing system module that implements efficient computing and parallel data processing operations based on pre-stored lookup tables; bit grouping is a data partitioning method that splits the large model weight sequence into fixed-size binary bit blocks; the maximum parallel operation bit width is the maximum number of bits processed at one time by the device instruction set obtained from the acceleration capability profile, such as 64 bits, which means that the device can process 64 bits of data at the same time; the fixed length of the bit grouping unit is a pre-set bit group unit size, for example For example, 8 bits are used as a basic unit; the number of bit grouping units is an integer value obtained through calculation, which represents the number of fixed-length units contained in each weight subset; the weight subset is a set of subunits after the large model weight sequence is divided, and each subset contains the same number of bit grouping units; the complete value range of the input value is the entire numerical interval that defines the possible input values ​​of the bit grouping unit, and the boundary is determined by the fixed length, for example, an 8-bit unit corresponds to an integer range of 0 to 255; the simulation output value set is a sequence of simulation operation results calculated in advance on each weight subset, and each result corresponds to an input value; the pre-calculated vector is a wide-bit vector data formed by splicing all the simulation output values ​​and arranging them in the order of the input values, which matches the bit width of the device instruction set to facilitate accelerated calculation.

[0037] In the embodiment of the present application, first, the maximum parallel operation bit width value is extracted from the acceleration capability profile through step 1021, and the preset bit grouping unit fixed length value is obtained; then, the number of bit grouping units is calculated. This calculation uses a division algorithm to implement integer operations. The specific formula is defined as: the number of bit grouping units is equal to the maximum parallel operation bit width divided by the bit grouping unit fixed length, that is, ,in Indicates the number of bit grouping units, Indicates the maximum parallel operation bit width, Indicates that the bit grouping unit has a fixed length; if the division result is a decimal, the system automatically performs a floor function to convert it to an integer to ensure that the number of units is an integer value; the entire calculation process is implemented by the system background service calling the mathematical operation function, such as the acceleration capability portrait of the target edge device showing the maximum parallel operation bit width =64 bits, the preset bit grouping unit has a fixed length =8 bits, the system reads first and numerical value, and then apply the formula , calculate the number of result units =8, this value is directly passed to the next step for dividing the weight sequence. During the process, the system will check whether the value is a positive integer to prevent incorrect input.

[0038] Secondly, step 1022 is used to cut and divide the large model weight sequence into multiple weight subsets based on the fixed length of the bit grouping unit and the number of units. The implementation process involves a data cutting algorithm and a sequence interception operation. First, the large model weight sequence is loaded, such as a binary data stream or a numerical array, and is cut into multiple equal-length continuous unit sequences according to a fixed length. For example, if a large model weight sequence is 1024 bits long and the fixed length is set to 8 bits, the cutting process is performed by dividing 1024 by 8, which is equal to 128, and a sequence of 128 continuous 8-bit units is obtained. Then, the number of units calculated from the beginning of the sequence according to step 1021 is as follows: Truncate 8 consecutive units as a set of weight subsets, such as intercepting units 1 to 8 as the first group, and then continue to intercept units 9 to 16 as the second group. The loop is executed until the end of the sequence. Finally, 128 divided by 8 equals 16 complete weight subsets. The division process uses the memory slicing function to ensure data boundary alignment, and each weight subset is stored as an independent array block for subsequent processing.

[0039] Next, a pre-calculation operation is performed on each weight subset in step 1023 to generate a set of simulated output values. The execution process includes the following steps: First, the complete range of input values ​​is determined. The maximum value of the range is calculated as ,in, Indicates the maximum value of the value range. Indicates that the bit grouping unit has a fixed length, and the minimum value is fixed to 0; for example, the fixed length =8, the calculation process is =28 1=256 1=255, the value range is all integers from 0 to 255; then, the value range is completely traversed in increasing order of integers, such as from 0 to 255, and each input value is passed as an input item to the unified lookup table vectorization engine; the engine synchronously calculates the independent output values ​​of the input item on all units of the weight subset based on pre-stored calculation rules such as multiplication or addition operations, and the length of the output value sequence is equal to the number of units obtained in step 1021 ; Finally, the output sequence is combined according to the physical storage order of the bit grouping unit to generate a set of analog output values ​​for the current input value; for example, a weight subset contains =8 units have a fixed length of 8 bits, and the unit weight values ​​are 01, which represents decimal 1, 02 represents decimal 2, and so on. When the input value is 5, the engine simultaneously applies 5 to each unit. Weight 01 outputs 5 times 1, which equals 5. Weight 02 outputs 5 times 2, which equals 10. This calculation is repeated 8 times to form a sequence, such as 5, 10, 15, 20, 25, 30, 35, 40, and is combined in unit order to form the output value set 5, 10, 15, 20, 25, 30, 35, 40. The entire process uses parallel processing technology to reduce latency and ensure efficiency.

[0040] Finally, step 1024 is used to splice the analog output value set of each input value into a vectorized result with matching bit width, and arrange them in the order of input values ​​to form a pre-calculated vector; the implementation process includes vector splicing and data sorting operations. First, the analog output value set corresponding to each input value generated in step 1023, such as the set 5, 10, 15, 20, 25, 30, 35, 40, is subjected to the splicing algorithm, that is, all output value sequences are converted into binary and merged into a single wide bit vector, and the bit width is equal to the maximum parallel operation bit width extracted in step 1021, such as bits; for example, the output value sequence of the input value 5 has 8 bits per value and a total of 8 outputs, which are spliced ​​into a 64-bit binary sequence such as 0101001010100010, a complete 64-bit vector; then, all vectorized results are arranged in ascending order of input values, such as 0 to 255, and the linear array write function is used to store the results in a continuous storage area such as a memory buffer or file to form the final pre-calculated vector; for example, all input value vectors are stored in an array in the order of 0, 1, 2, etc., for direct loading by subsequent model inference to utilize the device parallel capability to improve efficiency.

[0041] In actual applications, in an image recognition edge deployment scenario of a certain agricultural Internet of Things system, when optimizing large model inference based on the acceleration capability profile of device A, including vector loading instructions and 10 parallel units, the maximum parallel operation bit width supported, 128 bits, is first parsed from the profile. Combined with the preset 32-bit fixed length of the grouping unit, it is calculated that a single weight subset must contain 4 grouping units; then the weight sequence of the crop disease recognition model to be deployed is continuously cut into 70,000 basic unit sequences in units of 32 bits. According to the principle of each group of 4 units, the sequence is divided into 17,500 weight subsets and assigned to different parallel computing units for processing; for each weight subset, a complete vectorization engine is established in the unified lookup table. The input value enumeration space (0 to 4294967295) uses a parallel pipeline mechanism to synchronously calculate the operation results of each input value on the 4-bit grouping units. For example, when the input value is 137, the engine synchronously outputs the convolution kernel mapping value of this value in unit 1, the activation function prefetch value of unit 2, the feature scaling factor of unit 3 and the weight compensation parameter of unit 4, and combines them into a 4-dimensional output sequence in physical storage order; finally, the output sequence corresponding to each input value is spliced ​​into a 128-bit vectorized result by bit, and a structured pre-calculated vector matrix is ​​generated in a continuous storage area in the order of 0 to 4294967295, so that the dedicated instruction set of edge device C can directly call the vector to perform single instruction multiple data stream operations.

[0042] In the overall solution of step 102 above, based on the instruction type characteristics in the acceleration capability profile, the maximum parallel operation bit width is extracted and the number of bit grouping units contained in a single weight subset is calculated in combination with the preset fixed length of the bit grouping unit; after the large model weight sequence is cut into a continuous unit sequence according to the fixed length, multiple weight subsets are divided according to the calculated number of units; for each weight subset, the complete value range of the input value defined by the bit grouping unit length is determined in the unified lookup table vectorization engine, each input value in the range is traversed in increasing integer order, and the independent output value generated by the input value acting on all bit grouping units in the weight subset is synchronously calculated, forming an output value sequence matching the number of units and combining them into a set of analog output values ​​in physical storage order; finally, the set is converted into a vectorized result with a bit width equal to the maximum parallel operation bit width, and arranged as a pre-calculated vector in a continuous storage area in increasing order of input values, realizing an efficient instruction-level parallel computing structure that matches the large model weight. This process drives the weight structure reorganization and pre-calculation optimization through hardware instruction characteristics, significantly improving the model inference efficiency on edge devices while reducing the overhead of repeated calculations at runtime.

[0043] 103. Based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, compile the pre-computed vectors to generate parallel table lookup instruction blocks adapted to the target instruction set architecture, assign execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganize the pipeline operation sequence according to the instruction-level parallelism, and eliminate data dependency conflicts. Optionally, step 103 may specifically include the following steps: 1031. Analyze the continuous memory distribution characteristics of the pre-calculated vectors based on the instruction-level parallelism capability of the acceleration capability profile, compile the continuously stored fragments of the pre-calculated vectors into the same instruction block based on the continuity of the memory addresses, and generate an equal number of independent instruction blocks based on the number of parallel computing units. 1032. Fill in the table lookup opcode of the target instruction set architecture and the corresponding pre-computed vector starting address offset in each independent instruction block to obtain a parallel table lookup instruction block adapted to the target instruction set architecture, create an execution thread identifier equal to the number of parallel computing units, and assign each execution thread identifier to the parallel table lookup instruction block; 1033. Scan the output-input relationships between all the parallel table lookup instruction blocks, identify instruction block pairs with data dependencies, adjust the execution timing sequence of the identified instruction block pairs, insert synchronization control opcodes to eliminate data dependency conflicts, and rearrange the instruction block sequence according to the adjusted timing sequence.

[0044] In the above steps, the acceleration capability profile is a performance data structure extracted from the target edge device, including the number of parallel computing units and instruction-level parallel capability characteristics; the pre-computed vector is the vectorized data generated in step 102; compiling the pre-computed vector is to convert the data into instruction code executable by the device; the parallel table lookup instruction block adapted to the target instruction set architecture is the compiled instruction unit, which enables the device to efficiently query the pre-computed vector; allocating execution threads is to allocate independent processing units to each instruction block; instruction-level parallel capability is the feature of hardware supporting the simultaneous execution of multiple instructions, such as single instruction multiple data stream operations; the number of parallel computing units is the total number of units that the device can use to process tasks simultaneously ; Reorganizing the pipeline operation sequence is to adjust the instruction execution order to optimize the process; data dependency conflict is a conflict that occurs when an instruction requires the output result of other instructions; continuous memory distribution characteristics describe how pre-calculated vectors are stored continuously in memory; independent instruction blocks are instruction units that can run independently after compilation; execution thread identifiers are symbols used to uniquely mark each processing thread; data-dependent instruction block pairs are combinations of instruction blocks that have an output-input relationship; synchronization control opcodes are code units that insert instructions to ensure sequential execution; starting address offset is the position offset value of the pre-calculated vector in memory; table lookup opcodes are data query operation codes supported by the device instruction set.

[0045] In the embodiment of the present application, first, the continuous memory distribution characteristics of the pre-calculated vector are analyzed through step 1031, and equal independent instruction blocks are generated based on the number of parallel computing units. For example, the pre-calculated vector is stored in a continuous memory area such as an array data block with an address of 1000 to 2000. The system calls the analysis function to scan the continuity of the memory address, checks whether the start address and end address of each segment are continuous, and uses the address difference calculation algorithm to determine the fragment size. If the address is found to be discontinuous, the padding function is used to align to ensure continuity; then, based on the instruction-level parallel capability of the acceleration capability portrait, such as single instruction multiple data stream operation support, a segmentation algorithm is used to divide the memory address continuity as the basis for division. The system divides the pre-calculated vector fragments from the beginning of the address. The size of each fragment is determined by dividing the total length by the parallel count. The number of computing units is determined by integer operations; then, based on the number of parallel computing units, such as 8, the system performs a division to calculate the number of fragments 8 to generate an equal number of independent instruction blocks, each corresponding to a memory fragment address range; for example, a pre-calculated vector length is 1024 bytes and the number of parallel computing units is 8. The system calculates that the size of each fragment 1024 divided by 8 is equal to 128 bytes, and then divides the fragment into addresses 1000 to 1127 as the first instruction block corresponding to the memory, and then analogizes to generate 8 complete instruction blocks. All instruction blocks are stored in the instruction cache for subsequent use. The whole process ensures that the fragments are continuous and non-overlapping.

[0046] Next, in step 1032, the table lookup opcode and starting address offset are populated to obtain a parallel table lookup instruction block, and execution thread identifiers are created and assigned. For example, in each independent instruction block, the system loads a table lookup opcode supported by the target instruction set architecture, such as ARM, such as the data load instruction LDR. Next, the starting address offset of the corresponding pre-calculated vector is filled in, such as offset 0 for the first instruction block. The size of each fragment is subsequently increased by 128 bytes, so the offset of the second instruction block is 128. The calculation process uses a memory address calculation function to add the starting address to the fragment size to obtain the actual address value and write it into the instruction block. The system then creates execution thread identifiers equal to the number of parallel computing units, such as 8. A thread management function is used to generate unique identifiers, such as integers 1 to 8, respectively, marking thread 1, thread 2, and so on. Finally, each thread identifier is assigned to a parallel table lookup instruction block, such as thread 1 to the first instruction block, thread 2 to the second instruction block, and so on. The entire assignment process is based on a sequential mapping algorithm, ensuring that each thread has a corresponding instruction block. All instruction block and thread information is stored in the thread queue for execution.

[0047] Finally, step 1033 is used to scan the input and output relationships of all parallel table lookup instruction blocks, identify data-dependent instruction block pairs, adjust the timing to eliminate data dependency conflicts, and rearrange the instruction block sequence. For example, the system calls a scan function to check the output and input items of each instruction block, such as the matching of output variables and input variables, and uses a relationship detection algorithm to compare variable names or address ranges. If it finds that the output result of instruction block 1 is used as the input of instruction block 2, this is identified as a pair of instruction blocks with data dependency. Next, the execution timing is adjusted. For the identified instruction block pairs, the system uses a scheduling function to force instruction block 1 to execute before instruction block 2, ensuring that the output precedes the input and other non-dependent blocks execute in parallel. After that, synchronization control opcodes are inserted between the dependent pairs, such as the wait control instruction SYNC, to ensure that the subsequent blocks are executed only after the data is ready. Finally, all instruction block sequences are rearranged according to the adjusted timing, and the sequence reorganization function is used to write the new order into the execution queue, such as the sequential-first-then-parallel structure to eliminate conflicts. For example, if blocks A and B are dependent on each other and block C is independent, the system identifies A and B as a dependent pair and adjusts the order so that A is executed after B and the SYNC opcode is inserted before B. Finally, the sequence is sorted so that A is the synchronization instruction, B is the synchronization instruction, and C is the last one to be stored and directly used in the device execution pipeline to improve efficiency.

[0048] In a real-world application, during the edge inference deployment of an industrial AI model, device A implemented compilation optimization on precomputed vectors based on its 10 parallel computing units and data-level parallel instruction capabilities as identified in its acceleration capability profile. First, the precomputed vector's continuous memory distribution was analyzed based on instruction-level parallelism. The vector was linearly stored in a 256KB memory region starting at address 0x1000. Based on address continuity, the vector was divided into 10 independent segments of equal capacity, each occupying 25.6KB of memory, generating 10 independent instruction blocks. Each instruction block was then populated with a table lookup opcode specific to the target instruction set architecture, such as VLOAD, and associated with the corresponding memory segment starting address offset. For example, instruction block 1 was associated with address offset 0x1000, and instruction block 2 was associated with address offset 0x19600. This formed a set of parallel table lookup instruction blocks adapted to the architecture. Ten execution thread identifiers, numbered 1 to 10, were created and assigned to the instruction blocks in a 1:1 mapping. The subsequent process scans the input-output relationships of all instruction blocks and identifies instruction block combinations with data dependencies. A typical example is the output data of instruction block 3 serving as the input dependency source for instruction block 7. A timing adjustment mechanism advances the execution order of instruction block 7 to before instruction block 3, and SYNC synchronization control opcodes are inserted at key nodes. Ultimately, the reconstructed pipeline execution sequence is arranged in the order of instruction block 1, instruction block 2, instruction block 7, instruction block 3, instruction block 4, and so on. This parallel table lookup instruction block sequence eliminates data conflicts and significantly improves computing resource utilization.

[0049] In the overall solution of step 103 above, relying on the number of parallel computing units and instruction-level parallel capability characteristics in the acceleration capability profile, an equal number of independent instruction blocks are generated based on the address continuity according to the continuous memory distribution characteristics of the pre-calculated vector; the table lookup opcode of the target instruction set architecture and the starting address offset of the pre-calculated vector are filled in each instruction block to form a parallel table lookup instruction block, and at the same time, execution thread identifiers matching the number of parallel computing units are created and assigned to each instruction block; by scanning the output and input relationships between instruction blocks, data-dependent instruction block pairs are identified, their execution timing sequence is adjusted, and synchronization control opcodes are inserted to eliminate conflicts, and finally the pipeline structure is optimized according to the reorganized instruction block sequence. This process fully utilizes the characteristics of hardware parallel resources to achieve deep optimization of table lookup operations, effectively avoid data waiting delays between computing units, and maximize the large-model inference throughput of edge devices.

[0050] 104. Load the parallel table lookup instruction block into the shared memory area of ​​the processing unit and the field programmable gate array of the target edge device, configure the topological connection logic of the optical switch matrix based on the instruction type in the acceleration capability portrait, and switch the data transmission path between the processing unit and the field programmable gate array through the optical switch matrix during the hardware instruction execution cycle to complete the lightweight deployment and inference execution of the large model.

[0051] Optionally, step 104 may specifically include the following steps: 1041. Load the reorganized parallel table lookup instruction block into a shared memory area between a processing unit and a field programmable gate array of a target edge device, extract all vector opcode types from the instruction types in the acceleration capability profile, and generate a vector operation type set. 1042. Based on the vector operation type set, configure the topology connection logic of the first data channel in the optical switch matrix, the channel connecting the processing unit vector register and the field programmable gate array high-speed interface, and set the topology connection logic of the second data channel, the channel connecting the processing unit scalar register and the shared memory area; 1043. Monitor the opcode type in the hardware instruction execution cycle. When a vector opcode is detected, trigger the optical switch matrix to switch to the first data channel. When a non-vector opcode is detected, trigger the optical switch matrix to switch to the second data channel, thereby completing the lightweight deployment and inference execution of the large model.

[0052] Among them, step 1043 may specifically include the following processes: establishing an opcode type monitoring point after the instruction decoding stage, intercepting the opcode field of the current instruction through a hardware logic circuit, extracting a preset identification bit sequence of the opcode field, and matching the identification bit sequence with a predefined vector opcode identifier set; when the identification bit sequence matches the vector opcode identifier set, determining that the current instruction type is a vector opcode, and generating a first path switching control signal; when the identification bit sequence does not match the vector opcode identifier set, determining that the current instruction type is a non-vector opcode, and generating a second path switching control signal; inputting the path switching control signal into the physical channel selector of the optical switch matrix, wherein the first path switching control signal triggers the physical connection of the first transmission path, and the second path switching control signal triggers the physical connection of the second transmission path, thereby completing the lightweight deployment and inference execution of the large model.

[0053] In the above steps, the FPGA is a programmable hardware component used to accelerate computations for specific tasks, and its structure can be dynamically adjusted. The shared memory area is a shared storage space accessible to both the processing unit and the FPGA, facilitating data exchange. Instruction types distinguish different operation commands, such as addition or multiplication. Vector opcode types refer to specific command types for processing bulk data, such as vector operations. The vector operation type set is a list storing all supported vector opcodes. The optical switch matrix is ​​a hardware device that dynamically changes connection paths, enabling rapid switching of data transmission channels. The topology connection logic defines the physical wiring rules of the optical switch matrix, including specific connection points. The first data path is the direct path connecting the processing unit's vector registers to the FPGA's high-speed interface. Vector registers are used to store bulk data, while the high-speed interface is used for high-speed data transmission. The second data path is the alternate path connecting the processing unit's scalar registers to the shared memory area. Scalar registers are used to store single data points. The hardware instruction execution cycle is the total time from instruction fetch to execution. The opcode type indicates whether the instruction is a vector or scalar operation. The instruction decoding stage is when the processing unit interprets the meaning of the instruction. The opcode type monitoring point is set at a hardware location after the decoding stage to check the opcode type. The opcode field is a partial binary sequence that defines the operation in the instruction. The identification bit sequence is a fixed bit sequence extracted from the opcode field that indicates the operation type. The predefined vector opcode identifier set is a list of known vector opcode values ​​used for comparison and matching. The path switching control signal is an electrical signal that triggers the optical switch matrix to change the path. The first path switching control signal enables vector operations, and the second path switching control signal enables scalar operations. The physical channel selector is the component in the optical switch matrix that actually switches the physical connections. Lightweight deployment and inference execution of large models refers to the process of efficiently running model prediction tasks on resource-limited devices of large computing models.

[0054] In an embodiment of the present application, first, step 1041 is used to load the reorganized parallel table lookup instruction block into the shared memory area between the processing unit and the field programmable gate array of the target edge device. All vector opcode types are extracted from the instruction types in the acceleration capability profile to generate a vector operation type set. In a specific implementation, memory read and write commands are used to copy the reorganized instruction block into the shared memory area as basic data. Then, the acceleration capability profile is read. This profile is like a configuration file or table that stores details of supported instruction types. A parsing algorithm is used to filter all vector opcode types from it, such as by traversing the profile data to match keyword vectors to extract these types. Finally, the extracted vector opcode types are combined into a list as a vector operation type set. This process forms a complete flow: shared memory loading ensures data availability, and parsing the profile extracts instruction types to generate a set based on actual hardware capabilities. For example, the acceleration capability profile may list supported opcodes including VADD vector addition, VMUL vector multiplication, and ADD ordinary addition. During parsing, VADD and VMUL are extracted as vector types, generating a set [VADD; VMUL] for subsequent configuration steps.

[0055] Next, in step 1042, based on the vector operation type set, the topological connection logic for the first data channel is configured in the optical switch matrix, connecting the processing unit vector registers and the high-speed interface of the field programmable gate array. The topological connection logic for the second data channel is also configured, connecting the processing unit scalar registers and the shared memory area. In a specific implementation, the optical switch matrix logic rules are configured based on the vector operation type set to define the physical connection method, and configuration commands are written into the logic. The first data channel is configured to directly connect the processing unit vector register data output to the high-speed interface of the field programmable gate array to accelerate transmission; the second data channel is configured to connect the processing unit scalar registers via the data path of the shared memory area for use during normal operations. This process forms a complete flow: the set provides input to guide the configuration logic rules, ensuring optimized path configuration for different instruction types. For example, the optical switch matrix is ​​configured for the set [VADD;VMUL] so that during vector operations, the first channel is selected to directly transmit large amounts of data, while during non-vector operations such as ADD, the second channel is selected to cache data in the shared memory area to save resources.

[0056] Finally, step 1043 monitors the opcode type during the hardware instruction execution cycle. When a vector opcode is detected, the optical switch matrix is ​​triggered to switch to the first data channel. When a non-vector opcode is detected, the optical switch matrix is ​​triggered to switch to the second data channel, completing the lightweight deployment and inference execution of the large model. In a specific implementation, after the instruction decoding phase is completed, a hardware checkpoint is set and monitored. The opcode field portion of the current instruction is captured through hardware logic circuits. Then, a preset identification bit sequence in the opcode field is extracted. These bit sequences specify the operation type, such as obtaining the key bit position through a shift operation, for example, extracting the upper 4 bits from an 8-bit opcode as an identification sequence. Next, the identification bit sequence is compared with a predefined set of vector opcode identifiers, such as using logic gate circuits to match identical bit values. When the identification sequence matches the set, a first path switching control signal is generated, indicating a vector operation; otherwise, a second path switching control signal is generated. Finally, these signals are input to a physical channel selector, such as an electronic controller of the switch matrix, which triggers the actual physical connection to switch to the first or second channel. This optimizes the path before instruction execution, ensuring data transmission efficiency, and thus executing lightweight model inference. This process forms a complete flow: monitoring points trigger type check sequence extraction and matching control signal generation, and dynamically switch paths within the execution cycle. For example, assuming the opcode field is 1100 and the identification bit sequence is 11, the predefined sets are [11 vector] and [00 scalar]. Comparison reveals that the 11-matching set generates the first signal, triggering the optical switch matrix to switch to the first channel for fast vector data processing, achieving efficient model inference deployment and completing large model tasks.

[0057] In a practical application, when implementing a lightweight plant disease recognition model in edge computing device A within an agricultural IoT, the reorganized parallel table lookup instruction block is first loaded into a 256KB memory area shared by the processing unit and the field-programmable gate array (FPGA), starting at address 0x2000. A vector opcode type set, including eight operation types such as VLOAD vector load, VSHR logical shift, and VAND bitwise operation, is extracted from the acceleration capability profile. Based on this set, the topological connection logic of the optical switch matrix is ​​configured: the first data path establishes a direct link from the processing unit vector register bank to the FPGA high-speed interface, with a transmission bandwidth set to 128 bits per cycle; the second data path establishes access from the processing unit scalar register to the shared memory area. In the hardware instruction execution design, the hardware logic circuit intercepts the opcode field after the instruction decoding stage. When the opcode bits [5:3] are detected as a binary sequence of 101, this feature matches the predefined VLOAD opcode identifier, and the first path switching control signal is immediately generated. In this scenario, the optical switch matrix switches to the first data channel within 4 nanoseconds. For example, when executing a VLOAD instruction, the computational block of the field-programmable gate array directly accesses the weight parameters in the vector register through this channel, achieving zero-copy data transmission. Conversely, when opcode bits [5:3] are 010, matching the scalar operation type, the second path switching control signal is triggered, and the processing unit accesses the intermediate calculation results in shared memory through the second channel. This dynamic path switching mechanism performs 27 channel switching operations in a single inference cycle, ensuring that large models maintain millisecond-level inference latency in resource-constrained agricultural edge devices.

[0058] In the overall solution of step 104 above, the reorganized parallel table lookup instruction block is loaded into the shared memory area between the processing unit and the field programmable gate array. Based on the set of vector opcode types extracted from the acceleration capability profile, a first data path topology logic connecting the vector registers and the high-speed interface and a second data path topology logic connecting the scalar registers and the shared memory are configured in the optical switch matrix. After instruction decoding, the opcode field identification bit sequence is intercepted by the hardware logic circuit and matched in real time with a predefined set of vector opcode identifiers. When a vector opcode is detected, a first path switching control signal is generated to trigger the optical switch matrix to establish a physical connection between the vector registers and the high-speed interface. When a non-vector opcode is detected, a second path switching control signal is generated to trigger a physical connection between the scalar registers and the shared memory. In this way, the data transmission path between the two architectures is switched instantly during the hardware instruction execution cycle, achieving adaptive data routing optimization based on opcode type, and ultimately completing lightweight deployment and efficient inference execution of large models.

[0059] The following is a complete embodiment of steps 101 to 104: like Figure 2As shown in the figure, in a smart agriculture edge computing device deployment, when implementing a large model lightweighting solution on device A, the instruction set architecture identifier 0xAA01 is first read from the device's system configuration area. The architecture table is then queried to confirm support for eight instruction types, including vector multiply-add and bit-extend, and a 128-bit operation width. The CPU's four physical cores and the FPGA's six reconfigurable computing blocks are collected through the computational management interface to form an acceleration capability profile consisting of 10 parallel units (4 + 6) and specific instruction characteristics. Based on the 128-bit maximum bit width in the profile, the 2.5MB weights of the plant disease recognition model are segmented into 65,536 units with a fixed length of 32 bits. Each 4 units constitutes a weight subset (128 / 32 = 4), generating a total of 16,384 subsets. In the unified lookup table engine, the input values ​​from 0 to 4294967295 are fully enumerated for each subset. When the input value is 261, the engine synchronously calculates the output of this value in four units. Unit 1 outputs the convolution coefficient of 0.74, unit 2 outputs the feature scaling factor of 1.2, unit 3 outputs the bias compensation of -0.15, and unit 4 outputs the normalization parameter of 0.98, which are combined into a four-dimensional output sequence. Finally, a 128-bit pre-calculated vector matrix is ​​generated in ascending order of the input values ​​and stored in the continuous memory area 0x2000~0x3FFFF.

[0060] Based on the number of parallel units (10), the memory area was divided into ten equal 25.6KB segments (0x2000-0x7FFF, 0x8000-0xDFFF, etc.). Compilation generated ten instruction blocks: Block 1 was populated with the "VLDR128 R1, 0x2000" opcode, Block 2 was populated with "VLDR128 R2, 0x8000," and so on. Ten execution threads were assigned to the corresponding instruction blocks. Scanning revealed that the output of Block 3 (starting at 0x14000) was dependent on the input of Block 7 (starting at 0x2A000). By inserting a SYNC instruction and moving Block 7's execution ahead of Block 3, the pipeline sequence was restructured to Block 1 → 2 → 7 → SYNC → 3 → 4, etc.

[0061] During the hardware execution phase, the instruction block is loaded into the CPU-FPGA shared memory starting at 0x20000. Based on the vector operation types VLDR / VSHR / VFMA in the image, a dual-channel optical matrix topology is configured: Channel 1 directly connects the CPU vector registers to the FPGA high-speed interface (128 bits wide); Channel 2 connects the CPU scalar registers to the shared memory area. During the instruction decoding phase, if the lower three bits of the opcode are detected as 101 for a vector operation, the matrix switches to Channel 1 within 0.8ns. If the lower three bits are 010 for a scalar operation, the matrix switches to Channel 2. This achieves single-cycle dynamic path switching, enabling efficient model inference on agricultural edge devices.

[0062] Figure 3The present invention provides a schematic diagram of a lightweight deployment system for a large model on an edge computing device. Figure 3 As shown, the system includes: A construction module 31 is configured to extract the hardware instruction set architecture type and the number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and construct an acceleration capability profile including the instruction type and the number of parallel computing units; A matching module 32 is configured to divide the large model weights into multiple weight subsets by bit grouping using a unified lookup table vectorization engine based on the instruction type in the acceleration capability profile, perform a pre-computation operation on each weight subset, and generate a pre-computation vector that matches the target instruction set architecture; A generation module 33 is configured to compile the pre-calculated vector based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, generate parallel table lookup instruction blocks adapted to the target instruction set architecture, assign execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganize the pipeline operation sequence according to the instruction-level parallelism, and eliminate data dependency conflicts; The switching module 34 is used to load the parallel table lookup instruction block into the shared memory area of ​​the processing unit and the field programmable gate array of the target edge device, configure the topological connection logic of the optical switch matrix based on the instruction type in the acceleration capability profile, and switch the data transmission path between the processing unit and the field programmable gate array through the optical switch matrix during the hardware instruction execution cycle to complete the lightweight deployment and inference execution of the large model.

[0063] Figure 3 The lightweight deployment system of the large model on the edge computing device can execute Figure 1 The implementation principle and technical effects of the lightweight deployment method of the large model on the edge computing device described in the illustrated embodiment will not be repeated here. The specific manner in which each module and unit performs operations in the lightweight deployment system of the large model on the edge computing device in the above embodiment has been described in detail in the embodiment of the method and will not be elaborated here.

[0064] In one possible design, Figure 3 A lightweight deployment system of a large model on an edge computing device in the embodiment shown can be implemented as a computing device, such as Figure 4 As shown, the computing device may include a storage component 41 and a processing component 42; The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42 .

[0065] The processing component 42 is used for the above Figure 1 The embodiment provides a lightweight deployment method for a large model on an edge computing device.

[0066] The processing component 42 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0067] The storage component 41 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0068] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0069] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0070] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0071] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0072] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment provides a lightweight deployment method for a large model on an edge computing device.

[0073] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0075] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer or server) to execute the methods described in each embodiment or certain portions of the embodiments.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A lightweight deployment method for a large model on an edge computing device, characterized in that: include: Extract the hardware instruction set architecture type and the number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and construct an acceleration capability profile that includes the instruction type and the number of parallel computing units; Based on the instruction type in the acceleration capability profile, the large model weights are divided into multiple weight subsets by bit grouping through a unified lookup table vectorization engine, and a pre-calculation operation is performed on each weight subset to generate a pre-calculated vector that matches the target instruction set architecture; Based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, compile the pre-calculated vectors to generate parallel table lookup instruction blocks adapted to the target instruction set architecture, assign execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganize the pipeline operation sequence according to the instruction-level parallelism, and eliminate data dependency conflicts; The parallel table lookup instruction block is loaded into the shared memory area of ​​the processing unit and the field programmable gate array of the target edge device. Based on the instruction type in the acceleration capability portrait, the topological connection logic of the optical switch matrix is ​​configured. Through the optical switch matrix, the data transmission path between the processing unit and the field programmable gate array is switched during the hardware instruction execution cycle to complete the lightweight deployment and inference execution of the large model.

2. The method according to claim 1, characterized in that Based on the instruction type in the acceleration capability profile, the large model weights are divided into multiple weight subsets by bit grouping using a unified lookup table vectorization engine. A pre-calculation operation is performed on each weight subset to generate a pre-calculation vector that matches the target instruction set architecture, including: Extracting a maximum parallel operation bit width from the instruction type of the acceleration capability profile, obtaining a preset fixed length of a bit grouping unit, dividing the maximum parallel operation bit width by the fixed length of the bit grouping unit, and calculating the number of bit grouping units included in a single weight subset; Cutting the large model weight sequence into a continuous unit sequence according to the fixed length of the bit grouping unit, and intercepting an equal number of continuous units from the continuous unit sequence according to the number of the bit grouping units to divide the sequence into multiple weight subsets; For each weight subset, enumerate the complete value range of the input value in the unified lookup table vectorization engine, and for each input value of the weight subset within the complete value range, calculate the set of simulated output values ​​of the input value on all bit grouping units in the weight subset; The analog output value sets are sequentially spliced ​​into vectorized results with a bit width equal to the maximum parallel operation bit width, and the vectorized results are arranged in ascending order of input values ​​to form a pre-calculated vector in a continuous storage area.

3. The method according to claim 2, characterized in that For each weight subset, a complete value range of the input value is enumerated in a unified lookup table vectorization engine, and for each input value of the weight subset within the complete value range, a set of simulated output values ​​of the input value on all bit grouping units in the weight subset is calculated, including: Determining a complete range of input values ​​for the weight subset, where the complete range is defined by the boundaries of the binary value space corresponding to the fixed length of the bit grouping unit; Traverse each input value in the complete value range in increasing integer order, and use the current input value as the input item to enter the unified lookup table vectorization engine; Synchronously calculating independent output values ​​of the input item on all bit grouping units in the weight subset within the complete value range, and generating an output value sequence equal to the number of bit grouping units; The output value sequence is combined according to the physical storage order of the bit grouping units to form a set of analog output values ​​corresponding to the current input value.

4. The method according to claim 1, wherein The parallel table lookup instruction block is loaded into the shared memory area of ​​the processing unit and the field programmable gate array of the target edge device. Based on the instruction type in the acceleration capability profile, the topological connection logic of the optical switch matrix is ​​configured. The data transmission path between the processing unit and the field programmable gate array is switched through the optical switch matrix during the hardware instruction execution cycle to complete the lightweight deployment and inference execution of the large model, including: Loading the reorganized parallel table lookup instruction block into a shared memory area between a processing unit and a field programmable gate array of a target edge device, extracting all vector opcode types from the instruction type of the acceleration capability profile, and generating a vector operation type set; Based on the vector operation type set, configuring the topological connection logic of the first data channel in the optical switch matrix, the channel connecting the processing unit vector register and the field programmable gate array high-speed interface, and setting the topological connection logic of the second data channel, the channel connecting the processing unit scalar register and the shared memory area; Monitor the opcode type in the hardware instruction execution cycle. When a vector opcode is detected, trigger the optical switch matrix to switch to the first data channel. When a non-vector opcode is detected, trigger the optical switch matrix to switch to the second data channel, completing the lightweight deployment and inference execution of the large model.

5. The method according to claim 4, characterized in that Monitor the opcode type in the hardware instruction execution cycle. When a vector opcode is detected, trigger the optical switch matrix to switch to the first data channel. When a non-vector opcode is detected, trigger the optical switch matrix to switch to the second data channel. This completes the lightweight deployment and inference execution of the large model, including: An opcode type monitoring point is established after the instruction decoding stage, the opcode field of the current instruction is intercepted by a hardware logic circuit, a preset identification bit sequence of the opcode field is extracted, and the identification bit sequence is matched with a predefined vector opcode identifier set; When the identification bit sequence matches the vector opcode identifier set, determining that the current instruction type is a vector opcode, and generating a first path switching control signal; When the identification bit sequence does not match the vector opcode identifier set, determining that the current instruction type is a non-vector opcode, and generating a second path switching control signal; The path switching control signal is input into the physical channel selector of the optical switch matrix, wherein the first path switching control signal triggers the physical connection of the first transmission path, and the second path switching control signal triggers the physical connection of the second transmission path, thereby completing the lightweight deployment and inference execution of the large model.

6. The method according to claim 1, characterized in that Extract the hardware instruction set architecture type and number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and build an acceleration capability profile that includes the instruction type and the number of parallel computing units, including: Reading an identification code of a hardware instruction set architecture type from a system configuration storage area of ​​a target edge device, and querying a predefined architecture capability mapping table based on the identification code to obtain all opcodes and bit width parameters supported by the hardware instruction set architecture type; The number of independent computing cores in the processing unit is collected through the computing unit management interface of the target edge device, and the number of configurable computing blocks is collected through the configuration status interface of the field programmable gate array; Based on the opcode and bit width parameters, identify the instruction level parallelism capability characteristics, and add the number of the independent computing cores and the number of the configurable computing blocks to obtain the total number of parallel computing units; All the opcodes and bit width parameters, instruction-level parallel capability characteristics, and the overall number of parallel computing units are integrated to construct an acceleration capability profile that includes instruction types and the number of parallel computing units.

7. The method according to claim 1, characterized in that Based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, compile the pre-calculated vectors, generate parallel table lookup instruction blocks adapted to the target instruction set architecture, assign execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganize the pipeline operation sequence according to the instruction-level parallelism, and eliminate data dependency conflicts, including: Analyze the continuous memory distribution characteristics of the pre-calculated vectors based on the instruction-level parallelism capability of the acceleration capability profile, compile the continuously stored fragments of the pre-calculated vectors into the same instruction block based on the continuity of the memory addresses, and generate an equal number of independent instruction blocks based on the number of parallel computing units; Filling the table lookup opcode of the target instruction set architecture and the corresponding pre-calculated vector starting address offset in each of the independent instruction blocks to obtain a parallel table lookup instruction block adapted to the target instruction set architecture, creating an execution thread identifier equal to the number of parallel computing units, and assigning each of the execution thread identifiers to the parallel table lookup instruction block; Scan the output and input relationships between all the parallel table lookup instruction blocks, identify the instruction block pairs with data dependencies, adjust the execution timing sequence of the identified instruction block pairs, insert synchronization control operation codes to eliminate data dependency conflicts, and rearrange the instruction block sequence according to the adjusted timing sequence.

8. A lightweight deployment system for large models on edge computing devices, characterized by: include: A construction module is used to extract the hardware instruction set architecture type and the number of parallel computing units of the target edge device, identify the instruction-level parallelism capability based on the hardware instruction set architecture type, and construct an acceleration capability profile including the instruction type and the number of parallel computing units; a matching module, configured to divide the large model weights into a plurality of weight subsets by bit grouping using a unified lookup table vectorization engine according to the instruction type in the acceleration capability profile, perform a pre-computation operation on each of the weight subsets, and generate a pre-computation vector that matches the target instruction set architecture; a generation module, configured to compile the pre-calculated vectors based on the number of parallel computing units and instruction-level parallelism in the acceleration capability profile, generate parallel table lookup instruction blocks adapted to the target instruction set architecture, assign execution threads equal to the number of parallel computing units to the parallel table lookup instruction blocks, reorganize the pipeline operation sequence according to the instruction-level parallelism, and eliminate data dependency conflicts; A switching module is used to load the parallel table lookup instruction block into the shared memory area of ​​the processing unit and the field programmable gate array of the target edge device, configure the topological connection logic of the optical switch matrix based on the instruction type in the acceleration capability profile, and switch the data transmission path between the processing unit and the field programmable gate array through the optical switch matrix during the hardware instruction execution cycle to complete the lightweight deployment and inference execution of the large model.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a lightweight deployment method of a large model on an edge computing device as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, a lightweight deployment method of a large model on an edge computing device as described in any one of claims 1 to 7 is implemented.

Images