Scene adaptive model dynamic loading system, RISC-V processor and edge computing device
By using a scenario-adaptive model dynamic loading system and a RISC-V architecture, the flexibility and dynamic model loading issues of edge computing platforms are solved, enabling efficient and accurate edge inference, suitable for complex applications with multiple scenarios and tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-03-13
AI Technical Summary
Existing edge computing platforms lack flexibility and scalability, making it difficult to adapt to diverse application scenarios. They also lack efficient and intelligent dynamic model loading capabilities, affecting the balance between inference accuracy and real-time performance.
A scene-adaptive model dynamic loading system is introduced. By using scene ID, model ID, and parameter ID, the system can automatically select the most suitable deep learning model and load the corresponding parameters according to different scenes and tasks. Combined with the flexible design of the RISC-V architecture, it can achieve efficient and low-power edge computing inference.
It improves the efficiency and accuracy of edge reasoning, enhances the system's flexibility and adaptability, and enables it to handle complex and ever-changing task scenarios, ensuring that the most suitable model and parameters can be used for reasoning in each task scenario.
Smart Images
Figure CN121658100A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer and artificial intelligence technology, specifically to a scene adaptive model dynamic loading system, a RISC-V processor, and an edge computing device. Background Technology
[0002] In the context of rapid technological advancements, edge computing and deep learning technologies have been widely applied to various smart devices and IoT terminals, playing a crucial role, especially in applications requiring rapid data processing and real-time decision-making. By offloading computing tasks from the cloud to terminal devices, edge computing can significantly reduce communication latency, alleviate network load, and improve the overall efficiency of data processing.
[0003] Despite numerous research and technological solutions dedicated to this field, existing edge computing platforms and deep learning models still face several significant challenges in deployment. First, most existing systems rely on fixed hardware platforms and predefined model structures, lacking sufficient flexibility and scalability to adapt to the diverse needs of various application scenarios. Furthermore, existing dynamic model loading mechanisms are mostly focused on simple tasks, lacking efficient and intelligent model selection and dynamic loading capabilities when facing complex or multi-scenario tasks, thus affecting the balance between inference accuracy and real-time performance.
[0004] With the increasing demand for edge computing, technological development is evolving towards higher computing efficiency and lower power consumption. As an open-source, customizable instruction set architecture, the RISC-V architecture, with its modularity and flexibility, is gradually becoming an ideal choice for edge computing platforms and embedded systems. However, current RISC-V-based edge computing systems are mostly focused on processing static tasks or supporting basic model loading, lacking the ability to flexibly adapt to complex tasks and effective support for dynamic model loading mechanisms across multiple scenarios.
[0005] Therefore, the industry urgently needs a new system that can adaptively adjust to application scenarios and support dynamic loading and switching of deep learning models. Especially when handling high-precision inference tasks, this system should be able to assess task requirements in real time, intelligently select the optimal model, and configure corresponding parameters to meet the comprehensive requirements of efficiency, accuracy, and real-time performance for diverse edge applications. Summary of the Invention
[0006] This invention provides a scene-adaptive model dynamic loading system that can dynamically load the most suitable model according to different application scenarios, achieving efficient and accurate edge inference.
[0007] The present invention also provides a RISC-V processor for dynamic loading of scene adaptive models.
[0008] The present invention also provides an edge computing device for data processing and inference processes in edge computing.
[0009] Other objects and advantages of the present invention can be further understood from the technical features disclosed herein.
[0010] To achieve one or more of the above objectives or other objectives, the present invention provides a scene-adaptive model dynamic loading system, comprising: a data input module for receiving task data transmitted from an external device; a preprocessing module for receiving the input task data, determining the task scene to which the input task belongs based on an internal lightweight model, and outputting a scene ID; a model selection module for receiving the scene ID and selecting a model suitable for the current task, generating a model ID and a parameter ID; a context cache control module for receiving the scene ID, model ID, and parameter ID, and outputting a comparison result of the old and new IDs; a parameter storage control module for reading and transmitting model parameters from a memory according to the parameter ID, wherein the parameter storage control module receives the comparison result of the old and new IDs output by the context cache control module, and controls the parameter storage control module to perform memory access operations when the comparison results are different; and an execution module for running the loaded model on a hardware platform and performing inference tasks.
[0011] The hardware platform runs a RISC-V processor; the RISC-V instruction set is modified to create custom instructions: model ID instructions and parameter ID instructions; after the RISC-V processor retrieves the model ID instruction or parameter ID instruction, it parses and decodes the instruction function. During the execution phase, the scene ID information stored in the preprocessing module is retrieved through the LSU module, and the scene ID information is written into a register for the model selection module to read; the model selection module reads the scene ID information to generate a model ID and a parameter ID, and the RISC-V processor writes the model ID and parameter ID generated by the model selection module back to the corresponding registers.
[0012] The method for creating the model ID instruction and the parameter ID instruction is as follows: They are created within the R-type instruction format of the RISC-V instruction set; the same opcode is used as the identifier for both the model ID instruction and the parameter ID instruction; the funct7 and funct3 encoding fields are used to distinguish between the model ID instruction and the parameter ID instruction; the rs1 encoding field of both the model ID instruction and the parameter ID instruction represents the address of the storage register storing the current scene ID; the rd encoding field of the model ID instruction represents the register address where the model ID is written, and the rd encoding field of the parameter ID instruction represents the register address where the parameter ID is written.
[0013] The model selection module includes a parameter table and a model table; the parameter table records the mapping relationship between different scene IDs and parameter IDs, and the model table records the mapping relationship between different scene IDs and model IDs; the model selection module obtains the scene ID information written into the register of the RISC-V processor, performs a search process, and outputs the parameter ID and model ID.
[0014] It also includes a mode selection module, which determines the current task type based on the input data and selects automatic or manual mode; in automatic mode, the task data is directly input into the preprocessing module, and the model selection module performs a search process based on the scene ID and outputs the model ID and parameter ID; in manual mode, the model selection module allows the user to select a specific model ID and parameter ID to match the scene ID.
[0015] The model selection module also includes an ID model matcher. The lookup results output by the parameter table and the model table are input to the ID model matcher. The lookup results output by the parameter table and the model table are synchronously input to the context cache control module for comparison, and the new and old ID comparison results are output. The new and old ID comparison results are input to the ID model matcher. In automatic mode, the ID model matcher outputs an enable signal to enable the corresponding model based on the new and old ID comparison results, the input scene ID, and the lookup results output by the parameter table and the model table. The ID model matcher receives the manual mode signal and, in manual mode, allows the user to select a specific model ID and parameter ID to match the scene ID and enable the corresponding model.
[0016] When the comparison result of the old and new IDs output by the context cache control module is the same, the ID model matcher outputs the historical model ID and the historical parameter ID, and enables the corresponding model; the execution module reads and transmits the model parameters from the memory according to the historical parameter ID information; when the comparison result of the old and new IDs output by the context cache control module is different, the parameter storage control module performs a memory access operation and passes the address of the parameter information to the execution module for transmitting the model parameters.
[0017] The context cache control module includes a new ID information module, an old ID information module, and an ID comparison module. When the comparison results of the new and old IDs are different, the new ID information module triggers an ID update, updating the new ID information into the old ID information module.
[0018] The lightweight model of the preprocessing module is a micro-scene classifier structure with a total of 3500 parameters. It includes: an input layer that receives task data; a first convolutional layer, which includes a convolutional layer with a kernel size of 7×7×3×8, a pooling layer, a ReLU activation function layer, and a max pooling layer, used for preliminary feature extraction; a second convolutional block, which includes a convolutional layer with a kernel size of 3×3×8×16, a pooling layer, and a ReLU activation function layer, used to compress the preliminary features extracted by the first convolutional layer; a global average pooling layer, used to compress the feature map to a fixed size; and a classification layer, which includes a Dropout layer and a fully connected layer, used to output the raw score of the classified scene.
[0019] The lightweight model of the preprocessing module uses the Softmax function to convert the original scores of the classified scenes into a probability distribution, and determines the scene category based on the highest probability. Based on the scene category, the preprocessing module generates a scene ID.
[0020] Another technical solution of the present invention provides a RISC-V processor for dynamically loading scene adaptive models, comprising: an instruction fetch unit for fetching instructions from an instruction memory, wherein the RISC-V instruction set contains custom instructions: a model ID instruction and a parameter ID instruction; a decoding unit for parsing and decoding the instruction function after the RISC-V processor fetches the model ID instruction or parameter ID instruction; and an execution unit comprising a model dynamic loading module and an LSU module, wherein the LSU module fetches scene ID information stored in a preprocessing module and writes the scene ID information into a register for the model selection module to read, and simultaneously transmits the generated control signal to the model selection module to generate a model ID and parameter ID. Parameter ID; The model dynamic loading module executes the computation task of the corresponding model. The model dynamic loading module includes multiple computing units, each computing unit executes one type of operation. The external model enable controller receives the model enable signal, parses the type of operation required by the current task, and then the model dynamic loading module dynamically reconstructs the hardware path based on the operation type, so that the corresponding computing unit executes the computation task. The external enable signal is received by the external parameter distribution unit, which distributes the parameters to the corresponding computing units. The computing unit reads the task data from the register, performs accelerated inference operations, and outputs the inference result. Memory access unit is used to access the data memory. Write-back unit writes the processing result back to the register.
[0021] The method for creating the model ID instruction and the parameter ID instruction is as follows: The model ID instruction and the parameter ID instruction are created within the R-type instruction format of the RISC-V instruction set; the same opcode is used as the identifier for both the model ID instruction and the parameter ID instruction; the funct7 encoding field and the funct3 encoding field are used to distinguish between the model ID instruction and the parameter ID instruction; the rs1 encoding field of both the model ID instruction and the parameter ID instruction is the address of the storage register storing the current scene ID; the rd encoding field of the model ID instruction represents the register address where the model ID is written, and the rd encoding field of the parameter ID instruction represents the register address where the parameter ID is written.
[0022] Another technical solution of the present invention provides an edge computing device, which includes a scene adaptive model dynamic loading system as described above. The edge computing device is used for data processing and inference processes in edge computing.
[0023] Compared with existing technologies, the beneficial effects of this invention mainly include: 1. This invention introduces a scene-adaptive model dynamic loading mechanism. Through scene ID, model ID, and parameter ID, the system can automatically select the most suitable deep learning model according to different scenes and tasks, and load the corresponding parameters according to task requirements. This scheme greatly improves the efficiency and accuracy of edge inference, and is particularly suitable for complex applications with multiple scenes and tasks. 2. This invention, based on the flexible design of the RISC-V architecture, realizes an efficient and low-power edge computing inference process. 3. This invention, by introducing a mode selection module, can intelligently select automatic or manual mode according to the needs of different tasks. In automatic mode, the system can autonomously determine the task scene and select the corresponding processing flow, while manual mode allows for manual intervention. This design enhances the flexibility and adaptability of the system, enabling it to handle complex and variable task scenes. 4. This invention ensures accurate matching of models and parameters through the collaborative work of the parameter storage controller and the model selection module. By transmitting scene ID, model ID, and parameter ID, this invention ensures that each task scene can use the most suitable model and parameters for inference, greatly improving the accuracy and stability of the inference results.
[0024] To make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the specific embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is an architecture diagram of a scene adaptive model dynamic loading system according to the present invention.
[0027] Figure 2 This is a flowchart illustrating the workflow of a scene adaptive model dynamic loading system according to the present invention.
[0028] Figure 3 This is a schematic diagram of the preprocessing module of the present invention.
[0029] Figure 4 This is a schematic diagram of the lightweight model structure of the present invention.
[0030] Figure 5 This is a schematic diagram of the context cache control module of the present invention.
[0031] Figure 6 This is a schematic diagram of the model selection module of the present invention.
[0032] Figure 7 This is a schematic diagram of the custom instruction structure of the present invention.
[0033] Figure 8 This is a schematic diagram of the custom instructions of the present invention.
[0034] Figure 9 This is a schematic diagram of the RISC-V processor core of the present invention.
[0035] Figure 10 This is a schematic diagram of the internal architecture of the execution module of the present invention.
[0036] Figure 11 This is an application example diagram of the scene adaptive model dynamic loading system of the present invention. Detailed Implementation
[0037] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate preferred embodiments of the application. However, this application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.
[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
[0039] It should be understood that when an element or layer is referred to as being "on," "adjacent to," or "connected to" other elements or layers, it may be directly on, adjacent to, connected to, or coupled to other elements or layers, or there may be intervening elements or layers. It should be understood that although the terms first, second, third, etc., may be used to describe various elements, components, regions, layers, doping types, and / or portions, these elements, components, regions, layers, doping types, and / or portions should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer, doping type, or portion from another element, component, region, layer, doping type, or portion. Therefore, without departing from the teachings of this application, the first element, component, region, layer, doping type, or portion discussed below may be referred to as a second element, component, region, layer, or portion.
[0040] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that when the terms “comprise” and / or “comprising” are used in this specification, the presence of the stated feature, integer, step, operation, element, and / or part is established, but the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or groups is not excluded. Meanwhile, when used herein, the term “and / or” includes any and all combinations of the associated listed items.
[0041] Embodiments of the invention are described herein with reference to cross-sectional views illustrating preferred embodiments (and intermediate structures) of this application, thus allowing for the anticipation of variations in the illustrated shapes due to, for example, manufacturing techniques and / or tolerances. Embodiments of this application should not be limited to the specific shapes of the regions shown herein, but rather include shape deviations due to, for example, manufacturing techniques. Therefore, the regions shown in the figures are substantially schematic, and their shapes do not represent the actual shapes of regions of the device, nor do they limit the scope of this application.
[0042] Example 1 Example 1 provides a scene-adaptive model dynamic loading system, comprising: a data input module for receiving task data transmitted from an external device; a preprocessing module for receiving the input task data, determining the task scene to which the input task belongs based on an internal lightweight model, and outputting a scene ID; a model selection module for receiving the scene ID and selecting a model suitable for the current task, generating a model ID and a parameter ID; a context cache control module for receiving the scene ID, model ID, and parameter ID, and outputting a comparison result of the old and new IDs; a parameter storage control module for reading and transmitting model parameters from memory according to the parameter ID, wherein the parameter storage control module receives the comparison result of the old and new IDs output by the context cache control module, and controls the parameter storage control module to perform memory access operations when the comparison results are different; and an execution module for running the loaded model on a hardware platform and performing inference tasks.
[0043] This invention provides a scene-adaptive model dynamic loading system, which introduces a scene-adaptive model dynamic loading mechanism. Using scene ID, model ID, and parameter ID, the system can automatically select the most suitable deep learning model based on different scenes and tasks, and load the corresponding parameters according to task requirements. This solution greatly improves the efficiency and accuracy of edge inference, and is particularly suitable for complex applications involving multiple scenes and tasks.
[0044] The following text, in conjunction with the appendix Figure 1-10 A detailed explanation of the scene adaptive model dynamic loading system of the present invention is provided.
[0045] like Figure 2 As shown, the present invention discloses a scene-adaptive model dynamic loading system, including a data input module for receiving external task data, such as sensor data, image data, and environmental signals. The data input module receives data transmitted from external devices via a communication interface and transmits this task data to a mode selection module and a preprocessing module. The mode selection module determines the task type based on the task data and decides whether the system should use an automatic or manual mode. In automatic mode, the task data is directly transmitted to the preprocessing module for further processing, while in manual mode, the user can manually intervene and adjust the task processing method. Specifically, in manual mode, the user can manually select a model ID and parameter ID that match the scene ID based on the scene ID through the user interface.
[0046] The preprocessing module uses a lightweight model to perform preliminary analysis of the data, determine the task scenario type, generate a scenario ID based on the analysis results, and then pass the scenario ID to the context cache control module and the model selection module. For details, see [link to relevant documentation]. Figure 3The diagram illustrates the preprocessing module, which internally includes a lightweight model and an ID generation module. The lightweight model receives automatic mode signals and task data to output scene classification results, while the ID generation module outputs scene ID information based on the scene classification results. Figure 4 The diagram shows a lightweight model. The lightweight model of this invention adopts a micro scene classifier architecture, which is designed for real-time edge computing. The total number of parameters is 3500, which is significantly lower than that of traditional deep learning models, thereby achieving efficient inference on resource-constrained embedded devices. The lightweight model comprises the following hierarchical structure: the input layer receives task data (e.g., RGB image data with 3 input channels); the first convolutional block contains convolutional layers with kernel sizes of 7×7×3×8, pooling layers, ReLU activation function layers, and max pooling layers (with a kernel size of 4×4) for initial feature extraction; the second convolutional block contains convolutional layers with kernel sizes of 3×3×8×16, pooling layers, and ReLU activation function layers to further compress the feature dimensions; the global average pooling layer compresses the feature map to a fixed size; finally, the classification layer outputs the scene classification result through a Dropout layer (a regularization technique used in training deep learning models to prevent overfitting, which randomly discards a portion of neurons in the neural network in each iteration of the training process, i.e., temporarily setting their activation output to 0) and a fully connected layer, where the dropout rate of the Dropout layer is 0.1.
[0047] In operation, the lightweight model rapidly analyzes the input task data, performing forward propagation computation: the task data undergoes feature dimensionality reduction and abstraction sequentially through the first and second convolutional blocks, a global average pooling layer generates spatially invariant feature vectors, and a classification layer outputs raw scores for multiple scenarios (e.g., handwriting recognition, face recognition, license plate recognition, etc.). Subsequently, the model applies the Softmax function (a mathematical function that takes a vector containing arbitrary real numbers and converts it into a probability distribution, outputting a probability distribution with all probability values between 0 and 1, and the sum of all probability values equals 1) to convert the raw scores into a probability distribution, and determines the scene category based on the highest probability. Based on the classification results, the preprocessing module generates a scene ID (e.g., 0 corresponds to the handwriting recognition scene), which is used in subsequent model selection and inference processes to ensure the selection of the deep learning model most suitable for the current task.
[0048] The model selection module selects a suitable deep learning model for the current task based on the received scene ID, and generates a model ID and parameter ID. For example... Figure 6The schematic diagram of the model selection module shown includes a parameter table, a model table, and an ID model matcher. The parameter table records the mapping relationship between different scene IDs and parameter IDs, and the model table records the mapping relationship between different scene IDs and model IDs. When the model selection module receives a scene ID, in automatic mode, the parameter table and model table perform a lookup operation to find the model ID and parameter ID that match the scene ID, and then pass the model ID and parameter ID to the ID model matcher. In manual mode, the manual mode signal is directly input to the ID model matcher, which then performs manual selection of the model ID and parameter ID based on the scene ID. The ID model matcher outputs the corresponding model enable signal based on the action of various signals. Specifically, in automatic mode, after receiving the corresponding model ID and parameter ID, the ID model matcher enables the corresponding model and outputs an enable signal. In manual mode, it enables the corresponding model based on the user-selected model ID and parameter ID.
[0049] The model selection module obtains the scene ID information written into the register by the RISC-V processor (implemented by the RISC-V processor executing the RISC-V R-type custom instruction). The scene ID information is input into the parameter table and the model table. After the parameter table and the model table obtain the corresponding scene ID, they perform a lookup process and output the parameter ID and model ID to the ID model matcher.
[0050] The model ID and parameter ID output from the parameter table are passed to the context cache control module for comparison of the old and new IDs. Before returning the comparison result to the ID model matcher, the system waits. When the old and new IDs are the same, the ID model matcher outputs the historical model ID and historical parameter ID (this information is cached and does not require memory access) and enables the corresponding model. When the old and new IDs are different, the parameter storage module accesses memory and passes the found parameter storage address to the execution module for transmitting model parameters.
[0051] Simultaneously, when the old and new ID information differ, the context cache module control module will also update the old ID information (historical ID information) in the cache, updating the cached information in real time, such as... Figure 5 As shown, the context cache control module includes a new ID information module, an old ID information module, and an ID comparison module; when the comparison result of the new and old IDs shows that the new and old ID information do not match, the new ID information module triggers an ID update, updating the new ID information to the old ID information module.
[0052] The hardware platform runs a RISC-V processor. This application modifies the RISC-V instruction set to add custom instructions: model ID instructions and parameter ID instructions (created within the R-type instruction format of the RISC-V instruction set; the R-type instruction format is a customizable instruction format, as illustrated in the instruction format diagram below). Figure 7 As shown), the specific method is as follows: the same opcode is used as the identifier for the model ID instruction and the parameter ID instruction; the funct7 encoding field and the funct3 encoding field are used to distinguish the model ID instruction and the parameter ID instruction; the rs1 encoding field of the model ID instruction and the parameter ID instruction is the address of the storage register storing the current scene ID; the rd encoding field of the model ID instruction represents the register address where the model ID is written, and the rd encoding field of the parameter ID instruction represents the register address where the parameter ID is written.
[0053] For example: two custom directives providing the scene ID are named ModelID and ParamID, such as Figure 8 The diagram shows custom instructions. The instruction types for ModelID and ParamID are ModelID,rd,rs1 and ParamID,rd,rs1, respectively. The opcodes for both ModelID and ParamID are 0001011 (0x0B). The function7 and function3 for ModelID are 0000001 and 000, respectively, while those for ParamID are 0000011 and 001. Therefore, the instructions can be identified based on the opcodes, and further distinguished using the function7 and function3 encoding fields. The register address width for rd and rs1 is 5 bits. rs1 represents the address of the register storing the current scene ID, rd for ModelID represents the register address where the model ID output by the model selection module is written, and rd for ParamID represents the register address where the parameter ID output by the model selection module is written. The rs2 field is not set (blank).
[0054] The RISC-V processor provides scenario IDs to the parameter table and lookup table; for details, please refer to [link to relevant documentation]. Figure 9The diagram shows a RISC-V processor. The RISC-V processor includes a five-stage pipeline: fetch, decode, execute, memory access, and write-back. In the fetch stage, the corresponding instruction ID is fetched from the instruction memory based on the PC address generated by the program counter. In the decode stage, upon receiving the ModelID and ParamID instructions, the function of the instruction is parsed based on the opcode, function7, and function3 encodings. In the execute stage, the scene ID information stored in the preprocessing module is retrieved from the LSU module and written to a specific register for the model selection module to read. The model selection module reads the corresponding scene ID information from the register to generate the model ID and parameter ID, and then writes the generated model ID and parameter ID back to the corresponding registers.
[0055] The execution module performs the computational tasks of the corresponding model through the model dynamic loading module in the execution unit within the RISC-V processor and outputs the inference results.
[0056] The architecture of the execution module can be found in [reference needed]. Figure 10 As can be seen, the model dynamic loading module includes multiple computation units, each of which performs a type of operation (such as...). Figure 10 As shown, the model dynamic loading module is a reconfigurable computing array. Its internal computing units include processing / optimization computing units, sequence / fully connected computing units, and convolution computing units. Each computing unit can execute different computing tasks. The external model enable controller receives the model enable signal and parses the computing type required by the current task. At this time, the model dynamic loading module can dynamically reconfigure the hardware path based on the computing type required by the current task. Figure 10 In the reconfigurable computing array, different computing units are connected in hardware to start computing tasks, while units that do not need to participate in computing are disconnected in hardware to reduce power consumption. This allows the corresponding computing units to execute computing tasks. External enable signals are received by external parameter distribution units, which distribute parameters to the corresponding computing units. The computing units read task data from registers, execute accelerated inference operations of the model, and output inference results.
[0057] The following text combines Figure 1 The flowchart shown illustrates the operation process of a scene adaptive model dynamic loading system provided in Embodiment 1. Figure 1The system first receives data from sensors or image acquisition devices, then determines the task type. Based on the task type determination, the system generates three types of ID information (scene ID, model ID, and parameter ID). These generated IDs are cached in the system cache. Based on the generated model ID, the system selects and loads the corresponding deep learning model. The loaded model uses the stored parameter ID to extract the necessary parameter information from storage devices (such as memory or hard drives). After inference calculation is completed, the system outputs the inference result.
[0058] The above text, combined with the appendix Figure 1-10 This application provides a detailed explanation of a scene adaptive model dynamic loading system. Embodiment 1 will use a specific example to further explain the scene adaptive model dynamic loading system provided above.
[0059] like Figure 11 The specific application example diagram shown is as follows. Figure 11 The scene-adaptive model dynamic loading system shown is specifically applied to the intelligent and efficient traffic control at intersections. In the intelligent traffic intersection scenario, information such as vehicles, pedestrians, and traffic lights are input into the system through multiple sensors. Cameras transmit image streams in real time to the image acquisition unit, and simultaneously, the automatic mode is activated. In the preprocessing module, the system classifies traffic scenes. Assuming congestion occurs at the current intersection, the system will classify the scene ID as "traffic flow statistics." Subsequently, the system generates information with parameter ID 0530 in the model selection module and loads the traffic flow model. The RISC-V processor begins running the traffic flow model, and after hardware-accelerated inference, the system outputs traffic flow information of 66 vehicles per minute. Then, the traffic light duration is adjusted based on this information to alleviate intersection congestion.
[0060] All the above operations are performed in automatic mode. When the task changes and requires manual intervention, manual mode will be activated in the mode selection module. The user directly selects the task scenario and makes settings, such as a suspect search task. At this time, the system will load the face recognition model and generate parameter ID 0715. The RISC-V processor runs the face recognition model and obtains the result through hardware-accelerated inference. If a suspected target person is identified, the output control terminal will issue an alarm signal and return the captured target photo.
[0061] Example 2 Example 2 provides a RISC-V processor for dynamically loading scene-adaptive models, comprising: an instruction fetch unit for fetching instructions from an instruction memory, wherein the RISC-V instruction set contains custom instructions: a model ID instruction and a parameter ID instruction; a decoding unit for parsing and decoding the instruction function after the RISC-V processor fetches the model ID instruction or parameter ID instruction; and an execution unit comprising a model dynamic loading module and an LSU module, wherein the LSU module fetches scene ID information stored in a preprocessing module and writes the scene ID information into a register for the model selection module to read, and simultaneously transmits the generated control signal to the model selection module to generate the model ID and parameter ID. D; The model dynamic loading module executes the computational tasks of the corresponding model. The model dynamic loading module includes multiple computation units, each performing a different computation type. An external model enable controller receives the model enable signal, parses the computation type required by the current task, and then dynamically reconstructs the hardware path based on the computation type, enabling the corresponding computation unit to execute the computation task. The external enable signal is received by an external parameter distribution unit, which distributes the parameters to the corresponding computation units. The computation unit reads task data from registers, performs accelerated inference operations, and outputs the inference results. A memory access unit is used to access the data memory. A write-back unit writes the processing results back to the registers.
[0062] The method for creating the model ID instruction and the parameter ID instruction is as follows: The model ID instruction and the parameter ID instruction are created within the R-type instruction format of the RISC-V instruction set; the same opcode is used as the identifier for both the model ID instruction and the parameter ID instruction; the funct7 encoding field and the funct3 encoding field are used to distinguish between the model ID instruction and the parameter ID instruction; the rs1 encoding field of both the model ID instruction and the parameter ID instruction is the address of the storage register storing the current scene ID; the rd encoding field of the model ID instruction represents the register address where the model ID is written, and the rd encoding field of the parameter ID instruction represents the register address where the parameter ID is written.
[0063] The RISC-V processor provided in Embodiment 2 is used to execute the dynamic loading of the scene adaptive model in Embodiment 1. Specifically, it creates model ID instructions and parameter ID instructions, generates control signals during the decoding stage for controlling the model selection module, transmits the scene ID to the model selection module during the execution stage, and writes the parameter ID and model ID returned by the scene ID into the corresponding registers. The execution module also includes a model dynamic loading module, which performs the computational tasks of the corresponding model and outputs the inference results.
[0064] Example 3 Example 3 provides an edge computing device, which includes a scene adaptive model dynamic loading system as described in Example 1. The edge computing device is used for data processing and inference processes in edge computing.
[0065] The above provides a detailed description of the scene adaptive model dynamic loading system, RISC-V processor, and edge computing device provided by this invention. Specific examples have been used to illustrate the structure and working principle of this invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims of this invention.
Claims
1. A scene-adaptive model dynamic loading system, characterized in that, Includes a data input module for receiving task data transmitted from external devices; The preprocessing module receives the input task data, determines the task scenario to which the input task belongs based on the internal lightweight model, and outputs the scenario ID. The model selection module receives the scene ID, selects a model suitable for the current task, and generates a model ID and parameter ID. The context cache control module receives the scene ID, model ID, and parameter ID, and outputs the comparison result of the old and new IDs. The parameter storage control module reads and transmits model parameters from the memory according to the parameter ID. The parameter storage control module receives the comparison result of the old and new IDs output by the context cache control module, and controls the parameter storage control module to perform memory access operation when the comparison result is different. The execution module runs the loaded model on the hardware platform and performs inference tasks.
2. The scene adaptive model dynamic loading system according to claim 1, characterized in that, The hardware platform runs a RISC-V processor; Modify the RISC-V instruction set to create custom instructions: the model ID instruction and the parameter ID instruction; After the RISC-V processor retrieves the model ID instruction or parameter ID instruction, it parses and decodes the instruction function. During the execution phase, it retrieves the scene ID information stored in the preprocessing module through the LSU module and writes the scene ID information into the register for the model selection module to read. The model selection module reads the scene ID information to generate a model ID and a parameter ID, and the RISC-V processor writes the model ID and parameter ID generated by the model selection module back to the corresponding registers.
3. The scene adaptive model dynamic loading system according to claim 2, characterized in that, The method for creating the model ID instruction and the parameter ID instruction is to create them within the R-type instruction format of the RISC-V instruction set; The same opcode is used as the identifier for both the model ID instruction and the parameter ID instruction, and the funct7 and funct3 encoded fields are used to distinguish between the model ID instruction and the parameter ID instruction. The rs1 encoding field of the model ID instruction and the parameter ID instruction is the address of the storage register storing the current scene ID; The rd-encoded field of the model ID instruction represents the register address where the model ID is written, and the rd-encoded field of the parameter ID instruction represents the register address where the parameter ID is written.
4. The scene adaptive model dynamic loading system according to claim 2, characterized in that, The model selection module includes a parameter table and a model table; The parameter table records the mapping relationship between different scene IDs and parameter IDs, and the model table records the mapping relationship between different scene IDs and model IDs. The model selection module obtains the scene ID information written into the register by the RISC-V processor, performs a search process, and outputs the parameter ID and model ID.
5. The scene adaptive model dynamic loading system according to claim 4, characterized in that, It also includes a mode selection module, which determines the current task type based on the input data and selects automatic or manual mode; In automatic mode, task data is directly input into the preprocessing module, and the model selection module performs a search process based on the scene ID, outputting the model ID and parameter ID; In manual mode, the model selection module allows users to select a specific model ID and parameter ID to match the scene ID.
6. The scene adaptive model dynamic loading system according to claim 5, characterized in that, The model selection module also includes an ID model matcher, and the search results output by the parameter table and the model table are input to the ID model matcher. The lookup results output from the parameter table and the model table are synchronously input into the context cache control module for comparison, and the comparison results of the new and old IDs are output. The comparison results of the new and old IDs are input into the ID model matcher. In automatic mode, the ID model matcher outputs an enable signal to enable the corresponding model based on the comparison results of the old and new IDs, the input scene ID, and the search results output from the parameter table and model table. The ID model matcher receives the manual mode signal and, in manual mode, allows the user to select a specific model ID and parameter ID to match the scene ID and enable the corresponding model.
7. The scene adaptive model dynamic loading system according to claim 1, characterized in that, When the comparison result of the old and new IDs output by the context cache control module is the same, the ID model matcher outputs the historical model ID and the historical parameter ID, and enables the corresponding model. The execution module reads and transmits model parameters from memory based on historical parameter ID information; When the comparison result of the old and new IDs output by the context cache control module is different, the parameter storage control module performs memory access operation and passes the address of the parameter information to the execution module for transmitting model parameters.
8. The scene adaptive model dynamic loading system according to claim 1, characterized in that, The context cache control module includes a new ID information module, an old ID information module, and an ID comparison module; When the comparison results of the new and old IDs are different, the new ID information module triggers an ID update, updating the old ID information module with the new ID information.
9. A scene adaptive model dynamic loading system according to claim 1, characterized in that, The lightweight model of the preprocessing module is a micro-scene classifier structure with a total of 3500 parameters, including: The input layer receives task data; The first convolutional layer includes a convolutional layer with a kernel size of 7×7×3×8, a pooling layer, a ReLU activation function layer, and a max pooling layer. The first convolutional layer is used for preliminary feature extraction. The second convolutional block contains a convolutional layer with a kernel size of 3×3×8×16, a pooling layer, and a ReLU activation function layer, which compresses the preliminary features extracted by the first convolutional layer. Global average pooling layers are used to compress feature maps to a fixed size; The classification layer, which includes a Dropout layer and a fully connected layer, is used to output the raw scores of the scene after classification.
10. A scene adaptive model dynamic loading system according to claim 9, characterized in that, The lightweight model of the preprocessing module uses the Softmax function to convert the original scores of the classified scenes into a probability distribution, and determines the scene category based on the highest probability. Based on the scene category, the preprocessing module generates a scene ID.
11. A RISC-V processor for performing dynamic loading of scene adaptive models, characterized in that, include: The instruction fetch unit is used to fetch instructions from the instruction memory. The RISC-V instruction set includes custom instructions: model ID instructions and parameter ID instructions. The decoding unit parses and decodes the instruction function after the RISC-V processor retrieves the model ID instruction or parameter ID instruction. The execution unit includes a model dynamic loading module and an LSU module. The LSU module retrieves the scene ID information stored in the preprocessing module and writes the scene ID information into a register for the model selection module to read. At the same time, the generated control signal is transmitted to the model selection module to generate the model ID and parameter ID. The model dynamic loading module executes the calculation task of the corresponding model. The model dynamic loading module includes multiple computing units, each computing unit executes a type of operation. The external model enable controller receives the model enable signal, parses the type of operation required by the current task, and then the model dynamic loading module dynamically reconstructs the hardware path based on the type of operation, so that the corresponding computing unit executes the computing task. An external enable signal is received by an external parameter distribution unit, which then distributes the parameters to the corresponding arithmetic units. The arithmetic unit reads task data from the register, performs accelerated inference operations, and outputs the inference results; The memory access unit is used to access the data memory; Write back to the unit, writing the processing result back into the register.
12. A RISC-V processor according to claim 10, characterized in that, The method for creating the model ID instruction and the parameter ID instruction is to create the model ID instruction and the parameter ID instruction within the R-type type instruction format of the RISC-V instruction set; The same opcode is used as the identifier for both the model ID instruction and the parameter ID instruction, and the funct7 and funct3 encoded fields are used to distinguish between the model ID instruction and the parameter ID instruction. The rs1 encoding field of the model ID instruction and the parameter ID instruction is the address of the storage register storing the current scene ID; The rd-encoded field of the model ID instruction represents the register address where the model ID is written, and the rd-encoded field of the parameter ID instruction represents the register address where the parameter ID is written.
13. An edge computing device, characterized in that, The system includes a scene-adaptive model dynamic loading system as described in any one of claims 1-10, wherein the edge computing device is used for data processing and inference processes in edge computing.