Model weight solidification type inference chip-oriented native task adaptation inference method and system
By generating capability boundary maps and decomposing tasks into atomic operations on local terminal devices, and utilizing hard-wired inference units and lightweight incremental models, rapid task adaptation of model weight-fixed inference chips is achieved. This solves the problems of long task adaptation cycles and high costs in existing technologies, and is suitable for the high security, low latency, and real-time requirements of various types of terminal devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NINGBO YINGZHONGCHUANG TECHNOLOGY CO LTD
- Filing Date
- 2026-03-21
- Publication Date
- 2026-06-16
AI Technical Summary
Existing model weight-fixed inference chips require redesign or re-tape-out when adapting to new downstream tasks, resulting in long task adaptation cycles and high costs. Furthermore, there is a lack of task adaptation solutions that do not rely on the cloud or require hardware modifications, making it difficult to meet the usage requirements of offline, high-security, and high-real-time scenarios.
By scanning the chip's capability boundary map on the local terminal device, the target task is broken down into atomic inference operations. The incremental correction and knowledge fusion are completed at the output end using hard-wired inference units and lightweight incremental models, enabling rapid adaptation to new tasks. It supports online and offline dual-mode operation, enhancing the chip's versatility and practicality.
Without modifying the chip hardware or the weights of the basic model, it enables rapid adaptation to new tasks, reduces costs, shortens the cycle, and is suitable for high-security, network-free, and high-real-time scenarios. It expands the applicable scenarios of the chip and meets the flexible needs of various terminal devices.
Smart Images
Figure CN122222020A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to artificial intelligence inference, and more particularly to a native task-adaptive inference method and system for inference chips with fixed model weights. Background Technology
[0002] With the popularization of large model inference technology, dedicated inference chips are gradually developing towards lightweight, low power consumption, and high security. Among them, model weight-fixed inference chips, by fixing the basic model weights and inference circuits on the chip hardware, have the advantages of low inference latency, low power consumption, and high security, and are suitable for scenarios such as offline inference, high-concurrency inference, and confidential inference.
[0003] However, such chips have certain limitations in practical applications: since the weights of the basic model are fixed in the read-only memory unit, they cannot be modified, fine-tuned or replaced by software. When it is necessary to adapt to new downstream tasks, it is usually necessary to redesign the chip or re-fabricate it, resulting in a long task adaptation cycle and high cost, which to some extent limits the applicability and scalability of the chip.
[0004] Existing large-scale model adaptation methods are mostly based on programmable chips, completing task migration by adjusting model weights, training parameters, or recompiling the model. However, they are not suitable for fixed inference chips where weights cannot be modified. At the same time, existing technologies lack a task adaptation solution that can run independently on the local terminal, without relying on the cloud, without modifying hardware, and without changing fixed weights, making it difficult to meet the usage requirements of offline, high-security, and high-real-time scenarios. Summary of the Invention
[0005] Purpose of the invention: The purpose of this invention is to provide a native task adaptation inference method and system for inference chips with fixed model weights. The aim is to achieve rapid adaptation of new tasks through pure software without modifying the chip hardware structure, changing the fixed weights of the basic model, or relying on remote computing power in the cloud. This reduces adaptation costs, shortens the task adaptation cycle, and improves the versatility and practicality of the chip in native terminal scenarios.
[0006] Technical Solution: A native task-adaptive inference method for model-weight-fixed inference chips. The model-weight-fixed inference chip is mounted on a single local terminal device. The chip includes at least an on-chip read-only memory unit, a hard-wired inference unit corresponding to the basic large model, and an on-chip read-write memory unit. The weights of the basic large model are fixed in the on-chip read-only memory unit and cannot be modified by software. The entire inference process of the method is executed locally on the local terminal device; the cloud is only used for non-inference auxiliary operations and does not undertake any inference computation tasks. The method includes the following steps: S1. Scan the model weight-fixed inference chip in the local terminal device and generate the capability boundary map of the basic large model through a preset test case set; S2. Obtain the target task requirements input by the user, break down the target task into atomic inference operations that can be executed by the basic large model, as well as supplementary rules and proprietary knowledge that match the task. S3. Generate an inference execution pipeline based on the target task, schedule the hard-wired inference unit of the scheduling chip to perform atomic inference operations, and complete incremental correction and knowledge fusion at the output of the hard-wired inference unit to obtain the initial inference result. S4. Perform compliance and task consistency checks on the initial inference results. If the checks pass, output the final inference results. If the checks fail, perform secondary optimization inference.
[0007] Furthermore, the capability boundary map includes the atomic inference capability types, context window length, output format constraints, domain applicability, and inference latency metrics of the basic large model.
[0008] Furthermore, the supplementary rules are implemented through a scenario-based lightweight incremental model; the lightweight incremental model is deployed on the local terminal and performs correction and fusion at the output of the hardwired inference unit without entering the chip, changing the basic inference process, or modifying the fixed weights.
[0009] Furthermore, the proprietary knowledge is obtained through local retrieval enhancement. The user's proprietary knowledge base is pre-vectorized and stored on the local terminal device. During the reasoning process, local retrieval is performed according to task requirements and supplemented into the reasoning context.
[0010] Furthermore, the method supports both online and offline modes. The core inference steps are all completed locally on the local terminal device without relying on remote computing power in the cloud. In offline mode, all steps are executed in a network-disconnected environment, while in online mode, only non-computing power auxiliary operations are performed. In online mode, the following auxiliary operations can be performed with user authorization: synchronizing scene optimization strategies, updating incremental correction plugins, updating the local knowledge base, software version upgrades, and authorization management. These operations do not interfere with the core inference process of the chip.
[0011] Furthermore, after step S4, a closed-loop optimization step S5 is included: based on the user's feedback data on the inference results, the task decomposition strategy, incremental correction rules, and retrieval accuracy are iteratively optimized locally on the local terminal device.
[0012] Furthermore, the local multi-module collaborative inference supports multi-task parallel scheduling, which can simultaneously schedule the chip to execute atomic inference operations for multiple target tasks.
[0013] Furthermore, the local terminal device includes desktop computers, portable smart terminals, all-in-one office computers, vehicle-mounted smart terminals, industrial stand-alone equipment, security smart terminals, and edge computing terminals.
[0014] A native task-adaptive inference system for model-weight-fixed inference chips is disclosed. The system is deployed on a local terminal device equipped with the model-weight-fixed inference chip, and the entire inference process is completed locally without relying on cloud computing power. The chip includes at least an on-chip read-only memory unit, a hard-wired inference unit corresponding to the basic large model, and an on-chip read-write memory unit. The basic model weights are fixed in the on-chip read-only memory unit and cannot be modified by software. The system includes: Chip adaptation module, used to scan chips and generate capability boundary maps of basic large models; The task parsing module is used to obtain the target task requirements, break down the task into atomic reasoning operations, supplementary rules and proprietary knowledge, and generate a reasoning execution pipeline. The local inference scheduling module is used to schedule hard-wired inference units to perform atomic inference operations and to complete incremental correction and knowledge fusion at the output end. The result verification module is used to verify the compliance and task consistency of the initial inference results. If the verification passes, the final result is output; if the verification fails, a second optimization inference is triggered. The visual interaction module provides a user interface that supports task input, rule configuration, result display, and result feedback.
[0015] Furthermore, the system also includes a lightweight incremental correction module, a local retrieval enhancement module, and a local closed-loop optimization module. The lightweight incremental correction module is used to deploy a scenario-based lightweight incremental model and perform correction fusion at the output of the hard-wired inference unit without entering the chip or modifying the fixed weights. The local retrieval enhancement module is used to vectorize and store the user's exclusive knowledge base and perform local retrieval and context supplementation according to task requirements. The local closed-loop optimization module is used to optimize the task decomposition logic, correction strategy, and retrieval accuracy on the local terminal based on user feedback. The system supports both online and offline operation modes. In online mode, a cloud-based auxiliary module is configured to perform non-computational auxiliary operations.
[0016] Beneficial effects: (1) The present invention can achieve rapid adaptation of new tasks without modifying chip hardware, changing the fixed weights of the basic model, or re-fabricating the chip, thus significantly reducing the task adaptation cost and shortening the adaptation cycle.
[0017] (2) The entire reasoning process of this invention is completed on the local terminal, without relying on cloud computing power and network transmission, eliminating network transmission delay, and realizing low-latency response of reasoning. It is particularly suitable for scenarios with strict real-time requirements such as vehicle and industrial control. At the same time, user data does not leave the local area throughout the process, ensuring data privacy and security from the source, complying with the requirements of mainstream privacy protection regulations such as GDPR, and is suitable for high security, no network, and high real-time scenarios.
[0018] (3) This invention enables the solidified chip to adapt to diverse downstream tasks by atomizing the task and coordinating with multiple modules of the machine, thereby expanding the chip’s applicable scenarios and extending the chip’s life cycle.
[0019] (4) The present invention adopts output incremental correction, does not enter the hard-wired inference unit, does not modify the computation graph, is compatible with all hard-wired solidified inference chips, and has the widest hardware compatibility.
[0020] (5) This invention supports online and offline dual-mode operation, which can be flexibly switched according to the scenario, taking into account both stability and scalability, and is applicable to various types of local terminal devices such as consumer electronics, automotive, industrial, security, and government. Attached Figure Description
[0021] Figure 1 This is an overall flowchart of the method described in this invention; Figure 2 This is a module architecture diagram of the system described in this invention. Detailed Implementation
[0022] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0023] The model weight-fixed inference chip used in the embodiments of this invention is a dedicated inference chip with a fixed hardware structure and unmodifiable basic model weights. The chip includes at least an on-chip read-only memory unit, a hard-wired inference unit corresponding to the basic large model, and an on-chip read-write memory unit. The on-chip read-only memory unit is used to permanently store the basic large model weights, the hard-wired inference unit is used to perform fixed-logic model inference calculations, and the on-chip read-write memory unit is used to temporarily store intermediate results, lightweight incremental models, and externally fused data. The chip does not rely on external programmable logic units and can support the task adaptation method described in this invention without reconfiguration. The solution of this invention is implemented entirely on this fixed hardware architecture, without the need for external FPGA or GPU programmable units.
[0024] Example 1: Industrial Edge Computing Terminal Scenario This embodiment applies to an industrial edge computing terminal, which is equipped with a model weight-based inference chip. The basic model weights are stored in an on-chip read-only memory unit and cannot be modified by software. The terminal is deployed in a production workshop and operates in an offline, network-free environment for extended periods.
[0025] 1. After the system is powered on, it automatically scans the chip and generates a capability boundary map, which includes atomic reasoning capabilities such as text processing, instruction parsing, anomaly detection, and data classification.
[0026] 2. The operator inputs the equipment inspection task requirements, and the system breaks down the complex task into atomic inference operations that the chip can execute, and loads equipment maintenance rules as supplementary knowledge.
[0027] 3. The system scheduler chip hardwired inference unit executes core inference, while the lightweight incremental model (LoRA) completes industry terminology and format correction at the output end without entering the chip.
[0028] 4. The system performs compliance verification on the reasoning results and automatically outputs standardized equipment inspection conclusions.
[0029] 5. The terminal saves feedback data locally and completes task strategy iteration and optimization even when the network is disconnected.
[0030] This embodiment enables task self-adaptation of the embedded chip in an environment without network or human intervention, meeting the high stability and high security requirements of industrial scenarios.
[0031] Example 2: Offline Voice Interaction Scenario for In-Vehicle Smart Cockpit This embodiment is applied to an in-vehicle intelligent cockpit. The vehicle terminal is equipped with a low-power model weight fixed inference chip, which does not rely on cloud computing power and supports normal use of the vehicle in environments without network, such as tunnels and underground garages.
[0032] 1. After the system starts up, it scans the vehicle's embedded chips to generate a boundary map of capabilities such as voice understanding, command execution, scenario-based question answering, and vehicle control operation.
[0033] 2. When a user makes an interaction request via voice, the system breaks down the voice task into atomic inference operations and loads in-vehicle control rules and scene-specific knowledge.
[0034] 3. The chip hard-wired inference unit performs core semantic understanding, and the lightweight incremental model (LoRA) completes the adaptation and correction of tone, wording, and format at the output end.
[0035] 4. After the system completes the result verification, it outputs standardized response and control commands that conform to the vehicle scenario.
[0036] 5. The system performs closed-loop optimization locally on the vehicle's infotainment system based on user habits, improving subsequent recognition accuracy and response performance.
[0037] This embodiment enables the solidified chip to complete in-vehicle AI tasks under low power consumption, mobile, and offline conditions, without the need to replace the chip or re-fabricate.
[0038] Example 3: Security Monitoring Intelligent Analysis Terminal Scenario This embodiment is applied to a security monitoring intelligent analysis terminal. The device is equipped with a model weight fixed inference chip for high-security and high-compliance scenarios such as video text parsing, event judgment, compliance verification, and alarm generation.
[0039] 1. After the system starts up, it scans the embedded chip and generates atomic reasoning capabilities such as text extraction, content judgment, format normalization, and compliance filtering.
[0040] 2. The system acquires monitoring text analysis tasks, breaks down the tasks into atomic operations that the chip can execute, and loads security industry rules as supplementary knowledge.
[0041] 3. The chip executes core inference, and the lightweight incremental model (LoRA) completes industry terminology and compliance format correction at the output end without modifying the chip's fixed weights and internal calculation process.
[0042] 4. The system performs dual verification of the analysis results for consistency and compliance. After the verification is passed, standard alarms and event logs are output.
[0043] 5. The system continuously optimizes the task parsing logic based on local feedback data to improve recognition accuracy and adaptability.
[0044] This embodiment enables flexible task expansion of the embedded chip under confidential and high compliance requirements, meeting the offline, secure, and tamper-proof hardware needs of the security field.
[0045] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A native task-adaptive inference method for inference chips with fixed model weights, characterized in that, The model weight-fixed inference chip is mounted on a single local terminal device. The chip includes at least an on-chip read-only memory unit, a hard-wired inference unit corresponding to the basic large model, and an on-chip read-write memory unit. The weights of the basic large model are fixed in the on-chip read-only memory unit and cannot be modified by software. The entire inference process of the method is executed locally on the local terminal device; the cloud is only used for non-inference auxiliary operations and does not undertake any inference computation tasks. The method includes the following steps: S1. Scan the model weight-fixed inference chip in the local terminal device and generate the capability boundary map of the basic large model through a preset test case set; S2. Obtain the target task requirements input by the user, break down the target task into atomic inference operations that can be executed by the basic large model, as well as supplementary rules and proprietary knowledge that match the task. S3. Generate an inference execution pipeline based on the target task, schedule the hard-wired inference unit of the scheduling chip to perform atomic inference operations, and complete incremental correction and knowledge fusion at the output of the hard-wired inference unit to obtain the initial inference result. S4. Perform compliance and task consistency checks on the initial inference results. If the checks pass, output the final inference results. If the checks fail, perform secondary optimization inference.
2. The native task-adaptive inference method for model weight-fixed inference chips according to claim 1, characterized in that, The capability boundary map includes the atomic inference capability types, context window length, output format constraints, domain applicability, and inference latency metrics of the basic large model.
3. The native task-adaptive inference method for model weight-fixed inference chips according to claim 1, characterized in that, The supplementary rules are implemented through a scenario-based lightweight incremental model. The lightweight incremental model is deployed on the local terminal and performs correction and fusion at the output of the hardwired inference unit without entering the chip, changing the basic inference process, or modifying the fixed weights.
4. The native task-adaptive inference method for model weight-fixed inference chips according to claim 1, characterized in that, The proprietary knowledge is obtained through local retrieval enhancement. The user's proprietary knowledge base is pre-vectorized and stored on the local terminal device. During the reasoning process, local retrieval is performed according to task requirements and supplemented into the reasoning context.
5. The native task-adaptive inference method for model weight-fixed inference chips according to claim 1, characterized in that, The method supports both online and offline modes. The core inference steps are all completed locally on the terminal device and do not rely on remote computing power in the cloud. In offline mode, all steps are executed in a network-disconnected environment, while in online mode, only non-computing power auxiliary operations are performed. In online mode, the following auxiliary operations can be performed with user authorization: synchronizing scene optimization strategies, updating incremental correction plugins, updating the local knowledge base, software version upgrades, and authorization management. The above operations do not interfere with the core inference process of the chip.
6. The native task-adaptive inference method for model weight-fixed inference chips according to claim 1, characterized in that, Step S4 is followed by a closed-loop optimization step S5: based on user feedback data on the inference results, the task decomposition strategy, incremental correction rules, and retrieval accuracy are iteratively optimized locally on the local terminal device.
7. The native task-adaptive inference method for model weight-fixed inference chips according to claim 1, characterized in that, The local multi-module collaborative inference supports multi-task parallel scheduling, and can simultaneously schedule the chip to execute atomic inference operations for multiple target tasks.
8. The native task-adaptive inference method for model weight-fixed inference chips according to claim 1, characterized in that, The local terminal devices include desktop computers, portable smart terminals, all-in-one office computers, vehicle-mounted smart terminals, industrial stand-alone equipment, security smart terminals, and edge computing terminals.
9. A native task-adaptive inference system for model weight-fixed inference chips, characterized in that, The system is deployed on a local terminal device equipped with a model weight-fixed inference chip. The entire inference process is completed locally, without relying on cloud computing power. The chip includes at least an on-chip read-only memory unit, a hard-wired inference unit corresponding to the basic large model, and an on-chip read-write memory unit. The basic model weights are fixed in the on-chip read-only memory unit and cannot be modified by software. The system includes: Chip adaptation module, used to scan chips and generate capability boundary maps of basic large models; The task parsing module is used to obtain the target task requirements, break down the task into atomic reasoning operations, supplementary rules and proprietary knowledge, and generate a reasoning execution pipeline. The local inference scheduling module is used to schedule hard-wired inference units to perform atomic inference operations and to complete incremental correction and knowledge fusion at the output end. The result verification module is used to verify the compliance and task consistency of the initial inference results. If the verification passes, the final result is output; if the verification fails, a second optimization inference is triggered. The visual interaction module provides a user interface that supports task input, rule configuration, result display, and result feedback.
10. The native task-adaptive inference system for model weight-fixed inference chips according to claim 9, characterized in that, It also includes a lightweight incremental correction module, a local retrieval enhancement module, and a local closed-loop optimization module; the lightweight incremental correction module is used to deploy a scenario-based lightweight incremental model, and performs correction fusion at the output of the hard-wired inference unit without entering the chip or modifying the fixed weights; The local retrieval enhancement module is used to vectorize and store the user's exclusive knowledge base, and to perform local retrieval and context supplementation according to task requirements; the local closed-loop optimization module is used to optimize the task decomposition logic, correction strategy and retrieval accuracy on the local terminal based on user feedback; The system supports both online and offline operation modes. In online mode, a cloud-based auxiliary module is configured to perform non-computational auxiliary operations.