A method for deploying and reasoning of an end-side large model for a WiFi7-star flash intelligent terminal

Abstract

The application discloses a WiFi7-star flash intelligent terminal-oriented end-side large model deployment and reasoning method. The method comprises the following steps: collecting terminal running data and scene configuration data, and constructing a device capability knowledge base and a scene sample set; using a teacher large model for structured annotation, and obtaining a student model through multi-task distillation training; performing mixed quantization processing on the student model according to terminal resource constraints, and dividing the student model into multiple independently loadable model segments; after receiving a user instruction, searching the knowledge base to assemble a local context, loading corresponding model segments to perform local reasoning to generate a scene task graph; dividing the scene task graph into a first subtask and a second subtask, and executing the first subtask and the second subtask through a WiFi7 link and a star flash link respectively; updating scene template confidence according to an execution result, and updating a model version. The application reduces cloud dependence and data out-domain risk, and improves deployment stability and reasoning efficiency on a resource-constrained terminal.

Description

Technical Field

[0001] This application relates to the field of smart terminal control, and more specifically, to a method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals. Background Technology

[0002] With the development of smart homes, smart offices, smart manufacturing, and smart healthcare, smart terminals not only need to provide high-speed WiFi 7 connectivity and low-power Wi-Fi access, but also need to have local understanding capabilities for natural language, multimodal commands, and complex scene interactions. Large models can improve semantic understanding and task planning performance, but general-purpose large models typically have large parameter scales, high memory usage, and long inference times, making them difficult to deploy directly in resource-constrained devices such as home gateways, conference terminals, and campus edge nodes.

[0003] Existing solutions typically employ large cloud-based models for semantic understanding, then return the control results to the terminal for execution. While this approach yields strong semantic capabilities under favorable network conditions, it suffers from slow response times, data loss, service unavailability, and high costs in scenarios with weak networks, network outages, privacy concerns, or high real-time control requirements. For scenarios such as door locks, security, elderly care, conference confidentiality, and industrial sites, relying solely on large cloud-based models is not conducive to long-term stable operation.

[0004] Other solutions employ simple rule templates or small models for local execution. However, rule templates struggle to capture the diversity of natural user expressions, while small models often underperform in complex contexts, multi-device interactions, and abnormal scenario interpretation. Simply compressing large models without considering device capabilities, knowledge bases, scene templates, and terminal resource status can easily lead to issues such as decreased accuracy after compression, unreasonable model segment loading, excessively high memory spikes, and inference results that cannot be executed.

[0005] Therefore, a method for deploying and inferring large models on the edge for WiFi7-StarFlash smart terminals is needed to enable large model capabilities to run on local terminals with low resource consumption and form a closed loop with device access, scene control, privacy protection and operation feedback in the project. Summary of the Invention

[0006] The purpose of this application is to provide a method for deploying and inferring large models on the edge for WiFi7-StarFlash smart terminals, enabling smart terminals to complete local semantic understanding, task graph generation, device control parameter generation and security verification under resource-constrained conditions, thereby reducing dependence on large models in the cloud.

[0007] This invention provides a method for deploying and inferring a large-scale model on the edge of a WiFi7-StarFlash smart terminal, including: Collect operational data and scenario configuration data from smart terminals to build a device capability knowledge base and scenario sample set; The teacher's large model is used to perform structured annotation on the scene sample set to generate a distillation training dataset. The student model is then trained using a multi-task distillation method based on the distillation training dataset. Based on the resource constraints of the smart terminal, the trained student model is subjected to hybrid quantization processing, and the processed student model is divided into multiple independently loadable model segments according to function. After receiving a user input command, the system retrieves the local context from the device capability knowledge base, loads the corresponding model segment according to the scenario type of the current user input command, performs local reasoning based on the local context, and generates a scenario task graph. The scenario task graph is divided into a first subtask and a second subtask. The first subtask is sent and executed via a WiFi 7 link, and the second subtask is sent and executed via a StarFlash link. Based on the execution results and manual correction records, update the confidence level of the scenario template and manage the version of the student model.

[0008] In one optional implementation, the device capability knowledge base includes device type, communication method, executable actions, status field, installation location, user permission level, and security sensitivity level; the scenario sample set includes device capability description, user scenario instructions, historical linkage records, and wireless access status.

[0009] In one alternative implementation, the multi-task distillation training updates the parameters of the student model by minimizing the multi-task loss function; The multi-task loss function includes an intent classification loss term, a slot extraction loss term, a task graph structure loss term, and an output distribution divergence loss term, which are then summed according to preset weights.

[0010] In one optional implementation, hybrid quantization processing is performed on the trained student model according to the resource constraints of the smart terminal, specifically as follows: The sensitivity of each layer of the student model is calculated using the following expression:

[0011] In the formula, For the initial student model The sensitivity of the layer For unquantified model accuracy, To only the first The accuracy of the model after layer quantization For the first The storage amount of the layer is not quantified. For the first Storage size after layer quantization To prevent positive numbers with a denominator of zero; Under the constraint that the model accuracy loss, inference latency and runtime memory usage do not exceed their respective preset upper limits, a first quantization bit width is allocated to layers with sensitivity greater than or equal to a preset threshold, and a second quantization bit width is allocated to layers with sensitivity lower than a preset threshold, wherein the first quantization bit width is greater than the second quantization bit width. Quantization is performed on each layer of the student model according to the allocated quantization bit width.

[0012] In one optional implementation, the model segment includes a basic semantic segment, a device knowledge alignment segment, a task graph generation segment, a control parameter generation segment, and a security verification segment.

[0013] In one optional implementation, after receiving a user input command, the system retrieves the device capability knowledge base to assemble a local context, loads the corresponding model segment according to the scenario type corresponding to the current user input command, and further includes: Based on users' historical usage habits, the frequency of scene triggers, and the online status of devices, high-frequency model segments and high-frequency knowledge fragments are preloaded.

[0014] In one optional implementation, if the local inference fails to meet preset requirements or local knowledge is lacking, cloud-side knowledge supplementation is performed, specifically as follows: The intermediate features, including user input, device status, and historical records, are de-identified and compressed. The de-identification and compression ratio is the ratio of the length of the de-identified data to the length of the original data. When the cost of cloud-side collaboration is less than the cost of local execution and the cost of privacy leakage is lower than a preset threshold, the de-identified and compressed intermediate features or structured slots are uploaded to the cloud.

[0015] In one alternative implementation, the first subtask has a larger data volume than the second subtask, and the second subtask has a higher latency than the first subtask.

[0016] In one optional implementation, the scene template confidence level is updated based on the execution results and manual correction records, specifically as follows: The confidence level of the scenario template is updated based on the execution success rate and the number of manual corrections, expressed as follows:

[0017] In the formula, For the first The confidence level of the scene template after the latest update. For learning rate, For the first Confidence level of the scene template before the last update. For the first The success metric for each execution is set to 1 if successful and 0 if it fails. The penalty coefficient is... For the first The number of manual corrections performed per execution.

[0018] In one alternative implementation, version management of the student model is performed, specifically as follows: Calculate the degree of data drift of the smart terminal running sample relative to the benchmark sample. If the degree of data drift exceeds the preset drift threshold, perform incremental updates on the student model to obtain the candidate model version. The upgrade validation score for the candidate model version is calculated using the following expression:

[0019] In the formula, The score is used to validate the upgrade of the candidate model version. , , and These are the weighting coefficients for improved accuracy, reduced latency, reduced privacy risks, and risks associated with abnormal events, respectively. This represents the accuracy improvement of the candidate model version. Reduce latency for candidate model versions To reduce the amount of privacy risks associated with candidate model versions The abnormal event risk score for candidate model versions; If the upgrade verification score is greater than or equal to the preset verification score threshold, the student model will be updated to the candidate model version. If the upgrade verification score is lower than the preset verification score threshold, the student model version will be rolled back to the previous version.

[0020] This application has at least the following advantages or beneficial effects: This application reduces cloud reliance and improves availability and privacy protection in weak network conditions. By completing semantic understanding, task graph generation, control parameter generation, and security verification locally on the terminal, common scenarios and key control processes can still be executed in weak or offline environments. Furthermore, when supplementing from the cloud, the proportion of raw data leaving the domain is effectively reduced, protecting user privacy and system security. This application improves the deployment stability and inference efficiency of resource-constrained terminals. By using model layer sensitivity assessment and hybrid quantization processing, differentiated quantization bit widths are allocated to layers with different sensitivities under the comprehensive constraints of accuracy, latency, and memory. Combined with a hot-start caching mechanism and dynamic loading of model segments, a computable trade-off between accuracy, latency, and memory is achieved, reducing memory spikes and invalid loading, and ensuring that the model runs efficiently and stably on resource-constrained terminals. This application enhances the understanding of complex scenarios and supports long-term adaptive evolution of the model. Through multi-task distillation training, the student model learns multi-dimensional structured information simultaneously, directly generating executable task graphs, applicable to various complex scenarios such as home, office, industry, healthcare, and gateway self-diagnosis. Combined with a dynamic update and rollback mechanism for scenario template confidence, the edge model can continuously evolve with changes in user habits and scenario expansion, and automatically roll back to the previous stable version in case of upgrade anomalies. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for deploying and inferring a large-scale model on the edge of a WiFi7-StarFlash smart terminal, provided by the present invention. Figure 2 This is a flowchart of the local deployment and inference of the large model on the edge side in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In this article, "SparkLink," also known as "StarLink," is a next-generation short-range wireless communication technology. This technology combines the advantages of traditional Bluetooth Low Energy and Wi-Fi High Speed, featuring low latency, high reliability, high synchronization accuracy, support for massive concurrent device connections, and strong anti-interference capabilities. SparkLink technology includes two modes: SLB (Basic Access Technology, comparable to Wi-Fi, enabling high-speed, high-quality connections) and SLE (Small Energy Access Technology, comparable to Bluetooth, enabling low-power, lightweight connections).

[0025] In this paper, the "teacher model" refers to a large-scale pre-trained model in knowledge distillation technology, and the "student model" refers to a lightweight model that receives knowledge transfer from the teacher model during the knowledge distillation process. In this application, the student model learns the structured reasoning results of the teacher model through multi-task distillation training, and is finally deployed on the NPU (Neural Processing Unit) of the WiFi7-StarFlash smart terminal to perform local reasoning, enabling it to independently generate scene task graphs under weak network or network outage conditions. During long-term operation, the student model can also continuously evolve with changes in user habits and scenario expansion through incremental updates such as low-rank adaptation (LoRA), without the need for overall retraining.

[0026] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for deploying and inferring a large-scale edge model for WiFi7-StarFlash smart terminals, as proposed in one embodiment of this application. Figure 1 As shown, the method includes the following steps: Step S1: Collect the operation data and scene configuration data of the smart terminal to build a device capability knowledge base and scene sample set; Specifically, the device capability knowledge base includes device type, communication method, executable actions, status fields, installation location, user permission level, and security sensitivity level; the scenario sample set includes device capability descriptions, user scenario instructions, historical linkage records, and wireless access status.

[0027] In this embodiment, the smart terminal can be a home smart host, a carrier home gateway, a conference control terminal, a park edge node, or an industrial gateway, etc., and internally includes a main control processor, an NPU (neural network processing unit), a memory, a WiFi 7 communication module, a StarFlash communication module, and a local security unit.

[0028] During the data acquisition phase, operational data and scenario configuration data, such as device capability descriptions, user voice or text commands, sensor events, device status, scenario execution results, and manual correction logs, are collected from scenarios such as smart homes, smart offices, smart manufacturing, and smart healthcare.

[0029] The aforementioned data is not directly input as random text. Instead, it is first organized into a device capability table, a scene trigger table, a task dependency table, and a control parameter table to build a device capability knowledge base. Simultaneously, the collected scene-related data is organized into a scene sample set.

[0030] In one specific embodiment, the air conditioning device can record the target temperature, operating mode, air volume, on / off status, and room location; the door lock device can record the lock status, power level, security level, and allowed control roles.

[0031] It should be noted that training with data using the above structure allows the task graph output by the student model to be directly mapped into executable instructions.

[0032] Step S2: Call the teacher's large model to perform structured annotation on the scene sample set, generate a distillation training dataset, and perform multi-task distillation training on the student model based on the distillation training dataset; In this embodiment, the teacher model receives a scene sample set and device capability knowledge, and outputs standardized structured annotation results such as intent, slots, and task graphs.

[0033] For example, when a user sends the message "I'm home, make the living room more comfortable," the teacher model can label it as a home arrival scenario and break it down into sub-tasks such as adjusting the air conditioner, setting the lights, and checking the door lock status.

[0034] During the distillation training phase, the student model updates its parameters by minimizing the multi-task loss function.

[0035] The multi-task loss function includes an intent classification loss term, a slot extraction loss term, a task graph structure loss term, and an output distribution divergence loss term, which are then summed according to preset weights.

[0036] Unlike distilling only the final text response, this application's distillation is a structured intermediate result executed on a smart terminal. The student model simultaneously learns the teacher model's classification results, slot boundaries, task graph edge set, and control parameters, enabling it to generate an executable task graph locally, rather than simply generating a natural language explanation.

[0037] In some embodiments, the compression ratio between the number of parameters in the teacher model and the number of parameters in the student model is not lower than a preset lower limit of compression ratio to ensure the feasibility of local deployment; at the same time, the loss of scene understanding accuracy of the student model relative to the teacher model does not exceed a preset upper limit of accuracy loss to ensure scene understanding capability.

[0038] Step S3: Perform hybrid quantization processing on the trained student model according to the resource constraints of the smart terminal, and divide the processed student model into multiple independently loadable model segments according to their functions; In this embodiment, the hybrid quantization process specifically includes: First, calculate the sensitivity of each layer of the student model, the third... The sensitivity expression for a layer is:

[0039] In the formula, For the initial student model The sensitivity of the layer For unquantified model accuracy, To only the first The accuracy of the model after layer quantization For the first The storage amount of the layer is not quantified. For the first Storage size after layer quantization To prevent positive numbers with a denominator of zero; When the sensitivity is high, it means that the model layer will cause a large loss of accuracy while saving storage. It is preferable to retain a higher bit width. When the sensitivity is low, it means that the model layer is suitable for quantization with a lower bit width.

[0040] Then, under the constraint that the model accuracy loss, inference latency, and memory usage do not exceed their respective preset upper limits, a first quantization bit width is allocated to layers with sensitivity greater than or equal to a preset threshold, and a second quantization bit width is allocated to layers with sensitivity lower than a preset threshold, wherein the first quantization bit width is greater than the second quantization bit width. This allows for a calculable trade-off between model accuracy, inference latency, and memory usage, making the student model more suitable for deployment on intelligent terminals with NPUs but limited resources.

[0041] Finally, quantization is performed on each layer of the student model according to the allocated quantization bit width.

[0042] In one embodiment, the system evaluates the quantization sensitivity of the embedding layer, attention layer, feedforward layer, task graph generation layer, and control parameter output layer, respectively. For layers that have a significant impact on semantic understanding and task graph edge prediction, the system retains a higher bit width; for layers with high redundancy or a small impact on the control results, the system uses a lower bit width.

[0043] Regarding model segment loading, the model is divided into a basic semantic segment, a device knowledge alignment segment, a task graph generation segment, a control parameter generation segment, and a safety verification segment. Each model segment is stored in the terminal memory as an independent unit, supporting dynamic loading and unloading on demand. For simple switch-type instructions, only the basic semantic segment and the control parameter generation segment are loaded; for multi-device linkage instructions, the task graph generation segment is loaded; for high-risk instructions involving door locks, gas valves, power control, or industrial emergency stops, the safety verification segment is further loaded.

[0044] Step S4: After receiving the user input command, retrieve the local context assembled from the device capability knowledge base, load the corresponding model segment according to the scene type corresponding to the current user input command, perform local reasoning based on the local context, and generate a scene task graph. like Figure 2 The diagram shows a flowchart of the local deployment and inference of the large-scale model on the edge. In this embodiment, after receiving a user input command, the system first retrieves knowledge fragments related to the current command from the device capability knowledge base, including device capability information, historical scenario template information, user preference information, and security rule information.

[0045] The retrieved knowledge fragments are combined with user input commands to form a local context. The system dynamically determines the filling ratio of device knowledge, historical dialogues, user preferences, and security policies based on the available length of the context window, avoiding truncation or loss of key constraints due to excessively long contexts.

[0046] Furthermore, the system loads the corresponding model segment combination according to the scene type corresponding to the current user input command, performs local inference on the student model after quantizing the local context input, and generates a scene task graph.

[0047] To further improve the stability of edge operation, this application introduces a hot-start caching mechanism in addition to model segment loading. The system preloads high-frequency model segments and high-frequency knowledge fragments based on user habit time periods, scenario trigger frequency, device online status, and historical execution templates, so that common scenarios already have some inference context before user commands arrive.

[0048] Furthermore, the hot-start cache hit rate is defined as the ratio of the number of times a loaded model segment or retrieved knowledge fragment is hit within a preset time window to the number of requests for the model segment or knowledge fragment. When the cache hit rate remains below a preset threshold, it indicates that the preloaded set does not match the actual usage scenario, and the system will reselect the hot-start model segment based on the scenario frequency and execution benefits of the most recent time window.

[0049] The total local inference latency includes model segment loading latency, knowledge retrieval latency, NPU computation latency, CPU scheduling latency, and decoding output latency. The system adjusts the model segment loading strategy and / or the length of the local context based on the terminal's current resource status to meet preset latency constraints. The terminal resource status score is determined based on available NPU computing power, idle memory, storage read / write bandwidth, current operating power consumption, and chip temperature. When the resource status score is low, the system reduces the loading of non-critical model segments, shortens the historical context, or enables a lighter control template; when the resource status score is high, the system can load a more complete inference chain and more scene knowledge.

[0050] If local inference cannot meet the preset latency requirements or local knowledge is lacking, the system performs cloud-side knowledge supplementation. Specifically, this involves calculating the local execution cost and the cloud-side collaboration cost, and assessing the privacy leakage cost. Only when the cloud-side collaboration cost is less than the local execution cost and the privacy leakage cost is below a preset threshold, user input, device status, and historical records are anonymized and compressed. The anonymized and compressed intermediate features or structured slots are then uploaded to the cloud, and the enhanced results returned by the cloud are received to assist local inference.

[0051] The desensitization compression ratio is defined as the ratio of the length of the desensitized data to the length of the original data. If the desensitization compression ratio is lower than the preset upper limit, the cloud side will only receive the necessary summary features, search keywords or intermediate semantic vectors, and will not receive the complete original voice, images, device logs and user behavior records.

[0052] Step S5: Divide the scene task graph into a first subtask and a second subtask. Send the first subtask to be executed via the WiFi7 link and the second subtask to be executed via the StarFlash link. Specifically, after the scene task graph is generated, the system divides the scene task graph into a first subtask and a second subtask based on the data volume and latency sensitivity of each subtask. The first subtask has a larger data volume than the second subtask, and the second subtask has a higher latency sensitivity than the first subtask.

[0053] For example, in a home implementation, the user issues a voice command to "turn on Home Mode". The system retrieves the user's historical Home Mode settings, which typically include turning on the living room lights, setting the air conditioner to 26 degrees Celsius, partially opening the curtains, and announcing the door lock status. After generating the task graph locally, the student model divides the lightweight control tasks (light control, air conditioner adjustment, and door lock announcement) into a second subtask and assigns it to the StarFlash link for execution, while dividing the camera image synchronization and home service entry refresh into a first subtask and assigning it to the WiFi7 link.

[0054] For example, in one industrial implementation, a production line operator issues a "switch to night shift inspection mode" command. The system retrieves the night shift inspection template, sensor groups, AGV routes, and a list of high-risk devices, and generates an inspection task map. High-throughput tasks such as log aggregation, image transmission, and model status synchronization are classified as the first sub-task, and the system utilizes the WiFi 7 link. Critical tasks such as alarm confirmation, equipment start / stop, and safety interlocks are classified as the second sub-task, and the system prioritizes using the StarFlash control link and enables closed-loop confirmation.

[0055] Step S6: Based on the execution results and manual correction records, update the confidence level of the scene template and manage the version of the student model.

[0056] Specifically, the system records user input, retrieved knowledge, task graph, control results, failure reasons, and manual corrections for each task execution.

[0057] Furthermore, the confidence level of the scenario template is updated based on the execution success rate and the number of manual corrections, expressed as:

[0058] In the formula, For the first The confidence level of the scene template after the latest update. For learning rate, For the first Confidence level of the scene template before the last update. For the first The success metric for each execution is set to 1 if successful and 0 if it fails. The penalty coefficient is... For the first The number of manual corrections performed per execution.

[0059] When the scenario template executes successfully continuously without manual correction, the confidence level gradually increases; when the template causes erroneous execution or requires frequent correction, the confidence level decreases and the application enters a pending review state.

[0060] For example, if a user repeatedly changes "more comfortable" to "air conditioning at 26 degrees Celsius and medium fan speed", the system will increase the confidence of the user's preference template and prioritize calling the template in subsequent similar scenarios.

[0061] In terms of model version management, the degree of data drift of the smart terminal running sample relative to the benchmark sample is calculated. If the degree of data drift exceeds the preset drift threshold, the student model is incrementally updated to obtain the candidate model version. For example, the formula for calculating the degree of data drift is:

[0062] In the formula, For the degree of data drift, The weighting coefficients for the distribution distance term are... For the current data distribution Distribution with benchmark data distance, The weighting coefficient for the error rate term. For the scenario execution error rate, The weight coefficients for the number of unknown samples. This represents the number of samples that the model cannot recognize. The upgrade validation score for the candidate model version is calculated using the following expression:

[0063] In the formula, The score is used to validate the upgrade of the candidate model version. , , and These are the weighting coefficients for improved accuracy, reduced latency, reduced privacy risks, and risks associated with abnormal events, respectively. This represents the accuracy improvement of the candidate model version. Reduce latency for candidate model versions To reduce the amount of privacy risks associated with candidate model versions The abnormal event risk score for candidate model versions; When the upgrade verification score is greater than or equal to the preset verification score threshold, the student model is updated to the candidate model version, and the system gradually expands the scope of application of the new model; when the upgrade verification score is lower than the preset verification score threshold, the system reverts to the previous stable model version to avoid business interruption caused by a one-time replacement of the new model.

[0064] In summary, this application establishes a complete technical chain for the local deployability of large models in WiFi7-Starflash smart terminals, including scene sample construction, teacher model annotation, multi-task distillation, hybrid quantization, model segmented loading, knowledge-enhanced inference, de-identification protection, and feedback updates. This chain can support local intelligent services in various scenarios such as smart homes, smart offices, smart manufacturing, smart healthcare, and carrier gateways.

[0065] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0066] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0067] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0068] The above provides a detailed description of the edge-side large model deployment and inference method for WiFi7-StarFlash smart terminals provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals, characterized in that, include: Collect operational data and scenario configuration data from smart terminals to build a device capability knowledge base and scenario sample set; The teacher's large model is used to perform structured annotation on the scene sample set to generate a distillation training dataset. The student model is then trained using a multi-task distillation method based on the distillation training dataset. Based on the resource constraints of the smart terminal, the trained student model is subjected to hybrid quantization processing, and the processed student model is divided into multiple independently loadable model segments according to their functions. After receiving a user input command, the system retrieves the local context from the device capability knowledge base, loads the corresponding model segment according to the scenario type of the current user input command, performs local reasoning based on the local context, and generates a scenario task graph. The scenario task graph is divided into a first subtask and a second subtask. The first subtask is sent and executed via a WiFi 7 link, and the second subtask is sent and executed via a StarFlash link. Based on the execution results and manual correction records, update the confidence level of the scenario template and manage the student model for version control.

2. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, The device capability knowledge base includes device type, communication method, executable actions, status fields, installation location, user permission level, and security sensitivity level; the scenario sample set includes device capability description, user scenario instructions, historical linkage records, and wireless access status.

3. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, The multi-task distillation training updates the parameters of the student model by minimizing the multi-task loss function; The multi-task loss function includes an intent classification loss term, a slot extraction loss term, a task graph structure loss term, and an output distribution divergence loss term, which are then summed according to preset weights.

4. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, Based on the resource constraints of the smart terminal, the trained student model undergoes hybrid quantization processing, specifically as follows: The sensitivity of each layer of the student model is calculated using the following expression: In the formula, For the initial student model The sensitivity of the layer For unquantified model accuracy, To only the first The accuracy of the model after layer quantization For the first The storage amount of the layer is not quantified. For the first Storage size after layer quantization To prevent positive numbers with a denominator of zero; Under the constraint that the model accuracy loss, inference latency and runtime memory usage do not exceed their respective preset upper limits, a first quantization bit width is allocated to layers with sensitivity greater than or equal to a preset threshold, and a second quantization bit width is allocated to layers with sensitivity lower than a preset threshold, wherein the first quantization bit width is greater than the second quantization bit width. Quantization is performed on each layer of the student model according to the allocated quantization bit width.

5. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, The model segment includes a basic semantic segment, a device knowledge alignment segment, a task graph generation segment, a control parameter generation segment, and a security verification segment.

6. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, After receiving a user input command, the system retrieves the device capability knowledge base to assemble a local context, loads the corresponding model segment based on the scenario type corresponding to the current user input command, and further includes: Based on users' historical usage habits, the frequency of scene triggers, and the online status of devices, high-frequency model segments and high-frequency knowledge fragments are preloaded.

7. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, If the local inference fails to meet the preset requirements or local knowledge is lacking, cloud-based knowledge supplementation is performed, specifically as follows: The intermediate features, including user input, device status, and historical records, are de-identified and compressed. The de-identification and compression ratio is the ratio of the length of the de-identified data to the length of the original data. When the cost of cloud-side collaboration is less than the cost of local execution and the cost of privacy leakage is lower than a preset threshold, the de-identified and compressed intermediate features or structured slots are uploaded to the cloud.

8. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, The first subtask has a larger data volume than the second subtask, and the second subtask has a higher latency than the first subtask.

9. The method for deploying and inferring a large-scale model on the edge for WiFi7-StarFlash smart terminals according to claim 1, characterized in that, Based on the execution results and manual correction records, update the confidence level of the scenario template, specifically as follows: The confidence level of the scenario template is updated based on the execution success rate and the number of manual corrections, expressed as follows: In the formula, For the first The confidence level of the scene template after the latest update. For learning rate, For the first Confidence level of the scene template before the last update. For the first The success metric for each execution is set to 1 if successful and 0 if it fails. The penalty coefficient is... For the first The number of manual corrections performed per execution.

10. The method for deploying and inferring a large-scale model on the edge of a WiFi7-StarFlash smart terminal according to claim 1, characterized in that, Version management of student models is implemented as follows: Calculate the degree of data drift of the smart terminal running sample relative to the benchmark sample. If the degree of data drift exceeds the preset drift threshold, perform incremental updates on the student model to obtain the candidate model version. The upgrade validation score for the candidate model version is calculated using the following expression: In the formula, The score is used to validate the upgrade of the candidate model version. , , and These are the weighting coefficients for improved accuracy, reduced latency, reduced privacy risks, and risks of abnormal events, respectively. This represents the accuracy improvement of the candidate model version. Reduce latency for candidate model versions To reduce the amount of privacy risks associated with candidate model versions The abnormal event risk score for the candidate model version; If the upgrade verification score is greater than or equal to the preset verification score threshold, the student model will be updated to the candidate model version. If the upgrade verification score is lower than the preset verification score threshold, the student model version will be rolled back to the previous version.

Images