Model stream decoding and rectification method and device

Abstract

This application relates to the field of artificial intelligence technology, specifically to a model streaming decoding correction method and apparatus. The method includes: acquiring a first word segment output by the endpoint model, wherein the first word segment is a local word segment sequence output by the endpoint model during streaming decoding, and has not yet formed a complete device action instruction; determining device control information matching the first word segment, and the current state information of the target device associated with the device control information; correcting the initial probability distribution vector of the endpoint model before sampling in the current decoding step, provided that the current state information and the device control information are mutually exclusive; and sampling in the current decoding step based on the corrected probability distribution vector. The model streaming decoding correction method provided in this application can block the generation path of the illusion instruction before it is fully formed, significantly reducing computational resource consumption while eliminating the inherent high latency and interaction fragmentation defects of traditional post-interception mechanisms.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model streaming decoding correction method and apparatus. Background Technology

[0002] With the development of artificial intelligence technology, models are often deployed in resource-constrained home smart gateways or edge computing nodes to achieve natural language-based control of IoT devices. However, limited by edge computing power, small language models deployed in such scenarios are prone to illusions when lacking sufficient factual basis, easily outputting erroneous control commands that contradict the current physical state of the device. Current defense solutions, such as post-event text regularization validation, retrieval-enhanced injection, and dual-model cross-validation, generally have significant flaws. Post-event interception results in irreversible waste of computing power and severe interaction delays, attention drift of small models to prompt words causes anti-illusion mechanisms to fail, and dual-model solutions directly violate the premise of limited edge resources. Therefore, how to achieve timely and accurate correction of model illusions without increasing additional inference computing power and high latency waiting, and ensure that the model's smooth output conforms to the correct response of physical reality, is an urgent problem to be solved. Summary of the Invention

[0003] This application provides a model streaming decoding correction method, apparatus, electronic device, storage medium, and program product to solve the technical problem in the prior art that the end-side model is prone to producing illusions when there is a lack of sufficient factual basis, and is prone to outputting erroneous control commands that contradict the current physical state of the device.

[0004] This application provides a model streaming decoding correction method, including the following steps: Obtain the first word element output by the terminal model, wherein the first word element is a local word element sequence output by the terminal model during the streaming decoding process and has not yet formed a complete device action instruction; Determine the device control information that matches the first word, and the current status information of the target device associated with the device control information; If it is determined that the current state information and the device control information are mutually exclusive, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected. Sampling is performed for the current decoding step based on the corrected probability distribution vector.

[0005] According to the model streaming decoding correction method provided in this application, the initial probability distribution vector includes multiple candidate words and an initial output probability corresponding to each candidate word; Wherein, correcting the initial probability distribution vector of the end-side model before sampling in the current decoding step, when it is determined that the current state information and the device control information are mutually exclusive, includes: From the candidate lexical units, determine a second lexical unit that is mutually exclusive with the current state information and a third lexical unit that is compatible with the current state information condition; Decrease the initial output probability corresponding to the second word, and / or increase the initial output probability corresponding to the third word.

[0006] According to the model streaming decoding correction method provided in this application, determining the device control information matching the first lexical unit includes: The first word element is matched with a preset set of control information to obtain a matching result; If the matching result indicates that there is control information in the preset control information set that matches the first word, the matching control information is determined as the device control information that matches the first word.

[0007] According to the model streaming decoding correction method provided in this application, after performing matching processing on the first word and a preset control information set to obtain a matching result, the method further includes: If the matching result indicates that there is no control information in the preset control information set that matches the first word, sampling is performed for the current decoding step based on the initial probability distribution vector.

[0008] According to the model streaming decoding correction method provided in this application, the step of correcting the initial probability distribution vector of the end-side model before sampling in the current decoding step, when it is determined that the current state information and the device control information are mutually exclusive, includes: If it is determined that the current status information and the device control information are mutually exclusive, it is determined whether the device control information belongs to a preset risk control action, so as to obtain a judgment result; If the judgment result indicates that the device control information does not belong to the preset risk control action, the initial probability distribution vector of the end-side model before the current decoding step sampling is corrected.

[0009] According to the model streaming decoding correction method provided in this application, after determining whether the device control information belongs to a preset risk control action to obtain a determination result, the method further includes: If the judgment result indicates that the device control information belongs to the preset risk control action, the sampling of the current decoding step shall be stopped.

[0010] According to the model streaming decoding correction method provided in this application, determining the mutual exclusion of conditions between the current state information and the device control information includes: Determine the expected state information corresponding to the device control information; If the expected state information is consistent with the current state information, it is determined that the current state information and the device control information are mutually exclusive.

[0011] This application provides a model streaming decoding correction device, including the following modules: The acquisition module is used to acquire the first word element output by the terminal model, wherein the first word element is a local word element sequence output by the terminal model during the streaming decoding process and has not yet formed a complete device action instruction; The determination module is used to determine the device control information that matches the first word element, and the current status information of the target device associated with the device control information; The correction module is used to correct the initial probability distribution vector of the end-side model before sampling in the current decoding step when it is determined that the current state information and the device control information are mutually exclusive. The first sampling module is used to sample the current decoding step based on the corrected probability distribution vector.

[0012] According to the model streaming decoding correction device provided in this application, the initial probability distribution vector includes multiple candidate words and an initial output probability corresponding to each candidate word; The correction module is specifically used for: From the candidate lexical units, determine a second lexical unit that is mutually exclusive with the current state information and a third lexical unit that is compatible with the current state information condition; Decrease the initial output probability corresponding to the second word, and / or increase the initial output probability corresponding to the third word.

[0013] According to the model streaming decoding correction device provided in this application, the determining module is specifically used for: The first word element is matched with a preset set of control information to obtain a matching result; If the matching result indicates that there is control information in the preset control information set that matches the first word, the matching control information is determined as the device control information that matches the first word.

[0014] According to the model streaming decoding correction device provided in this application, it further includes: The second sampling module is used to sample the current decoding step according to the initial probability distribution vector after the first word is matched with the preset control information set to obtain a matching result, when the matching result indicates that there is no control information in the preset control information set that matches the first word.

[0015] According to the model streaming decoding correction device provided in this application, the correction module is specifically used for: If it is determined that the current status information and the device control information are mutually exclusive, it is determined whether the device control information belongs to a preset risk control action, so as to obtain a judgment result; If the judgment result indicates that the device control information does not belong to the preset risk control action, the initial probability distribution vector of the end-side model before the current decoding step sampling is corrected.

[0016] According to the model streaming decoding correction device provided in this application, it further includes: The third sampling module is used to stop sampling in the current decoding step after determining whether the device control information belongs to the preset risk control action and obtaining the determination result, if the determination result indicates that the device control information belongs to the preset risk control action.

[0017] According to the model streaming decoding correction device provided in this application, the correction module is specifically used for: Determine the expected state information corresponding to the device control information; If the expected state information is consistent with the current state information, it is determined that the current state information and the device control information are mutually exclusive.

[0018] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the model streaming decoding correction method as described above.

[0019] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model streaming decoding correction method as described above.

[0020] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the model streaming decoding correction method as described above.

[0021] This application provides a model streaming decoding correction method, device, electronic device, storage medium, and program product. By moving the timing of verifying the physical state of the Internet of Things from the post-processing complete text generation stage to the micro-time window of each first word generation during the streaming decoding process, and directly correcting the initial probability distribution vector before model sampling when it is determined that the physical state and the action intention are mutually exclusive, it achieves the mathematical blocking of the generation path of the illusion instruction before it is fully formed. This avoids the computational power and memory bandwidth consumed by matrix operations of invalid first word sequences. At the same time, since the correction occurs at the probability distribution level rather than after text generation, the model does not need to interrupt the generation process or return error feedback to the user. Only the semantic direction changes in real time. This significantly reduces the consumption of computing resources and eliminates the high latency and interaction fragmentation inherent in traditional post-processing interception mechanisms, providing users with a seamless and smooth natural interactive experience. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is one of the flowcharts illustrating the model streaming decoding and correction method provided in this application.

[0024] Figure 2 This is the second flowchart of the model streaming decoding and correction method provided in this application.

[0025] Figure 3 This is a schematic diagram of the architecture of the model streaming decoding and correction system provided in the embodiments of this application.

[0026] Figure 4 This is a schematic diagram of the model streaming decoding and correction device provided in this application.

[0027] Figure 5 This is a schematic diagram of the structure of the model streaming decoding and correction method device provided in this application.

[0028] Figure 6 A schematic diagram of the physical structure of an electronic device is provided. Detailed Implementation

[0029] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below with reference to the accompanying drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0030] Figure 1 is one of the schematic flow diagrams of the model streaming decoding error correction method provided by the present application, as Figure 1 shown in the figure, the method includes the following steps: Step 101: Obtain a first token output by the end-side model, where the first token is a partial token sequence output by the end-side model during the streaming decoding process and has not yet formed a complete device action instruction.

[0031] The end-side model refers to a Large Language Model (LLM) deployed on home Internet of Things edge nodes, such as smart gateways and home hubs, whose parameter quantity is usually at the level of 1B to 8B, and is limited by end-side computing power.

[0032] The first token (the token may also be referred to as Token hereinafter) refers to the minimum semantic unit for the model to process text in natural language processing, which may be a character, a word or a subword. In the embodiments of the present application, the first token specifically refers to one or more latest Tokens that have been generated by the model during the streaming decoding process, and these Tokens have not yet formed a complete and executable control instruction after being spliced. Specific examples of the first token include "turn", "turn on guest" and the like, which are not limited herein.

[0033] Streaming decoding is an autoregressive generation manner of the large language model, that is, predicting and outputting the next Token character by character / word by word, instead of generating all texts at one time.

[0034] In the embodiments of the present application, the end-side model may specifically be, for example, a model deployed on the side of a household smart device, such as an air conditioner, a refrigerator, a television, etc., which is not limited herein.

[0035] In the embodiments of the present application, the end-side model is in a streaming output process, so that the Tokens can be captured at the initial stage of their generation, ensuring that subsequent correction can be intervened when the sunk cost of an erroneous instruction is relatively small.

[0036] In the embodiments of this application, obtaining the first token output by the end-side model may be running a bypass monitoring thread on the Central Processing Unit (CPU) of the home smart gateway. When the end-side Neural Processing Unit (NPU) is executing the matrix operation for typing, the thread intercepts the latest output Token ID in real time, for example, the Token ID is 1024. The system decodes this ID back into the character string "da" (type), or obtaining the first token output by the end-side model may be that at the t-th step of streaming decoding, the model outputs the Token sequence [guan, bi, ke]. The system uses a sliding window mechanism to extract the last two Tokens "bi ke" as the first token, which is not limited herein.

[0037] Step 102: Determine device control information matching the first token, and current state information of a target device associated with the device control information.

[0038] The device control information refers to the action instruction content parsed from the first token and intended by the user to be generated by the model, which usually includes action verbs, such as open and close, and target device nouns, such as living room light, door lock, etc., which is not limited herein.

[0039] The current state information of the target device refers to the absolute state of the device in the real physical world recorded in the home Internet of Things physical state machine. For example, door lock status: 0 - closed / 1 - open, numerical value: current temperature of air conditioner: 26°C.

[0040] In some embodiments, determining the device control information matching the first token may be that the system is preconfigured with several instruction templates constructed based on regular expressions or context-free grammars, for example: action verb + device name. After obtaining the first token, the system performs character-by-character matching between the token string and the above templates. If the matching is successful, the specific action verb and device name are extracted according to the placeholders in the template as the device control information; if the matching fails, it is determined that the current token does not contain any valid device control intention.

[0041] Alternatively, a binary classification neural network model may be deployed on the end side. After obtaining the first token, the system inputs the semantic embedding vector of the token into the classifier, and the classifier outputs the probability distribution that the token belongs to various predefined home control scenarios, such as "light control", "temperature control adjustment", and "access control management", and determines the standardized control instruction corresponding to the scenario with the highest probability as the device control information matching the first token.

[0042] In some embodiments, determining the current state information of the target device associated with the device control information can be achieved by the system maintaining a unified home device state hash table in memory. This table uses the device's unique identifier as the key and the device's current physical state value as the value, and the table is kept synchronized with all online devices via a real-time message queue. After obtaining the device control information, the system parses the unique identifier of the target device from the hash table and directly looks up the current state information of that device.

[0043] Alternatively, the system may not maintain any local state cache. After obtaining device control information, the system extracts the target device's network address or communication handle and directly initiates a real-time status query request to the target device via a lightweight communication protocol for the home IoT. The system waits for and receives the device's current status response message, thereby obtaining the device's instantaneous state information in the real physical world.

[0044] Optionally, in some embodiments, determining the device control information matching the first lexical unit includes: The first word element is matched with a preset set of control information to obtain the matching result; If the matching result indicates that there is control information in the preset control information set that matches the first word, the matching control information is determined as the device control information that matches the first word.

[0045] The preset control information set can also be called a character-level action Trie tree. This set is built offline and contains only legal and key control verbs and device noun combinations in a home setting, eliminating a large number of irrelevant chatter words.

[0046] For example, if the first word is "I want", the Trie tree cannot match the path of the word "I" under the root node, and it is determined to be a non-control scenario. In another scenario, if the word is "open", the Trie tree quickly matches the "open" prefix in the unlocking path and determines that the device control information is unlocking.

[0047] In this embodiment, by performing prefix matching between the local word sequence generated in real time during streaming decoding and a preset control information set, the subsequent physical state verification and probability intervention logic is only activated when a match is found. This achieves an extremely low-overhead "on-demand triggering" listening mechanism in an environment with limited end-side resources. In everyday chat scenarios, the entire intervention process is bypassed directly due to a matching failure, resulting in near-zero CPU utilization and avoiding any unnecessary computational consumption. At the same time, the matching processing based on the preset set can complete intent recognition in microseconds, which is much faster than the milliseconds required for a large model to generate a single token. This provides ample time margin for completing physical state queries and probability distribution corrections before sampling the next token.

[0048] Step 103: If the current state information and device control information are mutually exclusive, correct the initial probability distribution vector of the end-side model before sampling in the current decoding step.

[0049] Mutual exclusion in this context refers to the physical and logical principle that an action cannot be applied to a device in its current state. For example, for a light that is currently on, turning it off is feasible; however, turning it on is not feasible because turning on an already on light would be physically redundant or logically incorrect.

[0050] The initial probability distribution vector (Logits vector) is located in the last layer of the LLM model, where the model outputs a real-valued vector with dimensions equal to the vocabulary size. After Softmax normalization, this vector becomes a probability distribution used to sample the next token.

[0051] Optionally, in some embodiments, determining that the current state information and device control information are mutually exclusive includes: Determine the expected status information corresponding to the equipment control information; If the expected state information is consistent with the current state information, determine that the current state information and the device control information are mutually exclusive.

[0052] The expected state information refers to the state the device should reach after performing the action. For example, the expected state for the action "close" is 0 - closed; the expected state for the action "open" is 1 - open. A comparison is performed with the current state information using an XOR logic; if the expected state of the action equals the current physical state, then they are mutually exclusive.

[0053] For example, if the device control information is to turn off the living room light, the system queries the state pool and finds the current state of the living room light is 0 (off). The expected state for this action is 0. Since both are the same (0=0), the condition is determined to be mutually exclusive, meaning the model is attempting to generate a phantom instruction to turn off a light that is already off. Alternatively, if the device control information is to open the front door, the state pool returns the front door state as 1 (open). The expected state 1 is the same as the current state 1, and the condition is determined to be mutually exclusive.

[0054] Optionally, in some embodiments, when it is determined that the current state information and device control information are mutually exclusive, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected, including: If the conditions between the current status information and the equipment control information are mutually exclusive, determine whether the equipment control information belongs to the preset risk control action to obtain the judgment result; If the judgment result indicates that the device control information does not belong to the preset risk control action, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected.

[0055] The preset risk control actions refer to control actions that may cause risks, such as unlocking, unbinding a device, restoring factory settings, or opening a gas valve, etc.

[0056] In the embodiments of the present application, the initial probability distribution vector includes a plurality of candidate tokens, and an initial output probability corresponding to each candidate token; Optionally, in some embodiments, when it is determined that the current state information and the device control information are conditionally mutually exclusive, modifying the initial probability distribution vector of the end-side model before sampling at the current decoding step includes: From the candidate tokens, determine a second token that forms conditional mutual exclusion with the current state information, and a third token that is conditionally compatible with the current state information; Reducing the initial output probability corresponding to the second token, and / or increasing the initial output probability corresponding to the third token.

[0057] The second token is a Token that causes the hallucination to continue. For example, in the scenario of turning off a light that is already off, the next Token with the highest probability of the model may be "already", to form "the light is already", and the third token is a Token that conforms to physical reality, for example, the next Token with a lower probability of the model may be "indeed", to form "the light is indeed".

[0058] In the embodiments of the present application, after determining through Boolean algebra that the current state information and the device control information are conditionally mutually exclusive, the system does not directly apply a unified probability correction to all mutually exclusive scenarios, but first compares the parsed device control information with the high-risk action set preconfigured in the end-side CPU memory to obtain a judgment result; if the judgment result indicates that the device control information does not belong to the high-risk action set, for example, it is only a common scenario such as turning off the living room light, then before the Softmax normalization operation at this decoding step is performed, a constant negative bias is applied to the candidate Token that causes the hallucination to continue in the initial Logits probability distribution vector to make its mathematical probability approach zero, and a positive reward coefficient is applied to the candidate Token that conforms to the real physical state, thereby guiding the model to sample to a text path that conforms to physical reality, allowing the user to perceive a natural and smooth semantic shift in an immersive experience with zero stutter and zero error.

[0059] In an example, the system determines that turning off the living room light is mutually exclusive, and turning off the light does not belong to the preset high-risk action. Then the system executes a standard correction process, applies a penalty to the hallucination path Token at the Logits layer, and guides the model to output that the light is off.

[0060] In this embodiment of the application, if it is determined that the current state information and the device control information are not mutually exclusive, the end-side model directly performs normal sampling based on the initial probability distribution vector of its current decoding step after Softmax normalization, and continues to generate subsequent content such as "How is it", "Today", "Suitable", and "Going out" token by token.

[0061] In a specific example, a user issues a voice command to turn on the living room light. During streaming decoding, the edge model outputs local word sequences such as "turn on," "open," and "living room light" token by token. The system identifies the device control information as "turn on the living room light" through Trie tree matching. It then queries the home state pool to obtain the current physical state of the living room light, which is 0. The expected state of this action is 1. The inconsistency between the two indicates that the conditions are not mutually exclusive, meaning that the "turn on" command conforms to the current physical logic. In this case, the intervention layer determines that the model's output direction is correct and does not perform any Logits modification operations. The model continues to sample subsequent tokens "living room" and "light" based on the initial probability distribution, and finally outputs the complete executable command to turn on the living room light and delivers it to the device for execution.

[0062] Step 104: Sample the current decoding step based on the corrected probability distribution vector.

[0063] Sampling refers to the process by which LLM selects a specific token from the vocabulary as the output of the current step, based on the corrected probability distribution and using a random or greedy strategy.

[0064] In this embodiment of the application, after completing the bias correction of the initial Logits probability distribution vector, the edge-side large model performs the sampling operation of the current decoding step based on the corrected probability distribution vector according to the standard autoregressive decoding process.

[0065] Specifically, the corrected Logits vector is input into the Softmax normalization function to convert the Logits values ​​in the real number field into a normalized probability distribution. Then, based on this distribution, a candidate token is selected from the vocabulary as the output token of the current step using a greedy search or random sampling strategy.

[0066] The model streaming decoding correction method in this application advances the timing of verifying the physical state of the Internet of Things from the post-process complete text generation stage to the micro-time window of each first word generation during the streaming decoding process. When it is determined that the physical state and the action intention are mutually exclusive, the initial probability distribution vector before model sampling is directly corrected. This achieves the mathematical blocking of the generation path of the illusion instruction before it is fully formed, avoiding the end-side computing power and memory bandwidth consumed by matrix operations of invalid first word sequences. At the same time, since the correction occurs at the probability distribution level rather than after text generation, the model does not need to interrupt the generation process or return error feedback to the user. Only the semantic direction changes in real time. This significantly reduces the consumption of computing resources and eliminates the high latency and interaction fragmentation inherent in traditional post-interception mechanisms, providing users with a seamless and smooth natural interaction experience.

[0067] Figure 2 This is the second flowchart of the streaming decoding and correction method for the model provided in this application, as shown below. Figure 2 As shown, the method includes the following: Step 201: Obtain the first word element output by the terminal model. The first word element is a local word element sequence output by the terminal model during the streaming decoding process, which has not yet formed a complete device action instruction.

[0068] Step 202: Perform matching processing on the first word element and the preset control information set to obtain the matching result.

[0069] Step 203: If the matching result indicates that there is control information in the preset control information set that matches the first word, the matching control information is determined as the device control information that matches the first word.

[0070] For a detailed description of steps 201-203, please refer to the above embodiments, which will not be repeated here.

[0071] Step 204: If the matching result indicates that there is no control information in the preset control information set that matches the first word, sample the current decoding step according to the initial probability distribution vector.

[0072] In this embodiment, if the matching result indicates that there is no control information matching the first lexical in the preset control information set, it means that the currently generated local lexical sequence does not match any predefined home device control command path, i.e., the lexical sequence belongs to casual conversation or non-control-related dialogue content. At this time, the system determines that there is no need to apply any physical state constraints to the model's generation process. The bypassed intervention layer remains silent and directly performs normal token sampling based on the initial Logits probability distribution vector output by the current decoding step of the edge model after Softmax normalization. The model continues to autoregressively generate according to its original language probability preferences. The entire inference process is consistent with the standard LLM decoding process without this intervention mechanism.

[0073] For example, when a user issues a voice command to their home smart gateway, "What's the weather like today?", the edge model outputs a sequence of local lexical units such as "today," "day," "weather," etc., token by token during streaming decoding. A streaming token prefix capturer mounted on the CPU intercepts these token IDs in real time and decodes them into strings. Then, each string prefix is ​​matched sequentially against a pre-built character-level action Trie tree. Since the Trie tree only pre-sets character combination paths for home device control actions such as "open," "close," and "adjust," the casual natural language prefix "Today's weather" cannot match any legitimate action node in this tree structure. The matching result indicates that there is no control information matching the first lexical unit in the preset control information set.

[0074] At this point, the system determines that the current decoding process does not involve any IoT device control intent. The physical state constraint intervention layer remains completely silent, neither querying the home state pool nor performing any Logits tensor modification operations. The edge model directly samples normally based on the initial probability distribution vector of its current decoding step after Softmax normalization, and continues to generate subsequent content such as "How is it?", "Today", "Suitable", and "Going out" token by token.

[0075] Step 205: If the current status information and the equipment control information are mutually exclusive, determine whether the equipment control information belongs to the preset risk control action to obtain the judgment result.

[0076] Step 206: If the judgment result indicates that the device control information does not belong to the preset risk control action, correct the initial probability distribution vector of the end-side model before sampling in the current decoding step.

[0077] For a detailed description of steps 205-206, please refer to the above embodiments, which will not be repeated here.

[0078] Step 207: If the judgment result indicates that the device control information belongs to the preset risk control action, stop sampling in the current decoding step.

[0079] Stopping sampling in the current decoding step means that the end-side large model no longer performs any operation to select lexical units from the vocabulary in the current autoregressive decoding step, that is, interrupting the entire streaming generation process.

[0080] In this embodiment, if Boolean algebra determines that the current state information and device control information are mutually exclusive, and further determination indicates that the mutually exclusive device control information belongs to a preset risk control action, the system determines that the current scenario has reached the highest security priority. For example, if the user intent or model output points to opening the door lock, and the state pool returns that the current physical state of the door lock is open, the two are mutually exclusive and belong to a high-risk scenario. At this time, the system does not execute the bias correction probability adjustment process, but directly sets the value of the terminator in the initial Logits probability distribution vector of the current decoding step to the maximum floating-point limit value before Softmax normalization is executed. Subsequently, the sampling module performs Softmax normalization. Since the Logits value corresponding to the terminator is much larger than the Logits values ​​of all other candidate tokens in the vocabulary, the entire streaming decoding process will be forcibly interrupted.

[0081] The foregoing embodiments, from the perspective of steps and procedures, elaborate in detail the model streaming decoding correction method provided in this application. To further demonstrate the engineering deployment of this method in a real-world hardware and software environment, and to illustrate the efficient collaboration and data flow mechanism between the proposed physical state constraint intervention layer, the edge-side large language model, and the external IoT environment, the following will combine... Figure 3 The overall system architecture for implementing the above-mentioned correction method is described in detail.

[0082] Figure 3 This is a schematic diagram of the architecture of the model streaming decoding and correction system provided in the embodiments of this application. Figure 3 As shown, the system mainly includes a voice interaction module, an edge-side LLM streaming inference engine, a physical state constraint intervention layer, and an external IoT environment.

[0083] Specifically, user voice commands are first converted into text input by an automatic speech recognition and speech-to-text module and then sent to the edge-side LLM streaming inference engine. During the word-by-word generation of responses by the streaming inference engine, a physical state constraint intervention layer acts as a bypass mechanism, and its internal processing logic is as follows: Streaming Token Prefix Capturer: Intercepts the original ID sequence flowing out of the streaming inference engine token by token in real time, extracts the latest N tokens for character-level decoding, and obtains the first word unit.

[0084] Character-level action Trie tree matcher: The decoded prefixes mentioned above are input into the Trie tree for efficient matching. When a specific action branch is matched, a callback request is triggered to the downstream branch; if no match is found, the system remains silent and does not consume additional computing power.

[0085] Home physical state machine memory pool: This memory pool maintains millisecond-level state synchronization with the device state shadow database in the external IoT environment. Upon receiving a callback from the matcher, the memory pool quickly returns the current state information of the target device through microsecond-level reading.

[0086] Logits Probability Dynamic Modifier: Upon receiving the physical state, if it is determined that the action the current model attempts to generate is conditionally mutually exclusive with the physical state, the Logits probability distribution of the next decoding step of the model is dynamically adjusted, and the modified Logits tensor is re-injected into the edge-side LLM streaming inference engine to complete the correction from a mathematical perspective.

[0087] After real-time correction by the physical state constraint intervention layer, the output of the edge-side LLM streaming inference engine is divided into two paths: Control Execution Path: Outputs only the final, valid JSON instructions without illusions to the smart home execution gateway in the external IoT environment, ensuring the safe and stable execution of physical devices.

[0088] Voice feedback path: The corrected word stream is sent to the text-to-speech module, and finally the smart speaker reads it naturally, providing users with a seamless, smooth and physically realistic interactive experience.

[0089] To more clearly illustrate the probability distribution vector correction mechanism and the collaborative workflow between software and hardware modules in the embodiments of this application, the following will be combined with... Figure 4 Taking a specific smart home control scenario as an example, the above streaming decoding and correction process will be explained in detail. Figure 4 This is an interaction timing diagram of a model streaming decoding correction method provided in this application embodiment in a specific scenario. Taking a scenario where the user inputs a command to close the door, and the door is actually already closed, as an example... Figure 4 As shown, the interaction process specifically includes the following steps: (1) The user inputs "Prompt: Close the door" into the edge-side LLM inference engine. After receiving the input, the edge-side LLM inference engine begins the normal autoregressive generation of the preceding token; (2) The initial Token ID sequence is output from the edge-side LLM inference engine. The Logits tamperer deployed on the CPU in this application captures this sequence and performs prefix matching through a Trie tree, successfully hitting the close action branch; (3) The Logits tamper extracts the target entity gate from the prefix, and sends a request to the home physical state machine to read the status bit. The home physical state machine returns the current state at high speed, where DOOR_STATUS = CLOSED. The Logits tamper detects a contradiction between the expected action CLOSE and the current physical state CLOSED, and immediately initiates the Logits tensor tampering process; (4) At the stage before Softmax sampling is performed, the edge-side LLM inference engine exposes the original Logits vector of the next Token. The Logits tamper analyzes the original Logits probability distribution and finds that: the original score of Token s that causes the logical error is 8.5, while the original score of Token that conforms to physical reality is 2.1; (5) Probability tampering at the mathematical level: Reverse suppression: the Logits tamper penalizes the wrong Token, reducing its Logits score to a negative value; Positive guidance: the Logits tamper rewards the correct Token, increasing its Logits score; (6) The Logits tamper returns the modified Logits vector to the Softmax layer of the edge-side LLM inference engine. After Softmax calculation, the sampling probability of the correct Token is increased to close to 1.0, and the model naturally samples the correct character without the user's awareness; (7) Final text generation: The edge-side LLM inference engine smoothly changes the semantic direction, and finally outputs the corrected legal text to the user: It has been confirmed that the door is currently closed.

[0090] The following describes the model streaming decoding deviation correction apparatus provided in the present application. The model streaming decoding deviation correction apparatus described below and the model streaming decoding deviation correction method described above may be referenced in correspondence with each other.

[0091] Figure 5 is a schematic structural diagram of the model streaming decoding deviation correction apparatus provided in the present application, as shown in Figure 5 the figure, the apparatus includes the following components: An acquisition module 501, configured to acquire a first token output by the edge-side model, where the first token is a partial token sequence output by the edge-side model during the streaming decoding process and has not yet formed a complete device action instruction; A determination module 502, configured to determine device control information matching the first token, and current state information of a target device associated with the device control information; A correction module 503, configured to correct the initial probability distribution vector of the edge-side model before sampling at the current decoding step when it is determined that the current state information and the device control information are conditionally mutually exclusive; The first sampling module 504 is used to sample the current decoding step based on the corrected probability distribution vector.

[0092] According to the model streaming decoding correction device provided in this application, the initial probability distribution vector includes multiple candidate words and the initial output probability corresponding to each candidate word. Correction module 503 is specifically used for: From the candidate lexical units, determine the second lexical unit that is mutually exclusive with the current state information and the third lexical unit that is compatible with the current state information condition; Decrease the initial output probability corresponding to the second word, and / or increase the initial output probability corresponding to the third word.

[0093] According to the model streaming decoding and correction device provided in this application, the determining module 502 is specifically used for: The first word element is matched with a preset set of control information to obtain the matching result; If the matching result indicates that there is control information in the preset control information set that matches the first word, the matching control information is determined as the device control information that matches the first word.

[0094] According to the model streaming decoding correction device provided in this application, it further includes: The second sampling module (not shown in the figure) is used to perform sampling for the current decoding step based on the initial probability distribution vector after matching the first word with the preset control information set to obtain the matching result. If the matching result indicates that there is no control information in the preset control information set that matches the first word, the module will perform sampling for the current decoding step.

[0095] According to the model streaming decoding correction device provided in this application, the correction module is specifically used for: If the conditions between the current status information and the equipment control information are mutually exclusive, determine whether the equipment control information belongs to the preset risk control action to obtain the judgment result; If the judgment result indicates that the device control information does not belong to the preset risk control action, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected.

[0096] According to the model streaming decoding correction device provided in this application, it further includes: The third sampling module (not shown in the figure) is used to stop sampling in the current decoding step after determining whether the device control information belongs to the preset risk control action and obtaining the judgment result.

[0097] According to the model streaming decoding correction device provided in this application, the correction module 503 is specifically used for: Determine the expected status information corresponding to the equipment control information; If the expected state information is consistent with the current state information, determine that the current state information and the device control information are mutually exclusive.

[0098] The model streaming decoding correction device in this application forwards the timing of verifying the physical state of the Internet of Things from the post-process complete text generation stage to the micro-time window of each first word generation during the streaming decoding process. When it is determined that the physical state and the action intention are mutually exclusive, the device directly corrects the initial probability distribution vector before the model sampling. This achieves the mathematical blocking of the generation path of the illusion instruction before it is fully formed, avoiding the end-side computing power and memory bandwidth consumed by the matrix operation of invalid first word sequences. At the same time, since the correction occurs at the probability distribution level rather than after text generation, the model does not need to interrupt the generation process or return error feedback to the user. Only the semantic direction changes in real time. This significantly reduces the consumption of computing resources and eliminates the high latency and interaction fragmentation inherent in traditional post-interception mechanisms, providing users with a seamless and smooth natural interaction experience.

[0099] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a model streaming decoding correction method, which includes: Obtain the first word element output by the edge model, where the first word element is a local word element sequence output by the edge model during streaming decoding and has not yet formed a complete device action instruction; Determine the device control information that matches the first word element, and the current status information of the target device associated with the device control information; Given that the current state information and device control information are mutually exclusive, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected. Sampling is performed for the current decoding step based on the corrected probability distribution vector.

[0100] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the model streaming decoding correction method provided by the above methods, the method including: Obtain the first word element output by the edge model, where the first word element is a local word element sequence output by the edge model during streaming decoding and has not yet formed a complete device action instruction; Determine the device control information that matches the first word element, and the current status information of the target device associated with the device control information; Given that the current state information and device control information are mutually exclusive, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected. Sampling is performed for the current decoding step based on the corrected probability distribution vector.

[0102] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the model streaming decoding correction method provided by the above methods, the method comprising: Obtain the first word element output by the edge model, where the first word element is a local word element sequence output by the edge model during streaming decoding and has not yet formed a complete device action instruction; Determine the device control information that matches the first word element, and the current status information of the target device associated with the device control information; Given that the current state information and device control information are mutually exclusive, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected. Sampling is performed for the current decoding step based on the corrected probability distribution vector.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A model streaming decoding and correction method, characterized in that, include: Obtain the first word element output by the terminal model, wherein the first word element is a local word element sequence output by the terminal model during the streaming decoding process and has not yet formed a complete device action instruction; Determine the device control information that matches the first word, and the current status information of the target device associated with the device control information; If it is determined that the current state information and the device control information are mutually exclusive, the initial probability distribution vector of the end-side model before sampling in the current decoding step is corrected. Sampling is performed for the current decoding step based on the corrected probability distribution vector.

2. The method according to claim 1, characterized in that, The initial probability distribution vector includes multiple candidate words and an initial output probability corresponding to each candidate word; Wherein, correcting the initial probability distribution vector of the end-side model before sampling in the current decoding step, when it is determined that the current state information and the device control information are mutually exclusive, includes: From the candidate lexical units, determine a second lexical unit that is mutually exclusive with the current state information and a third lexical unit that is compatible with the current state information condition; Decrease the initial output probability corresponding to the second word, and / or increase the initial output probability corresponding to the third word.

3. The method according to claim 1, characterized in that, The determination of the device control information matching the first lexical term includes: The first word element is matched with a preset set of control information to obtain a matching result; If the matching result indicates that there is control information in the preset control information set that matches the first word, the matching control information is determined as the device control information that matches the first word.

4. The method according to claim 3, characterized in that, After performing matching processing on the first lexical unit and the preset control information set to obtain a matching result, the method further includes: If the matching result indicates that there is no control information in the preset control information set that matches the first word, sampling is performed for the current decoding step based on the initial probability distribution vector.

5. The method according to claim 1, characterized in that, The step of correcting the initial probability distribution vector of the end-side model before sampling in the current decoding step, when it is determined that the current state information and the device control information are mutually exclusive, includes: If it is determined that the current status information and the device control information are mutually exclusive, it is determined whether the device control information belongs to a preset risk control action, so as to obtain a judgment result; If the judgment result indicates that the device control information does not belong to the preset risk control action, the initial probability distribution vector of the end-side model before the current decoding step sampling is corrected.

6. The method according to claim 5, characterized in that, After determining whether the device control information belongs to a preset risk control action and obtaining a determination result, the method further includes: If the judgment result indicates that the device control information belongs to the preset risk control action, the sampling of the current decoding step shall be stopped.

7. The method according to claim 1, characterized in that, The step of determining the mutual exclusion between the current state information and the device control information includes: Determine the expected state information corresponding to the device control information; If the expected state information is consistent with the current state information, it is determined that the current state information and the device control information are mutually exclusive.

8. A model streaming decoding correction device, characterized in that, include: The acquisition module is used to acquire the first word element output by the terminal model, wherein the first word element is a local word element sequence output by the terminal model during the streaming decoding process and has not yet formed a complete device action instruction; The determination module is used to determine the device control information that matches the first word element, and the current status information of the target device associated with the device control information; The correction module is used to correct the initial probability distribution vector of the end-side model before sampling in the current decoding step when it is determined that the current state information and the device control information are mutually exclusive. The first sampling module is used to sample the current decoding step based on the corrected probability distribution vector.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the model streaming decoding correction method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the model streaming decoding correction method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the model streaming decoding correction method as described in any one of claims 1 to 7.

Images