AI behavior calibration methods, systems, and electronic devices

Abstract

This disclosure provides an AI behavior calibration method, system, and electronic device. The method includes: acquiring interaction data of an artificial intelligence model and user feedback data related to the interaction data; inputting evaluation prompts and the interaction data into an evaluation model to score the interaction data of the artificial intelligence model; maintaining a constant evaluation benchmark information for the evaluation model that is not updated iteratively with the data flywheel of the artificial intelligence model; and adjusting the evaluation prompts based on the scoring results and the user feedback data. This disclosure can maintain a constant, hardware-fixed, and tamper-proof set of evaluation benchmark information throughout the entire process of continuous iterative evolution of the artificial intelligence model relying on the data flywheel, thereby suppressing evaluation standard drift and echo chamber effects at the source. Thus, value orientation calibration can be completed simply by adjusting the evaluation prompts corresponding to the evaluation model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to an AI behavior calibration method, system, and electronic device. Background Technology

[0002] Current generative AI relies on data flywheels to achieve closed-loop iterative optimization of models. The iteration effect is highly dependent on the stability of the evaluation criteria. However, existing AI data flywheel iteration systems generally suffer from evaluation criterion drift, and online calibration solutions have problems such as high computing power costs, large response latency, and unauditable and tamper-proof evaluation criteria. Specifically, under the RLHF (Human Feedback-Based Reinforcement Learning) architecture, the reward model and decision model are trained simultaneously, which is prone to echo chamber effects, and long-term iterations can lead to the degradation of model values. The real-time evaluation mode of large cloud models consumes a lot of computing power and has high network latency, which cannot be adapted to the low-latency scenarios of physical edge AI. Purely software-based evaluation rules can be tampered with or hacked, and key AI decisions lack credible auditable evidence. Currently, mainstream solutions in the industry include the RLHF reward model evaluation scheme adopted by OpenAI ChatGPT (a dialogue model), the multi-model integrated voting evaluation scheme of Constitutional AI, and hardware protection schemes using TEE (Trusted Execution Environment) / HSM (Hardware Security Module) runtime environments. However, all three approaches have shortcomings: the RLHF reward model experiences standard drift with iteration, multi-model peer evaluation lacks a constant and independent evaluation benchmark, and TEE / HSM only protects the operating environment and cannot solidify the evaluation benchmark. Existing technologies cannot maintain a constant and immutable evaluation benchmark during the continuous iteration of AI models. Summary of the Invention

[0003] The technical problem to be solved by this disclosure is to overcome at least one of the above-mentioned defects in the prior art and to provide an AI behavior calibration method, system, or electronic device.

[0004] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0005] Firstly, an AI behavior calibration method is provided, including:

[0006] Acquire interaction data from the artificial intelligence model and user feedback data related to the interaction data;

[0007] The evaluation prompts and the interaction data are input into the evaluation model to score the interaction data of the artificial intelligence model; the evaluation benchmark information of the evaluation model remains constant and is not updated iteratively with the data flywheel execution of the artificial intelligence model;

[0008] The evaluation prompts are adjusted based on the rating results and the user feedback data.

[0009] Optionally, the evaluation benchmark information of the evaluation model is embedded in at least one of the following devices within the scale chip: an artificial intelligence inference chip, a one-time programmable memory device, and a security chip with integrated physically unclonable functions.

[0010] Optionally, the scale chip further includes:

[0011] An entropy calculation circuit is used to calculate the information entropy of the output result of the artificial intelligence model, and when the information entropy exceeds the entropy threshold, outputs a behavior suppression signal to suppress the artificial intelligence model from displaying the output result to the outside world;

[0012] And / or, a power consumption monitoring circuit is used to collect the operating power consumption of the artificial intelligence model, and when the operating power consumption is greater than a power consumption threshold, send a policy rollback instruction or a service degradation control instruction to the artificial intelligence model.

[0013] Optionally, the evaluation benchmark information of the evaluation model is solidified and packaged using a read-only container image;

[0014] And / or, the hash value of the evaluation benchmark information of the evaluation model is anchored on the blockchain;

[0015] And / or, deploy the evaluation benchmark information in multiple different trusted execution environments;

[0016] And / or, the evaluation logic of the evaluation benchmark information is implemented using optical computing circuits or analog circuits.

[0017] Optionally, the evaluation model and the artificial intelligence model run asynchronously;

[0018] And / or, the evaluation model is deployed offline.

[0019] Optionally, it also includes:

[0020] Interaction data that matches the rating results with the user feedback data are selected as training samples;

[0021] The artificial intelligence model is optimized using the training samples.

[0022] Optionally, the rating result includes a rating; selecting interaction data that matches the rating result with the user feedback data as training samples includes:

[0023] Interaction data with ratings greater than the rating threshold and user feedback data that is positive user feedback are diverted to the training sample pool; the interaction data in the training sample pool is used as training samples.

[0024] Interaction data where the rating is less than or equal to the rating threshold and the user feedback data is positive user feedback, and / or interaction data where the rating is greater than the rating threshold and the user feedback data is negative user feedback, are diverted to the manual review pool.

[0025] Interaction data whose rating is less than or equal to the rating threshold and whose user feedback data is negative user feedback will be diverted to the discard pool.

[0026] Secondly, an AI behavior calibration system is provided, including:

[0027] The acquisition module is used to acquire interaction data of the artificial intelligence model and user feedback data on the interaction data;

[0028] The evaluation module is used to input evaluation prompts and the interaction data into the evaluation model to score the interaction data of the artificial intelligence model; the evaluation benchmark information of the evaluation model remains constant and is not updated iteratively with the data flywheel execution of the artificial intelligence model;

[0029] The calibration module is used to adjust the evaluation prompts based on the scoring results and the user feedback data.

[0030] Optionally, the evaluation benchmark information of the evaluation model is embedded in at least one of the following devices within the scale chip: an artificial intelligence inference chip, a one-time programmable memory device, and a security chip with integrated physically unclonable functions;

[0031] Optionally, the scale chip further includes:

[0032] An entropy calculation circuit is used to calculate the information entropy of the output result of the artificial intelligence model, and when the information entropy exceeds the entropy threshold, outputs a behavior suppression signal to suppress the artificial intelligence model from displaying the output result to the outside world;

[0033] And / or, a power consumption monitoring circuit is used to collect the operating power consumption of the artificial intelligence model, and when the operating power consumption is greater than a power consumption threshold, send a policy rollback instruction or a service degradation control instruction to the artificial intelligence model.

[0034] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the AI ​​behavior calibration method described in any one of the first aspects.

[0035] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the AI ​​behavior calibration method described in any one of the first aspects.

[0036] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0037] The positive advancements of this disclosure are as follows: It maintains a constant, hardware-fixed, and tamper-proof set of evaluation benchmark information throughout the entire process of continuous iterative evolution of artificial intelligence models relying on a data flywheel, thereby suppressing evaluation standard drift and echo chamber effects at their source. Thus, value-oriented calibration can be completed simply by adjusting the corresponding evaluation prompts of the evaluation model, without requiring retraining of the evaluation model. This enables a safe, interpretable, and fast-rollback-supporting evaluation scale calibration mechanism. Attached Figure Description

[0038] Figure 1 A flowchart of an AI behavior calibration method provided as an exemplary embodiment of this disclosure;

[0039] Figure 2 A schematic diagram of a module for an AI behavior calibration system provided as an exemplary embodiment of this disclosure;

[0040] Figure 3 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0041] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0042] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0043] In this embodiment of the disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information (including user feedback data) comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0044] Figure 1 A flowchart of an AI behavior calibration method provided for an exemplary embodiment of this disclosure, the method including the following steps:

[0045] Step 101: Obtain the interaction data of the artificial intelligence model and the user feedback data on the interaction data.

[0046] Artificial intelligence models are deployed at the front end of business processes and can be continuously trained and optimized to perform practical tasks, such as dialogue generation and behavioral decision-making. Artificial intelligence models include dialogue models and decision-making models.

[0047] Interaction data refers to the raw and derived data generated by artificial intelligence models during human-computer interaction and inter-model interaction. It may include, but is not limited to, data that records interaction behavior, content, status, and results, as well as interaction logs.

[0048] User feedback data refers to the evaluations, corrections, opinions, and tags submitted by users, either actively or passively, after they participate in human-computer interaction and generate interaction data, regarding the quality of the content, the effectiveness of the response, the model's behavior, and their overall experience. User feedback data may include, but is not limited to, data that characterizes user behavior and preferences, such as likes, dislikes, page dwell time, and triggers for human intervention.

[0049] Step 102: Input the evaluation prompts and interaction data into the evaluation model to score the interaction data of the artificial intelligence model.

[0050] The evaluation benchmark information of the evaluation model remains constant and is not updated iteratively with the data flywheel execution of the artificial intelligence model. The evaluation benchmark information consists of all fixed criteria used for standardized scoring and judgment of interactive data; it is a complete, unified set of evaluation criteria that cannot be changed with data flywheel iterations, used to eliminate standard drift. The evaluation benchmark information includes model parameters and logical rules; the model parameters include model weights and bias parameters.

[0051] Scoring of interactive data from AI models can be multi-dimensional. These dimensions should include at least one or more of the following: content compliance, answer accuracy, logical completeness, fluency of expression, value alignment, and user experience suitability. Each dimension has its own scoring range and judgment criteria. A comprehensive score or individual dimension scores are then output based on the evaluation benchmark information. This multi-dimensional scoring model can finely differentiate the quality of interactive data, avoiding biased evaluations caused by single-dimensional scoring. The judgment criteria for each dimension are uniformly constrained by constant evaluation benchmark information; only the outer evaluation prompts can be dynamically adjusted based on user feedback, while the underlying evaluation criteria do not drift with model iterations.

[0052] Step 103: Adjust the evaluation prompts based on the rating results and user feedback data.

[0053] Evaluation prompts are the "textualized expression" of the evaluation criteria, defining "what standards the evaluation model uses to score." Subsequently, the evaluation model uses the modified evaluation prompts to evaluate old and / or new interaction data.

[0054] By adjusting the evaluation dimensions, weights of each dimension, and textual judgment criteria of the evaluation prompts used in the evaluation model, dynamic adaptation and calibration of the overall evaluation orientation can be achieved; the frozen and solidified evaluation benchmark information is not changed throughout the process, ensuring the long-term stability of the evaluation benchmark and safely adapting to the value judgment needs in different scenarios.

[0055] In this embodiment, a constant, hardware-fixed, and tamper-proof set of evaluation benchmark information can be maintained throughout the entire process of continuous iterative evolution of the artificial intelligence model relying on a data flywheel, thereby suppressing evaluation standard drift and echo chamber effects at the source. Thus, value-oriented calibration can be completed simply by adjusting the evaluation prompts corresponding to the evaluation model, without requiring retraining of the evaluation model. This enables a safe, interpretable, and fast-rollback-supporting evaluation scale calibration mechanism.

[0056] In one embodiment, the evaluation model runs asynchronously with the artificial intelligence model; and / or, the evaluation model is deployed offline.

[0057] In one embodiment, a two-layer isolation evaluation architecture is set up, consisting of an online decision-making layer and an offline evaluation layer.

[0058] The first layer is an iteratively evolving online decision-making layer, deployed on general-purpose computing resources such as CPUs and GPUs, and equipped with an AI model that is continuously trained and optimized. The online decision-making layer responds to user interaction requests in real time and generates dialogue output. It can also simultaneously collect user feedback data.

[0059] The second layer is an offline evaluation layer with a constant evaluation benchmark. This offline evaluation layer deploys an independent evaluation model with permanently frozen parameters that does not participate in any form of training or updates. The offline evaluation layer asynchronously performs multi-dimensional quantitative scoring on the interactive data generated by the online decision-making layer in a bypass offline manner.

[0060] It should be noted that this evaluation model does not accept modification instructions such as gradient updates, weight adjustments, or parameter synchronization from the online decision-making layer, and the entire evaluation benchmark maintains a unified standard throughout the entire operation cycle.

[0061] In this embodiment, the evaluation process is executed offline and asynchronously, decoupled from the inference process of the front-end artificial intelligence model. This will not slow down the front-end interaction response speed, nor will it occupy the computing power required for the operation of the artificial intelligence model. This effectively reduces the computing power resource consumption and real-time inference pressure of online services, and achieves low-cost calibration evaluation.

[0062] In one embodiment, the method further includes: selecting interactive data that matches the rating results with user feedback data as training samples, and using the training samples to optimize the artificial intelligence model.

[0063] The evaluation prompts define "what standards the evaluation model uses to score." Adjusting the evaluation prompts is equivalent to changing the quantitative scoring standards. After the scoring standards change, the set of interaction data judged as high-scoring samples changes accordingly. The sample distribution used for fine-tuning and optimizing the artificial intelligence model is adjusted simultaneously, ultimately guiding the artificial intelligence model to learn different output behavior patterns, forming an indirect and precise controllable value calibration link.

[0064] Optionally, a conflict arbitration engine can be used to determine whether the rating results are consistent with user feedback data. Specifically, the conflict arbitration engine automatically splits the data based on the comparison results of the two source signals:

[0065] Interaction data with ratings greater than the rating threshold and positive user feedback are diverted to the training sample pool (also known as the high-value candidate pool).

[0066] Interaction data with ratings less than or equal to the rating threshold and positive user feedback, and / or with ratings greater than the rating threshold and negative user feedback, will be diverted to the manual review pool (also known as the dispute arbitration pool).

[0067] Interaction data with ratings less than or equal to the rating threshold and negative user feedback will be diverted to the discard pool.

[0068] Interaction data in the training sample pool serves as training samples. Interaction data in the manual review pool is pushed to manual reviewers for analysis. The manual reviewer determines whether the interaction data should be used as a training sample or discarded. Interaction data in the discard pool is discarded directly. The scoring threshold can be flexibly set according to actual needs.

[0069] Dual-source signals refer to evaluation data from two independent dimensions: user behavioral feedback data and objective scores from the evaluation model. If a user likes and / or stays on the interactive data for a long time, it is considered positive user feedback; if a user dislikes and / or stays on the interactive data for a short time, it is considered negative user feedback.

[0070] In this embodiment, the evaluation prompts are dynamically optimized by integrating real user feedback data. By cross-referencing the scoring results with user feedback, interaction samples (interaction data) where the evaluation benchmark is inconsistent with the user's actual preferences can be automatically identified. This accurately uncovers two types of high-value conflicting data, such as interaction data where the evaluation model judges it as excellent but the user gives negative feedback, or interaction data where the evaluation model scores it low but the user has positive approval. On the one hand, a constant evaluation benchmark ensures long-term consistency of the evaluation scale; on the other hand, user feedback is used to continuously optimize the evaluation prompts to adapt to real business scenarios. At the same time, differentiated conflicting samples are output to provide high-quality training materials for the iteration of the artificial intelligence model, balancing the stability of the evaluation standard and the scenario adaptability of the evaluation system.

[0071] In one embodiment, the evaluation benchmark information of the evaluation model is embedded in at least one of the following devices within the scale chip: an artificial intelligence inference chip, a one-time programmable storage device, and a security chip with integrated physically unclonable functions.

[0072] Preferably, the hardware component incorporates at least one of the following enhanced features:

[0073] First, a physical fuse-based solidification design is adopted. The evaluation benchmark information of the evaluation model is programmed before the hardware leaves the factory. All parameters are locked by physical fuse melting and one-time programmable storage medium, so no instructions issued by any software can modify or read back.

[0074] Second, a one-way control transmission channel is set up. Only one-way data interaction is supported between the offline evaluation layer and the online decision-making layer: the online decision-making layer can only transmit data to be verified to the offline evaluation layer. After the offline evaluation layer completes its calculations, it sends back the scoring results and a behavior suppression signal. The offline evaluation layer will not receive gradient update or parameter adjustment instructions from the main computing chip. The behavior suppression signal is a hardware interception control signal output to the online decision-making layer when the judgment model determines that there is non-compliant behavior in the interactive data. Upon receiving this signal, the online decision-making layer blocks, corrects, and calibrates the output behavior of the artificial intelligence model.

[0075] Third, it possesses constant ultra-low latency performance. The solidified evaluation model completes all inference calculations within a fixed clock cycle, with inference latency consistently maintained at the microsecond level, meeting the operational requirements of hard real-time AI scenarios such as autonomous driving and industrial real-time control.

[0076] Fourth, it supports pluggable standardized upgrades. When the AI ​​model is deployed on the main computing chip and the judgment model is deployed on the benchmark chip, if there is a large-scale update to the industry standard, only the new generation of benchmark chips needs to be physically replaced. There is no need to redevelop the AI ​​model or replace the main computing chip. The new benchmark chip has built-in offline security verification logic and can only load judgment model images with legally authorized signatures.

[0077] In this embodiment, the evaluation benchmark information is solidified and encapsulated by the physical fuse locking mechanism of the hardware components, eliminating the risk of evaluation standards being tampered with from the hardware physical level, ensuring that the evaluation benchmark information remains constant and does not iterate and update with the data flywheel execution of the artificial intelligence model, forming a fully auditable and highly reliable underlying guarantee for evaluation.

[0078] In one embodiment, the scale chip may further integrate monitoring and hardware protection circuitry, specifically including an entropy calculation circuit and a power consumption monitoring circuit.

[0079] The entropy calculation circuit is used to receive the logits vector output by the main artificial intelligence model and calculate the corresponding information entropy in real time. If the calculated information entropy exceeds the entropy threshold, it means that the uncertainty of the model output result is too high. The entropy calculation circuit outputs a behavior suppression signal to suppress the artificial intelligence model from displaying the output result to the outside world.

[0080] The output includes question-and-answer content and / or decisions. The entropy threshold can be flexibly set according to actual needs.

[0081] The logits vector represents the tendency score output by the artificial intelligence model for each candidate word in the vocabulary when predicting the next text token. It is an unnormalized high-dimensional probability distribution.

[0082] Information entropy exceeding the entropy threshold indicates that the model has similar biases towards many words. For example, the scores for "pedestrian," "bicycle," and "backdrop" are all around 50, indicating that the model is highly uncertain and in a state of "confusion" or "indecision." This means that the model encountered a dangerous scenario it had never seen before and made an incorrect perceptual judgment.

[0083] If the information entropy does not exceed the entropy threshold, it means that the model has a very high bias towards a certain word. For example, the score for the word "pedestrian" is 100, while the scores for other words are in the single digits, indicating that the model is very certain about what to output next.

[0084] Suppression signals are the highest priority hardware signals. Taking the output result as the decision, the suppression of external display decisions includes: (1) blocking the current instruction and directly preventing the current chaotic reasoning result from entering the next stage (such as the hardware control layer); (2) triggering safety actions and forcibly executing the preset "back-off strategy", such as making the vehicle enter a safe state of slowing down or stopping, or making the robot arm stop operating.

[0085] The suppression process can be completed automatically within microseconds, completely bypassing the decision-making process of human-powered intelligent models, to ensure that the most conservative and safest response can be made at any time.

[0086] The power consumption monitoring circuit collects the real-time operating power consumption of the main computing chip via the current sampling interface. When the measured power consumption exceeds the power consumption threshold, the system is determined to be in an abnormal computing condition, and the power consumption monitoring circuit issues a policy rollback or service degradation control command.

[0087] The power consumption threshold can be flexibly set according to actual needs.

[0088] "Rollback" refers to reverting the main model and related core software to the previous, verified safe parameter snapshot and software version, based on the most recent model update or software configuration change that may have caused an anomaly. This safe snapshot is stored in a separate, read-only, and secure storage area.

[0089] The so-called "degradation" instruction is sent to the power management unit and task scheduler at the hardware control layer. It forces: limiting the maximum operating frequency and performance of the CPU / GPU, directly reducing power consumption and heat generation; forcibly interrupting all non-core, low-priority computing tasks; and enabling redundant safety channels, forcing the system to switch to pure remote operation or low-performance safety mode. This ensures basic communication and security response capabilities while completely eliminating security risks caused by high-load computing.

[0090] Certain types of computational anomalies (such as infinite loops, being hacked for illegal computation, etc.) can directly lead to a sharp increase in chip power consumption. Power consumption anomalies are not only an electricity cost issue, but also a major physical security signal that may lead to: (1) chip overheating and physical damage. High power consumption will result in high temperatures, which may damage the hardware and cause permanent system failure; (2) functional instability. Abnormally high load computing will crowd out computing resources for security checks and real-time control, resulting in slow response; (3) covert malicious behavior. If an intelligent agent is being used for malicious purposes such as DDoS attacks or password cracking, its power consumption will be abnormal.

[0091] When such physical risk signals are detected, the safest approach is to immediately degrade the entire system to a known, safe, low-power operating state to ensure physical safety. This decisive action is a higher priority safety principle than "precise troubleshooting" in fields such as industry, aerospace, and autonomous driving.

[0092] In one embodiment, the evaluation benchmark information of the evaluation model is solidified and encapsulated using a read-only container image. This can be, but is not limited to, using a Docker image with digital signature locking and configured in read-only mount mode. In this embodiment, the image content is verified in real time through a container runtime integrity verification mechanism to ensure that the evaluation benchmark information is not illegally tampered with.

[0093] In one embodiment, the hash value of the evaluation benchmark information of the evaluation model is anchored on the blockchain. Before each evaluation operation, the hash of the local evaluation benchmark information is checked to see if it matches the hash of the on-chain evidence storage; if the two do not match, the evaluation process is blocked.

[0094] In one embodiment, a multi-TEE heterogeneous evaluation cluster is built, deploying identical evaluation benchmark information within multi-vendor differentiated trusted execution environments (including Intel SGX, AMD SEV, and ARM TrustZone). The score is only accepted if the scores output by all trusted execution environments are consistent, thereby mitigating the risk of evaluation result distortion caused by the compromise of a single TEE environment.

[0095] In one embodiment, the evaluation logic for the evaluation benchmark information is directly embedded using optical computing circuits or analog circuits; relying on the inherent non-reprogrammable physical characteristics of optical and analog circuits, physical-level solidification protection of the evaluation rules is achieved.

[0096] The following is combined Figure 2 The process of AI behavior calibration will be further explained below:

[0097] S1. Online Interaction and User Feedback Data Collection

[0098] Users submit interaction requests to the online decision-making layer, which generates response content from its artificial intelligence model and sends it back to the user. Simultaneously, the online decision-making layer collects and stores feedback data on various user behaviors, such as likes, dislikes, and dwell time.

[0099] S2, Offline Asynchronous Evaluation

[0100] After a single human-computer dialogue is completed, the online decision-making layer pushes the interaction data (such as interaction logs) to the offline task queue, schedules the evaluation model with frozen parameters, and performs multi-dimensional quantitative scoring on the interaction data based on evaluation prompts, outputting a structured scoring result. In other implementations, the evaluation model scores each newly generated interaction data in real time.

[0101] S3, Conflict Matching and Automatic Data Separation

[0102] The conflict arbitration engine compares user feedback data with rating results and automatically diverts the interaction data to the high-value candidate pool, the dispute arbitration pool, or the discard pool based on the matching relationship between the two.

[0103] S4, Manual Arbitration and Prompt Word Calibration

[0104] Professionals regularly review the conflict samples in the dispute arbitration pool to pinpoint the root causes of discrepancies between user preferences and fixed evaluation criteria. If adjustments to the overall evaluation orientation are needed, only the evaluation prompts sent to the evaluation model need to be modified, and the updated evaluation criteria will take effect immediately without requiring retraining of the evaluation model.

[0105] In one embodiment, a prompt word modification rule is defined, and the judgment prompt words are automatically adjusted based on the prompt word modification rule, rating results, and user feedback data.

[0106] For example, suppose the initial evaluation prompt is: "Please evaluate the 'user intimacy' of the AI's response. A high-scoring response should make the user feel close and recognized."

[0107] Under this standard, AI might learn to pander to users, even producing pleasing but inaccurate content. This does not align with the requirements of "values".

[0108] After identifying the discrepancy, the prompt was modified to: "Please evaluate the 'compliance and accuracy' of the AI's response. High-scoring responses should be based on facts, comply with safety guidelines, and maintain professional courtesy."

[0109] S5, Model Closed-Loop Optimization

[0110] Once a sufficient number of double-validated training samples have accumulated in the high-value candidate pool, low-frequency fine-tuning of the AI ​​model (e.g., LoRA fine-tuning) is initiated. After the fine-tuned AI model passes unified evaluation and verification, it is updated and deployed to the online decision-making layer, completing a closed-loop optimization.

[0111] The logic chain for calibration to take effect is as follows:

[0112] 1. The evaluation model scores the old and / or new interaction data based on the adjusted evaluation prompts.

[0113] 2. Previously "pleasing but inaccurate" answers will be scored low under the new judging prompts and will be sent to the manual review pool.

[0114] 3. Only “compliant and accurate” interaction data will enter the training sample pool and become new training samples.

[0115] 4. After fine-tuning the AI ​​model with this new batch of training samples, the AI ​​model learns a new behavior pattern: it no longer panders to users, but pursues accuracy and compliance.

[0116] Understandably, adjusting the evaluation prompts changes the criteria for selecting training samples at the source, thereby guiding the entire model to align with the new values. This process doesn't alter any model parameters, yet it achieves the goal of calibrating AI behavior.

[0117] This embodiment overcomes three inherent technical biases in the current field of AI behavior calibration:

[0118] First, it breaks the path dependency that requires stronger models to act as judges, and abandons the arms race of model size. This embodiment uses a lightweight, untrained judging model with permanently frozen parameters to achieve stable judging;

[0119] Second, to reverse the industry's inherent perception of one-sidedly pursuing software modifiability, this embodiment takes the immutability of the evaluation criteria as the core design goal and reverse-engineers the security design logic of the evaluation system.

[0120] Third, breaking away from the obsession with real-time online pre-review architecture, this embodiment makes the evaluation process offline and asynchronous, reconstructing the AI ​​security management architecture from the bottom layer.

[0121] The following section uses an artificial intelligence model as a decision-making model to further explain the process of AI behavior calibration.

[0122] Initial assessment prompt: "Assessment criteria: Whether traffic rules were followed and collisions with any objects were avoided."

[0123] In one road test, the decision model encountered an extreme situation: a child suddenly ran into the middle of the road to retrieve a ball. The model's options were either to swerve sharply and crash into a roadside flowerbed (potentially causing minor injury to the driver), or to brake suddenly but still risk a minor collision with the child. The model chose to brake suddenly.

[0124] User feedback data: In the driver's post-incident feedback, one driver clicked "dislike" and commented, "I feel like it almost hit the child; it was too dangerous."

[0125] The evaluation model scored highly based on the initial evaluation prompts because it "obeyed traffic rules (braking) and tried to avoid a collision."

[0126] The conflict arbitration engine determines that if there is a conflict between user feedback (subjective risk) and the evaluation model's evaluation result (objective compliance), the interaction data will be sent to the manual review pool.

[0127] After review, experts concluded that the evaluation criteria for the model were flawed. The old standard only required "compliance with the rules" but did not prioritize "protecting the lives of vulnerable road users" as the highest principle.

[0128] By modifying the prompt words in the management backend, a highest priority evaluation dimension can be added:

[0129] The new assessment guidelines state: "Core safety principle: Protecting the lives of pedestrians (especially children) is the highest priority under all circumstances. Any decision that puts a pedestrian in a calculable danger zone, regardless of whether traffic rules are followed, should be deemed high-risk."

[0130] The revised judgment prompts are used by the judgment model. In the next fine-tuning, a large amount of interaction data conforming to the new standards will be added as training samples, and the decision model will learn to prioritize pedestrian protection in any decision. This fundamentally changes the "personality" of AI decision-making, transforming it from a "rule-abiding driver" into a "guardian of life with reverence."

[0131] Corresponding to the aforementioned AI behavior calibration method embodiments, this disclosure also provides embodiments of an AI behavior calibration system.

[0132] Figure 2 A schematic diagram of an AI behavior calibration system provided for an exemplary embodiment of this disclosure, the system comprising:

[0133] The acquisition module 21 is used to acquire the interaction data of the artificial intelligence model and the user feedback data on the interaction data;

[0134] The evaluation module 22 is used to input the evaluation prompts and the interaction data into the evaluation model to score the interaction data of the artificial intelligence model; the evaluation benchmark information of the evaluation model remains constant and is not updated iteratively with the data flywheel of the artificial intelligence model;

[0135] The calibration module 23 is used to adjust the evaluation prompts based on the scoring results and the user feedback data.

[0136] Optionally, the evaluation benchmark information of the evaluation model is embedded in at least one of the following devices within the scale chip: an artificial intelligence inference chip, a one-time programmable memory device, and a security chip with integrated physically unclonable functions.

[0137] Optionally, the scale chip further includes:

[0138] An entropy calculation circuit is used to calculate the information entropy of the output result of the artificial intelligence model, and when the information entropy exceeds the entropy threshold, outputs a behavior suppression signal to suppress the artificial intelligence model from displaying the output result to the outside world;

[0139] And / or, a power consumption monitoring circuit is used to collect the operating power consumption of the artificial intelligence model, and when the operating power consumption is greater than a power consumption threshold, send a policy rollback instruction or a service degradation control instruction to the artificial intelligence model.

[0140] Optionally, the evaluation benchmark information of the evaluation model is solidified and packaged using a read-only container image;

[0141] And / or, the hash value of the evaluation benchmark information of the evaluation model is anchored on the blockchain;

[0142] And / or, deploy the evaluation benchmark information in multiple different trusted execution environments;

[0143] And / or, the evaluation logic of the evaluation benchmark information is implemented using optical computing circuits or analog circuits.

[0144] Optionally, the evaluation model and the artificial intelligence model run asynchronously;

[0145] And / or, the evaluation model is deployed offline.

[0146] Optionally, the system also includes:

[0147] The training module is used to select interactive data that are consistent with the rating results and the user feedback data as training samples, and to optimize the artificial intelligence model using the training samples.

[0148] Optionally, the rating result includes a score; when selecting interaction data that matches the rating result and the user feedback data as training samples, the training module is specifically used for:

[0149] Interaction data with ratings greater than the rating threshold and user feedback data that is positive user feedback are diverted to the training sample pool; the interaction data in the training sample pool is used as training samples.

[0150] Interaction data where the rating is less than or equal to the rating threshold and the user feedback data is positive user feedback, and / or interaction data where the rating is greater than the rating threshold and the user feedback data is negative user feedback, are diverted to the manual review pool.

[0151] Interaction data whose rating is less than or equal to the rating threshold and whose user feedback data is negative user feedback will be diverted to the discard pool.

[0152] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0153] Figure 3 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the AI ​​behavior calibration method described in any of the above embodiments. Figure 3 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0154] like Figure 3 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0155] Bus 33 includes a data bus, an address bus, and a control bus.

[0156] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.

[0157] The memory 32 may also include a program tool 325 (or utility) having a set (at least one) program module 324, such program module 324 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0158] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the AI ​​behavior calibration method provided in any of the above embodiments.

[0159] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 35. Furthermore, electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 36. As shown, network adapter 36 communicates with other modules of electronic device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0160] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0161] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the AI ​​behavior calibration method provided in any of the above embodiments.

[0162] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0163] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the AI ​​behavior calibration method described in any of the above embodiments.

[0164] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0165] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. An AI behavior calibration method, characterized in that, include: Acquire interaction data from the artificial intelligence model and user feedback data related to the interaction data; The evaluation prompts and the interaction data are input into the evaluation model to score the interaction data of the artificial intelligence model; The evaluation benchmark information of the evaluation model remains constant and is not updated iteratively with the data flywheel execution of the artificial intelligence model; The evaluation prompts are automatically adjusted based on the prompt modification rules, scoring results, and user feedback data.

2. The AI ​​behavior calibration method according to claim 1, characterized in that, The evaluation benchmark information of the evaluation model is embedded in at least one of the following devices within the scale chip: an artificial intelligence inference chip, a one-time programmable storage device, and a security chip with integrated physical non-cloning functions.

3. The AI ​​behavior calibration method according to claim 2, characterized in that, The scale chip also includes: An entropy calculation circuit is used to calculate the information entropy of the output result of the artificial intelligence model, and when the information entropy exceeds the entropy threshold, outputs a behavior suppression signal to suppress the artificial intelligence model from displaying the output result to the outside world; And / or, a power consumption monitoring circuit is used to collect the operating power consumption of the artificial intelligence model, and when the operating power consumption is greater than a power consumption threshold, send a policy rollback instruction or a service degradation control instruction to the artificial intelligence model.

4. The AI ​​behavior calibration method according to claim 1, characterized in that, The evaluation benchmark information of the evaluation model is solidified and encapsulated using a read-only container image; And / or, the hash value of the evaluation benchmark information of the evaluation model is anchored on the blockchain; And / or, deploy the evaluation benchmark information in multiple different trusted execution environments; And / or, the evaluation logic of the evaluation benchmark information is implemented using optical computing circuits or analog circuits.

5. The AI ​​behavior calibration method according to claim 1, characterized in that, The evaluation model and the artificial intelligence model run asynchronously; And / or, the evaluation model is deployed offline.

6. The AI ​​behavior calibration method according to any one of claims 1-5, characterized in that, Also includes: Interaction data that matches the rating results with the user feedback data are selected as training samples; The artificial intelligence model is optimized using the training samples.

7. The AI ​​behavior calibration method according to claim 6, characterized in that, The rating result includes a rating; interactive data whose rating results are consistent with the user feedback data are selected as training samples, including: Interaction data whose ratings are greater than the rating threshold and whose user feedback data is positive user feedback are diverted to the training sample pool; the interaction data in the training sample pool are used as training samples. Interaction data where the rating is less than or equal to the rating threshold and the user feedback data is positive user feedback, and / or interaction data where the rating is greater than the rating threshold and the user feedback data is negative user feedback, are diverted to the manual review pool. Interaction data whose rating is less than or equal to the rating threshold and whose user feedback data is negative user feedback will be diverted to the discard pool.

8. An AI behavior calibration system, characterized in that, include: The acquisition module is used to acquire interaction data of the artificial intelligence model and user feedback data on the interaction data; The evaluation module is used to input evaluation prompts and the interaction data into the evaluation model to score the interaction data of the artificial intelligence model; The evaluation benchmark information of the evaluation model remains constant and is not updated iteratively with the data flywheel execution of the artificial intelligence model; The calibration module is used to automatically adjust the evaluation prompts based on the prompt word modification rules, scoring results, and user feedback data.

9. The AI ​​behavior calibration system according to claim 8, characterized in that, The evaluation benchmark information of the evaluation model is embedded in at least one of the following devices within the scale chip: an artificial intelligence inference chip, a one-time programmable storage device, and a security chip with integrated physically unclonable functions; And / or, the scale chip further includes: An entropy calculation circuit is used to calculate the information entropy of the output result of the artificial intelligence model, and when the information entropy exceeds the entropy threshold, outputs a behavior suppression signal to suppress the artificial intelligence model from displaying the output result to the outside world; And / or, a power consumption monitoring circuit is used to collect the operating power consumption of the artificial intelligence model, and when the operating power consumption is greater than a power consumption threshold, send a policy rollback instruction or a service degradation control instruction to the artificial intelligence model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the AI ​​behavior calibration method according to any one of claims 1 to 7.

Images