Embodied intelligent edge unit self-programming evolution method and system based on ai agent

Abstract

The application relates to an AI Agent-based embodied intelligent edge unit self-programming evolution method and system, which comprises deploying an intelligent control unit RoBOX on the device edge side, and internally dividing into a real-time control domain and an intelligent evolution domain; an AI Agent is run in the intelligent evolution domain, RGB and depth images are acquired through a controlled interface, a scene description is generated by using a visual language model, objects are located and distance and yaw angle are calculated in combination with a target detection model; strategy instructions are output through reasoning of a large language model thinking chain and a majority voting output strategy, and self-programming logic such as automatically generated control code, parameters or interaction strategies is generated; after consistency verification and shadow mode safety verification of the self-programming logic, the self-programming logic is effectively applied to the real-time control domain through atomic switching without interruption, and is automatically rolled back in an abnormal state; meanwhile, running data is collected to generate feedback, and continuous evolution is driven.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and system for the self-programming evolution of embodied intelligent edge units based on AI Agents. Background Technology

[0002] With the widespread application of embodied intelligent devices (such as drones and robots) in complex environments, the requirements for intelligent decision-making, autonomous adaptation, and remote low-latency control capabilities are continuously increasing. Traditional embodied intelligent systems mostly use pre-configured control logic or fixed functional modules, and their intelligent capabilities are limited in the design phase, making it difficult to continuously evolve according to the actual operating environment and task requirements after the device is deployed.

[0003] Currently, the continuous evolution of tasks is mainly achieved through remote intelligent control using large cloud-based models. In this method, embodied intelligent devices upload collected sensor data to a cloud server via a network. A large language model or visual language model deployed in the cloud processes and infers the data, generating control commands which are then sent to the device for execution. For example, in a drone inspection mission, the device captures images and uploads them to the cloud. The cloud model identifies the target and plans a path, then sends control commands back to the drone. This leverages the powerful computing capabilities of the cloud to achieve relatively complex intelligent decision-making and has already seen initial applications in real-world scenarios.

[0004] Regarding the aforementioned technologies, in scenarios where the network is unstable or disconnected, the performance of locally deployed quantization models degrades and lacks verification fallback, resulting in insufficient system robustness and an inability to guarantee the security and reliability of real-time control.

[0005] Based on this, this application provides a method and system for the self-programming evolution of embodied intelligent edge units based on AI Agent. Summary of the Invention

[0006] To address the issues of performance degradation and lack of verification fallback in locally deployed quantization models under unstable or disconnected network conditions, resulting in insufficient system robustness and inability to guarantee the security and reliability of real-time control, this application provides a self-programming evolution method and system for embodied intelligent edge units based on AI Agents.

[0007] Firstly, this application provides a self-programming evolution method for embodied intelligent edge units based on AI Agents, employing the following technical solution: a RoBOX deployed at the edge of an intelligent device, wherein the RoBOX comprises a logically isolated real-time control domain and an intelligent evolution domain, including: Launch an AI Agent instance in the intelligent evolution domain and bind it to the basic remote control module in the real-time control domain through a controlled interface; the AI ​​Agent is loaded with a visual language model (VLM), a large language model (LLM), an object detection model, and a device API library; RGB and depth images are acquired from the real-time control domain through a controlled interface, a scene description is generated using VLM, and detection information of the target object is generated based on the target detection model; based on the depth image and the detection information, the relative distance and yaw angle between the smart device and the target object are calculated. The scene description, detection information, relative distance, yaw angle, and natural language task instructions are fused into a multidimensional prompt input LLM; the LLM outputs structured policy instructions through thought chain reasoning and majority voting after multiple independent reasonings; the AI ​​Agent generates self-programmed logic based on the policy instructions through the LLM, and the self-programmed logic includes control code, control parameters, control flow, or interaction strategy; The self-programmed logic is stored in the temporary storage area of ​​the intelligent evolution domain. After RoBOX consistency and security verification, the self-programmed logic is applied to the real-time control domain through an atomic switching operation. If an operational abnormality is detected after verification, RoBOX automatically reverts to the control logic of the previous stable version. RoBOX collects data on device operating status, task completion quality, and AI Agent behavior evaluation after the self-programmed logic takes effect, generating feedback information to guide subsequent self-programming evolution.

[0008] Preferably, the step of acquiring RGB and depth images from the real-time control domain through a controlled interface and generating a scene description using VLM includes: The AI ​​Agent sends rotation commands to the real-time control domain through a controlled interface, controlling the smart device to rotate sequentially to multiple preset angles. At each rotation angle, the AI ​​Agent collects RGB and depth images of the current viewpoint through the controlled interface and stores the collected image pairs in association with the corresponding angles. An RGB image from the same viewpoint is input into a VLM, and a preset structured prompt is also input into the VLM. The structured prompt includes output format requirements, requiring the VLM to output the scene description in a preset spatial order. The VLM performs multiple inferences on the same RGB image to generate multiple scene description candidates. The AI ​​Agent performs semantic matching between multiple scene description candidates and the current natural language task instructions, and selects the scene description candidate with the highest matching degree as the scene description from the corresponding perspective. If the scene description contains a target object that matches the task instruction or a candidate object with matching attributes, the rotation is terminated and the depth image corresponding to that viewpoint is used as the current depth image; otherwise, the acquisition of the next angle continues. If no target object or candidate object is matched after all preset angles have been collected, the AI ​​Agent will summarize the scene descriptions of each viewpoint and input them into the LLM. The LLM will then infer the associated viewpoint with the highest semantic correlation to the task instruction and control the smart device to rotate to the associated orientation corresponding to the associated viewpoint, and use the depth image of the associated viewpoint as the current depth image.

[0009] Preferably, the step of generating target object detection information based on the target detection model; and calculating the relative distance and yaw angle between the smart device and the target object based on the depth image and the detection information, includes: The RGB image acquired from the current viewpoint and the natural language task instructions are simultaneously input into the object detection model. The object detection model outputs the bounding box of at least one target object and its corresponding confidence score. The bounding box includes the coordinates of the center point, width, and height of the bounding box. From the current depth image, extract the depth value of the pixel corresponding to the center point coordinate of the bounding box, as the first distance between the smart device and the target object; extract the depth value of the corresponding position of the same pixel in the planar depth image, as the second distance from the smart device to the phase plane; Based on the ratio of the first distance to the second distance, the yaw angle between the smart device and the target object is calculated using the inverse cosine function, where the yaw angle represents the horizontal deflection angle of the target object relative to the current orientation of the smart device. When the target detection model outputs bounding boxes of multiple target objects, the target object with the highest confidence is selected as the main target, and its corresponding first distance, second distance and yaw angle are used as the effective navigation parameters of the current viewpoint; if the confidence is lower than a preset threshold, the bounding box, the first distance and yaw angle are marked as unverified.

[0010] Preferably, after marking the bounding box, the first distance, and the yaw angle as pending verification if the confidence level is lower than a preset threshold, the method further includes: The RGB image region corresponding to the bounding box is cropped and input into the VLM, which generates a detailed object description of the RGB image region. At the same time, the detailed object description and the natural language task instructions are input into the LLM, which performs attribute matching and reasoning to determine whether it is the target object. If the LLM determines that an object within the RGB image region is the target object or a candidate object with matching attributes, the pending verification state is lifted, and the bounding box, the first distance, and the yaw angle are converted into valid navigation parameters. If the LLM determines that the object is not the target object, the corresponding detection result is discarded and a rotation acquisition step is triggered to find other perspectives; If the LLM cannot determine the result clearly, then based on the first distance and yaw angle, the intelligent device is controlled to move forward a preset trial distance in the current direction, and then the image is re-acquired to obtain a new confidence level; if the new confidence level is higher than the preset threshold, the new detection result is adopted; otherwise, the target is discarded and the rotation acquisition step is triggered.

[0011] Preferably, the scene description, detection information, relative distance, yaw angle, and natural language task instructions are fused into a multidimensional prompt input LLM; the LLM, after thought chain reasoning and majority voting following multiple independent reasoning, outputs structured policy instructions, including: The scene description, detection information, relative distance, yaw angle, and natural language task instructions are fused together according to a preset template to construct a multidimensional prompt. The multidimensional prompt also includes output format requirements, requiring the LLM output to include inference steps and structured results. The structured result is a data set containing four fields, which represent the matching type, relative distance, yaw angle, and confidence level, respectively. Among them, the matching type can be a first value, a second value, or a third value. The first value indicates an exact match, the second value indicates an attribute inference match, and the third value indicates no match. The multidimensional prompts are input into the LLM, which identifies key attributes in the natural language task instructions, compares them item by item with the objects in the scene description, and determines whether there is an exact match or an attribute similarity match based on the detection information. For the same multidimensional prompt, perform independent LLM reasoning and thought chain generation several times. Each reasoning is sampled independently, and each reasoning outputs a complete structured result and its corresponding reasoning chain. Collect the structured results of several inference outputs, count the number of times the matching type values ​​are the same, and select the matching type with the highest frequency and its corresponding relative distance, yaw angle and confidence as the policy instruction; If multiple matching types receive the same number of votes, the final result is selected according to the rule that the first value takes precedence over the second value, and the second value takes precedence over the third value, thus obtaining the strategy instruction.

[0012] Preferably, the AI ​​Agent generates self-programmable logic based on the policy instructions using an LLM. The self-programmable logic includes control code, control parameters, control flow, or interaction strategies, including: The matching type, relative distance, yaw angle, and confidence level in the policy instruction are combined with a predefined device API library to construct code generation prompts. The code generation prompts include the function signatures, function descriptions, and call examples of the API library. Input the code generation prompt into LLM, and LLM will generate an executable Python policy code snippet; the policy code snippet only allows calls to functions in the predefined API library and basic functions in the standard Python library, and prohibits calls to undefined external functions; When the matching type in the strategy instruction is a first value or a second value, and the confidence level is lower than the preset high confidence threshold, parameter adjustment suggestions are generated through LLM. The parameter adjustment suggestions include modifying the motion speed threshold, steering sensitivity, and target approach distance threshold in the device control parameter table. After the parameter adjustment suggestions are verified for consistency, they are written to the parameter storage area of ​​the real-time control domain through the controlled interface. By performing semantic analysis on historically successfully executed control code fragments using LLM, reusable action sequence patterns are identified. Multiple basic control actions are combined into composite functions according to temporal logic, and these composite functions are registered in the device API library as newly added callable functions. The composite functions include sequential execution, conditional branching, or loop control structures. When the matching type in the policy instruction is a third value, an interactive policy update suggestion is generated through LLM, which modifies the triggering conditions of abnormal alarms, adjusts the presentation format or content of natural language feedback information, and updates the status reporting frequency in the communication protocol with the remote operator RoCA; the updated interactive policy takes effect through the controlled interface after verification.

[0013] Preferably, the step of storing the self-programmed logic in the temporary storage area of ​​the intelligent evolution domain, and after RoBOX consistency verification and security verification, applying the self-programmed logic to the real-time control domain through an atomic switching operation, further includes: The self-programming logic is stored in a versioned manner in the temporary storage area of ​​the intelligent evolution domain, and each version is assigned a unique version identifier and timestamp. RoBOX performs consistency checks on the self-programmed logic in the staging area: it calls the locally deployed audit language model to perform static analysis on the control code to detect whether there are undefined API calls, type mismatches, syntax errors or logic vulnerabilities; if it is a control parameter, it checks whether the parameter value is within the predefined legal value range; if it is a control flow or interaction strategy, it checks whether it conforms to the preset syntax specifications. After the consistency check passes, a safety check is performed using RoBOX: the self-programmed logic is run in shadow mode in an isolated simulation environment, and the deviation between its output actions and the output actions of the current stable version is compared; at the same time, it is checked whether the self-programmed logic violates the preset safety constraint set, which includes the upper limit of device movement speed, the boundary of the restricted space, and the upper limit of execution time. If the consistency check or security check fails, RoBOX discards the self-programmed logic in the current temporary storage area, maintains the stable version in the current real-time control domain, and generates a check failure report. If both the consistency check and the security check pass, RoBOX performs an atomic switch operation: it marks the currently running control logic version in the real-time control domain as to be decommissioned, and switches the control logic version in the temporary storage area to the executable state of the real-time control domain in one go through pointer redirection or atomic memory swap; the control thread of the real-time control domain is not interrupted during the switch, and the response delay of the basic remote control command does not increase by more than one control cycle. After the atomic switch is completed, RoBOX will retain the retired control logic version in the history storage area.

[0014] Secondly, this application discloses an AI Agent-based embodied intelligent edge unit self-programming evolution device, employing the following technical solution: deployed on the edge side of an intelligent device in a RoBOX, the RoBOX comprising a logically isolated real-time control domain and an intelligent evolution domain, the device comprising: The intelligent evolution module is used to launch AI Agent instances in the intelligent evolution domain and interact with and bind them to the basic remote control module in the real-time control domain through a controlled interface; the AI ​​Agent is loaded with a visual language model (VLM), a large language model (LLM), an object detection model, and a device API library; The target detection module is used to acquire RGB images and depth images from the real-time control domain through a controlled interface, generate scene descriptions using VLM, and generate detection information of target objects based on the target detection model; and calculate the relative distance and yaw angle between the smart device and the target object based on the depth image and the detection information. The programming strategy module is used to integrate the scene description, the detection information, the relative distance, the yaw angle, and the natural language task instructions into a multidimensional prompt input LLM; the LLM outputs structured strategy instructions after thinking chain reasoning and majority voting after multiple independent reasonings; the AI ​​Agent generates self-programming logic based on the strategy instructions through the LLM, and the self-programming logic includes control code, control parameters, control flow, or interaction strategy; The real-time verification module stores the self-programmed logic in the temporary storage area of ​​the intelligent evolution domain. After RoBOX consistency verification and security verification, the self-programmed logic is applied to the real-time control domain through an atomic switching operation. If an operational abnormality is detected after verification, RoBOX automatically reverts to the control logic of the previous stable version. The information feedback module is used to collect data on device operating status, task completion quality, and AI Agent behavior evaluation after the self-programmed logic takes effect through RoBOX, generate feedback information, and guide subsequent self-programming evolution.

[0015] Thirdly, this application also provides a control device, the device comprising: It includes a memory and a processor, wherein the memory stores computer programs that can be loaded and executed by the processor, such as the above-described AI Agent-based embodied intelligent edge unit self-programming evolution method.

[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above in the AI ​​Agent-based embodied intelligent edge unit self-programming evolution method.

[0017] In summary, this application deploys a RoBOX unit comprising a real-time control domain and an intelligent evolution domain at the device edge. An AI Agent runs in the intelligent evolution domain, interacting isolated from the real-time control domain through a controlled interface. The Agent drives the device to rotate and acquire multi-angle RGB and depth images, generates a structured scene description using a visual language model, and calls an open-vocabulary object detection model to locate target objects, calculating relative distance and yaw angle based on the depth images. When the detection confidence is low, the Agent performs attribute reasoning or trial movement verification using a large language model. Subsequently, the scene description, detection information, distance, yaw angle, and natural language commands are fused into a multi-dimensional prompt, input into the large language model to perform thought chain reasoning and majority voting after multiple independent reasonings, outputting a policy command. The Agent generates self-programmed logic based on this command through the large language model, stores it in a versioned manner, and then RoBOX performs consistency and security checks. If successful, it atomically switches to the real-time control domain; otherwise, it automatically rolls back. The entire evolution process is isolated from the real-time control domain in terms of resources and timing, ensuring uninterrupted remote low-latency control. This enables local autonomy, secure isolation, and continuous evolution of edge intelligence, significantly reducing reliance on the cloud and improving the real-time performance and robustness of remote control. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a self-programming evolution method for embodied intelligent edge units based on AI Agents.

[0019] Figure 2 This is a structural block diagram of an embodied intelligent edge unit self-programming evolution device based on an AI Agent. Detailed Implementation

[0020] The following combination Figures 1-2 This application will be described in further detail.

[0021] In this application, the executing entity is the RoBOX intelligent control unit deployed at the edge of the embodied intelligent device, which is the control system of AIAgen. RoBOX is internally divided into a real-time control domain and an intelligent evolution domain: the former is responsible for maintaining a low-latency control channel with the remote client; the latter runs an AI Agent instance, loading a visual language model, a large language model, and a target detection model. The AI ​​Agent acquires sensor data through a controlled interface, completes environmental perception, target localization, and distance calculation, and then autonomously generates control logic code using the code generation capabilities of the large language model. After security verification, it atomically switches to the real-time control domain for execution, while simultaneously collecting feedback data to drive continuous evolution.

[0022] Reference Figure 1 The embodiments of this application include at least steps S10 to S50.

[0023] S10: Launch an AI Agent instance in the intelligent evolution domain and bind it to the basic remote control module in the real-time control domain through a controlled interface; the AI ​​Agent is loaded with a visual language model (VLM), a large language model (LLM), an object detection model, and a device API library.

[0024] S20 acquires RGB and depth images from the real-time control domain through a controlled interface, generates scene descriptions using VLM, and generates target object detection information based on the target detection model; based on the depth image and detection information, it calculates the relative distance and yaw angle between the smart device and the target object.

[0025] S30 integrates scene description, detection information, relative distance, yaw angle, and natural language task instructions into a multi-dimensional prompt input LLM; the LLM outputs structured policy instructions through thought chain reasoning and majority voting after multiple independent reasonings; the AIAgent generates self-programmed logic based on the policy instructions through the LLM, and the self-programmed logic includes control code, control parameters, control flow, or interaction strategy.

[0026] S40 stores the self-programmed logic in the temporary storage area of ​​the intelligent evolution domain. After RoBOX consistency and security verification, the self-programmed logic is applied to the real-time control domain through an atomic switching operation. If an operational abnormality is detected after verification, RoBOX automatically reverts to the control logic of the previous stable version.

[0027] The S50 uses RoBOX to collect data on device operating status, task completion quality, and AIAgent behavior evaluation after the self-programmed logic takes effect, generating feedback information to guide subsequent self-programming evolution.

[0028] Meanwhile, the control intelligent evolution domain and the real-time control domain are kept isolated in terms of computing resources and execution timing, so that the real-time control domain executes basic control instructions at fixed cycles.

[0029] Specifically, an isolated intelligent evolution domain is built at the device edge. The AI ​​Agent securely acquires sensor data through controlled interfaces, uses visual and language models to understand the environment, and calculates the target's location. After reasoning through thought chains and majority voting, the Agent automatically generates self-programmed logic such as control code or parameters. After consistency verification and shadow mode testing, it atomically switches to the real-time control domain, automatically reverting in case of anomalies. This achieves secure isolation between the autonomous evolution of edge intelligence and real-time control, reduces cloud dependence, and ensures the robustness of remote control.

[0030] Next, after the self-programmed logic takes effect, RoBOX collects data on device operating status, task completion quality, and AI Agent behavior evaluation. This data is then stored in a structured manner in chronological order as feedback records. Local calculations are performed on evaluation metrics such as success rate, path efficiency weighted success rate, and confidence-success rate correlation. Based on historical feedback, the AI ​​Agent adjusts subsequent strategy priorities, increases the number of LLM inferences, or optimizes the verification process. Simultaneously, the data can be encrypted and not synchronized to the cloud in real time for auditing and model fine-tuning.

[0031] In step S50, the statistical correlation between the AI ​​Agent output confidence level and the actual task success rate is calculated, using the following formula: ; in, The Pearson correlation coefficient ranges from -1 to 1, with positive values ​​indicating a positive correlation and negative values ​​indicating a negative correlation; N is the total number of task instances participating in the statistics. Output the confidence level of the policy instructions for the AI ​​Agent in the i-th task; The arithmetic mean of the confidence scores for all tasks; This is a binary flag indicating whether the i-th task was successful (1 for success, 0 for failure). The numerator is the arithmetic mean of the success rates. The covariance of confidence level and success rate reflects the trend of their joint change; The denominator is... The product of the confidence level standard deviation and the success rate standard deviation is used for normalization. The correlation coefficient can be used to assess the self-calibration capability of the AI ​​agent. If the value is too low, the LLM inference parameters or prompt template need to be adjusted.

[0032] In some embodiments, step S20 specifically includes the following steps: The AI ​​Agent sends a rotation command to the real-time control domain through a controlled interface, controlling the smart device to rotate sequentially to multiple preset angles. At each rotation angle, the AI ​​Agent acquires the RGB image and depth image of the current viewpoint through the controlled interface, and stores the acquired image pairs associating them with the corresponding angles. The AI ​​Agent inputs the RGB image from the same viewpoint into the VLM, and simultaneously inputs a preset structured prompt into the VLM. The structured prompt includes output format requirements, requiring the VLM to output scene descriptions according to a preset spatial order. The VLM performs multiple inferences on the same RGB image to generate multiple scene description candidates. The AI ​​Agent performs semantic matching between multiple scene description candidates and the current natural language task instruction, selecting the scene description candidate with the highest matching degree as the scene description under the corresponding viewpoint. If the scene description contains a target object that matches the task instruction or a candidate object with matching attributes, the rotation is terminated, and the depth image corresponding to that viewpoint is used as the current depth image; otherwise, the acquisition of the next angle continues. If no target object or candidate object is matched after all preset angles have been acquired, the AI ​​Agent summarizes the scene descriptions of each viewpoint and inputs them into the LLM. The LLM infers the associated viewpoint with the highest semantic correlation with the task instruction and controls the intelligent device to rotate to the associated orientation corresponding to the associated viewpoint, using the depth image of the associated viewpoint as the current depth image.

[0033] During rotational acquisition, the same target object may be detected from multiple viewpoints. It is necessary to fuse the confidence scores from different viewpoints to improve reliability. The specific formula is as follows: ; in, The confidence level after fusion is [0, 1]; M is the total number of viewpoints that detected the same target object, and M is an integer greater than 1. Let be the confidence score output by the target detection model from the j-th viewpoint; This represents the probability of detection failure at the j-th viewpoint. This fusion formula assumes independent detection at each viewpoint, effectively utilizing multi-view information to improve detection reliability in low-confidence scenarios.

[0034] Specifically, a scene description is generated using a Visual Model (VLM) in a specified spatial order, and the same image is inferred multiple times to improve stability. Semantic matching of each angle description with natural language instructions is performed. Once a target or candidate object is detected, rotation stops and the current depth image is locked. If no match is found after a full rotation, the LLM synthesizes all descriptions to infer the most relevant angle, driving the device to turn to that orientation and use the corresponding depth image. This enables active viewpoint selection and target orientation locking in unknown environments.

[0035] Furthermore, step S20 also includes the following steps: simultaneously inputting the RGB image acquired from the current viewpoint and the natural language task instruction into the object detection model, the object detection model outputs the bounding box of at least one target object and its corresponding confidence score, the bounding box including the coordinates of the center point of the bounding box, width and height; extracting the depth value of the pixel corresponding to the center point coordinates of the bounding box from the current depth image, as the first distance between the smart device and the target object; extracting the depth value of the corresponding position of the same pixel in the planar depth image, as the second distance from the smart device to the phase plane; calculating the yaw angle between the smart device and the target object using the inverse cosine function based on the ratio of the first distance to the second distance, where the yaw angle represents the horizontal deflection angle of the target object relative to the current orientation of the smart device; when the object detection model outputs the bounding boxes of multiple target objects, selecting the target object with the highest confidence score as the main target, and using its corresponding first distance, second distance and yaw angle as the effective navigation parameters of the current viewpoint; if the confidence score is lower than a preset threshold, marking the bounding box, first distance and yaw angle as unverified.

[0036] Let the first distance between the smart device and the target object be . The second distance from the smart device to the phase plane is The formula for the yaw angle is: ; in, The yaw angle, expressed in degrees, represents the horizontal deflection angle of the target object relative to the current orientation of the smart device. The second distance is the vertical distance from the smart device to the camera plane (imaging plane), and the unit is the same as that of the depth image. The first distance is the straight-line distance from the smart device to the target object, and the unit is the same as that of the depth image.

[0037] Specifically, the system uses an object detection model to locate target objects from RGB images, outputting bounding boxes and confidence scores. Combining this with a depth image, the depth value of the box's center point is extracted as the first distance from the device to the object, and the planar depth value is extracted as the second distance to the camera plane. The yaw angle is calculated using an inverse cosine function to determine the target's horizontal orientation relative to the device. When multiple objects are detected, the one with the highest confidence score is prioritized as the primary target, and its navigation parameters are obtained. If the confidence score is below a threshold, the result is marked as pending verification to ensure reliability.

[0038] To further verify the perception results, the corresponding processing steps are as follows: The RGB image region corresponding to the bounding box is cropped and input into the VLM (Visual Learning Model), which generates a detailed object description of the RGB image region. Simultaneously, the detailed object description and the natural language task instruction are input into the LLM (Launch Management Model), which performs attribute matching inference to determine if it is the target object. If the LLM determines that the object within the RGB image region is the target object or a candidate object with matching attributes, the pending verification state is lifted, and the bounding box, first distance, and yaw angle are converted into valid navigation parameters. If the LLM determines that it is not the target object, the corresponding detection result is discarded, and a rotation acquisition step is triggered to find other perspectives. If the LLM cannot clearly determine the result, based on the first distance and yaw angle, the intelligent device is controlled to move forward a preset trial distance along the current direction, and then the image is re-acquired to obtain a new confidence level. If the new confidence level is higher than a preset threshold, the new detection result is adopted; otherwise, the target is discarded, and a rotation acquisition step is triggered.

[0039] Specifically, when the target detection confidence is low, the corresponding image region is cropped and fed into the VLM to generate a detailed description. Then, the LLM combines the description with the task instructions to perform attribute inference: if a match is found, the verification is removed and the navigation parameters are adopted; if a match is not found, the target is discarded and the rotation exploration is restarted; if no determination can be made, the device is controlled to advance along the estimated direction and test a distance before re-detecting. If the new confidence meets the standard, the target is adopted; otherwise, the target is abandoned and the exploration is restarted. This multi-level verification effectively reduces the risk of false detections and improves the reliability of target recognition in complex environments.

[0040] In some embodiments, step S30 specifically includes the following steps: fusing scene description, detection information, relative distance, yaw angle, and natural language task instructions into a multidimensional prompt according to a preset template. The multidimensional prompt also includes output format requirements, requiring the LLM output to include inference steps and structured results. The structured results are a data set containing four fields, which represent the matching type, relative distance, yaw angle, and confidence level, respectively. The matching type takes the value of a first value, a second value, or a third value, where the first value indicates an exact match, the second value indicates an attribute inference match, and the third value indicates no match. The multidimensional prompts are input into the LLM (Linguistic Model). The LLM identifies key attributes in the natural language task instructions, compares them item by item with objects in the scene description, and determines whether there is an exact match or an attribute similarity match based on the detection information. The same multidimensional prompt is repeatedly executed several times for independent LLM inference and thought chain generation. Each inference is sampled independently, and each inference outputs a complete structured result and its corresponding inference chain. The structured results output from several inferences are collected, and the number of times the same matching type value appears is counted. The matching type with the highest frequency and its corresponding relative distance, yaw angle, and confidence score are selected as the policy instruction. If multiple matching types have the same vote, the final result is selected according to the rule that the first value takes precedence over the second value, and the second value takes precedence over the third value, thus obtaining the policy instruction.

[0041] After multiple independent inferences in LLM, traditional majority voting treats each inference result equally. To further improve decision accuracy, a confidence-weighted voting mechanism is introduced. Let the matching type in the structured result output by the k-th inference be... The corresponding confidence level is The formula for weighted voting score is: ; in, is the weighted vote count for match type t; t is the value of the match type, which can be the first value (exact match), the second value (attribute inference match), or the third value (no match); N is the total number of independent inferences, where N is an odd number greater than 3; The confidence level in the output of the kth inference is generated by LLM based on a self-consistency mechanism. As an indicator function, when the matching type of the k-th inference... The value is 1 if it equals the currently calculated type t, and 0 otherwise. This formula calculates the sum of the confidence scores for all inferences under each matching type. The matching type with the highest weighted vote is then selected as the matching type in the final strategy instruction. If multiple types have the same number of votes, the first value takes precedence over the second, and the second value takes precedence over the third. This weighted voting method, compared to simple majority voting, better reflects the influence of high-confidence inference results.

[0042] Specifically, the scenario description, detection information, distance, yaw angle, and task instructions are integrated into a multi-dimensional cue, requiring the LLM output to include an inference chain and structured results. The LLM compares object attributes using the thought chain to determine whether it's an exact match, attribute inference, or no match. To overcome randomness, multiple independent inferences are performed on the same cue, and the results are collected. The most frequent match type and its corresponding parameters are selected by majority voting. If the votes are tied, the final policy instruction is determined by prioritizing exact matches over attribute inferences over no matches, significantly improving the stability of inference and the reliability of decision-making.

[0043] Furthermore, step S30 also includes the following steps: combining the matching type, relative distance, yaw angle, and confidence level in the policy instruction with a predefined device API library to construct a code generation prompt. The code generation prompt includes the function signature, function description, and call examples of the API library. The code generation prompt is input into the LLM, which generates an executable Python policy code snippet. The policy code snippet is only allowed to call functions in the predefined API library and basic functions in the standard Python library; calling undefined external functions is prohibited. When the matching type in the policy instruction is a first or second value, and the confidence level is lower than a preset high confidence threshold, parameter adjustment suggestions are generated through the LLM. These suggestions include modifying the motion speed threshold, steering sensitivity, and target approach distance threshold in the device control parameter table. After consistency verification, the parameter adjustment suggestions are written to the parameter storage area of ​​the real-time control domain through a controlled interface. By performing semantic analysis on historically successfully executed control code fragments using LLM, reusable action sequence patterns are identified. Multiple basic control actions are combined into composite functions according to temporal logic, and these composite functions are registered in the device API library as newly added callable functions. These composite functions include sequential execution, conditional branching, or loop control structures. When the matching type in the policy instruction is a third value, an interactive policy update suggestion is generated through LLM. This suggestion modifies the triggering conditions for abnormal alarms, adjusts the presentation format or content of natural language feedback information, and updates the status reporting frequency in the communication protocol with the remote operator RoCA. The updated interactive policy is then validated and takes effect through the controlled interface.

[0044] Specifically, based on policy instructions and predefined API libraries, code hints are generated, and the LLM generates safe code that only calls legal functions. If the match type is exact or the attribute match is low confidence, parameter adjustment suggestions are generated and written to the real-time control domain. By analyzing historical successful code, basic actions are combined into compound functions with branches or loops and registered as new APIs. When no match is found, an interaction policy update is generated, adjusting alarm conditions, feedback format, or communication frequency, thereby achieving automatic generation of control code, parameters, processes, and interaction policies.

[0045] In some embodiments, step S40 specifically includes the following steps: storing the self-programmed logic in a versioned manner in the temporary storage area of ​​the intelligent evolution domain, and assigning a unique version identifier and timestamp to each version; performing consistency verification on the self-programmed logic in the temporary storage area through RoBOX: calling the locally deployed audit language model to perform static analysis on the control code to detect whether there are undefined API calls, type mismatches, syntax errors or logic vulnerabilities; if it is a control parameter, verifying whether the parameter value is within the predefined legal value range; if it is a control flow or interaction strategy, verifying whether it conforms to the preset syntax specifications; After the consistency check passes, a safety check is performed via RoBOX: the self-programmed logic is run in shadow mode in an isolated simulation environment, and its output actions are compared with the output actions of the current stable version; at the same time, it is checked whether the self-programmed logic violates the preset safety constraint set, which includes the upper limit of device movement speed, the boundary of the restricted space, and the upper limit of execution time; if the consistency check or the safety check fails, RoBOX discards the self-programmed logic in the current temporary storage area, maintains the stable version in the current real-time control domain, and generates a check failure report; If both consistency and security checks pass, RoBOX performs an atomic switch operation: marking the currently running control logic version in the real-time control domain as awaiting retirement, and switching the control logic version in the temporary storage area to the executable state of the real-time control domain in one go through pointer redirection or atomic memory swapping; the control thread of the real-time control domain is not interrupted during the switch, and the response latency of basic remote control commands does not increase by more than one control cycle; after the atomic switch is completed, RoBOX retains the retired control logic version in the historical storage area.

[0046] Specifically, the self-programmed logic is versioned and stored. First, the validity of the code or parameters is analyzed statically using a review model. Then, in shadow mode, the output deviation between the new logic and the stable version is compared, and safety constraints such as speed and restricted areas are verified. If any verification fails, the new logic is discarded and the current stable version is maintained, while a failure report is generated. Once all verifications pass, pointer redirection or atomic memory swapping is used to achieve uninterrupted switching, ensuring that the response latency of the real-time control thread does not exceed one cycle. The retired version is retained in the history area for rollback. This ensures the security and high reliability of intelligent evolution.

[0047] The implementation principle of the self-programming evolution method for an embodied intelligent edge unit based on an AI Agent in this application is as follows: An AI Agent runs in the intelligent evolution domain and interacts with the real-time control domain in isolation through a controlled interface. The Agent drives the device to rotate and acquire multi-angle RGB and depth images, generates a structured scene description using a visual language model, and calls an open-vocabulary object detection model to locate target objects, calculating the relative distance and yaw angle based on the depth images. When the detection confidence is low, the Agent performs attribute reasoning or trial movement verification through a large language model. Subsequently, the scene description, detection information, distance, yaw angle, and natural language commands are fused into a multi-dimensional prompt, which is input into the large language model to perform thought chain reasoning and majority voting after multiple independent reasonings, and outputs a policy command. The Agent generates self-programming logic based on this command through the large language model, stores it in a versioned manner, and performs consistency and security checks by RoBOX. If successful, it is atomically switched to the real-time control domain; otherwise, it automatically rolls back. The entire evolution process is isolated from the real-time control domain in terms of resources and timing, ensuring uninterrupted remote low-latency control. This enables local autonomy, secure isolation, and continuous evolution of edge intelligence, significantly reducing reliance on the cloud and improving the real-time performance and robustness of remote control.

[0048] Figure 1 This is a flowchart illustrating a self-programming evolution method for embodied intelligent edge units based on an AI Agent in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless explicitly stated otherwise, there is no strict order requirement for the execution of these steps, and they can be executed in other orders; and Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0049] Based on the same technical concept, referring to Figure 2 This application also provides an AI Agent-based embodied intelligent edge unit self-programming evolution device, which adopts the following technical solution: a RoBOX deployed at the edge of an intelligent device, wherein the RoBOX includes a logically isolated real-time control domain and an intelligent evolution domain, and the device includes: The intelligent evolution module is used to launch AI Agent instances in the intelligent evolution domain and interact with and bind them to the basic remote control module in the real-time control domain through a controlled interface; the AI ​​Agent is loaded with a visual language model (VLM), a large language model (LLM), an object detection model, and a device API library; The target detection module is used to acquire RGB and depth images from the real-time control domain through a controlled interface, generate scene descriptions using VLM, and generate target object detection information based on the target detection model; based on the depth image and detection information, it calculates the relative distance and yaw angle between the smart device and the target object. The programming strategy module is used to integrate scene description, detection information, relative distance, yaw angle and natural language task instructions into a multidimensional prompt input LLM; the LLM outputs structured strategy instructions after thinking chain reasoning and majority voting after multiple independent reasonings; the AI ​​Agent generates self-programmed logic based on the strategy instructions through the LLM, and the self-programmed logic includes control code, control parameters, control flow or interaction strategy; The real-time verification module stores the self-programmed logic in the temporary storage area of ​​the intelligent evolution domain. After RoBOX consistency verification and security verification, the self-programmed logic is applied to the real-time control domain through an atomic switching operation. If an operational abnormality is detected after verification, RoBOX automatically reverts to the control logic of the previous stable version. The information feedback module is used to collect data on device operating status, task completion quality, and AI Agent behavior evaluation after the self-programmed logic takes effect through RoBOX, generate feedback information, and guide subsequent self-programming evolution.

[0050] In some embodiments, the target detection module is specifically used by the AI ​​Agent to send rotation commands to the real-time control domain through a controlled interface, control the smart device to rotate sequentially to multiple preset angles, and at each rotation angle, acquire RGB images and depth images of the current viewpoint through the controlled interface, and associate and store the acquired image pairs with the corresponding angles. An RGB image from the same viewpoint is input into a VLM, along with a pre-defined structured prompt. The structured prompt includes output format requirements, requiring the VLM to output scene descriptions in a pre-defined spatial order. The VLM performs multiple inferences on the same RGB image to generate multiple scene description candidates. The AI ​​Agent performs semantic matching between multiple scene description candidates and the current natural language task instructions, and selects the scene description candidate with the highest matching degree as the scene description from the corresponding perspective. If the scene description contains a target object that matches the task instruction or a candidate object with matching attributes, the rotation is terminated and the depth image corresponding to that viewpoint is used as the current depth image; otherwise, the acquisition of the next angle continues. If no target or candidate object is found after all preset angles have been collected, the AI ​​Agent will summarize the scene descriptions of each viewpoint and input them into the LLM. The LLM will then infer the associated viewpoint with the highest semantic relevance to the task instruction and control the smart device to rotate to the associated orientation corresponding to the associated viewpoint, using the depth image of the associated viewpoint as the current depth image.

[0051] In some embodiments, the object detection module is specifically used to simultaneously input the RGB image acquired from the current viewpoint and the natural language task instruction into the object detection model. The object detection model outputs the bounding box of at least one object and the corresponding confidence score. The bounding box includes the coordinates of the center point of the bounding box, the width and the height. From the current depth image, extract the depth value of the pixel corresponding to the center point coordinate of the bounding box, as the first distance between the smart device and the target object; extract the depth value of the corresponding position of the same pixel in the planar depth image, as the second distance from the smart device to the phase plane; Based on the ratio of the first distance to the second distance, the yaw angle between the smart device and the target object is calculated using the inverse cosine function, where the yaw angle represents the horizontal deflection angle of the target object relative to the current orientation of the smart device. When the target detection model outputs bounding boxes of multiple target objects, the target object with the highest confidence is selected as the main target, and its corresponding first distance, second distance, and yaw angle are used as the effective navigation parameters of the current viewpoint; if the confidence is lower than the preset threshold, the bounding box, first distance, and yaw angle are marked as unverified.

[0052] In some embodiments, the target detection module is specifically used to crop the RGB image region corresponding to the bounding box and input it into the VLM, which generates a detailed object description of the RGB image region; at the same time, the detailed object description and the natural language task instructions are input into the LLM, which performs attribute matching reasoning to determine whether it is a target object. If the LLM determines that an object within the RGB image region is the target object or a candidate object with matching attributes, the pending verification state is lifted, and the bounding box, first distance, and yaw angle are converted into valid navigation parameters. If the LLM determines that the object is not the target object, the corresponding detection result is discarded and a rotation acquisition step is triggered to find other perspectives; If the LLM cannot determine the result clearly, it controls the smart device to move forward a preset trial distance along the current direction based on the first distance and yaw angle, and then re-acquires the image to obtain a new confidence level; if the new confidence level is higher than the preset threshold, the new detection result is adopted; otherwise, the target is discarded and the rotation acquisition step is triggered.

[0053] In some embodiments, the programming strategy module is specifically used to fuse scene description, detection information, relative distance, yaw angle, and natural language task instructions according to a preset template to construct a multidimensional prompt. The multidimensional prompt also includes output format requirements, requiring the LLM output to include inference steps and structured results. The structured result is a data set containing four fields, which represent the matching type, relative distance, yaw angle, and confidence level, respectively. Among them, the matching type takes the value of a first value, a second value, or a third value, where the first value indicates an exact match, the second value indicates an attribute inference match, and the third value indicates no match. The multidimensional prompts are input into the LLM, which identifies key attributes in the natural language task instructions, compares them item by item with the objects in the scene description, and determines whether there is an exact match or an attribute similarity match based on the detection information. For the same multidimensional prompt, perform independent LLM reasoning and thought chain generation several times. Each reasoning is sampled independently, and each reasoning outputs a complete structured result and its corresponding reasoning chain. Collect the structured results of several inference outputs, count the number of times the matching type values ​​are the same, and select the matching type with the highest frequency and its corresponding relative distance, yaw angle and confidence as the policy instruction; If multiple matching types receive the same number of votes, the final result is selected according to the rule that the first value takes precedence over the second value, and the second value takes precedence over the third value, thus obtaining the strategy instruction.

[0054] In some embodiments, the programming strategy module is specifically used to combine the matching type, relative distance, yaw angle and confidence level in the strategy instruction with a predefined device API library to construct code generation prompts. The code generation prompts include the function signature, function description and call examples of the API library. Input the code generation prompt into LLM, and LLM will generate an executable Python strategy code snippet; the strategy code snippet only allows calls to functions in the predefined API library and basic functions in the standard Python library, and prohibits calls to undefined external functions; When the matching type in the strategy instruction is the first value or the second value, and the confidence level is lower than the preset high confidence threshold, parameter adjustment suggestions are generated through LLM. The parameter adjustment suggestions include modifying the motion speed threshold, steering sensitivity, and target approach distance threshold in the device control parameter table. After the parameter adjustment suggestions are verified for consistency, they are written to the parameter storage area of ​​the real-time control domain through the controlled interface. By performing semantic analysis on historically successfully executed control code fragments using LLM, reusable action sequence patterns are identified. Multiple basic control actions are combined into composite functions according to temporal logic, and these composite functions are registered in the device API library as newly added callable functions. The composite functions include sequential execution, conditional branching, or loop control structures. When the matching type in the policy instruction is the third value, an interactive policy update suggestion is generated through LLM, which modifies the triggering conditions of the abnormal alarm, adjusts the presentation format or content of the natural language feedback information, and updates the status reporting frequency in the communication protocol with the remote operator RoCA; the updated interactive policy takes effect through the controlled interface after verification.

[0055] In some embodiments, the real-time verification module is specifically used to store the self-programmed logic in a versioned manner in the temporary storage area of ​​the intelligent evolution domain, and assign a unique version identifier and timestamp to each version. RoBOX performs consistency checks on the self-programmed logic in the staging area: it calls the locally deployed audit language model to perform static analysis on the control code to detect whether there are undefined API calls, type mismatches, syntax errors or logic vulnerabilities; if it is a control parameter, it checks whether the parameter value is within the predefined legal value range; if it is a control flow or interaction strategy, it checks whether it conforms to the preset syntax specifications. After the consistency check passes, a safety check is performed using RoBOX: the self-programmed logic is run in shadow mode in an isolated simulation environment, and the deviation between its output actions and the output actions of the current stable version is compared; at the same time, it is checked whether the self-programmed logic violates the preset safety constraint set, which includes the upper limit of device movement speed, the boundary of the restricted space, and the upper limit of execution time. If the consistency check or security check fails, RoBOX discards the self-programmed logic in the current temporary storage area, maintains the stable version in the current real-time control domain, and generates a check failure report. If both the consistency check and the security check pass, RoBOX performs an atomic switch operation: it marks the currently running control logic version in the real-time control domain as to be decommissioned, and switches the control logic version in the temporary storage area to the executable state of the real-time control domain in one go through pointer redirection or atomic memory swap; the control thread of the real-time control domain is not interrupted during the switch, and the response delay of the basic remote control command does not increase by more than one control cycle. After the atomic switch is completed, RoBOX will retain the retired control logic version in the history storage area.

[0056] This application also discloses a control device.

[0057] Specifically, the control device includes a memory and a processor, with the memory storing a computer program that can be loaded and executed by the processor to perform the aforementioned AI Agent-based embodied intelligent edge unit self-programming evolution method.

[0058] This application also discloses a computer-readable storage medium.

[0059] Specifically, the computer-readable storage medium stores a computer program that can be loaded and executed by a processor, such as the self-programming evolution method of the embodied intelligent edge unit based on the AI ​​Agent described above. The computer-readable storage medium includes, for example, various media that can store program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0060] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A self-programming evolution method for embodied intelligent edge units based on AI Agents, characterized in that, RoBOX, deployed at the edge of smart devices, comprises logically isolated real-time control and intelligent evolution domains, including: Launch an AI Agent instance in the intelligent evolution domain and bind it to the basic remote control module in the real-time control domain through a controlled interface; the AI ​​Agent is loaded with a visual language model (VLM), a large language model (LLM), an object detection model, and a device API library; RGB and depth images are acquired from the real-time control domain through a controlled interface, a scene description is generated using VLM, and detection information of the target object is generated based on the target detection model; based on the depth image and the detection information, the relative distance and yaw angle between the smart device and the target object are calculated. The scene description, detection information, relative distance, yaw angle, and natural language task instructions are fused into a multidimensional prompt input LLM; the LLM outputs structured policy instructions through thought chain reasoning and majority voting after multiple independent reasonings; the AIAgent generates self-programmed logic based on the policy instructions through the LLM, and the self-programmed logic includes control code, control parameters, control flow, or interaction strategy; The self-programmed logic is stored in the temporary storage area of ​​the intelligent evolution domain. After RoBOX consistency and security verification, the self-programmed logic is applied to the real-time control domain through an atomic switching operation. If an operational abnormality is detected after verification, RoBOX automatically reverts to the control logic of the previous stable version. RoBOX collects data on device operating status, task completion quality, and AI Agent behavior evaluation after the self-programmed logic takes effect, generating feedback information to guide subsequent self-programming evolution.

2. The self-programming evolution method for embodied intelligent edge units based on AI Agent according to claim 1, characterized in that, The process of acquiring RGB and depth images from the real-time control domain via a controlled interface and generating a scene description using VLM includes: The AI ​​Agent sends rotation commands to the real-time control domain through a controlled interface, controlling the smart device to rotate sequentially to multiple preset angles. At each rotation angle, the AI ​​Agent collects RGB and depth images of the current viewpoint through the controlled interface and stores the collected image pairs in association with the corresponding angles. An RGB image from the same viewpoint is input into a VLM, and a preset structured prompt is also input into the VLM. The structured prompt includes output format requirements, requiring the VLM to output the scene description in a preset spatial order. The VLM performs multiple inferences on the same RGB image to generate multiple scene description candidates. Semantically match multiple scene description candidates with the current natural language task instruction, and select the scene description candidate with the highest matching degree as the scene description under the corresponding perspective; If the scene description contains a target object that matches the task instruction or a candidate object with matching attributes, the rotation is terminated and the depth image corresponding to that viewpoint is used as the current depth image; otherwise, the acquisition of the next angle continues. If no target object or candidate object is matched after all preset angles have been collected, the AI ​​Agent will summarize the scene descriptions of each viewpoint and input them into the LLM. The LLM will then infer the associated viewpoint with the highest semantic correlation to the task instruction and control the smart device to rotate to the associated orientation corresponding to the associated viewpoint, and use the depth image of the associated viewpoint as the current depth image.

3. The self-programming evolution method for embodied intelligent edge units based on AI Agent according to claim 2, characterized in that, The target object detection information is generated based on the target detection model; Based on the depth image and the detection information, the relative distance and yaw angle between the smart device and the target object are calculated, including: The RGB image acquired from the current viewpoint and the natural language task instructions are simultaneously input into the object detection model. The object detection model outputs the bounding box of at least one target object and its corresponding confidence score. The bounding box includes the coordinates of the center point, width, and height of the bounding box. From the current depth image, extract the depth value of the pixel corresponding to the center point coordinate of the bounding box, as the first distance between the smart device and the target object; extract the depth value of the corresponding position of the same pixel in the planar depth image, as the second distance from the smart device to the phase plane; Based on the ratio of the first distance to the second distance, the yaw angle between the smart device and the target object is calculated using the inverse cosine function, where the yaw angle represents the horizontal deflection angle of the target object relative to the current orientation of the smart device. When the target detection model outputs bounding boxes of multiple target objects, the target object with the highest confidence is selected as the main target, and its corresponding first distance, second distance and yaw angle are used as the effective navigation parameters of the current viewpoint; if the confidence is lower than a preset threshold, the bounding box, the first distance and yaw angle are marked as unverified.

4. The self-programming evolution method for embodied intelligent edge units based on AI Agent according to claim 3, characterized in that, After marking the bounding box, the first distance, and the yaw angle as pending verification if the confidence level is lower than a preset threshold, the method further includes: The RGB image region corresponding to the bounding box is cropped and input into the VLM, which generates a detailed object description of the RGB image region. At the same time, the detailed object description and the natural language task instructions are input into the LLM, which performs attribute matching and reasoning to determine whether it is the target object. If the LLM determines that an object within the RGB image region is the target object or a candidate object with matching attributes, the pending verification state is lifted, and the bounding box, the first distance, and the yaw angle are converted into valid navigation parameters. If the LLM determines that the object is not the target object, the corresponding detection result is discarded and a rotation acquisition step is triggered to find other perspectives; If the LLM cannot determine the result clearly, then based on the first distance and yaw angle, the intelligent device is controlled to move forward a preset trial distance in the current direction, and then the image is re-acquired to obtain a new confidence level; if the new confidence level is higher than the preset threshold, the new detection result is adopted; otherwise, the target is discarded and the rotation acquisition step is triggered.

5. The self-programming evolution method for embodied intelligent edge units based on AI Agent according to claim 3, characterized in that, The scene description, detection information, relative distance, yaw angle, and natural language task instructions are fused into a multidimensional prompt input LLM; the LLM, after thought chain reasoning and majority voting following multiple independent reasoning, outputs structured policy instructions, including: The scene description, detection information, relative distance, yaw angle, and natural language task instructions are fused together according to a preset template to construct a multidimensional prompt. The multidimensional prompt also includes output format requirements, requiring the LLM output to include inference steps and structured results. The structured result is a data set containing four fields, which represent the matching type, relative distance, yaw angle, and confidence level, respectively. Among them, the matching type can be a first value, a second value, or a third value. The first value indicates an exact match, the second value indicates an attribute inference match, and the third value indicates no match. The multidimensional prompts are input into the LLM, which identifies key attributes in the natural language task instructions, compares them item by item with the objects in the scene description, and determines whether there is an exact match or an attribute similarity match based on the detection information. For the same multidimensional prompt, perform independent LLM reasoning and thought chain generation several times. Each reasoning is sampled independently, and each reasoning outputs a complete structured result and its corresponding reasoning chain. Collect the structured results of several inference outputs, count the number of times the matching type values ​​are the same, and select the matching type with the highest frequency and its corresponding relative distance, yaw angle and confidence as the policy instruction; If multiple matching types receive the same number of votes, the final result is selected according to the rule that the first value takes precedence over the second value, and the second value takes precedence over the third value, thus obtaining the strategy instruction.

6. The self-programming evolution method for embodied intelligent edge units based on AI Agents according to claim 5, characterized in that, The AI ​​Agent generates self-programmable logic using LLM based on the policy instructions. This self-programmable logic includes control code, control parameters, control flow, or interaction strategies, including: The matching type, relative distance, yaw angle, and confidence level in the policy instruction are combined with a predefined device API library to construct code generation prompts. The code generation prompts include the function signatures, function descriptions, and call examples of the API library. Input the code generation prompt into LLM, and LLM will generate an executable Python policy code snippet; the policy code snippet only allows calls to functions in the predefined API library and basic functions in the standard Python library, and prohibits calls to undefined external functions; When the matching type in the strategy instruction is a first value or a second value, and the confidence level is lower than the preset high confidence threshold, parameter adjustment suggestions are generated through LLM. The parameter adjustment suggestions include modifying the motion speed threshold, steering sensitivity, and target approach distance threshold in the device control parameter table. After the parameter adjustment suggestions are verified for consistency, they are written to the parameter storage area of ​​the real-time control domain through the controlled interface. By performing semantic analysis on historically successfully executed control code fragments using LLM, reusable action sequence patterns are identified. Multiple basic control actions are combined into composite functions according to temporal logic, and these composite functions are registered in the device API library as newly added callable functions. The composite functions include sequential execution, conditional branching, or loop control structures. When the matching type in the policy instruction is a third value, an interactive policy update suggestion is generated through LLM, which modifies the triggering conditions of abnormal alarms, adjusts the presentation format or content of natural language feedback information, and updates the status reporting frequency in the communication protocol with the remote operator RoCA; the updated interactive policy takes effect through the controlled interface after verification.

7. The self-programming evolution method for embodied intelligent edge units based on AI Agent according to claim 1, characterized in that, The process of storing the self-programmable logic in the temporary storage area of ​​the intelligent evolution domain, and then applying the self-programmable logic to the real-time control domain through an atomic switching operation after RoBOX consistency and security verification includes: The self-programming logic is stored in a versioned manner in the temporary storage area of ​​the intelligent evolution domain, and each version is assigned a unique version identifier and timestamp. RoBOX performs consistency checks on the self-programmed logic in the staging area: it calls the locally deployed audit language model to perform static analysis on the control code to detect whether there are undefined API calls, type mismatches, syntax errors or logic vulnerabilities; if it is a control parameter, it checks whether the parameter value is within the predefined legal value range; if it is a control flow or interaction strategy, it checks whether it conforms to the preset syntax specifications. After the consistency check passes, a safety check is performed using RoBOX: the self-programmed logic is run in shadow mode in an isolated simulation environment, and the deviation between its output actions and the output actions of the current stable version is compared; at the same time, it is checked whether the self-programmed logic violates the preset safety constraint set, which includes the upper limit of device movement speed, the boundary of the restricted space, and the upper limit of execution time. If the consistency check or security check fails, RoBOX discards the self-programmed logic in the current temporary storage area, maintains the stable version in the current real-time control domain, and generates a check failure report. If both the consistency check and the security check pass, RoBOX performs an atomic switch operation: it marks the currently running control logic version in the real-time control domain as to be decommissioned, and switches the control logic version in the temporary storage area to the executable state of the real-time control domain in one go through pointer redirection or atomic memory swap; the control thread of the real-time control domain is not interrupted during the switch, and the response delay of the basic remote control command does not increase by more than one control cycle. After the atomic switch is completed, RoBOX will retain the retired control logic version in the history storage area.

8. A self-programming evolution device for an embodied intelligent edge unit based on an AI agent, characterized in that, RoBOX, deployed at the edge of smart devices, comprises a logically isolated real-time control domain and an intelligent evolution domain. The device includes: The intelligent evolution module is used to launch AI Agent instances in the intelligent evolution domain and interact with and bind them to the basic remote control module in the real-time control domain through a controlled interface; the AI ​​Agent is loaded with a visual language model (VLM), a large language model (LLM), an object detection model, and a device API library; The target detection module is used to acquire RGB images and depth images from the real-time control domain through a controlled interface, generate scene descriptions using VLM, and generate detection information of target objects based on the target detection model; and calculate the relative distance and yaw angle between the smart device and the target object based on the depth image and the detection information. The programming strategy module is used to integrate the scene description, the detection information, the relative distance, the yaw angle, and the natural language task instructions into a multidimensional prompt input LLM; the LLM outputs structured strategy instructions after thinking chain reasoning and majority voting after multiple independent reasonings; the AI ​​Agent generates self-programming logic based on the strategy instructions through the LLM, and the self-programming logic includes control code, control parameters, control flow, or interaction strategy; The real-time verification module stores the self-programmed logic in the temporary storage area of ​​the intelligent evolution domain. After RoBOX consistency verification and security verification, the self-programmed logic is applied to the real-time control domain through an atomic switching operation. If an operational abnormality is detected after verification, RoBOX automatically reverts to the control logic of the previous stable version. The information feedback module is used to collect data on device operating status, task completion quality, and AI Agent behavior evaluation after the self-programmed logic takes effect through RoBOX, generate feedback information, and guide subsequent self-programming evolution.

9. A control device, characterized in that, The device includes: A memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 7.

Images