Dynamic adjustment method of embodied intelligence system, electronic device and storage medium

Abstract

This application discloses a dynamic adjustment method, electronic device, and storage medium for an embodied intelligence system. The method includes: acquiring real-time status data of target computing hardware deployed on an embodied intelligence carrier, and a target task execution plan; generating a scheduling strategy to guide resource allocation for the target computing hardware based on the real-time status data and the target task execution plan; dynamically adjusting the structure of a target neural network model running on the target computing hardware based on the scheduling strategy; and generating control instructions to drive the embodied intelligence carrier to execute the target task execution plan using the dynamically adjusted target neural network model. This application can dynamically adjust the structure of the target neural network model running on the computing hardware according to the real-time operating status of the computing hardware on the embodied intelligence carrier, enabling the embodied intelligence system to operate adaptively in resource-constrained environments.

Description

Technical Field

[0001] This application relates to the field of embodied intelligence technology, and in particular to a dynamic adjustment method, electronic device and storage medium for an embodied intelligence system. Background Technology

[0002] To enable embodied intelligent vehicles to perceive, plan, and act through real-time interaction with their physical environment, deploying complex, large-scale neural network models (such as multimodal large models) on edge devices such as robots and autonomous vehicles has become a growing trend. These devices typically feature dedicated, high-performance artificial intelligence processors, such as brain processing units (BPUs), to meet the demands for high energy efficiency and low latency computing.

[0003] However, when complex large neural network models (multimodal large models) are mounted on BPU chips or BPU-based embedded platforms, the large neural network models cannot perform inference due to the computing power limitations of the BPU chips or BPU-based embedded platforms; when hardware resources are scarce, the system cannot operate stably. Summary of the Invention

[0004] This application provides a dynamic adjustment method for an embodied intelligence system, which can dynamically adjust the target neural network model structure running on the computing hardware according to the real-time operating status of the computing hardware on the embodied intelligence carrier, so as to ensure the stable operation of the system and solve the above-mentioned technical problems.

[0005] In a first aspect, this application provides a dynamic adjustment method for an embodied intelligence system, comprising: acquiring real-time status data of a target computing hardware deployed on an embodied intelligence carrier, and a target task execution plan; generating a scheduling strategy for guiding the allocation of resources on the target computing hardware based on the real-time status data and the target task execution plan; dynamically adjusting the structure of a target neural network model running on the target computing hardware based on the scheduling strategy; wherein the dynamic adjustment of the structure of the target neural network model includes at least one of the following: activating or skipping one or more network layers in the target neural network model, adjusting the computational precision of one or more network layers in the target neural network model, adjusting the data resolution input to the target neural network model; and using the dynamically adjusted target neural network model to generate control instructions to drive the embodied intelligence carrier to execute the target task execution plan.

[0006] This application introduces a dynamic adjustment mechanism based on real-time hardware status, enabling the system to flexibly adjust the neural network structure according to the current load, temperature, and other conditions of the computing hardware. This avoids the performance drop or even failure of traditional statically deployed models under resource constraints, significantly improving the system's real-time performance and robustness. Furthermore, this application achieves optimal matching between task requirements and hardware resources by fusing the target task execution plan with hardware status to generate a scheduling strategy, maximizing hardware utilization efficiency while ensuring task completion quality. Moreover, this application uses dynamically adjusted model generation control instructions to ensure that the embodied intelligent carrier's action response in the physical environment is adapted to the current computing power, thereby resolving the contradiction between "large model computing requirements" and "limited computing power of embedded platforms."

[0007] In some embodiments, obtaining the target task execution plan includes: receiving a user's task instruction; generating multiple candidate task execution plans based on the task instruction and a preset world model; and determining a target task execution plan from the multiple candidate execution plans based on the real-time status data.

[0008] This application generates multiple candidate solutions based on a world model, enabling the system to seek the optimal solution from different execution paths. This avoids the limitations that may exist with a single solution and improves the flexibility and success rate of task execution. Furthermore, this application introduces real-time state data as a basis for solution selection, ensuring that the selected solution is not only logically feasible but also executable at the hardware resource level, avoiding solution failure due to insufficient resources. In addition, this application also introduces a dual decision-making mechanism of world model and real-time state data, realizing the intelligent transformation from abstract task instructions to specific execution solutions, enabling the system to adaptively adjust task planning according to dynamic changes in hardware status.

[0009] In some embodiments, generating multiple candidate task execution schemes includes: parsing the task instructions to determine multiple subtasks and the logical dependencies between each subtask, and constructing a task dependency graph; and generating the multiple candidate task execution schemes based on the task dependency graph and the world model.

[0010] This application identifies subtasks and their logical dependencies by parsing task instructions, transforming high-level abstract instructions into a quantifiable and manageable set of subtasks, providing a clear granular basis for subsequent hardware resource allocation. Simultaneously, based on task dependency graphs and world model deductions, this application can identify critical paths and potential bottlenecks in task execution, thereby prioritizing optimization of key aspects when generating candidate solutions, improving overall execution efficiency. Furthermore, it enables the system to handle highly complex multi-step tasks (such as assembly and navigation), significantly expanding the application scenarios and commercial value of this invention.

[0011] In some embodiments, a scheduling strategy for guiding the allocation of target computing hardware resources is generated based on the real-time state data and the target task execution plan, including: encoding the real-time state data into a state feature vector; encoding the target task execution plan into a task feature vector; fusing the state feature vector and the task feature vector to form a joint decision feature; and outputting the scheduling strategy according to the joint decision feature.

[0012] This application encodes real-time status data and task execution plans into feature vectors, enabling heterogeneous numerical data (temperature, load) and structured data (task sequences) to be mathematically expressed and computed in the same feature space, laying the foundation for subsequent fusion decision-making. In addition, this application forms joint decision features through feature fusion, which can capture the deep-seated correlation between hardware status and task requirements (such as "high-precision tasks should be degraded under high temperature conditions"), giving the generation of scheduling strategies the ability to learn and generalize. Moreover, the approach of this application to output scheduling strategies based on joint decision features ensures that the generation of strategies has clear input basis and interpretability, avoiding the rigidity and shortcomings of traditional rule engines.

[0013] In some embodiments, dynamically adjusting the structure of a target neural network model running on the target computing hardware based on the scheduling strategy includes: generating corresponding control parameters according to the scheduling strategy; and inputting the control parameters into the target neural network model to drive the target neural network model to perform dynamic structural adjustment.

[0014] This application transforms abstract scheduling strategies into specific, executable control parameters (such as "skip layers 5-8" and "use INT8 precision"), giving model adjustments clear operational instructions and quantification standards, thus avoiding the semantic gap between strategy and execution. Furthermore, the control parameters are input into the target neural network model, enabling real-time and precise manipulation of the model structure. This allows the model to complete structural adjustments within milliseconds, meeting the stringent requirements of embodied intelligent systems for low-latency response and facilitating modular design and subsequent functional expansion of the system.

[0015] In some embodiments, the target neural network model is formed from an initial neural network model through a cross-device collaborative training method. The cross-device collaborative training method includes: during the training of the initial neural network model, assigning training tasks related to perception processing to the target computing hardware for execution, and assigning training tasks related to inference and decision-making to a collaborative computing device with higher computing power than the target computing hardware for execution; monitoring the resource status of the target computing hardware and the collaborative computing device, and adjusting the load distribution of training tasks between the two based on a preset allocation strategy.

[0016] This application assigns tasks related to perception processing to target computing hardware (such as a BPU) for execution, enabling the model to learn how to run efficiently on specific hardware during training, thus avoiding the accuracy loss that may occur due to traditional post-training compression. It also assigns inference and decision-making tasks to collaborative devices with higher computing power, fully utilizing heterogeneous computing resources and improving training efficiency while ensuring training effectiveness. Furthermore, by monitoring resource status and dynamically adjusting load allocation, it achieves adaptive scheduling of training tasks, ensuring the final generation of a target neural network model that can run efficiently on the target hardware, thereby enhancing the feasibility of this invention.

[0017] In some embodiments, the method further includes: dividing the inference computation graph of the target neural network model into multiple computational units that can be executed in parallel or pipelined on the target computing hardware according to the real-time generated scheduling strategy; and dynamically adjusting the execution dependencies and data communication links between the computational units based on the real-time generated scheduling strategy.

[0018] This application fully utilizes the parallel computing capabilities of the target computing hardware by dividing the inference computation graph into computational units that can be executed in parallel or pipelined, significantly improving the throughput and response speed of model inference. Based on the scheduling strategy, the execution dependencies and data communication links between computational units are dynamically adjusted, enabling the computation flow to be flexibly reorganized according to the real-time hardware status and task requirements, avoiding the efficiency decline of the static computation graph when resources fluctuate. A two-layer optimization mechanism (model structure adjustment + computation graph scheduling) is formed to jointly ensure the high-performance operation of the system in complex dynamic environments.

[0019] In some embodiments, the method further includes: adjusting the computation step size and parallelism of each computing unit according to the timeliness requirements of the target task execution scheme.

[0020] This application can dynamically adjust the calculation step size according to the timeliness requirements of the target task execution plan (such as the need for millisecond-level response in emergency obstacle avoidance), which can save computing resources for non-critical tasks while ensuring the real-time performance of critical tasks. By adjusting the parallelism of each computing unit, it realizes the flexible allocation of hardware computing resources. Under high load, the parallelism is appropriately reduced to balance power consumption and performance, and under low load, the parallelism is increased to accelerate processing. This enables the system to meet the differentiated needs of hard real-time tasks (such as security control) and soft real-time tasks (such as environmental modeling), which significantly improves the application scope and commercial value of this invention.

[0021] In a second aspect, this application also provides an electronic device, comprising: a memory storing one or more computer programs; and a processor connected to the memory for executing one or more computer programs stored in the memory, wherein when the processor executes one or more of the computer programs, the electronic device performs the method described in any of the above embodiments.

[0022] This application enables the dynamic adjustment method of the present invention to be plug-and-play in various embedded devices (such as robot controllers and autonomous driving domain controllers) by storing the computer program in the memory and executing it by the processor, which greatly improves the portability and commercial potential of the technology; by implementing dynamic adjustment through the processor executing the program, the system can continuously optimize the scheduling strategy through software upgrades and has long-term technological evolution capabilities.

[0023] Thirdly, this application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by an electronic device, cause the electronic device to perform the method described in any of the above embodiments.

[0024] The computer-readable storage medium of this application, as an independent commodity, enables the technical solution of this invention to be sold, licensed, and distributed in the form of software products, thus broadening the commercialization path of the patent.

[0025] Additional aspects and advantages of this application will be described or shown in part in the following description, or illustrated by practice of the embodiments of this application. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating the dynamic adjustment method of the embodied intelligence system in some embodiments of this application; Figure 2 This is a flowchart illustrating the process of obtaining the target task execution plan in some embodiments of this application; Figure 3 This is a flowchart illustrating the generation of multiple candidate task execution schemes in some embodiments of this application; Figure 4 This is a flowchart illustrating the process of generating a scheduling strategy to guide the allocation of target computing hardware in some embodiments of this application; Figure 5 This is a flowchart illustrating the process of dynamically adjusting the structure of a target neural network model running on the target computing hardware based on a scheduling strategy in some embodiments of this application. Figure 6 This is a flowchart illustrating a method for cross-device collaborative training of a target neural network in some embodiments of this application.

[0028] Figure 7 This is a flowchart illustrating the dynamic adjustment of execution dependencies and data communication links among various computing units in some embodiments of this application. Figure 8 This is a flowchart illustrating the adjustment of the computation step size and parallelism of each computing unit in some embodiments of this application; Figure 9 This is a schematic diagram of the structure of an electronic device in some embodiments of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0030] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, and all are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0031] Furthermore, this embodiment is an exemplary description of a specific implementation of the dynamic adjustment method for an embodied intelligent system. In this implementation, for ease of description, the functional steps involved in the dynamic adjustment method of the embodied intelligent system are concretized into corresponding functional modules. These modules include a perception module, a decision-making module, a model adjustment module, a vision module, a path planning module, and a grasping control module, etc. It should be clarified that these modules are merely a physical or logical division of the dynamic adjustment method of the embodied intelligent system in the specific implementation of this application. These modules can be independent hardware units or different software functional modules running on the same hardware unit (such as a CPU core), and their specific implementation methods are well known to those skilled in the art. The core of this invention does not lie in these modules themselves, but in the complete method defined in this application where these modules work collaboratively according to a specific process. Those skilled in the art can implement the technical solution of this invention based on the description of this embodiment and in combination with various hardware modules in the prior art, but the dynamic adjustment method of the embodied intelligent system in this application is not limited to the specific naming and division of these modules.

[0032] Based on this, please refer to Figure 1 An embodiment of this application provides a method for dynamically adjusting an embodied intelligence system, comprising: S1. Obtain real-time status data of the target computing hardware deployed on the embodied intelligent carrier, as well as a target task execution plan; S2. Based on the real-time status data and the target task execution plan, generate a scheduling strategy to guide the allocation of target computing hardware resources; S3. Based on the scheduling strategy, dynamically adjust the structure of the target neural network model running on the target computing hardware; wherein, the method of dynamically adjusting the structure of the target neural network model includes at least one of the following: activating or skipping one or more network layers in the target neural network model, adjusting the computational precision of one or more network layers in the target neural network model, and adjusting the data resolution input to the target neural network model; S4. Using the target neural network model with dynamically adjusted structure, generate control instructions to drive the embodied intelligent carrier to execute the target task execution scheme.

[0033] Specifically, please refer to Figure 1This embodiment uses a six-axis robotic arm equipped with a BPU chip as an example to detail the complete implementation process of the dynamic adjustment method of the embodied intelligent system in this embodiment. The robot's control system includes three types of core chips: a general-purpose computing chip (using an ARM Cortex-A series CPU) responsible for task planning and high-level decision-making; a dedicated AI acceleration chip (using a Horizon Journey series BPU) responsible for inference calculation of the neural network model; and a real-time control chip (using an ARM Cortex-M series MCU) responsible for real-time drive and force feedback control of the joint motors. The above chips are interconnected through high-speed buses (such as PCIe, EtherCAT) and work together to form a complete embodied intelligent control system. It should be noted that the chip models mentioned in this embodiment are only specific examples given for ease of description. Those skilled in the art can select other chips with corresponding functions to implement this invention according to actual needs, such as integrating the CPU and BPU into the same SoC chip, or using an FPGA to replace some functions, all of which do not depart from the protection scope of this invention. This robot needs to perform the repetitive task of "grabbing workpieces on a conveyor belt and placing them into a designated material box".

[0034] First, the functional unit responsible for hardware status monitoring in the robot control system (hereinafter referred to as the perception module) continuously monitors the real-time operating status of its core computing hardware—the BPU chip (the core hardware processor that performs neural network model inference calculations, which is a specific implementation of the "target computing hardware").

[0035] At a certain moment, the perception module collected the following real-time status data about the BPU chip: the current computing power utilization rate of the BPU chip is 92%, the memory bandwidth utilization rate is 85%, and the chip temperature has reached 78°C, approaching the temperature threshold of 85°C. Simultaneously, the robot control system received a target task execution plan. This plan was generated by the task planning functional unit (hereinafter referred to as the upper-level planning module) based on the task instruction of "grabbing the workpiece." Specifically, it involves: visually identifying the target workpiece on the conveyor belt and obtaining its three-dimensional coordinates; planning a collision-free path for the robotic arm from its current position to the gripping point; controlling the gripper to grasp the workpiece with appropriate force; moving the workpiece above the designated material frame and releasing it; the plan also specifies a timeliness requirement of no more than 500 milliseconds for a single gripping cycle to meet the conveyor belt's rhythm, and an accuracy requirement of a gripping positioning error of no more than 2 millimeters.

[0036] Subsequently, the functional unit responsible for scheduling decisions (hereinafter referred to as the decision module) receives the aforementioned real-time status data and target task execution plan for fusion analysis. The decision module first assesses the matching degree between the current BPU chip hardware status and task requirements: the current BPU temperature of 78°C is close to the threshold, and the computing power utilization rate is as high as 92%. If it continues to run at full load, it may trigger hardware frequency reduction protection in the following cycles, resulting in a sharp drop in processing speed; while the target task requires a single cycle of no more than 500 milliseconds, which is a high real-time requirement, but the requirement for visual positioning accuracy (no more than 2 mm) can be moderately relaxed to 5 mm in this scenario and is still acceptable. Based on this assessment, the decision module generates the following scheduling strategy: the priority of computing resources is set to ensure real-time performance is higher than maintaining accuracy, and it is recommended to alleviate hardware pressure by reducing the computational load of visual processing; specific instructions include switching the visual positioning network from high-precision mode to fast mode, reducing the input image resolution from 1080p to 720p, and skipping the last three fine classification layers in the object detection network; the expected effect is to reduce the BPU load by about 25%, slow down the rate of temperature rise, and thus ensure that processing is completed within a 500-millisecond cycle.

[0037] The functional unit responsible for model structure adjustment (hereinafter referred to as the model adjustment module) receives the above scheduling strategy and converts it into specific adjustment instructions for the target neural network model. The currently running target neural network is a multi-task model, which includes multiple sub-networks such as visual localization, path planning, and grasping control. The model adjustment module performs the following operations: First, the visual positioning subnetwork originally contained a 12-layer convolutional neural network for workpiece recognition and positioning. According to the scheduling strategy, the model adjustment module generates instructions to skip layers 8 to 10 (i.e., fine feature extraction layers), allowing the inference computation path to bypass these three layers and directly connect from the output of layer 7 to layer 11. This adjustment reduces the computational load of the visual subnetwork by approximately 30%. Second, the entire model originally ran at FP16 precision. The model adjustment module generates instructions to adjust the computational precision of layers 1 to 7 (basic feature extraction layers) from FP16 to INT8. Since INT8 computation has higher throughput and lower power consumption on the BPU, this adjustment further reduces the computational load. Finally, the original input image resolution processed by the visual module was 1920×1080. The model adjustment module sets parameters in the data preprocessing stage to downsample the image to 1280×720 before inputting it into the network, reducing the amount of input data by approximately 55% and significantly reducing memory bandwidth usage. The above three adjustments were completed within 50 milliseconds, and the new model structure began to take effect. However, the path planning and grasping control sub-networks were not adjusted to ensure the accuracy of the robotic arm's movements.

[0038] The adjusted target neural network model began processing real-time sensor data. The vision processing unit (hereinafter referred to as the vision module) received 1920×1080 images from the camera, downsampled them to 720p according to the new configuration, and input them into the adjusted vision positioning network. This network successfully identified the workpiece on the conveyor belt and output its three-dimensional coordinates (positioning error of 3.2 mm, still within the acceptable range required by the task) even after skipping three fine layers and using INT8 precision. The path planning unit (hereinafter referred to as the path planning module) planned a collision-free path within 30 milliseconds based on the visual output coordinates. The gripping control unit (hereinafter referred to as the gripping control module) adjusted the gripper force in real time based on the feedback from the tactile sensor after the robotic arm reached the gripping point, successfully gripping the workpiece. The entire process, from image input to the completion of the gripping action, took a total of 460 milliseconds, meeting the timeliness requirement of no more than 500 milliseconds. At the same time, the BPU temperature only rose by 1°C during this period, stabilizing at 79°C, without triggering the frequency reduction protection.

[0039] Through the above steps, this embodiment fully demonstrates the dynamic adjustment method of embodied intelligence systems. Under conditions of limited hardware resources, by dynamically adjusting the neural network structure (including skipping layers, reducing precision, and reducing resolution), it avoids a sharp drop in system performance caused by hardware frequency reduction while ensuring the completion of core tasks, thus achieving stable and real-time operation in resource-constrained environments. The core value of this method lies in enabling embodied intelligence systems to have the adaptive ability to perceive their own state and actively adjust, thereby solving the inherent defects of large models statically deployed on embedded platforms and unable to adapt to changes.

[0040] It should be noted that this embodiment only uses a six-axis robotic arm robot as an example for illustration, but the dynamic adjustment method of the embodied intelligent system in this embodiment is not limited to this. It can be applied to any embodied intelligent carrier equipped with computing hardware, such as wheeled mobile robots, humanoid robots, autonomous vehicles, drones, etc. As long as these carriers need to perform tasks in real time under dynamically changing hardware conditions, they can all achieve adaptive optimization through this method.

[0041] Furthermore, to facilitate a deeper understanding of the overall technical solution of the dynamic adjustment method of the embodied intelligent system in this embodiment, the BPU chip in this embodiment can specifically be a dedicated AI processor chip or a heterogeneous computing SoC. These chips integrate computing units specifically designed for neural network acceleration (such as the TAE tensor acceleration engine-9 in the BPU), which can efficiently perform inference calculations of the target neural network model. The perception module is manifested at the hardware level as sensors (temperature, voltage) and performance counter registers (computational load, memory bandwidth) inside the chip, as well as dedicated circuitry for reading this data. The perception module can also be a related module with hardware monitoring functions integrated into the BPU chip; the perception module can also be an independent state acquisition chip. The upper-level planning module is manifested at the hardware level as a general-purpose CPU core in the target computing hardware (located on the same chip as the BPU or an independent main control chip), running planning algorithm software responsible for task parsing, task dependency graph construction, and world model deduction. The decision module and the upper-level planning module typically run on the same CPU core, but execute different software functions. The decision module is responsible for fusing the target task plan with the hardware state data collected by the perception module to generate a specific scheduling strategy. The model tuning module can be represented as runtime management software running on the CPU core. Its core function is to generate specific control parameters based on the scheduling strategy and drive the target neural network model to complete the corresponding structural adjustments. The vision module is not a separate hardware component but rather a part of the target neural network model deployed on the BPU chip; specifically, it's a sub-network responsible for visual perception processing, and it consumes the BPU's computing resources at the hardware level. The path planning module can be a neural network sub-network deployed on the BPU chip or a traditional planning algorithm running on the CPU, depending on the implementation. The grasping control module typically consists of two parts: a control algorithm running on the real-time MCU core, and a power drive circuit for actuators such as drive motors.

[0042] In some embodiments, obtaining the target task execution plan includes: S11, Receive the user's task instructions; S12. Based on the task instructions and the preset world model, generate multiple candidate task execution schemes; S13. Based on the real-time status data, determine a target task execution scheme from the multiple candidate execution schemes.

[0043] Specifically, please refer to Figure 2Before the robot begins performing the repetitive task of "grabbing workpieces from the conveyor belt and placing them into a designated material frame," the robot control system first receives a user task instruction from the upper-level management system. This instruction is expressed in structured text as: "Task: Continuous gripping; Target: Metal workpieces on the conveyor belt; Placement point: Left material frame; Priority: High; Continuously run until a stop instruction is received." The upper-level management system is specifically the upper-level control unit of the embodied intelligent system, which can be implemented as an industrial-grade host computer, an embedded main control terminal, or a cloud-based control platform. Its core functions are to receive abstract task requirements input by the user, perform global monitoring of the operating status of the entire embodied intelligent carrier (a six-axis robotic arm robot in this embodiment), issue task and stop instructions, and receive task execution results from the robot control system. It serves as the core interactive bridge between the user and the robot control system and can also achieve collaborative control of multiple embodied intelligent carriers (in this embodiment, only a single six-axis robotic arm robot is controlled). The upper-level management system establishes a communication connection with the general-purpose computing chip (ARM Cortex-A series CPU) of the robot control system through the EtherCAT bus to ensure high-speed and stable transmission of task instructions. Its hardware can be an industrial-grade microcontroller or a small server. The software layer is equipped with basic functional modules such as task instruction editing, status monitoring, and data feedback. The specific implementation method is well known to those skilled in the art.

[0044] This abstract instruction needs to be transformed into a specific executable target task execution plan, and this transformation process is the responsibility of the upper-level planning module running on the CPU core.

[0045] After receiving a user's task instruction, the upper-level planning module verifies the completeness and validity of the task instruction (confirming that key information such as task objectives, placement points, and priorities are not missing). If the verification passes, the instruction is temporarily stored in the cache unit to provide input for subsequent steps. If the verification finds that the instruction information is missing, it will be fed back to the upper-level management system to prompt the user to supplement and improve the instruction.

[0046] Next, based on the task instruction and the preset world model, multiple candidate task execution schemes are generated. After receiving and verifying the task instruction, the upper-layer planning module calls the preset world model and, combined with the core requirements of the task instruction, generates multiple candidate task execution schemes that meet the task objectives and possess different performance characteristics. The world model is a pre-trained neural network model capable of predicting the execution results of different action sequences based on the current environmental state. The inputs to the world model include current environmental perception data (such as conveyor belt speed, workpiece distribution density, and the current position of the robotic arm) and candidate action sequences (the candidate action sequences are based on the core requirements obtained after the upper-layer planning module parses the task instruction, combined with the operational logic of the embodied intelligent carrier, generating multiple specific action combinations that can achieve the task objectives). The output of the world model is the predicted execution time, success rate, energy consumption, and other indicators corresponding to each candidate action sequence. Based on these predicted indicators, the upper-layer planning module selects and forms multiple feasible candidate task execution schemes (the specific candidate scheme generation logic will be explained in detail in subsequent embodiments).

[0047] Finally, the upper-layer planning module calls the BPU chip's operating status data collected in real time by the perception module. This operating status data includes: current computing power utilization rate of 85%, memory bandwidth utilization rate of 78%, and chip temperature of 75℃. The upper-layer planning module integrates and analyzes this real-time status data with the predicted performance indicators (execution time, hardware load, success rate, etc.) of each candidate task execution scheme, and selects the scheme that meets the core requirements of the task (high priority, continuous capture) and is adapted to the current hardware operating status as the target task execution scheme.

[0048] The specific analysis process is as follows: The upper-level planning module first clarifies the core constraints of the task—high priority, ensuring the stability of continuous capture, a single capture cycle not exceeding 500 milliseconds, and avoiding hardware frequency reduction protection due to excessive load. Based on this, several candidate solutions are evaluated: some candidate solutions, although highly accurate and successful, predict hardware loads exceeding the safe operating range of the current BPU, could easily lead to a rapid increase in chip temperature and trigger frequency reduction, affecting task continuity; some candidate solutions, although low in hardware load and fast in execution, have too low a predicted success rate, failing to meet the stability requirements of high-priority tasks; finally, a candidate solution that achieves a balance between execution speed, success rate, and hardware load is selected as the target task execution solution, and its specific parameters (execution rhythm, accuracy requirements, hardware resource allocation requirements, etc.) are synchronously transmitted to the decision module.

[0049] It should be noted that this embodiment only uses the grasping task of a six-axis robotic arm as an example for illustration, but the method for obtaining the target task execution plan is not limited to this. Any embodied intelligent system that needs to transform abstract task instructions into specific execution plans, such as an autonomous vehicle needing to transform the "navigate to destination" instruction into a specific path planning plan, a service robot needing to transform the "clean the desktop" instruction into a specific cleaning step sequence, and a drone needing to transform the "inspect area" instruction into a specific waypoint sequence, can achieve intelligent and adaptive task planning through this method. Those skilled in the art can adjust the parsing method of task instructions, the type and structure of the world model, the generation strategy of candidate solutions, and the selection criteria according to the actual application scenario, without departing from the protection scope of this invention.

[0050] In some embodiments, generating multiple candidate task execution schemes includes: S121. Parse the task instructions to determine multiple subtasks and the logical dependencies between each subtask, and construct a task dependency graph; S122. Based on the task dependency graph and the world model, generate the multiple candidate task execution schemes.

[0051] Specifically, please refer to Figure 3 The upper-level management system of the six-axis robotic arm issues user task instructions, which are represented in structured text as follows: "Task: Continuous gripping; Target: Metal workpiece on the conveyor belt; Placement point: Left material box; Priority: High; Continuously run until a stop instruction is received." This instruction is received and processed by the upper-level planning module (running on an ARM Cortex-A series CPU core). The core process of generating multiple candidate task execution schemes is specifically completed independently by the upper-level planning module. This process only calls a preset world model, combines the task instruction parsing results with the task dependency graph, and ensures that the generated candidate schemes are feasible, diverse, and adaptable to different scenario requirements.

[0052] The upper-level planning module receives task instructions from the upper-level management system and first starts the instruction parsing submodule. This submodule uses a combination of natural language processing algorithms and task template matching to decompose and parse the abstract task instructions. The core purpose is to extract the core elements of the task, break down specific subtasks, and clarify the execution order of each subtask. It does not involve any content related to hardware resource scheduling or model adjustment.

[0053] The specific parsing process is as follows: The instruction parsing submodule of the upper-level planning module first identifies the core information in the task instruction, including the task type (continuous grabbing), target object (metal workpiece on the conveyor belt), execution target (place to the left material box), task priority (high) and termination condition (receive a stop instruction). Then, the abstract instruction of "continuous grabbing" is decomposed into a series of sub-tasks that can be executed independently and have clear logical relationships. Based on the operational logic of the six-axis robotic arm and the actual requirements of the gripping task, the sub-tasks are broken down as follows: Sub-task A (visual positioning of the workpiece), which involves identifying the metal workpiece on the conveyor belt and obtaining its three-dimensional coordinates through the vision processing unit; Sub-task B (planning the gripping path), which involves planning a collision-free path from the current position of the robotic arm to the workpiece gripping point based on the workpiece coordinates output by Sub-task A and the kinematic model of the robotic arm; Sub-task C (controlling the gripping action), which involves controlling the robotic arm to move to the gripping point based on the path planned by Sub-task B and completing the workpiece gripping action through the grippers; Sub-task D (transferring and placing the workpiece), which involves controlling the robotic arm to carry the gripped workpiece, move it to the top of the left material frame, release the workpiece, and then reset the robotic arm to prepare for the next gripping cycle.

[0054] After determining the above sub-tasks, the upper-level planning module further analyzes the logical dependencies between each sub-task. Through the task dependency analysis algorithm, the execution order of each sub-task is clarified: Sub-task A is the foundation of the entire task flow. Only after completing the visual positioning of the workpiece (sub-task A) can the grasping path be planned based on the positioning coordinates (sub-task B). Therefore, sub-task A is a prerequisite for sub-task B. After sub-task B is completed, the robotic arm can move to the grasping point according to the planned path and perform the grasping action (sub-task C). Therefore, sub-task B is a prerequisite for sub-task C. After sub-task C is completed, the workpiece can be transferred to the material frame and placed (sub-task D). Therefore, sub-task C is a prerequisite for sub-task D. After sub-task D is completed, the robotic arm resets and re-executes sub-task A, forming a continuous grasping loop logic.

[0055] Based on the aforementioned logical dependencies, the upper-level planning module constructs a task dependency graph. This graph uses nodes to represent subtasks (node ​​A corresponds to subtask A, node B to subtask B, node C to subtask C, and node D to subtask D), and directed edges to represent the dependencies between subtasks: a point from node A to node B indicates that subtask B can be executed after subtask A is completed; a point from node B to node C indicates that subtask C can be executed after subtask B is completed; a point from node C to node D indicates that subtask D can be executed after subtask C is completed; a point from node D to node A indicates that the robotic arm resets after subtask D is completed, restarting subtask A, thus achieving a continuous grasping loop. This task dependency graph clearly presents the execution logic and loop relationships of each subtask, clarifying the complete process of "visual positioning—path planning—grasping action—transfer and placement—reset loop," providing a core logical framework for the generation of subsequent candidate task execution schemes, ensuring that the generated candidate schemes all conform to the basic execution logic of the task and do not have logical conflicts.

[0056] Subsequently, based on the task dependency graph and the world model, the multiple candidate task execution schemes are generated. The world model, as defined earlier, is a pre-trained neural network model deployed on the CPU core of the upper-layer planning module. It pre-stores core information such as conveyor belt operating parameters, robotic arm kinematics model, material box spatial layout, performance indicators of different vision and control algorithms, and environmental perception data. Based on the current environmental state and task execution logic, it can predict key indicators such as execution time, success rate, and energy consumption for different execution methods, providing data support for the generation of candidate schemes. It does not involve any prediction or control related to hardware resource allocation or model structure adjustment.

[0057] The core logic of the upper-level planning module in generating candidate task execution schemes is based on the clearly defined sub-task execution logic in the task dependency graph. It involves diverse combinations of different implementation methods, parameter configurations, and execution rhythms for each sub-task, combined with the prediction results of the world model, to select multiple candidate schemes that meet the core requirements of the task (continuous fetching, high priority) and possess different performance characteristics. The specific generation process is as follows: Based on the task dependency graph constructed by the upper-level planning module, it is ensured that all candidate solutions follow the cyclic logic of "subtask A → subtask B → subtask C → subtask D → subtask A" without breaking the prerequisite dependencies between subtasks. Under this premise, the implementation methods of each subtask are designed differently, and different execution combinations are generated by combining the parameter library of the world model to form multiple candidate solutions.

[0058] For example, candidate solution one (high-precision standard solution): Subtask A uses high-precision visual positioning to ensure that the workpiece positioning error does not exceed 1.5 mm; Subtask B uses a traditional path planning algorithm to ensure that the path is collision-free and the motion trajectory is smooth; Subtask C uses a constant force grasping mode to keep the grasping force constant; Subtask D uses a uniform speed transfer mode to ensure that the workpiece is placed stably. The upper-level planning module calls the world model and inputs the current environmental perception data (conveyor belt speed 0.5 m / s, workpiece distribution density 2 / s, the current position of the robotic arm is the initial reset position), predicting that the single grasping cycle of this solution is 480 milliseconds, the success rate is 98%, and the energy consumption is 0.5 joules, which fully meets the high precision and high success rate requirements of the task.

[0059] Candidate Solution 2 (Balanced Optimization Solution): Subtask A employs a medium-precision visual positioning method, with positioning errors controlled within 3 mm; Subtask B uses a simplified path planning algorithm to shorten path calculation time; Subtask C uses a variable force feedback grasping mode, adjusting the grasping force in real time based on tactile sensor feedback; Subtask D uses a variable speed transfer mode, increasing transfer speed while ensuring stability. According to world model predictions, this solution has a single grasping cycle of 420 milliseconds, a success rate of 95%, and energy consumption of 0.55 joules, achieving a balance between real-time performance and accuracy.

[0060] Candidate Solution 3 (Fast and Efficient Solution): Subtask A employs a lightweight visual positioning method, skipping some fine processing steps, with positioning errors controlled within 5 mm; Subtask B uses a preset path combined with real-time fine-tuning to reduce path planning computation; Subtask C uses a fast grasping mode to simplify the force feedback adjustment process; Subtask D uses a fast transfer mode to improve overall execution efficiency. According to the world model prediction, this solution has a single grasping cycle of 380 milliseconds, a success rate of 90%, and energy consumption of 0.4 joules, offering optimal real-time performance, but with a slight decrease in accuracy and success rate.

[0061] The three candidate task execution schemes above are all generated based on the logical framework of the task dependency graph and have completed performance index prediction through the world model. Each has its own characteristics and is suitable for different scenario requirements. They provide a variety of choices for the subsequent upper-layer planning module to combine the real-time status data of the BPU chip to select and determine the target task execution scheme.

[0062] It should be noted that this embodiment only uses the continuous grasping task of a six-axis robotic arm robot as an example for illustration, but the method of generating multiple candidate task execution schemes is not limited to this. Any embodied intelligent system that needs to generate multiple candidate task execution schemes, such as autonomous vehicles, service robots, drones, etc., can follow the logic of this embodiment: by parsing the task instructions to break down subtasks, constructing a task dependency graph, and then combining it with a world model for deduction, multiple candidate schemes that conform to the task logic and have different performance characteristics can be generated. Those skilled in the art can adjust the subtask splitting method, the task dependency graph construction logic, the world model parameter configuration, and the candidate scheme generation strategy according to the actual application scenario, all of which do not depart from the protection scope of this invention. At the same time, the terminology used in this embodiment (such as upper-level management system, upper-level planning module, world model, subtask AD, etc.) is completely consistent with the embodiments of claims 1 and 2, and the expression style and detailed presentation method are also consistent, ensuring the coherence and consistency of the entire patent embodiment, and does not involve any subsequent essential technical details.

[0063] In some embodiments, based on the real-time status data and the target task execution plan, a scheduling strategy is generated to guide the allocation of target computing hardware resources, including: S21. Encode the real-time status data into a status feature vector; S22. Encode the target task execution scheme into a task feature vector; S23. Merge the state feature vector and the task feature vector to form a joint decision feature; S24. Output the scheduling strategy based on the joint decision characteristics.

[0064] For details, please refer to Figure 4 The upper-level planning module of the six-axis robotic arm has completed the reception and parsing of user task instructions, constructed a task dependency graph and generated multiple candidate task execution schemes. Finally, combined with the real-time status data of the BPU chip collected by the perception module, a candidate scheme that achieves a balance between execution speed, success rate and hardware load is determined as the target task execution scheme.

[0065] The perception module continuously monitors the real-time operating status of the target computing hardware (BPU chip), collecting and outputting a set of structured real-time status data. In this embodiment, the real-time status data specifically includes: BPU chip current computing power utilization rate of 85%, memory bandwidth utilization rate of 78%, chip temperature of 75°C, data transmission latency of 12ms, and current operating power consumption of 15W. Since the above data are all continuous or discrete values, they cannot be directly recognized and processed by the fusion algorithm of the decision module. Therefore, they need to be converted into standardized state feature vectors by the encoding module built into the decision module.

[0066] The specific encoding process is as follows: The encoding module first normalizes the collected real-time status data, mapping each indicator data to the [0,1] interval (for example, the chip temperature of 75℃ is normalized based on the normal operating temperature range of the BPU chip of 40℃-85℃, and the normalized value is (75-40) / (85-40)=0.78); then, according to the preset indicator order (computing power utilization, memory bandwidth utilization, chip temperature, data transmission delay, and operating power consumption), the normalized indicator values ​​are arranged in sequence to form a state feature vector with a dimension of 1×5. In this embodiment, the final generated state feature vector is [0.85, 0.78, 0.78, 0.12, 0.60] (where the data transmission delay of 12ms corresponds to 0.12 after normalization and the operating power consumption of 15W corresponds to 0.60 after normalization, and the normalization basis is the normal operating range of the corresponding indicator). This state feature vector can accurately and concisely represent the current operating state of the BPU chip, providing standardized input for subsequent feature fusion.

[0067] Next, the target task execution plan determined by the upper-level planning module includes structured information such as core task requirements, execution requirements of each sub-task, and performance constraints. Specifically, this includes: task type (continuous grasping), target object (metal workpiece on a conveyor belt), single grasping cycle constraint (≤500ms), positioning accuracy constraint (≤3mm), task priority (high), and sub-task execution mode (medium-precision visual positioning, simplified path planning, variable force feedback grasping, variable speed transfer). This information is non-numerical or multi-dimensional discrete information and needs to be transformed into a task feature vector that matches the dimension of the state feature vector and can be fused through the encoding module of the decision module.

[0068] The specific encoding process is as follows: The encoding module uses a combination of one-hot encoding and numerical mapping to encode the core information of the target task execution plan: task priority (high) is mapped to 1.0, task type (continuous grasping) is mapped to 0.8, single grasping cycle constraint (≤500ms) is mapped to 0.7, positioning accuracy constraint (≤3mm) is mapped to 0.8, and subtask execution mode (medium accuracy and simplified mode) is mapped to 0.7. These are arranged in a preset order (task priority, task type, cycle constraint, accuracy constraint, subtask execution mode) to form a 1×5 task feature vector. In this embodiment, the final generated task feature vector is [1.0, 0.8, 0.7, 0.8, 0.7]. This task feature vector can clearly characterize the core requirements and performance constraints of the target task execution plan, achieving accurate matching and fusion with the state feature vector.

[0069] Then, the decision module calls the built-in feature fusion algorithm to fuse the generated state feature vector ([0.85,0.78, 0.78, 0.12, 0.60]) and the task feature vector ([1.0, 0.8, 0.7, 0.8, 0.7]). The core purpose is to deeply associate the hardware operating state with the task execution requirements to form a joint decision feature that can support the generation of scheduling strategies.

[0070] In this embodiment, the feature fusion algorithm adopts a weighted summation fusion method. Based on task priority and hardware state importance, preset weights are assigned to each corresponding dimension of the two feature vectors (each dimension of the state feature vector has a weight of 0.4, each dimension of the task feature vector has a weight of 0.6, and the total weight is 1.0; this can be adjusted according to the actual scenario without affecting the implementation of this method). The specific fusion calculation process is as follows: Value of each dimension of the joint decision feature = value of the corresponding dimension of the state feature vector × 0.4 + value of the corresponding dimension of the task feature vector × 0.6. After calculation, the final joint decision feature is [0.94, 0.79, 0.73, 0.53, 0.66]. This joint decision feature includes both the current operating state information of the BPU chip and the core requirements of the target task, providing a unique decision basis for the subsequent output scheduling strategy. It should be noted that the weighted summation fusion method used in this embodiment is only an illustrative example. Those skilled in the art can use other fusion methods such as splicing fusion or attention mechanism fusion, all of which do not depart from the protection scope of this invention.

[0071] Finally, the decision-making module has a built-in preset strategy generation model, which is a pre-trained classification model that can output a scheduling strategy that matches the current hardware status and task requirements based on the numerical distribution of joint decision features. The core of the scheduling strategy is the specific instructions that guide the target computing hardware (BPU chip) to allocate resources.

[0072] The specific process is as follows: The decision module inputs the generated joint decision features [0.94, 0.79, 0.73, 0.53, 0.66] into the strategy generation model. The model analyzes the meaning of each dimension's value (e.g., the first dimension 0.94 corresponds to high task priority, the second dimension 0.79 corresponds to continuous grasping task type, the third dimension 0.73 corresponds to relatively relaxed period constraints, the fourth dimension 0.53 corresponds to low data transmission latency, and the fifth dimension 0.66 corresponds to medium hardware power consumption), and combines it with preset strategy rules to output the following scheduling strategy: 1. Prioritize allocating BPU chip computing resources to visual positioning sub-tasks to ensure positioning accuracy meets the constraint of ≤3mm; 2. Reasonably allocate memory bandwidth to prioritize data transmission efficiency between sub-tasks and reduce transmission latency; 3. Control the operating power consumption of the BPU chip to avoid excessive power consumption leading to rapid temperature rise; 4. Reserve 15% computing power redundancy to cope with hardware state fluctuations during task execution and ensure the stability of continuous grasping tasks.

[0073] This scheduling strategy clarifies the priority, specific allocation ratio, and constraints of BPU chip resource allocation. It can guide the target computing hardware to allocate resources reasonably, meeting the core requirements of the target task execution plan (high priority, continuous capture, accuracy and cycle constraints) and adapting to the current operating status of the BPU chip (computing power utilization of 85% and temperature of 75℃).

[0074] It should be noted that this embodiment only uses a six-axis robotic arm robot's continuous grasping task and the BPU chip as the target computing hardware as an example for illustration, but the method for generating the scheduling strategy is not limited to this. Any embodied intelligent system with target computing hardware that needs to generate resource allocation and scheduling strategies based on the hardware's real-time state and task execution plan, such as autonomous vehicles (target computing hardware is an onboard AI chip), service robots (target computing hardware is an embedded AI chip), and drones (target computing hardware is a flight control chip), can use the logic of this embodiment: converting real-time state data and task execution plans into feature vectors through encoding, and outputting the scheduling strategy after fusion processing. Those skilled in the art can adjust the encoding method, feature fusion algorithm, strategy generation model, and specific content of the scheduling strategy according to the actual application scenario, all of which do not depart from the protection scope of this invention.

[0075] In some embodiments, based on the scheduling strategy, dynamically adjusting the structure of the target neural network model running on the target computing hardware includes: S31. Generate corresponding control parameters according to the scheduling strategy; S32. Input the control parameters into the target neural network model to drive the target neural network model to perform dynamic structural adjustment.

[0076] Specifically, please refer to Figure 5 The decision-making module has generated a scheduling strategy to guide the allocation of BPU chip resources based on the real-time status data of the BPU chip and the target task execution plan. The core content is as follows: prioritize the allocation of BPU chip computing resources to the visual positioning sub-task to ensure that the positioning accuracy meets the constraint of ≤3mm; reasonably allocate memory bandwidth to reduce data transmission latency; control the power consumption of BPU chip to avoid rapid temperature rise; and reserve 15% computing power redundancy to cope with hardware status fluctuations.

[0077] Therefore, since the scheduling strategy output by the decision-making module belongs to the high-level resource allocation guidelines and is an abstract instruction, it cannot be directly recognized and executed by the target neural network model deployed on the BPU chip. Thus, the core task of the model adjustment module is to interpret and decompose the scheduling strategy, transforming it into a set of specific, quantifiable, and standardized control parameters. These control parameters need to be compatible with the runtime interface of the target neural network model to ensure that they can be accurately recognized and executed by the model.

[0078] In this embodiment, the target neural network model deployed on the BPU chip is a multi-task model, integrating functional modules such as a visual localization subnetwork and a path planning subnetwork. Its runtime environment supports real-time configuration of the network structure via a parameterized interface. Control parameters are encapsulated in a standardized key-value pair format, facilitating data transmission with the BPU chip via a high-speed bus (such as PCIe). The model adjustment module generates the following three types of core control parameters based on the core requirements of the scheduling strategy, perfectly matching the resource allocation orientation of the scheduling strategy: 1. Network Layer Activation / Skip Control Parameter: The parameter key is "Layer_Control", and the parameter value is a Boolean array `[1,1,1,1,1,1,1,0,0,0,1,1]`. This array corresponds one-to-one with the 12 convolutional layers of the visual localization sub-network, where "1" indicates activation of the corresponding network layer and "0" indicates skipping the corresponding network layer. In accordance with the scheduling strategy of "reserving 15% computational redundancy," this parameter specifies skipping layers 8, 9, and 10 (fine feature extraction layers). These network layers have a large computational load; skipping them effectively reduces the BPU's computational load and achieves computational redundancy reservation.

[0079] 2. Precision Control Parameters: The parameter key is "Precision_Config", and the parameter value is a dictionary structure `{"Layers_1-7": "INT8", "Layers_11-12": "FP16"}`. According to the scheduling strategy's requirement to "control BPU power consumption," the computation precision of the basic feature extraction layers (layers 1-7) in the visual positioning sub-network is adjusted to INT8 precision. INT8 precision offers lower power consumption and higher throughput on the BPU chip, effectively controlling power consumption. Simultaneously, the FP16 precision of the final coordinate regression layers (layers 11-12) is retained to ensure that the positioning accuracy meets the constraint of ≤3mm, aligning with the scheduling strategy's requirement to "guarantee positioning accuracy."

[0080] 3. Input data resolution control parameter: The parameter key is "Input_Resolution", and the parameter value is the tuple `(1280, 720)`. In accordance with the scheduling strategy's requirement of "rationally allocating memory bandwidth," the image resolution input to the visual positioning sub-network is adjusted from the original 1080p to 720p, reducing the amount of input data, lowering memory bandwidth usage, improving data transmission efficiency, and alleviating the bandwidth pressure on the BPU chip.

[0081] The three types of control parameters mentioned above comprehensively cover the core requirements of the scheduling strategy, which not only optimizes the allocation of hardware resources but also ensures the core requirements of the target task. Furthermore, all parameters conform to the interface specifications between the target neural network model and the BPU chip, providing precise underlying instructions for subsequent adjustments to the driving model structure.

[0082] Next, after generating and encapsulating the control parameters, the model adjustment module uses the runtime programming interface provided by the BPU chip to batch send the standardized key-value pair control parameters to the target computing hardware (BPU chip), achieving high-speed and stable transmission of control parameters. The transmission process is completed through the PCIe bus, with the delay controlled within 10ms, ensuring the real-time performance of the model adjustment.

[0083] In this embodiment, the BPU chip's model runtime environment has a built-in online reconfiguration engine. This engine can receive and parse control parameters in real time without interrupting the overall operation of the target neural network model, and quickly perform model structure adjustment operations. The specific adjustment process is as follows: 1. Parameter Validation and Loading: The online reconfiguration engine first validates the received control parameters to ensure that the network layer indexes are not out of bounds, the computational precision mode is supported by the BPU hardware, and the input resolution meets the model preprocessing requirements. After the validation is passed, all control parameters are loaded into the model's configuration register to complete parameter initialization. 2. Network layer execution path adjustment: Based on the “Layer_Control” parameter, the engine reconstructs the computation path of the visual positioning sub-network, cuts off the connection between the 7th and 8th layers, and establishes a direct data path between the output of the 7th layer and the input of the 11th layer. This physically realizes the skip operation of the 8th, 9th and 10th layers without recompiling the model. 3. Precision mode switching: Based on the "Precision_Config" parameter, the engine switches the tensor operation units corresponding to layers 1-7 to INT8 calculation mode, while keeping layers 11-12 in FP16 calculation mode. This switching is completed at the hardware circuit level, with fast response speed and no impact on the overall execution efficiency of the model. 4. Input preprocessing update: Based on the “Input_Resolution” parameter, the engine notifies the front-end image preprocessing unit to downsample the original 1080p image captured by the camera to 720p before inputting it into the target neural network model, ensuring that the input data is consistent with the control parameter requirements.

[0084] The entire dynamic structure adjustment process is completed within 50 milliseconds. During this time, the BPU chip only pauses the inference calculation for the current batch. Once the adjustment is complete, it immediately resumes operation and begins processing sensor data with the new model structure. After the adjustment, the structure of the target neural network model perfectly matches the scheduling strategy requirements, which reduces the computational load and power consumption of the BPU chip, reserves computational redundancy, and ensures visual positioning accuracy.

[0085] It should be noted that this embodiment only uses a six-axis robotic arm robot, a BPU chip, and specific control parameters as examples for illustration. However, the method of dynamically adjusting the structure of a target neural network model running on the target computing hardware based on a scheduling strategy is not limited to this. Any embodied intelligent system that has target computing hardware and a target neural network model and needs to dynamically adjust the model structure based on a scheduling strategy, such as autonomous vehicles, service robots, and drones, can use the logic of this embodiment: the scheduling strategy is converted into control parameters through a model adjustment module, and then the control parameters are input into the model to drive structural adjustment. Those skilled in the art can adjust the type, form, adjustment method, and implementation logic of the control parameters and model reconfiguration according to the actual application scenario, all of which do not depart from the protection scope of this invention.

[0086] In some embodiments, the target neural network model is formed from an initial neural network model through a cross-device collaborative training method, the cross-device collaborative training method comprising: S301. During the training of the initial neural network model, training tasks related to perception processing are assigned to the target computing hardware for execution, and training tasks related to reasoning and decision-making are assigned to a collaborative computing device with higher computing power than the target computing hardware for execution. S302. Monitor the resource status of the target computing hardware and the collaborative computing device, and adjust the load distribution of the training task between them based on a preset allocation strategy.

[0087] Specifically, please refer to Figure 6 In this embodiment, the target neural network model deployed on the BPU chip of the six-axis robotic arm is not trained directly through a single device. Instead, it is optimized based on the initial neural network model through a cross-device collaborative training method. The core purpose of this cross-device collaborative training method is to fully utilize the computing power advantages of different devices, improve training efficiency, reduce the training load on the target computing hardware (BPU chip) while ensuring training accuracy, and avoid training interruptions caused by hardware overheating or insufficient computing power during training. The entire process does not involve subsequent important technical details such as model structure adjustment, computational graph partitioning, and parallel execution.

[0088] In this embodiment, cross-device collaborative training involves two types of core devices. The two devices establish a communication connection through high-speed Ethernet to realize the real-time transmission of training data and intermediate results. The specific device configuration is as follows: First, the target computing hardware, namely the BPU chip (Horizon Robotics Journey series) carried by the six-axis robotic arm robot, is responsible for undertaking part of the training tasks. Its computing power and storage resources are relatively limited, and it is better at performing basic perception-type calculations of neural networks. Second, the collaborative computing device adopts a high-performance server (equipped with NVIDIA A100 GPU). Its computing power and memory bandwidth are higher than those of the BPU chip. It is good at performing complex inference and decision-making calculations and is responsible for undertaking tasks with high computing power requirements during the training process.

[0089] In this embodiment, considering the training scenario of the target neural network model of a six-axis robotic arm, the detailed implementation process of the cross-device collaborative training method is as follows: First, during the training of the initial neural network model, training tasks related to perception processing are assigned to the target computing hardware, while training tasks related to inference and decision-making are assigned to collaborative computing devices with higher computing power than the target computing hardware. The initial neural network model is an unoptimized basic multi-task model integrating sub-networks such as visual localization and path planning. The training process requires simultaneously completing multiple tasks, including perceptual feature extraction and inference and decision-making logic optimization. The computing power requirements for different tasks vary significantly. Therefore, targeted task allocation is performed based on the computing power advantages of the two types of devices to achieve efficient resource utilization.

[0090] The specific task allocation logic is as follows: 1. Training tasks assigned to the target computing hardware (BPU chip): These mainly consist of training tasks related to perception processing. These tasks have relatively simple computational logic and moderate computing power requirements, which align with the hardware characteristics of the BPU chip. Specifically, they include: training the basic feature extraction layer of the visual positioning sub-network, training tasks related to input image preprocessing, feature encoding training of perception data (such as workpiece images and position coordinates), and simple perception result verification tasks. For example, in the training of the visual positioning sub-network, the BPU chip is responsible for extracting features from a large number of workpiece image samples, training the parameters of the basic convolutional layers, and transmitting the extracted basic features to the collaborative computing device for subsequent inference and decision-making training.

[0091] 2. Training tasks assigned to collaborative computing devices (high-performance GPU servers): These mainly involve training tasks related to inference and decision-making. These tasks have complex computational logic and high computing power requirements, necessitating the use of high-performance equipment. Specifically, they include: training the inference logic of the path planning sub-network, training collaborative decision-making among sub-networks, calculating the training loss function and optimizing it through backpropagation, simulating task decision-making training in complex scenarios, and fusing and optimizing the parameters of the overall model. For example, the collaborative computing device receives perceptual features transmitted from the BPU chip, trains the path planning inference logic based on these features, determines the decision rules for the robotic arm's grasping path, and feeds back the optimized parameters to the BPU chip through the backpropagation algorithm, synchronously updating the parameters of the initial neural network model.

[0092] The above task allocation method not only fully leverages the high efficiency of the BPU chip in perception processing, but also utilizes the high computing power of collaborative computing devices, avoiding the inefficiency or hardware overload caused by a single device undertaking all training tasks, and ensuring that the training of the initial neural network model can proceed efficiently and stably.

[0093] Next, during cross-device collaborative training, the resource status of the two types of devices will change dynamically with the training progress. If the task allocation ratio is fixed, there may be a situation where one device is overloaded and the other device is idle. Therefore, it is necessary to monitor the resource status in real time and dynamically adjust the load allocation to ensure training efficiency and device stability.

[0094] The specific implementation process is as follows: 1. Resource Status Monitoring: The cross-device collaborative training system incorporates a resource monitoring module, deployed on both the target computing hardware (BPU chip) and the collaborative computing device (GPU server), to collect core resource status data from both types of devices in real time. Specifically, the BPU chip's resource status data includes: computing power utilization, memory bandwidth usage, chip temperature, and training task execution latency; the GPU server's resource status data includes: GPU computing power utilization, video memory usage, server CPU load, and data transmission bandwidth. The monitoring module collects data every 100 milliseconds and feeds it back to the collaborative training control unit (deployed on the GPU server) in real time via a communication link.

[0095] 2. Load Allocation Adjustment: The collaborative training control unit has a built-in preset load allocation strategy. This strategy is based on the core principle of "balanced utilization of resources between the two types of devices and optimal training efficiency". It presets resource utilization thresholds (BPU chip computing power utilization threshold is 80%, GPU server computing power utilization threshold is 85%) and dynamically adjusts the allocation ratio of training tasks based on the monitored resource status data.

[0096] In this embodiment, the load adjustment scenario during training is as follows: In the early stage of training, the initial neural network model parameters are not optimized, the load of perception processing tasks is low, the BPU chip computing power utilization rate is 65%, and the GPU server computing power utilization rate is 82% due to undertaking inference and decision training. At this time, the initial task allocation ratio is maintained. As training progresses, the load of perception processing tasks increases, and the BPU chip computing power utilization rate is monitored to rise to 88%, exceeding the preset threshold, and the chip temperature rises to 78°C, approaching the safety threshold. The collaborative training control unit immediately initiates load adjustment: some perception processing-related training tasks (such as some basic feature extraction training) are transferred to the GPU server for execution, while reducing the computing power allocation of some non-core inference and decision training tasks on the GPU server. After adjustment, the BPU chip computing power utilization rate drops to 72%, the temperature drops to 73°C, and the GPU server computing power utilization rate rises to 84%, which is within a reasonable range, ensuring the stable progress of the training process.

[0097] As the training task nears completion and the model parameters stabilize, the computational demands of various training tasks decrease. When the GPU server's computational utilization drops to 60% and resources become idle, the collaborative training control unit readjusts the load distribution: the perception processing training tasks previously transferred to the GPU server are reassigned to the BPU chip for execution, making full use of the BPU chip's resources to improve training efficiency until the initial neural network model training is completed. Finally, the optimized target neural network model is obtained and deployed on the BPU chip for the execution of continuous grasping tasks by the six-axis robotic arm.

[0098] The above-mentioned cross-device collaborative training method not only solves the problem that the target computing hardware (BPU chip) has limited computing power and cannot independently and efficiently complete the training of complex models, but also avoids the idle resources of collaborative computing devices, achieving a dual improvement in training efficiency and device stability. The final target neural network model can better adapt to the hardware environment and task requirements of the six-axis robotic arm.

[0099] It should be noted that this embodiment only illustrates cross-device training of a six-axis robotic arm robot, a BPU chip, and a GPU server, but the application scenarios of the cross-device collaborative training method are not limited to this. Any embodied intelligent system that needs to generate a target neural network model through cross-device collaborative training, such as autonomous vehicles, service robots, and drones, can follow the logic of this embodiment: assign tasks to devices with different computing power according to task type, monitor resource status in real time, and adjust load allocation. Those skilled in the art can adjust the type of collaborative devices, task allocation logic, resource monitoring indicators, and load adjustment strategies according to actual application scenarios, without departing from the protection scope of this invention.

[0100] In some embodiments, the method further includes: S241. According to the real-time generated scheduling strategy, the inference computation graph of the target neural network model is divided into multiple computation units that can be executed in parallel or in pipeline on the target computing hardware. S242. Based on the real-time generated scheduling strategy, dynamically adjust the execution dependencies and data communication links between each computing unit.

[0101] Specifically, please refer to Figure 7 The decision-making module has generated a scheduling strategy to guide the allocation of BPU chip resources. The model adjustment module has also completed the dynamic adjustment of the target neural network model structure based on the scheduling strategy, so that the model can adapt to the current operating status of the BPU chip and the target task requirements.

[0102] In this embodiment, the inference computation process of the target neural network model (after structural adjustment) deployed on the BPU chip can be decomposed into multiple independent computation stages. By dividing the inference computation graph into computation units that can be executed in parallel or pipelined, and dynamically adjusting their execution dependencies and communication links, the multi-core computing power of the BPU chip can be fully utilized, resource idleness can be avoided, and the resource allocation requirements of the scheduling strategy can be adapted.

[0103] In this embodiment, considering a six-axis robotic arm's continuous grasping scenario, the detailed implementation process of the method for "dynamically adjusting the execution dependencies and data communication links between computing units" is as follows: First, based on the real-time generated scheduling strategy, the inference computation graph of the target neural network model is divided into multiple computational units that can be executed in parallel or pipelined on the target computing hardware. The inference computation graph of the target neural network model is a logical representation of the model's inference process, containing multiple interconnected computational nodes. Direct execution would cause some computational steps to wait sequentially, wasting hardware resources. The scheduling execution module, based on the real-time scheduling strategy output by the decision module (prioritizing visual positioning subtasks, controlling power consumption, and reserving computational redundancy), splits the inference computation graph, dividing it into multiple computational units adapted to the multi-core architecture of the BPU chip, capable of parallel or pipelined execution. The specific division process is as follows: The scheduling and execution module has a built-in computation graph analysis unit. First, it performs a comprehensive analysis of the inference computation graph of the adjusted target neural network model, identifies the independent computation nodes and their relationships, and, in conjunction with the resource allocation priority of the scheduling strategy, divides the inference computation graph into four core computation units (computation unit 1, computation unit 2, computation unit 3, and computation unit 4) according to the principles of "functional independence, load balancing, and hardware adaptation". All computation units can be executed in parallel or in pipeline on different computation cores of the BPU chip.

[0104] The computational unit 1 (image preprocessing unit) receives image data from the camera, performs downsampling and normalization preprocessing operations according to the adjusted input resolution (1280, 720), and outputs standardized image data. This unit corresponds to the scheduling strategy's requirement of "reasonable allocation of memory bandwidth," and its independent execution can reduce blocking of subsequent computational stages. The computational unit 2 (basic feature extraction unit) performs convolution calculations on layers 1-7 (INT8 precision) of the visual positioning subnetwork, extracting basic workpiece features from the image. This unit has a large computational load, corresponding to the scheduling strategy's requirement of "prioritizing the allocation of computational resources to the visual positioning subtask." Its independent computational unit ensures sufficient computing power. The computational unit 3 (coordinate regression unit) performs calculations on layers 11-12 (FP16 precision) of the visual positioning subnetwork, regressing the workpiece's three-dimensional coordinates based on the feature data output by the basic feature extraction unit. This unit has a data dependency on computational unit 2, but can be executed pipelinedly after computational unit 2 outputs some feature data. The computing unit 4 (path planning auxiliary unit) is responsible for performing the basic calculations of the path planning sub-network and assisting in generating the preliminary trajectory of the robotic arm's grasping path. The computing load of this unit is moderate and can be executed in parallel with computing units 2 and 3, making full use of the multi-core resources of the BPU chip. At the same time, it meets the scheduling strategy of "reserving computing power redundancy" and does not occupy the computing power of the core vision positioning task.

[0105] The above division method ensures that each computing unit is functionally independent and has a balanced load, while also being fully compatible with the multi-core architecture of the BPU chip, enabling parallel or pipelined execution and effectively improving inference efficiency.

[0106] Next, based on the real-time generated scheduling strategy, the execution dependencies and data communication links between each computing unit are dynamically adjusted. After the computing units are partitioned, their initial execution dependencies and data communication links are fixed configurations, which cannot adapt to the dynamic changes in the scheduling strategy and fluctuations in hardware status. Therefore, the scheduling execution module needs to dynamically adjust these two based on the real-time scheduling strategy to ensure the collaborative execution efficiency between computing units while adapting to resource allocation requirements. The specific adjustment process is as follows: 1. Execution Dependency Adjustment: Based on the core requirements of the scheduling strategy, the scheduling execution module adjusts the execution dependency priority of each calculation unit. Initially, calculation unit 4 (path planning auxiliary unit) must wait for calculation unit 3 (coordinate regression unit) to output the workpiece coordinates before it can execute. Based on the scheduling strategy's requirement of "prioritizing the visual positioning subtask," the scheduling execution module adjusts the dependency relationship, allowing calculation unit 4 to perform some basic path planning calculations in advance after calculation unit 2 (basic feature extraction unit) outputs basic features, without waiting for the complete output of calculation unit 3. It only depends on the coordinate data of calculation unit 3 when generating the final trajectory. This reduces the waiting time of calculation unit 4, improves the overall execution efficiency, and does not affect the core process of the visual positioning subtask.

[0107] 2. Data Communication Link Adjustment: The BPU chip contains multiple high-speed data communication links. Initially, the communication links between computing units are fixedly allocated, which may result in some links being congested and others being idle. The scheduling execution module, based on the scheduling strategy of "rationally allocating memory bandwidth and reducing transmission latency," monitors the load status of each communication link in real time and dynamically adjusts the allocation of communication links between computing units: switching the communication link between computing unit 1 and computing unit 2 to a higher bandwidth channel to ensure rapid transmission of preprocessed image data; adjusting the communication link between computing unit 3 and computing unit 4 to be allocated on demand to avoid wasting bandwidth due to idle links; and simultaneously allocating a dedicated temporary data buffer for each computing unit to reduce data transmission conflicts and lower transmission latency.

[0108] In this embodiment, the adjusted execution dependencies and data communication links enable efficient collaboration among the computing units: after completing image preprocessing, computing unit 1 immediately transmits the data to computing unit 2 via a high-speed link; while computing unit 2 performs basic feature extraction, computing unit 4 simultaneously initiates basic path planning calculations; after computing unit 2 outputs partial feature data, computing unit 3 initiates pipelined execution; and after computing unit 3 outputs the workpiece coordinates, computing unit 4 quickly completes the final path planning. Throughout the process, the computing units execute in parallel or pipelined, fully utilizing the multi-core resources and communication bandwidth of the BPU chip, improving inference efficiency by 30% compared to before the adjustment, while simultaneously meeting the resource allocation requirements of the scheduling strategy, ensuring balanced load on the BPU chip, and preventing rapid temperature rise.

[0109] Through the above steps, the inference process of the target neural network model was further optimized. By partitioning the computation graph, adjusting communication links, and adjusting execution dependencies, the performance advantages of the target computing hardware were fully utilized, and the model inference efficiency was improved.

[0110] It should be noted that this embodiment only uses a six-axis robotic arm robot, a BPU chip, and a specific computing unit partitioning as examples for illustration, but the method of "dynamically adjusting the execution dependencies and data communication links between computing units" in this application is not limited to this. Any embodied intelligent system with target computing hardware and a target neural network model that needs to optimize model inference efficiency, such as autonomous vehicles, service robots, and drones, can use the logic of this embodiment: partitioning computing units according to a scheduling strategy and dynamically adjusting execution dependencies and communication links. Those skilled in the art can adjust the number of computing units, the partitioning logic, the dependency adjustment method, and the communication link allocation strategy according to the actual application scenario, all of which do not depart from the protection scope of this invention.

[0111] In some embodiments, the method further includes: S243. Adjust the computation step size and parallelism of each computing unit according to the timeliness requirements of the target task execution plan.

[0112] Specifically, please refer to Figure 8 In this embodiment, the scheduling and execution module has completed the division of the target neural network model inference computation graph, resulting in four computation units that can be executed in parallel or in pipeline. It has also dynamically adjusted the execution dependencies and data communication links between the computation units, thus achieving preliminary optimization of the model inference efficiency.

[0113] However, in this embodiment, since the initial calculation step size of each computing unit is a fixed value, it cannot adapt to the dynamic changes in hardware state fluctuations and task accuracy requirements. If the calculation step size is too large, it will lead to a decrease in inference accuracy, failing to meet the accuracy constraint of visual positioning ≤3mm; if the calculation step size is too small, it will increase the computational load, resulting in excessive load on the BPU chip, violating the scheduling strategy's requirement of "controlling power consumption and reserving computing power redundancy." Therefore, it is necessary to dynamically adjust the calculation step size based on the real-time scheduling strategy and hardware state, and ensure that the accuracy does not deviate from the task requirements through an accuracy compensation mechanism.

[0114] In this embodiment, the computation step size of each computing unit is dynamically adjusted based on the real-time generated scheduling strategy and the real-time status data of the target computing hardware. The scheduling execution module receives the scheduling strategy output by the decision module (prioritizing visual positioning subtasks, controlling power consumption, and reserving computing power redundancy) and the real-time status data of the BPU chip collected by the perception module (computing power utilization, chip temperature, and memory bandwidth occupancy). Combining the functional priority and computing load of each computing unit, the module dynamically adjusts the computation step size of each computing unit to achieve a dynamic balance between efficiency and accuracy. The specific adjustment process includes: the scheduling execution module has a built-in computation step size adjustment unit that pre-sets the computation step size adjustment range and priority weight (based on the resource allocation priority of the scheduling strategy) for each computing unit. Among them, the visual positioning related computing units (computing unit 2 and computing unit 3) have the highest priority, the path planning auxiliary unit (computing unit 4) has the second highest priority, and the image preprocessing unit (computing unit 1) has the lowest priority. The adjustment range is preset according to the functional characteristics of each computing unit.

[0115] The initial calculation step size of computation unit 1 (image preprocessing unit) is 2 (downsampling and normalization calculation is performed once for every 2 pixels), with an adjustment range of 1-4. This unit has a relatively small impact on the overall inference accuracy and prioritizes meeting the scheduling requirements of "controlling power consumption". When the BPU chip computing power utilization is ≥80% and the temperature is ≥75℃, the calculation step size is adjusted to 4 to reduce the amount of computation and reduce the hardware load; when the computing power utilization is ≤60% and resources are idle, the calculation step size is adjusted to 1 to improve the preprocessing accuracy and provide more accurate input data for subsequent computation units. The initial calculation step size of computation unit 2 (basic feature extraction unit) is 1 (convolution calculation is performed once for every feature map pixel), with an adjustment range of 1-2. This unit is the core of visual positioning and must prioritize ensuring accuracy while also considering the computing power load. When the BPU chip's computing power utilization is ≥85% and the scheduling strategy emphasizes "power consumption control," the calculation step size is adjusted to 2 to reduce the number of convolution calculations and lower computing power consumption. Simultaneously, a subsequent accuracy compensation mechanism compensates for accuracy loss. When the computing power utilization is ≤70%, the calculation step size is maintained at 1 to ensure the accuracy of basic feature extraction and provide reliable support for coordinate regression. The initial calculation step size of calculation unit 3 (coordinate regression unit) is 1 (one coordinate regression calculation is performed for each set of feature data), with an adjustment range of 1-1.5 (a step size of 1.5 means that every other set of feature data undergoes interpolation calculation between adjacent sets before regression). This unit directly determines the visual positioning accuracy, and its adjustment must strictly adhere to the "accuracy priority" scheduling requirement. Only when the BPU chip's computing power utilization is ≥90% and the temperature is close to the safe threshold (80℃) is the calculation step size adjusted to 1.5 to balance accuracy and efficiency through interpolation calculations. Otherwise, the calculation step size is maintained at 1 to ensure that the workpiece coordinate regression error is ≤3mm. The initial calculation step size of calculation unit 4 (path planning auxiliary unit) is 2 (path planning auxiliary calculation is performed once every 2 sets of basic feature data), and the adjustment range is 1-3. This unit has low accuracy requirements and prioritizes meeting the scheduling requirements of "improving efficiency and reserving computing power". When the BPU chip computing power is sufficient (utilization rate ≤65%), the calculation step size is adjusted to 1 to improve the accuracy of path planning assistance; when the computing power load is high (utilization rate ≥80%), the calculation step size is adjusted to 3 to significantly reduce the amount of calculation and reserve computing power redundancy for the core visual positioning unit.

[0116] In this embodiment, the triggering condition for dynamic adjustment is: the real-time status data and scheduling strategy of the BPU chip are synchronized every 100 milliseconds. When the computing power utilization rate, chip temperature exceeds the preset threshold, or the scheduling strategy is dynamically adjusted, the calculation step size adjustment is started immediately. The adjustment process is completed within 10 milliseconds, without affecting the collaborative execution of each computing unit, thus ensuring the real-time performance of the task.

[0117] Next, during the adjustment of the calculation step size, a preset accuracy compensation mechanism is used to compensate for the inference accuracy of the target neural network model. When the calculation step size is increased, the inference accuracy of some calculation units will decrease slightly (e.g., when the step size of calculation unit 2 is adjusted to 2, the basic feature extraction accuracy decreases by about 5%). If no compensation is performed, the visual positioning error may exceed the constraint of ≤3mm. Therefore, the accuracy compensation module needs to perform accuracy compensation operation synchronously to ensure that the inference accuracy meets the task requirements and conforms to the scheduling strategy of "prioritizing the visual positioning subtask".

[0118] The specific compensation process includes: the accuracy compensation module and the scheduling execution module work synchronously, monitoring the adjustment of the calculation step size of each calculation unit in real time. Different compensation methods are adopted for the accuracy loss characteristics of different calculation units. All compensation operations are executed efficiently on the BPU chip without adding excessive computational load. Specifically, the compensation for calculation unit 2 (basic feature extraction unit) is as follows: when the calculation step size is adjusted from 1 to 2, a "feature interpolation compensation" method is used. Linear interpolation is performed using the pixel values ​​of adjacent feature points to supplement skipped feature data, restore some feature details, and control the accuracy loss within 2%, ensuring the integrity of basic feature extraction and providing reliable support for the coordinate regression of calculation unit 3. The compensation for calculation unit 3 (coordinate regression unit) is as follows: when the calculation step size is adjusted from 1 to 1.5, a "coordinate correction compensation" method is used. Based on historical workpiece coordinate data and current feature data, a correction model is established to correct the regression coordinates obtained from the interpolation calculation, controlling the coordinate regression error within 3mm, fully meeting the accuracy constraints of visual positioning. For the compensation of calculation unit 1 and calculation unit 4, when the step size of calculation unit 1 is increased, the "edge enhancement compensation" method is adopted to enhance the edge of the preprocessed image to make up for the loss of detail caused by downsampling; when the step size of calculation unit 4 is increased, the "path smoothing compensation" method is adopted to smooth the initially generated path trajectory to avoid trajectory deviation caused by excessive calculation step size and ensure that the accuracy of subsequent path planning is not affected.

[0119] In this embodiment, the accuracy compensation module presets a compensation threshold (accuracy loss ≤ 5%). When the accuracy loss of a certain computing unit exceeds the threshold, it automatically triggers a calculation step size callback (reducing it by one level) and increases the compensation intensity to ensure a balance between accuracy and efficiency. For example, when the step size of computing unit 2 is adjusted to 2, the accuracy loss reaches 6%, exceeding the compensation threshold. At this time, the scheduling execution module callbacks its step size to 1.5, and the accuracy compensation module simultaneously adopts a dual compensation method of "feature interpolation + detail completion" to control the accuracy loss within 3%, which satisfies the accuracy requirements while reducing the computing load.

[0120] Through the above steps, this embodiment further optimizes the inference process of the target neural network model. Based on the division of computing units and adjustment of communication links, the dynamic adjustment of computing step size and accuracy compensation not only further improve the model inference efficiency and control the hardware load, but also ensure that the inference accuracy does not deviate from the task requirements.

[0121] It should be noted that this embodiment only uses a six-axis robotic arm robot, a BPU chip, and a specific computing unit as an example for step size adjustment and compensation, but the method is not limited to this. Any embodied intelligent system with target computing hardware and a target neural network model that needs to balance inference efficiency and accuracy, such as autonomous vehicles, service robots, and drones, can use the logic of this embodiment: dynamically adjust the calculation step size according to the scheduling strategy and hardware status, and ensure inference accuracy through an accuracy compensation mechanism. Those skilled in the art can adjust the adjustment range, triggering conditions, compensation methods, and compensation thresholds of the calculation step size according to the actual application scenario, without departing from the protection scope of this invention.

[0122] In some embodiments, this application also provides an electronic device, including: a memory storing one or more computer programs; and a processor connected to the memory for executing one or more computer programs stored in the memory, wherein when the processor executes one or more of the computer programs, the electronic device causes the electronic device to implement the method described in any of the above embodiments.

[0123] Specifically, please refer to Figure 9 An electronic device according to an embodiment of this application includes a memory, a processor, and a computer program stored in the memory. The computer program is connected to the processor via a bus. When the processor executes the computer program, it implements the dynamic adjustment method of the embodied intelligent system in any of the above embodiments.

[0124] Specifically, the processor is configured to support the electronic device in performing the corresponding functions in the methods described in the above method embodiments. The processor may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0125] Memory is used to store program code, etc. Memory can include volatile memory (VM), such as random access memory (RAM); memory can also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory can also include combinations of the above types of memory.

[0126] The memory can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the dynamic adjustment method of the embodied intelligence system in the embodiments of this application.

[0127] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and an application program required for at least one function.

[0128] In some embodiments, the memory may include memory remotely configured relative to the processor, and this remote memory may be connected via a network to means of implementing the dynamic adjustment method of the embodied intelligence system. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0129] The electronic devices in this application embodiment may specifically be ultra-mobile personal computer devices, smart displays or all-in-one machines, servers or server clusters, etc.

[0130] In some embodiments, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by an electronic device, cause the electronic device to perform the method described in any of the above embodiments.

[0131] Specifically, one embodiment of this application provides a storage medium storing a computer program configured as the dynamic adjustment method of the embodied intelligence system in any of the above embodiments.

[0132] The storage medium in this application refers to a non-transitory computer-readable storage medium containing computer programs. For example, an industrial-grade embedded storage medium (eMMC 5.1 specification) is suitable for industrial applications of six-axis robotic arms. It features anti-interference, high temperature resistance, large storage capacity, and fast read / write speed. It can adapt to the complex environment of industrial sites (such as temperature fluctuations and electromagnetic interference), ensuring that the stored computer program instructions are not lost or damaged, while meeting the requirements for high-speed reading of program instructions and adapting to the collaborative working rhythm of the BPU chip and CPU core.

[0133] On this storage medium, a complete computer program for executing the aforementioned dynamic adjustment method of the embodied intelligence system is stored in binary code; the program contains a set of logical instructions for implementing all steps of the dynamic adjustment method of the embodied intelligence system in any of the aforementioned embodiments, specifically including: 1. Task instruction receiving and parsing program, which drives the upper-level planning module to receive user task instructions (such as "continuously grab metal workpieces on the conveyor belt and place them in the left material box") issued by the upper-level management system, performs integrity verification and core information parsing of the instructions, extracts key parameters such as task type, target object, and priority, and provides program support for obtaining the subsequent target task execution plan. 2. The candidate solution generation and target solution determination program is used to drive the upper-level planning system to parse task instructions, construct task dependency graphs, call the world model to generate multiple candidate task execution solutions, and combine the real-time status data of the BPU chip collected by the perception module to screen and determine the target task execution solution that meets both task requirements and hardware status.

[0134] 3. Scheduling strategy generation program, which drives the decision module to encode the real-time status data of the BPU chip into a status feature vector and the target task execution plan into a task feature vector. It generates joint decision features through feature fusion algorithm and finally outputs a scheduling strategy to guide the allocation of BPU chip resources. 4. Dynamic Model Structure Adjustment Program: This program drives the model adjustment module to interpret the scheduling strategy into standardized control parameters (such as network layer activation control, computational precision configuration, input resolution adjustment, etc.), and sends them to the BPU chip to drive the target neural network model to complete dynamic structural adjustments. 5. Cross-device collaborative training program, used to drive the collaborative training control unit, allocate the training task of the initial neural network model to the BPU chip (perception task) and collaborative computing device (inference and decision-making task), monitor the resource status of the two types of devices in real time, dynamically adjust the load distribution, and finally generate the optimized target neural network model. 6. The inference computation optimization program is used to drive the scheduling and execution module to complete the partitioning of the inference computation graph, the adjustment of the execution dependencies and data communication links of each computation unit, and the dynamic adjustment of the computation step size. At the same time, it drives the accuracy compensation module to perform accuracy compensation operations to balance the model inference efficiency and inference accuracy. 7. The overall collaborative control program is used to coordinate the execution order of the above-mentioned functional programs, ensure that each step proceeds logically and orderly, and realize the complete process of "receiving task instructions - determining target solutions - generating scheduling strategies - adjusting model structure - optimizing inference execution". At the same time, it realizes the collaborative interaction between program instructions and hardware and functional modules such as BPU chip, perception module, and decision module to ensure stable task execution.

[0135] It should be noted that this embodiment only uses an industrial-grade eMMC 5.1 storage chip and a six-axis robotic arm as examples for illustration, but the storage medium in this embodiment is not limited to this. Any computer-readable storage medium capable of storing computer program instructions and executable by a processor to implement the method described in claim 1, such as SD cards, solid-state drives (SSDs), flash memory, hard disks, etc., can be used with the logic of this embodiment. Those skilled in the art can adjust the type of storage medium, hardware parameters, and programming language of the program instructions according to the actual application scenario, without departing from the protection scope of this invention.

[0136] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A dynamic adjustment method for an embodied intelligent system, characterized in that, include: Acquire real-time status data of the target computing hardware deployed on the embodied intelligent carrier, as well as a target task execution plan; Based on the real-time status data and the target task execution plan, a scheduling strategy is generated to guide the allocation of target computing hardware resources; Based on the scheduling strategy, the structure of the target neural network model running on the target computing hardware is dynamically adjusted; wherein, the method of dynamically adjusting the structure of the target neural network model includes at least one of the following: activating or skipping one or more network layers in the target neural network model, adjusting the computational precision of one or more network layers in the target neural network model, and adjusting the data resolution input to the target neural network model; Using a target neural network model with a dynamically adjusted structure, control instructions are generated to drive the embodied intelligent carrier to execute the target task execution plan.

2. The method according to claim 1, characterized in that, Obtaining the target task execution plan includes: Receive task instructions from the user; Based on the task instructions and the preset world model, multiple candidate task execution schemes are generated; Based on the real-time status data, a target task execution plan is determined from the multiple candidate execution plans.

3. The method according to claim 2, characterized in that, The generation of multiple candidate task execution schemes includes: The task instructions are parsed to determine multiple subtasks and the logical dependencies between them, and a task dependency graph is constructed. Based on the task dependency graph and the world model, the multiple candidate task execution schemes are generated.

4. The method according to claim 1, characterized in that, Based on the real-time status data and the target task execution plan, a scheduling strategy is generated to guide the allocation of target computing hardware resources, including: The real-time status data is encoded into a status feature vector; The target task execution plan is encoded into a task feature vector; The state feature vector and the task feature vector are fused to form a joint decision feature; Based on the joint decision-making characteristics, the scheduling strategy is output.

5. The method according to claim 1, characterized in that, Based on the scheduling strategy, the structure of the target neural network model running on the target computing hardware is dynamically adjusted, including: Based on the scheduling strategy, generate corresponding control parameters; The control parameters are input into the target neural network model, thereby driving the target neural network model to perform dynamic structural adjustments.

6. The method according to claim 1, characterized in that, The target neural network model is formed from an initial neural network model through a cross-device collaborative training method, which includes: During the training of the initial neural network model, training tasks related to perception processing are assigned to the target computing hardware for execution, while training tasks related to reasoning and decision-making are assigned to a collaborative computing device with higher computing power than the target computing hardware for execution. Monitor the resource status of the target computing hardware and the collaborative computing device, and adjust the load distribution of training tasks between them based on a preset allocation strategy.

7. The method according to claim 1, characterized in that, The method further includes: According to the real-time generated scheduling strategy, the inference computation graph of the target neural network model is divided into multiple computation units that can be executed in parallel or in pipeline on the target computing hardware. Based on the real-time generated scheduling strategy, the execution dependencies and data communication links between each computing unit are dynamically adjusted.

8. The method according to claim 7, characterized in that, The method further includes: Adjust the computation step size and parallelism of each computing unit according to the timeliness requirements of the target task execution plan.

9. An electronic device, characterized in that, include: Memory, which stores one or more computer programs; A processor, connected to the memory, is configured to execute one or more computer programs stored in the memory, and when the processor executes one or more of the computer programs, causes the electronic device to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by an electronic device, cause the electronic device to perform the method as described in any one of claims 1-8.

Images