Heterogeneous calculation control fusion chip control system and method, robot and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence chip and robot control, and relates to a heterogeneous calculation and control fusion chip control system and method, a robot and a storage medium. A GPU, an NPU, a BPU and an MCU are integrated in a central large core cluster, a communication network subunit and a memory control subunit are matched, and a low-delay data channel is constructed in a chip, so that perception, cognition and control instructions can be directly interacted, the cross-chip communication delay is reduced, and the real-time performance is improved. The three-level cache module is accessed through the memory control subunit, high-bandwidth sharing and centralized scheduling are achieved, data migration is reduced, the bandwidth bottleneck is relieved, and the complex task processing efficiency is improved. A trust root subunit is arranged in each unit, chip-level trusted verification and access isolation are provided, and the reliability under key action control and security sensitive scenes is enhanced.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence chip and robot control technology, specifically relating to a heterogeneous computing and control fusion chip control system, method, robot and storage medium. Background Technology

[0002] In complex scenarios such as robotic systems, intelligent equipment, and automated production lines, devices typically need to simultaneously perform multiple heterogeneous computing tasks, including environmental perception, task recognition and decision-making, and fine motion control. To accommodate the differences in computing types, response latency, and data processing paths among various tasks, existing products generally employ a computing architecture composed of multiple independent chips working together, with each chip undertaking functions such as perception processing, intelligent computing, and underlying execution control.

[0003] However, in the process of developing this application, the inventors discovered at least the following problems in the prior art: First, the perception computing unit and the motion control unit are physically deployed separately, and they usually rely on cross-chip buses or the main processor for data interaction, resulting in significant communication latency between cognitive processing and control execution, making it difficult to meet the requirements of high real-time scenarios. Second, the data access path between different computing units often requires scheduling by a general-purpose processor (such as a CPU), resulting in numerous data transfers and high bandwidth consumption, further causing access bottlenecks in the overall system and thus limiting task processing efficiency. Third, existing multi-chip collaborative systems lack a unified hardware isolation mechanism for security management, and the access boundaries between computing units are difficult to effectively define at the chip level, posing potential security risks when used in applications involving critical motion control. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a heterogeneous computing and control fusion chip control system, method, robot, and storage medium. Under a unified architecture, it enables collaboration among multiple types of computing units and provides system-level protection in terms of data interconnection and access security, thereby improving overall processing performance, reducing communication latency, and enhancing system security and reliability.

[0005] To address the aforementioned technical problems, one technical solution adopted in this application is as follows: a heterogeneous computing and control fusion chip control system is provided, comprising: a central core cluster module and a three-level cache module; the central core cluster module includes a computing unit, a control unit, and an interconnection and memory control unit; the computing unit includes a graphics processing unit and a neural network processing unit; the control unit includes a behavior processing unit and a microcontroller unit; the graphics processing unit and the neural network processing unit are used to process target data to obtain target perception data; the behavior processing unit is used to process a control instruction set to obtain target control instructions; the microcontroller unit is used to drive a motor to work according to the target control instructions; wherein, the graphics processing unit, the neural network processing unit, the behavior processing unit, and the microcontroller unit each have a built-in root trust subunit; the interconnection and memory control unit includes a memory control subunit and a communication network subunit; the computing unit and the control unit access the three-level cache module through the memory control subunit; wherein, the graphics processing unit, the neural network processing unit, the behavior processing unit, and the microcontroller unit communicate and interconnect through the communication network subunit.

[0006] In some embodiments, the central core cluster module includes a scheduling unit, which is communicatively connected to the computing unit and the control unit. The scheduling unit is used for task distribution and security policy management.

[0007] In some embodiments, the behavior processing unit includes an instruction prefetch subunit and a dual-mode decoder subunit. The instruction prefetch subunit and the dual-mode decoder subunit are communicatively connected. The instruction prefetch subunit is used to preload the control instruction set to obtain the target control instruction. The dual-mode decoder subunit is used to perform dual-mode decoding processing according to the task type set corresponding to the control instruction set.

[0008] In some embodiments, the image processing unit, the neural network processing unit, and the behavior processing unit each include a microinstruction decoder subunit, which is used to obtain the task type and computational requirements based on the task instructions sent by the receiving scheduling unit.

[0009] In some embodiments, the communication network subunit includes a tensor on-chip network subunit. The tensor on-chip network subunit has a built-in dual-channel design and is used to optimize communication transmission between the graphics processing unit, the neural network processing unit, the behavior processing unit, and the microcontroller unit based on the dual-channel design.

[0010] In some embodiments, the neural network processing unit includes an array control unit, which is used to perform dual-mode switching of the processing unit array.

[0011] To solve the above-mentioned technical problems, another technical solution adopted in this application is: providing a heterogeneous computing and control fusion chip control method, which is applied to the above-mentioned heterogeneous computing and control fusion chip control system. The method includes: receiving target data; processing the target data based on a graphics processing unit and a neural network processing unit to obtain target perception data; processing the target perception data to obtain a control instruction set; processing the control instruction set based on a behavior processing unit to obtain a target control instruction, and sending the target control instruction to a microcontroller unit so that the microcontroller unit drives the motor to work according to the target control instruction.

[0012] In some embodiments, target data is processed based on a graphics processing unit and a neural network processing unit to obtain target perception data, including: performing environmental modeling processing on the target data based on the graphics processing unit to obtain target environment data; and performing semantic analysis processing on the target environment data based on the neural network processing unit to obtain target perception data.

[0013] To solve the above-mentioned technical problems, another technical solution adopted in the embodiments of this application is: to provide a robot, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned method.

[0014] To solve the above-mentioned technical problems, another technical solution adopted in the embodiments of this application is: to provide a non-volatile computer-readable storage medium that stores computer-executable instructions, which, when executed by a robot, cause the robot to perform the above-mentioned method.

[0015] Unlike related technologies, this application provides a heterogeneous computing and control fusion chip control system, method, robot, and storage medium. By integrating a graphics processing unit, a neural network processing unit, a behavior processing unit, and a microcontroller unit within a central large-core cluster, and providing supporting communication network subunits and memory control subunits, a high-speed, low-latency unified data interaction path can be built within the chip. This allows for direct transmission of perceived data, cognitive results, and control commands between different units, avoiding the communication latency caused by forwarding via external buses or the main processor in traditional multi-chip architectures, thereby significantly improving the overall real-time response capability of the system. Secondly, the computing unit and control unit access the L3 cache module through the memory control subunit, enabling unified scheduling and high-bandwidth shared access to various types of data within the chip. This effectively reduces the number of data transfers across modules, lowers the pressure on main memory access, improves bandwidth bottlenecks in the processing chain, and ultimately enhances the overall data throughput efficiency during complex task execution. Furthermore, by embedding trust root subunits in the graphics processing unit, neural network processing unit, behavior processing unit, and microcontroller unit, each unit can establish a hardware-based trusted root verification mechanism during startup, interconnection access, and instruction interaction. This enables access isolation and permission control at the chip level, solving the problem of lacking a unified security boundary when multiple units collaborate in traditional systems. This enhances the system's reliability and anti-attack capabilities in applications involving critical motion control or security sensitivity. Through this approach, this application achieves deep collaboration between perception computing, task decision-making, and motion control within a unified chip architecture. It addresses issues such as high cross-chip communication latency, bandwidth limitations due to complex data access paths, and the lack of a unified hardware-level security mechanism, realizing a computational control fusion system architecture with high real-time performance, high performance, and high security. Attached Figure Description

[0016] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0017] Figure 1 This is a schematic diagram of the structure of a heterogeneous computing and control fusion chip control system provided in an embodiment of this application; Figure 2 This is a flowchart of a heterogeneous computing and control fusion chip control method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of a robot provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0019] It should be noted that, unless otherwise specified, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. When an element is described as "connected" to another element, it can be directly connected to the other element, or there may be one or more intervening elements between them.

[0020] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0021] Please see Figure 1 , Figure 1 This is a schematic diagram of a heterogeneous computing and control fusion chip control system provided in an embodiment of this application. Figure 1 As shown, the heterogeneous computing and control fusion chip control system 100 includes: a central big core cluster module 10 and a three-level cache module 20; The central core cluster module 10 includes a computing unit 11, a control unit 12, and an interconnect and memory control unit 13. The computing unit 11 includes a graphics processing unit 111 and a neural network processing unit 112; the control unit 12 includes a behavior processing unit 121 and a microcontroller unit 122; wherein the graphics processing unit 111, the neural network processing unit 112, the behavior processing unit 121 and the microcontroller unit 122 each have a built-in root trust subunit (such as root trust subunit 1111, root trust subunit 1121, root trust subunit 1211 and root trust subunit 1221). The interconnection and memory control unit 13 includes a memory control subunit 131 and a communication network subunit 132; the computing unit 11 and the control unit 12 access the level 3 cache module 20 through the memory control subunit 131; the graphics processing unit 111, the neural network processing unit 112, the behavior processing unit 121 and the microcontroller unit 122 communicate and interconnect through the communication network subunit 132.

[0022] Specifically, the central big-core cluster module 10 is the core computing and control platform of the heterogeneous computing and control fusion chip control system 100. It consists of 4 to 8 ARM A7XX big cores and undertakes key functions such as task distribution, data processing, cognitive decision-making, and real-time control. Internally, this module is divided into a computing unit 11, a control unit 12, and an interconnection and memory control unit 13. Each unit achieves low-latency collaboration through a high-speed communication network subunit 132 and a memory control subunit 131. The central big-core cluster module 10 is responsible for receiving external sensor data and task instructions in the system and dynamically allocating computing resources according to task type. This enables visual perception, semantic analysis, motion control, and micro-control tasks to be processed in parallel within a single chip, effectively avoiding the latency exceeding 5ms caused by cross-chip communication in traditional multi-chip architectures.

[0023] During operation, the central core cluster module 10 processes complex image and neural network tasks through the computing unit 11, generates precise control commands through the control unit 12, and completes high-speed data transmission and shared cache access through the interconnection and memory control unit 13. Furthermore, this module implements a hardware-level security mechanism through a built-in root of trust subunit, ensuring that each unit undergoes trusted verification when accessing data or executing tasks, thereby enhancing the system's reliability in critical action control and security-sensitive applications.

[0024] Through this architecture, the central core cluster module 10 not only achieves deep integration of perception, cognition and control, but also optimizes the on-chip data path and bandwidth usage, improves the overall processing efficiency of complex tasks, and provides a unified security boundary and access isolation mechanism, providing a solid foundation for the system to achieve high real-time performance, high performance and high security.

[0025] The computing unit 11 is the main computing execution unit within the central core cluster module 10, consisting of a graphics processing unit 111 (GPU) and a neural network processing unit 112 (NPU). The GPU 111 is primarily responsible for processing environmental perception data, including 3D point cloud computing, image recognition, and visual information preprocessing, providing foundational data for subsequent cognitive analysis and control decisions. The neural network processing unit 112 is used for semantic analysis, task decision-making, and high-performance neural network inference, generating structured cognitive results from image, voice, or other sensor inputs. To ensure the security and trustworthiness of on-chip data, each processing unit in the computing unit 11 has built-in trust root subunits 1111 and 1121, achieving hardware-level access isolation and trusted verification. The inputs to the computing unit 11 include task instructions distributed by the central core cluster module 10 and sensor data, and it accesses shared data in the three-level cache module 20 through the memory control subunit 131 to achieve high-speed read and write of task data. The outputs include processed cognitive results, which are transmitted to the control unit 12 or the communication network subunit 132 for subsequent control calculations or cross-unit data transmission.

[0026] Through this design, the computing unit 11 can complete closed-loop processing of environmental perception and cognitive reasoning within a single chip, reduce cross-chip communication latency, improve data throughput, and ensure the security of perception and computing data during transmission and processing through the built-in root of trust subunit, providing a reliable and efficient computing foundation for complex tasks.

[0027] The control unit 12 includes a behavior processing unit 121 (BPU) and a microcontroller unit 122 (MCU). Its main responsibility is to convert the cognitive results generated by the computing unit 11 into executable control commands and drive mechanical or electronic systems to achieve target actions. The behavior processing unit 121 efficiently processes the control command set and sends it to the microcontroller unit 122, ensuring microsecond-level response capability for real-time tasks. After receiving the control commands, the microcontroller unit 122 directly drives the actuator, such as a motor or robotic arm, and monitors force sensor feedback in real time to trigger hardware-level interrupts, ensuring immediate response to abnormal states. The control unit 12 also incorporates trust root subunits 1211 and 1221 for access permission isolation and security verification, ensuring the security of critical control logic and command transmission. The control unit 12 accesses the three-level cache module 20 through the memory control subunit 131 to obtain task data and intermediate calculation results, and communicates with the computing unit 11 and other units with low latency through the communication network subunit 132.

[0028] Through this design, the control unit 12 can realize closed-loop control from calculation results to actual execution within a single chip, improving the real-time performance and reliability of the system, and effectively solving the latency and data transfer problems caused by multi-chip collaboration in traditional solutions, providing precise and safe control support for complex task execution.

[0029] The Interconnect and Memory Control Unit 13, comprising a Memory Control Subunit 131 (HMCU) and a Communication Network Subunit 132 (TCC-NOC), is a key module within the Central Big Data Cluster Module 10, enabling high-bandwidth data access and high-speed unit interconnection. The Memory Control Subunit 131 manages access to the L3 cache module 20 by the computing unit 11 and the control unit 12, achieving unified data scheduling and shared access. This allows each unit to efficiently read and write task data, cognitive results, and control instructions, thereby reducing main memory access pressure, alleviating bandwidth bottlenecks, and improving the efficiency of complex task processing. Secondly, the Communication Network Subunit 132 enables low-latency data transmission between the computing unit 11, the control unit 12, and the microcontroller unit 122, allowing for rapid on-chip interaction of perception, cognition, and control data, significantly reducing cross-module communication latency. Furthermore, the Interconnect and Memory Control Unit 13, in conjunction with the built-in trust root subunits in each unit, achieves secure isolation and access control, ensuring that sensitive data is not accessed without authorization during on-chip transmission. Through this design, the interconnect and memory control unit 13 improves data transmission efficiency while providing hardware-level security for the system, supporting deep computing and control integration within a single chip.

[0030] It should be noted that the graphics processing unit 111, neural network processing unit 112, behavior processing unit 121, and microcontroller unit 122 each have their own built-in root of trust subunits (such as root of trust subunit 1111, root of trust subunit 1121, root of trust subunit 1211, and root of trust subunit 1221). In this embodiment, each unit corresponds to an independent hardware security island (ROT) for implementing on-chip security isolation and trusted verification. It is understood that each root of trust subunit integrates an independent encryption engine and secure storage module within each unit, enabling hardware-level encryption and integrity verification of sensitive data processed by that unit, ensuring data security during on-chip transmission, caching, and cross-unit communication. The graphics processing unit 111 and neural network processing unit 112 encrypt environmental perception data, image features, and cognitive results through the root of trust subunits, and provide hardware protection through a TEE (Trusted Execution Environment) before outputting the data to the control unit 12 or communication network subunit 132, ensuring the confidentiality and integrity of the data when transmitted to the behavior processing unit 121 or microcontroller unit 122. The root of trust subunits of the behavior processing unit 121 and the microcontroller unit 122 not only decrypt and verify input command data and control parameters, but also implement dynamic access control for hardware resources. For example, the central core cluster can restrict the NPU to only read GPU output data and prohibit writing to MCU registers through the root of trust subunit, while ensuring that the MCU has exclusive access to force sensors and actuators, thereby preventing unauthorized data modification or command interference. After receiving encrypted control commands, the microcontroller unit 122 decrypts and verifies the integrity of the commands through its built-in ROT, ensuring that actions such as motor control, joint angle adjustment, and emergency braking are strictly executed according to the commands, preventing execution deviations or system malfunctions.

[0031] By independently configuring a trust root subunit for each unit and combining hardware encryption and dynamic permission management, data security isolation and trusted interaction between processing units are achieved in a single-chip heterogeneous computing and control architecture. This effectively prevents unauthorized access or tampering of on-chip data and solves the problems of lack of hardware-level protection for critical motion control instructions, vulnerability to attacks, and unauthorized data reading in traditional systems. As a result, the system is guaranteed to have high reliability and security in complex perception, decision-making, and motion control tasks.

[0032] The Level 3 cache module 20 (L3 cache) is a crucial shared storage resource in the heterogeneous computing and control fusion chip control system 100, connecting the computing unit 11, control unit 12, and interconnect and memory control unit 13. It enables high-bandwidth data exchange and unified data scheduling within the chip. This module connects to the various functional units of the central core cluster module 10 via the memory control subunit 131 (HMCU), providing a high-speed cache space for unified access by the GPU, NPU, BPU, and MCU. The main functions of the L3 cache module 20 include: storing intermediate feature data, cognitive results, and task instructions processed by the computing unit 11; caching target control instructions generated by the control unit 12 and real-time data required by the microcontroller unit 122; and providing a low-latency, schedulable shared access platform for data transmission between units. Through the L3 cache module 20, the computing unit 11 and control unit 12 can directly read and write shared data without multiple transfers through the CPU or external main memory, significantly reducing data movement, alleviating bandwidth bottlenecks, and improving the overall processing efficiency of complex tasks within a single chip.

[0033] In the actual workflow, the three-level cache module 20 receives environmental perception data from the graphics processing unit 111 and cognitive results from the neural network processing unit 112, and caches this data for rapid access by the behavior processing unit 121 and the microcontroller unit 122. It also supports dynamic reading, writing, and updating of the cached content by each unit, ensuring that high-priority data for real-time control tasks can be processed immediately. The three-level cache module 20 also works with the built-in trust root subunit to implement access control, ensuring that each unit can only access authorized data areas, thereby providing hardware-level security isolation and trusted verification to prevent critical control data from being read or tampered with by unauthorized units.

[0034] Through the design of the three-level cache module 20, this system realizes high-speed exchange of on-chip perception, cognition and control data, avoiding the latency and bandwidth bottlenecks caused by external buses or CPU relays in traditional multi-chip architectures. At the same time, the reliability and security of data access are enhanced through hardware security mechanisms, providing a unified data platform with high real-time performance, high bandwidth and high security for the heterogeneous computing and control fusion chip system 100, supporting the efficient execution of complex tasks within a single chip.

[0035] In this embodiment, the basic working principle of the heterogeneous computing and control fusion chip control system 100 is as follows: the heterogeneous computing and control fusion chip control system 100 achieves deep collaboration of perception, cognition, and control through the central big core cluster module 10 and the three-level cache module 20. The central big core cluster module 10 includes a computing unit 11, a control unit 12, and an interconnection and memory control unit 13. The computing unit 11 consists of a graphics processing unit 111 (GPU) and a neural network processing unit 112 (NPU), which are responsible for environmental perception data processing, 3D point cloud computing, image recognition, semantic analysis, and high-performance neural network inference tasks, respectively. The control unit 12 includes a behavior processing unit 121 (BPU) and a microcontroller unit 122 (MCU), which mainly undertake motion control, joint trajectory calculation, and actuator driving tasks. Each unit has a built-in root of trust subunit. Through hardware encryption engine and security isolation mechanism, it realizes trusted verification of on-chip data and dynamic permission management, ensuring secure transmission of perception data, cognition results, and control commands between different units and preventing unauthorized access and data tampering.

[0036] The interconnect and memory control unit 13 includes a memory control subunit 131 and a communication network subunit 132. The former provides unified access to the L3 cache module 20 by the computing unit 11 and the control unit 12, enabling high-bandwidth sharing and centralized scheduling, reducing the number of cross-module data transfers, and alleviating data transmission bottlenecks. The latter establishes a high-speed, low-latency data channel between units through a Tensor Network on-Chip (TCC-NOC), enabling the GPU, NPU, BPU, and MCU to directly exchange task instructions, perception information, and control results, avoiding the communication latency that occurs in traditional multi-chip architectures via the CPU or bus.

[0037] The workflow of the heterogeneous computing and control fusion chip control system 100 is as follows: After receiving task instructions and sensor inputs, the central core cluster module 10 allocates tasks to the computing unit 11 or the control unit 12, and dynamically schedules resources between GPU / NPU or BPU / MCU according to the task type and computing requirements; after the graphics processing unit 111 and the neural network processing unit 112 process the environmental perception data and cognitive analysis results, they securely transmit the processing results to the behavior processing unit 121 or the microcontroller unit 122 through the communication network subunit 132; the microcontroller unit 122 drives the actuator in real time according to the decrypted control instructions to implement precise mechanical operations or emergency braking, while the root trust subunit strictly manages the execution data and permissions to achieve hardware-level security isolation. Through this mechanism, the heterogeneous computing and control fusion chip control system 100 forms a unified computing and control collaboration architecture within a single chip, effectively solving the problems of high communication latency, complex data access paths, and lack of security mechanisms in traditional multi-chip solutions, and achieving high real-time performance, high bandwidth, and high security computing and control fusion operation.

[0038] In some embodiments, the central core cluster module 10 further includes a scheduling unit 14, which is communicatively connected to the computing unit 11 and the control unit 12. The scheduling unit 14 is used for task distribution and security policy management.

[0039] The scheduling unit 14 is the core of task distribution and security policy management in the central big core cluster module 10. It mainly consists of two ARM A7XX big cores and can run a real-time operating system or other real-time task management software. The scheduling unit 14 receives external task instructions and sensor data, generates scheduling policies based on task type, priority, and computational requirements, and allocates tasks to the computing unit 11 or control unit 12. Simultaneously, it determines access permissions and security policies, achieving chip-level resource scheduling and task management. The scheduling unit 14 maintains high-speed connections with various functional units, enabling rapid distribution of task instructions, cognitive data, and control instructions, forming a closed-loop feedback loop. Its core functions include: parsing task types, generating accelerator binding tags to achieve dynamic resource binding between big cores and accelerators; managing memory access permissions and coordinating the shared scheduling of the L3 cache by the memory control subunit 131; and monitoring task execution status to ensure that real-time tasks are completed with priority.

[0040] Through the unified management of the scheduling unit 14, the central core cluster module 10 can achieve efficient task allocation, low-latency execution, and secure and reliable resource access, thereby solving the problems of slow task scheduling, complex cross-chip communication, and weak security mechanisms in traditional multi-chip architectures.

[0041] In some embodiments, the image processing unit 111, the neural network processing unit 112, and the behavior processing unit 121 each further include a microinstruction decoder subunit (such as microinstruction decoder subunit 1112, microinstruction decoder subunit 1123, and microinstruction decoder subunit 1212), which is used to obtain the task type and computational requirements according to the task instructions sent by the scheduling unit 14.

[0042] Understandably, the microinstruction decoder subunit, acting as a software and hardware intermediary layer within the central core cluster module 10, primarily parses task instructions issued by the scheduling unit 14 and transforms the instruction content into specific computational task types, computing power requirements, and data access schemes. During system operation, the scheduling unit 14 first generates a microinstruction stream based on external input or robot task planning. These microinstructions include information such as task category (e.g., visual recognition, semantic analysis, motion planning), computational priority (e.g., real-time high priority or non-real-time low priority), data input / output paths, cache usage strategies, and on-chip communication requirements. Upon receiving a task instruction, the microinstruction decoder subunit parses the task type and computational requirements. For example, if the task instruction is identified as a vision-related task, the microinstruction decoder subunit 1112 marks the task as TASK_VISION and generates a corresponding hardware binding tag, binding the task to the graphics processing unit 111. For semantic analysis tasks, the microinstruction decoder subunit 1123 parses the instruction and binds the task to the neural network processing unit 112. For motion control or behavioral decision-making tasks, the microinstruction decoder subunit 1212 binds the task to the behavior processing unit 121. During the parsing process, the microinstruction decoder subunit will also provide a basis for the establishment of subsequent data paths based on the computing power density, latency constraints and data scale of the task, and notify the communication network subunit 132 and the memory control subunit 131 to prepare the task-specific data transmission channel and cache area, thereby realizing efficient collaboration of on-chip heterogeneous computing units.

[0043] In some embodiments, the communication network subunit 132 includes a tensor on-chip network subunit 1321. The tensor on-chip network subunit 1321 has a built-in dual-channel design and is used to optimize the communication transmission between the graphics processing unit 111, the neural network processing unit 112, the behavior processing unit 121, and the microcontroller unit 122 based on the dual-channel design.

[0044] Understandably, the communication network subunit 132 further integrates the tensor on-chip network subunit 1321. The tensor on-chip network subunit 1321 employs a dual-channel design, including a high-bandwidth channel and a low-latency channel. The high-bandwidth channel is primarily used for large-scale data transmission. For example, when transmitting 3D point cloud data generated by the graphics processing unit 111 or intermediate feature maps from the neural network processing unit 112 to other computing units or control units, high-speed transmission is achieved through a 256-bit wide channel, supporting the 120fps transmission requirement for 4K image tensors. The low-latency channel is used to transmit control commands and behavioral data. For instance, when action commands generated by the BPU are sent to the microcontroller unit 122, the latency is less than 1μs, ensuring real-time response for motion control tasks. The tensor on-chip network subunit 1321 not only supports direct data exchange between computing units but also transparently transmits task commands parsed by the micro-instruction decoder and intermediate calculation results to each unit, achieving seamless on-chip collaboration between the cognitive cluster (GPU+NPU) and the control cluster (BPU+MCU).

[0045] By combining the microinstruction decoder subunit and the tensor on-chip network subunit to construct a heterogeneous computing unit collaborative mechanism, the central big core cluster module 10 can achieve the following functions: First, through the microinstruction parsing and hardware binding mechanism, the system dynamically generates computing clusters according to task type. The GPU / NPU processes perception and cognition tasks, and the BPU / MCU executes behavior and motion control tasks, realizing task-specific hardware acceleration. Second, through the dual-channel optimization of the tensor on-chip network, a low-latency and predictable data path is established within the computing cluster and between the computing cluster and the control cluster, avoiding the communication bottleneck caused by the reliance on external buses or CPU relays in traditional multi-chip architectures. Third, combined with the memory control subunit 131, the L3 cache module 20 is partitioned and locked to achieve task-specific cache access, ensuring high-bandwidth sharing and zero-copy transmission of data within the chip, thereby improving the overall processing efficiency of complex tasks. Finally, each unit has a built-in root of trust subunit, which, through the joint control of the microinstruction decoder and the tensor on-chip network, can provide hardware-level access isolation and trusted verification during task execution, ensuring the security and reliability of perception, cognition, and control data during on-chip transmission. Through the above mechanisms, this system can achieve dynamic binding of heterogeneous computing units according to task type, high-speed data collaboration within the chip, cache sharing optimization, and security isolation, significantly improving the system's real-time performance, computing power utilization, and operational security in complex robotics or artificial intelligence application scenarios.

[0046] In some embodiments, the behavior processing unit 121 includes an instruction prefetch subunit 1213 and a dual-mode decoder subunit 1214. The instruction prefetch subunit 1213 and the dual-mode decoder subunit 1214 are communicatively connected. The instruction prefetch subunit 1213 is used to preload the control instruction set to obtain the target control instruction. The dual-mode decoder subunit 1214 is used to perform dual-mode decoding processing according to the task type set corresponding to the control instruction set.

[0047] A three-stage decoding pipeline mechanism is designed using an instruction prefetch subunit 1213 and a dual-mode decoder subunit 1214. The instruction prefetch subunit 1213, as the first stage of the three-stage decoding pipeline, primarily preloads the control instruction set, enabling early acquisition of target control instructions. Essentially, the instruction prefetch subunit 1213 monitors the BPU instruction queue, identifies high-priority control instructions (such as robot joint angle adjustments, force feedback control, or emergency obstacle avoidance instructions like the 0xFE emergency stop code), and preloads these instructions from the three-stage cache module 20 into the zero-wait-state cache (ZWS-Cache). During preloading, the instruction prefetch subunit 1213 utilizes a hardware-level tag matching circuit to quickly compare the tags of instructions in the instruction queue, locating the physical address region of the instruction in the cache, thereby achieving zero-wait access to the target instruction and significantly reducing response latency. The operation of the instruction prefetch subunit 1213 not only ensures that high-frequency control instructions are available in a timely manner, but also provides accurate and stable input data for the subsequent decoding stage, ensuring the real-time performance and reliability of motion control.

[0048] The dual-mode decoder subunit 1214 serves as the second and third stage of the three-stage decoding pipeline. Its primary responsibility is to perform dual-mode decoding based on the target control instructions provided by the instruction prefetch subunit 1213, combined with the corresponding task type set. Understandably, the dual-mode decoder subunit 1214 contains two parallel channels: a static decoding channel and a dynamic bypass channel. The static decoding channel primarily handles routine motion instructions, such as robot trajectory planning and periodic motion control, employing a three-stage pipeline structure (fetch → decode → execute) to ensure accurate sequential execution of instructions. The dynamic bypass channel, on the other hand, handles high-priority, real-time control instructions, such as emergency stop signals or force feedback trigger instructions. It bypasses the conventional pipeline through a physical direct connection, directly transmitting the instructions to the microcontroller unit 122 or the execution drive module, achieving sub-microsecond response times. The dual-mode decoder subunit 1214 obtains preloaded instructions in real time through a communication connection with the instruction prefetch subunit 1213, and selects the appropriate channel for decoding according to the instruction type, thereby ensuring decoding efficiency while taking into account the real-time nature of the task and the overall system scheduling strategy.

[0049] It should be noted that the collaborative mechanism between the instruction prefetch subunit 1213 and the dual-mode decoder subunit 1214 is embedded in the heterogeneous computing unit collaborative mechanism of the central big core cluster module 10. This collaborative mechanism, centered on the scheduling unit 14, is responsible for binding computing clusters according to task type (e.g., GPU+NPU for processing perception and cognitive tasks, BPU+MCU for motion control), and establishing an on-chip high-speed, low-latency data channel through the communication network subunit 132 and the memory control subunit 131. Through this mechanism, the instruction prefetch subunit 1213 can dynamically adjust the order and cache area of ​​preloaded instructions based on task priority and computing power allocation strategies, while the dual-mode decoder subunit 1214 selects the appropriate decoding channel for execution based on the microinstruction parsing results and real-time task requirements, thereby achieving high throughput, low latency, and predictable execution capabilities for the behavior processing unit 121 under complex tasks.

[0050] Through the above design, this embodiment not only optimizes the efficiency of BPU instruction acquisition and decoding, but also realizes data collaboration and real-time control between the behavior processing unit 121, the microcontroller unit 122, and other computing units 11. This provides a reliable guarantee for the robot to perform precision movements, force feedback control, and safety-sensitive operations, and solves the problems of slow instruction response, low decoding efficiency, and unpredictable control delay in traditional multi-core systems. At the same time, it ensures the security and stability of instructions during on-chip transmission and processing.

[0051] In some embodiments, the neural network processing unit 112 includes an array control unit 1122, which is used to perform dual-mode switching of the processing unit array.

[0052] The Neural Processing Unit (NPU) 112 integrates an array control unit 1122. This unit is responsible for the dynamic management and dual-mode switching of the Processing Element Array (PE array) within the NPU to achieve efficient and reliable neural network computation. The NPU itself consists of multiple processing units, which can independently perform tensor operations such as convolution, matrix multiplication, or sparse data processing tasks. The array control unit 1122 serves as the control core of the entire PE array, flexibly switching the array's operating mode according to the characteristics of the input data and the type of computation task, thereby optimizing on-chip data flow efficiency and improving overall computational throughput.

[0053] Specifically, the array control unit 1122's dual-mode switching includes two operating states: dense mode and sparse mode. In dense mode, when the input data is a high-density image or multi-channel feature map, the array control unit 1122 activates all PE units of the NPU, enabling each processing unit to undertake the corresponding computational task and supporting mixed-precision computation (such as the combined use of INT8 and FP16) to balance computational accuracy and performance. In this mode, the array control unit 1122 directly distributes input data to each PE through the on-chip high-speed memory interface and Tensor Network on-Chip (TCC-NOC), ensuring that data can flow rapidly between processing units and achieving highly parallel matrix operations and convolution calculations. In sparse mode, when the input data contains a large amount of blank space or event-driven features (such as event camera data or sparse sensor signals), the array control unit 1122 shuts down unused PE units, keeping only the necessary processing units running, thereby significantly reducing power consumption. In this mode, the array control unit 1122 still accurately allocates valid data to active PEs through its internal communication network, ensuring that computing tasks do not suffer data loss or delay. It also reduces bandwidth consumption caused by invalid data moving within the chip, improving overall system efficiency. Furthermore, the array control unit 1122 integrates a fault-tolerant management mechanism. When a single PE unit experiences a failure or anomaly during computation, the array control unit 1122 automatically triggers redundant link reconstruction, redistributing the computing task to adjacent PE units and rebuilding the data transmission link to ensure the continuity and stability of the entire PE array's computation. Through this dynamic scheduling and fault-tolerant strategy, the NPU can achieve reliable and efficient computing performance when facing complex neural network inference tasks or large-scale feature map computations, while ensuring controllable power consumption.

[0054] It should be noted that the array control unit 1122 works closely with the scheduling unit 14 in the central core cluster module 10: the scheduling unit 14 first parses the task instructions, determines the characteristics of the input data and the computational requirements, and then transmits the task information to the NPU through the communication network subunit 132; after receiving the information, the array control unit 1122 selects dense mode or sparse mode according to the task type and data density, allocates PE array computation tasks, manages the on-chip tensor transfer path, and triggers the fault tolerance mechanism when necessary to achieve efficient computation and reliable operation of the PE array.

[0055] Through the above design, the neural network processing unit 112 can not only dynamically optimize the use of the PE array according to the data characteristics, improve the on-chip data flow efficiency and computing throughput, but also ensure computing reliability through the fault tolerance mechanism. This solves the problems of low efficiency, high latency and poor reliability of traditional single-mode PE arrays in sparse data processing or abnormal situations, and realizes low latency and high performance neural network inference capabilities.

[0056] It should be noted that the central large-core cluster module 10 consists of multiple ARM A7XX large cores, with the GPU, NPU, BPU, and MCU each corresponding to one or more large cores for parallel processing of different types of tasks. Understandably, cognitive scheduling tasks are allocated to two large cores by the scheduling unit 14, responsible for receiving external sensor data, parsing task types, generating accelerator binding tags, and managing security policies; computationally intensive tasks, such as graphics processing and neural network inference, are allocated to 2-4 large cores by the computing unit 11. These cores directly mount the GPU and NPU, accessing the L3 cache module 20 through the memory control subunit 131 (HMCU) to achieve high-bandwidth sharing and unified scheduling of task-specific cache areas; control interface tasks are allocated to 1-2 large cores by the control unit 12, responsible for instruction decoding, control instruction generation, and real-time execution by the behavior processing unit 121 and the microcontroller unit 122. The scheduling unit 14 dynamically binds the large cores corresponding to the GPU / NPU / BPU / MCU into cognitive clusters or control clusters according to the task characteristics, thereby achieving efficient allocation of task instructions and resource optimization. This ensures that the system can achieve high parallelism, high real-time performance and low latency computing and control collaborative processing in multi-task scenarios such as visual recognition, semantic analysis, path planning and execution control.

[0057] The heterogeneous computing and control fusion chip control system 100 provided in this application mainly consists of a central big core cluster module 10 and a three-level cache module 20. The central big core cluster module 10 includes a computing unit 11, a control unit 12, an interconnect and memory control unit 13, and a scheduling unit 14. The system achieves deep collaboration between perception computing, task decision-making, and motion control through on-chip high-speed, low-latency data channels, solving the problems of large communication latency, complex data paths, and lack of security mechanisms in traditional multi-chip architectures.

[0058] Specifically, the scheduling unit 14 of the central core cluster module 10 is responsible for task reception, parsing, and distribution, while also managing security policies. The scheduling unit 14 generates accelerator binding tags based on task type (such as visual recognition, semantic analysis, or motion control) and computational requirements, distributes tasks to each unit via the communication network subunit 132, and performs hardware-level encryption on the data to ensure security. The computing unit 11 includes a graphics processing unit 111 (GPU) and a neural network processing unit 112 (NPU). The graphics processing unit 111 is responsible for environmental perception, image recognition, and point cloud computing. Its internal micro-instruction decoder subunit 1112 parses task instructions and ensures data security through the root of trust subunit 1111. The neural network processing unit 112 performs semantic analysis and neural network inference. The micro-instruction decoder subunit 1123 parses tasks, the array control unit 1122 implements dual-mode switching and fault-tolerant reconfiguration of the PE array, and the root of trust subunit 1121 ensures computational trustworthiness. The control unit 12 includes a behavior processing unit 121 (BPU) and a microcontroller unit 122 (MCU). The processing unit 121 implements the preloading and fast decoding of control instructions through the instruction prefetch subunit 1213 and the dual-mode decoder subunit 1214, dynamically responding to real-time tasks. The microcontroller unit 122 (MCU) is responsible for the specific execution of action instructions, and the root of trust subunit 1221 ensures the secure transmission of control data. The interconnect and memory control unit 13 accesses the L3 cache module 20 through the memory control subunit 131, realizing high-bandwidth sharing and centralized scheduling. The communication network subunit 132, through the dual-channel design of the tensor on-chip network subunit 1321, realizes low-latency and high-efficiency communication between the GPU, NPU, BPU, and MCU.

[0059] During system operation, the scheduling unit 14, combined with the micro-instruction decoder subunit and the dual-channel tensor on-chip network, realizes task instruction parsing, heterogeneous computing unit collaborative scheduling and high-speed communication optimization, enabling deep collaboration of vision, semantics, behavior control and execution tasks within a single chip, thereby achieving a computing and control fusion system architecture with high real-time performance, high computing efficiency and high security.

[0060] Please see Figure 2 , Figure 2 This is a flowchart illustrating a heterogeneous computing and control fusion chip control method provided in an embodiment of this application. This method is applied to the aforementioned heterogeneous computing and control fusion chip control system 100. Figure 2 As shown, the method includes steps S11-S14: S11: Receive target data.

[0061] S12: Based on the graphics processing unit and the neural network processing unit, the target data is processed to obtain target perception data.

[0062] Specifically, the target data is processed by the graphics processing unit and the neural network processing unit to obtain target perception data, including: environmental modeling of the target data by the graphics processing unit to obtain target environment data; and semantic analysis of the target environment data by the neural network processing unit to obtain target perception data.

[0063] S13: Process the target perception data to obtain a set of control instructions.

[0064] S14: Based on the behavior processing unit, the control instruction set is processed to obtain the target control instruction, and the target control instruction is sent to the microcontroller unit so that the microcontroller unit drives the motor to work according to the target control instruction.

[0065] Specifically, the heterogeneous computing and control fusion chip control system 100 receives target data through sensors or other data interfaces. The target data may include environmental images, three-dimensional point cloud information or other perception information, which are used to describe the working scene and object characteristics of the robot or equipment.

[0066] The graphics processing unit 111 (GPU) and neural network processing unit 112 (NPU) in the computing unit 11 work together to process the received target data. Understandably, the GPU first performs environment modeling on the target data, generating target environment data, such as constructing a 3D spatial point cloud model or image feature map. Subsequently, the neural network processing unit 112 performs semantic analysis on the target environment data, identifying the category, location, and attributes of the target object, and outputs target perception data. To ensure processing efficiency, the GPU and NPU establish a high-speed dual-channel transmission through the communication network subunit 132, and, in conjunction with the array control unit 1122, switch between dense and sparse modes of the processing unit array, thereby optimizing energy consumption and latency while ensuring computational accuracy.

[0067] The behavior processing unit 121 in the control unit 12 receives target perception data, processes it further, and generates a corresponding set of control instructions. The behavior processing unit 121 uses the instruction prefetching subunit 1213 to preload high-frequency control instructions, and uses the dual-mode decoder subunit 1214 to decode and schedule the control instructions according to the task type, so as to realize the rapid differentiation and execution preparation of real-time and conventional control instructions.

[0068] The behavior processing unit 121 sends the processed control commands to the microcontroller unit 122 (MCU). The MCU 122 drives the actuator, such as a motor or servo motor, to complete the motion operation according to the target control command. During execution, the MCU 122 can combine force sensors or other feedback sensors for real-time monitoring. When an abnormality or emergency is detected, a hardware-level interrupt can be triggered to achieve emergency braking or motion adjustment, ensuring system safety and reliability. Throughout the process, the trust root subunits (1111, 1121, 1211, 1221) of each processing unit ensure the security and trustworthiness of data during on-chip transmission and execution. At the same time, the memory control subunit 131 accesses the three-level cache module 20 to achieve high bandwidth sharing and unified scheduling, improving the real-time performance and computational efficiency of the method.

[0069] This application provides a heterogeneous computing and control fusion chip control method. In the perception phase, the graphics processing unit 111 and the neural network processing unit 112 collaboratively process environmental data and target object information. Through the high-speed dual-channel communication network subunit 132 and the array control unit 1122, data flow efficiency is optimized, enabling rapid parsing and semantic analysis of large-scale 3D point cloud and image data, reducing latency caused by cross-chip transmission in traditional multi-chip architectures. In the decision-making phase, the behavior processing unit 121 utilizes the instruction prefetching subunit 1213 and the dual-mode decoder subunit 1214 to preload the control instruction set and identify task types, achieving efficient differentiation and scheduling of real-time and regular instructions, ensuring rapid generation of complex action decisions. In the execution phase, the microcontroller unit 122 drives the motor or actuator according to the target control instructions, combining sensor feedback to achieve microsecond-level response and emergency interrupt handling, improving action accuracy and safety reliability. By incorporating built-in trust root subunits 1111, 1121, 1211, and 1221 in each unit, the system establishes a hardware-level trusted verification and access isolation mechanism during data transmission and instruction execution, ensuring security and access control during operation. Simultaneously, by accessing the L3 cache module 20 through the memory control subunit 131, high-bandwidth data sharing and unified scheduling within the chip are achieved, reducing cross-module data movement, alleviating bandwidth bottlenecks, and significantly improving the efficiency of complex task processing. This method not only solves the problems of high communication latency, complex data paths, and lack of security mechanisms in traditional systems, but also enables perception computing, task decision-making, and motion control to form a complete closed loop within a single chip, achieving high real-time performance, high performance, and high security in computational and control fusion control.

[0070] The following example will further illustrate the heterogeneous computing and control fusion chip control method provided in this application.

[0071] Taking the heterogeneous computing and control fusion chip control system 100 as an example for controlling an intelligent robotic arm to perform precision grasping tasks.

[0072] First, the graphics processing unit 111 (GPU) receives target environment data from the camera, including high-resolution 4K images and 3D point cloud information. The GPU parses task instructions using its built-in micro-instruction decoder 1112, and, in conjunction with the tensor on-chip network subunit 1321, writes the data to the L3 cache module 20. Simultaneously, it triggers the neural network processing unit 112 (NPU) to enter dense mode, performing semantic analysis and target recognition on the environmental data to obtain target perception data, including workpiece position, pose, and feature information. During processing, the trust root subunits 1111 and 1121 within the GPU and NPU ensure secure on-chip data transmission and access isolation, preventing unauthorized access or data tampering.

[0073] Subsequently, the scheduling unit 14 generates accelerator binding tags based on task characteristics, binding the visual processing task to the cognitive cluster (GPU+NPU), and optimizes data transmission through the dual-channel design of the tensor on-chip network subunit 1321 to ensure high-bandwidth, low-latency communication. After processing, the neural network processing unit 112 transmits the target perception data to the behavior processing unit 121 (BPU). The instruction prefetching subunit 1213 of the behavior processing unit 121 preloads the grasping instructions, and the dual-mode decoder subunit 1214 distinguishes between regular trajectory planning instructions and real-time control instructions according to the task type, performing dual-mode decoding processing on the target control instructions. During this process, the trust root subunit 1211 within the behavior processing unit 121 ensures the trustworthiness of the instructions and access isolation.

[0074] Next, the microcontroller unit 122 (MCU) receives the target control command and drives the robotic arm to perform the grasping operation. The MCU ensures the safe execution of the command through its built-in root of trust subunit 1221, while simultaneously monitoring the force sensor feedback data in real time. Once it detects a force exceeding a preset threshold or an abnormal state, the MCU immediately triggers a hardware-level interrupt, interrupting BPU control to achieve emergency braking or obstacle avoidance. Throughout the process, the computing unit 11 and the control unit 12 access the level 3 cache module 20 through the memory control subunit 131, achieving high-speed data sharing and unified scheduling within the chip, reducing cross-module data transfer, and improving overall task execution efficiency.

[0075] In this example, the system can achieve a complete closed-loop operation from environmental perception, semantic parsing, task decision-making to execution control within a single chip, ensuring that the robotic arm's motion accuracy reaches the 0.1mm level, while achieving microsecond-level response and safety and reliability. This effectively solves the problems of large communication latency, complex data paths, and insufficient security in traditional multi-chip architectures.

[0076] This application also provides a robot 200, please refer to... Figure 3The diagram illustrates the hardware structure of a robot 200 capable of performing the methods described in the embodiments above. The robot 200 includes: at least one processor 210; and a memory 220 communicatively connected to the at least one processor 210. Figure 3 Taking a processor 210 as an example, the memory 220 stores instructions executable by at least one processor 210. These instructions are executed by the at least one processor 210 to enable it to perform the heterogeneous computing and control fusion chip control method described in the above embodiment. The processor 210 and the memory 220 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0077] The memory 220, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the heterogeneous computing and control fusion chip control method in the embodiments of this application. The processor 210 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 220, thereby implementing the heterogeneous computing and control fusion chip control method of the above embodiments.

[0078] Memory 220 may include a stored program area and a stored data area, wherein the stored program area may store the operating system and applications required for at least one function; the stored data area may store data created based on the use of the computing device, etc. Furthermore, memory 220 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 220 may optionally include memory remotely located relative to processor 210, and these remote memories may be connected to the computing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0079] One or more modules are stored in memory 220, and when executed by one or more processors 210, they execute the heterogeneous computing and control fusion chip control method of the above embodiment.

[0080] The above-described product can execute the method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the heterogeneous computing and control fusion chip control method of the embodiments of this application.

[0081] This application provides a non-volatile computer-readable storage medium storing computer-executable instructions. These instructions are executed by one or more processors to enable at least one processor to perform the heterogeneous computing and control fusion chip control method described in the above embodiments. For example, the non-volatile computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A heterogeneous control and compute fusion chip control system, comprising: The system comprises a central large core cluster module and a three-level cache module; The central large core cluster module comprises a computing unit, a control unit and an interconnection and memory control unit; The computing unit comprises a graphics processing unit and a neural network processing unit; the control unit comprises a behavior processing unit and a micro control unit; the graphics processing unit and the neural network processing unit are used for processing target data to obtain target perception data; the behavior processing unit is used for processing a control instruction set to obtain a target control instruction; and the micro control unit is used for driving a motor to work according to the target control instruction; The graphics processing unit, the neural network processing unit, the behavior processing unit and the micro control unit each have a root-of-trust subunit built-in; The interconnection and memory control unit comprises a memory control subunit and a communication network subunit; The computing unit and the control unit access the three-level cache module through the memory control subunit; and the graphics processing unit, the neural network processing unit, the behavior processing unit and the micro control unit are interconnected through the communication network subunit.

2. The heterogeneous control-fusion chip control system of claim 1, wherein, The central large core cluster module comprises a scheduling unit, which is in communication connection with the computing unit and the control unit, and is used for task distribution and security policy management.

3. The heterogeneous control-fusion chip control system of claim 1, wherein, The behavior processing unit comprises an instruction prefetching subunit and a dual-mode decoder subunit, which are in communication connection; the instruction prefetching subunit is used for preloading the control instruction set to obtain the target control instruction; and the dual-mode decoder subunit is used for dual-mode decoding processing according to a task type set corresponding to the control instruction set.

4. The heterogeneous control-fusion chip control system of claim 2, wherein, The graphics processing unit, the neural network processing unit and the behavior processing unit each comprise a micro instruction decoder subunit, which is used for obtaining a task type and a computing requirement according to a task instruction received from the scheduling unit.

5. The heterogeneous control-fusion chip control system of claim 1, wherein, The communication network subunit comprises a tensor on-chip network subunit, which has a dual-channel design built-in and is used for optimizing communication transmission among the graphics processing unit, the neural network processing unit, the behavior processing unit and the micro control unit based on the dual-channel design.

6. The heterogeneous control-fusion chip control system of claim 1, wherein, The neural network processing unit comprises an array control unit, which is used for dual-mode switching of a processing unit array.

7. A control method of a heterogeneous compute-control fusion chip, characterized by, The method is applied to the heterogeneous algorithm and control fusion chip control system of any one of claims 1 to 6, and the method comprises: receiving target data; processing the target data based on the graphics processing unit and the neural network processing unit to obtain target perception data; processing the target perception data to obtain a control instruction set; processing the control instruction set based on the behavior processing unit to obtain a target control instruction, and sending the target control instruction to the micro control unit, so that the micro control unit drives a motor to work according to the target control instruction.

8. The heterogeneous control-fusion chip control method of claim 7, wherein, The target data is processed based on the GPU and the neural network processing unit to obtain target perception data, including: The target data is processed based on the GPU to obtain target environment data through environment modeling processing; The target environment data is processed based on the neural network processing unit to obtain the target perception data through semantic analysis processing.

9. A robot, characterized in that Comprise: At least one processor; And, The memory is in communication connection with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 7-8.

10. A non-transitory computer readable storage medium, comprising: The non-volatile computer readable storage medium stores computer executable instructions, when the computer executable instructions are executed by the robot, the robot executes the method of any one of claims 7-8.

Images