Multi-level heterogeneous integrated chip task processing method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to service scenes of pension service, financial science and technology, medical health and the like, and discloses a multi-level heterogeneous integrated chip task processing method, device, equipment and medium, and the method comprises the steps: constructing a multi-level heterogeneous integrated chip composed of a perception processing layer, an intelligent decision-making layer and a driving control layer, the layers are connected through a vertical interconnection structure; receiving multi-modal task data and extracting features to generate feature vectors; inputting the feature vector into a neuromorphic processing unit to determine a task decision result; converting the decision result into a driving signal to control an execution device; adjusting synaptic weights based on the feedback signal; and monitoring chip operation state parameters and dynamically adjusting processing frequency and structure parameters. By integrating multi-modal sensing, neuromorphic decision and a dynamic feedback mechanism, data sensing, decision and execution processing are completed in a chip, and the real-time performance and the calculation efficiency are improved by combining operation state monitoring and adjusting frequency and structure.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multi-level heterogeneous integrated chip task processing method, apparatus, device, and storage medium. Background Technology

[0002] In the development of embodied intelligent systems, intelligent agents typically need to achieve efficient perception, rapid decision-making, and precise control of multimodal task data in dynamic environments. However, existing chip architectures face several technical bottlenecks in handling multimodal data processing and decision control, mainly manifested in insufficient coupling of the integrated structure, high inter-layer communication latency, and poor collaborative efficiency of heterogeneous functional units, making it difficult to meet the high requirements of embodied intelligent systems in terms of processing efficiency, response speed, and power consumption control.

[0003] In the fintech sector, embodied intelligence technologies are increasingly being applied to smart teller machines, human-machine collaboration terminals, and interactive service systems. These systems require rapid fusion and real-time response to data from various sensors, including vision and touch, to perform tasks such as customer identification, behavior assessment, and risk command response. However, the decentralized design of existing chip architectures in the data flow path leads to significant latency in the overall processing chain, making it difficult to guarantee the real-time transmission of critical control signals and the rapid issuance of task commands. This impacts the system's service efficiency and interactive reliability in high-concurrency financial business scenarios.

[0004] In the healthcare sector, intelligent wearable devices, intelligent rehabilitation assistive systems, and remote medical robots widely employ multimodal sensors for environmental perception and physiological monitoring. However, existing chips suffer from inflexible computational resource scheduling mechanisms when processing large-scale visual and tactile sensory data. They often fail to provide stable sensing computing capabilities under low power consumption constraints, particularly during long-term operation or high-frequency use, leading to power accumulation and overheating issues. This limits the practicality of such systems in scenarios with extremely high requirements for response time and stability, such as bedside monitoring and surgical assistance. Summary of the Invention

[0005] The main objective of this invention is to provide a multi-level heterogeneous integrated chip task processing method, apparatus, device, and storage medium, aiming to solve the technical problem that the existing technology lacks a multi-level heterogeneous integrated chip architecture that can achieve efficient collaborative processing of perception, decision-making, and execution within the chip, and supports dynamic optimization of neuromorphic structure and processing parameters based on task feedback.

[0006] To achieve the above objectives, the present invention provides a multi-level heterogeneous integrated chip task processing method, comprising:

[0007] Construct a multi-level heterogeneous integrated chip that includes a perception processing layer, an intelligent decision-making layer, and a drive control layer connected through a vertical interconnect structure;

[0008] The perception processing layer receives multimodal task data and performs parallel feature extraction on the multimodal task data to generate multimodal feature vectors.

[0009] The multimodal feature vector is input into the neuromorphic processing unit of the intelligent decision layer, and the multimodal feature vector is processed based on synaptic weights in the neuromorphic processing unit to obtain the task decision result;

[0010] The drive control layer converts the task decision result into a drive control signal, and controls the target execution device to perform actions according to the drive control signal;

[0011] Based on the task feedback signal returned after the action is performed, the synaptic weights of the neuromorphic processing unit are adjusted in real time.

[0012] The operating status parameters of the multi-level heterogeneous integrated chip are monitored, and the processing frequency and structural parameters of the multi-level heterogeneous integrated chip are dynamically adjusted based on the operating status parameters.

[0013] Furthermore, to achieve the above objectives, the present invention provides a multi-level heterogeneous integrated chip task processing device, comprising:

[0014] Chip-level building module, used to build a multi-level heterogeneous integrated chip including a perception processing layer, an intelligent decision-making layer and a drive control layer connected by a vertical interconnect structure;

[0015] The multimodal perception module is used to receive multimodal task data through the perception processing layer, and to perform parallel feature extraction on the multimodal task data to generate multimodal feature vectors.

[0016] The neural decision-making module is used to input the multimodal feature vector into the neuromorphic processing unit of the intelligent decision-making layer, and process the multimodal feature vector based on synaptic weights in the neuromorphic processing unit to obtain the task decision result;

[0017] An execution control module is used to convert the task decision result into a drive control signal through the drive control layer, and control the target execution device to perform actions according to the drive control signal;

[0018] The synaptic weight learning module is used to adjust the synaptic weights of the neuromorphic processing unit in real time based on the task feedback signal returned after the action is performed.

[0019] An adaptive operation status module is used to monitor the operation status parameters of the multi-level heterogeneous integrated chip and dynamically adjust the processing frequency and structural parameters of the multi-level heterogeneous integrated chip based on the operation status parameters.

[0020] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a multi-level heterogeneous integrated chip task processing program stored in the memory and executable on the processor, wherein when the multi-level heterogeneous integrated chip task processing program is executed by the processor, it implements the steps of the multi-level heterogeneous integrated chip task processing method as described above.

[0021] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a multi-level heterogeneous integrated chip task processing program, wherein when the multi-level heterogeneous integrated chip task processing program is executed by a processor, it implements the steps of the multi-level heterogeneous integrated chip task processing method described above.

[0022] Beneficial Effects: This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as elderly care services, fintech, and healthcare. It discloses a multi-level heterogeneous integrated chip task processing method, apparatus, device, and medium, comprising: constructing a multi-level heterogeneous integrated chip, which consists of a perception processing layer, an intelligent decision-making layer, and a drive control layer connected by a vertical interconnect structure; receiving multimodal task data and performing parallel feature extraction to generate multimodal feature vectors; inputting the multimodal feature vectors into a neuromorphic processing unit to determine the task decision result; converting the task decision result into a drive control signal to control the target execution device to perform actions; adjusting synaptic weights based on task feedback signals; monitoring chip operating status parameters and dynamically adjusting the chip's processing frequency and structural parameters based on these parameters. This invention integrates multimodal perception, neuromorphic decision-making, and dynamic feedback optimization mechanisms to achieve closed-loop processing of data perception, intelligent decision-making, and control execution within the chip. Simultaneously, by combining operating status monitoring with dynamic adjustment of the chip's frequency and structure, it improves the real-time performance, computational efficiency, and system adaptability of data processing, meeting the needs of embodied intelligence for high-performance and low-power chips. Attached Figure Description

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0024] Figure 1 This is a schematic diagram of an application environment for a multi-level heterogeneous integrated chip task processing method according to an embodiment of the present invention;

[0025] Figure 2 This is a flowchart illustrating an embodiment of the multi-level heterogeneous integrated chip task processing method of the present invention;

[0026] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the multi-level heterogeneous integrated chip task processing device of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0028] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0030] The multi-level heterogeneous integrated chip task processing method provided in this invention can be applied to, for example... Figure 1 In this application environment, the user terminal communicates with the server via a network. The server can construct a multi-level heterogeneous integrated chip through the user terminal. This chip consists of a perception processing layer, an intelligent decision-making layer, and a drive control layer connected by a vertical interconnect structure. It receives multimodal task data and performs parallel feature extraction to generate multimodal feature vectors. The multimodal feature vectors are input into the neuromorphic processing unit to determine the task decision result. The task decision result is converted into drive control signals to control the target execution device to perform actions. Synaptic weights are adjusted based on task feedback signals. The chip's operating status parameters are monitored, and the chip's processing frequency and structural parameters are dynamically adjusted based on these parameters. This invention achieves closed-loop processing of data perception, intelligent decision-making, and control execution within the chip by integrating multimodal perception, neuromorphic decision-making, and dynamic feedback optimization mechanisms. Simultaneously, by combining operating status monitoring with dynamic adjustment of the chip's frequency and structure, it improves the real-time performance, computational efficiency, and system adaptability of data processing, meeting the needs of embodied intelligence for high-performance and low-power chips. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster composed of multiple servers. The present invention will now be described in detail through specific embodiments.

[0031] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the multi-level heterogeneous integrated chip task processing method provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0032] like Figure 2 As shown, the multi-level heterogeneous integrated chip task processing method proposed in this invention includes the following steps:

[0033] S10, constructing a multi-level heterogeneous integrated chip including a perception processing layer, an intelligent decision-making layer, and a drive control layer connected through a vertical interconnect structure;

[0034] In this embodiment, when constructing a multi-level heterogeneous integrated chip consisting of a perception processing layer, an intelligent decision-making layer, and a drive control layer, the three layers need to be connected into a single functional module through a vertical interconnect structure. In terms of hierarchical structure, the perception processing layer is used for data interaction with external sensing devices, undertaking data access and preprocessing tasks; the intelligent decision-making layer receives feature data from the perception processing layer and understands and judges the task state based on neuromorphic structures; the drive control layer controls the target execution device based on signals generated by the intelligent decision-making layer. In implementing this multi-level structure, the chip's functional areas can be stacked within the same chip package using a vertical stacking process, thereby significantly reducing signal latency, improving response speed, and saving lateral area. The vertical interconnect structure can be formed using through-silicon via (TSV) technology, which allows for the establishment of vertical conductive paths between multiple chip layers and supports high-density, high-speed data interaction.

[0035] The sensing processing layer can be implemented based on nanoscale carbon nanotube field-effect transistor arrays. These devices possess excellent electrical performance and miniaturization capabilities, making them suitable for parallel processing of multimodal sensing data. The sources of sensing data include, but are not limited to, visual images, tactile pressure signals, and inertial measurement unit outputs. After receiving different types of sensing data, the sensing processing layer can be configured with multiple data input interfaces and a signal normalization module. Through data format conversion and signal preprocessing, synchronous fusion of multi-channel data can be achieved. To meet the processing requirements of different modalities, parallel spatial convolutional networks and temporal convolutional network arrays can be designed to process static image-like information and dynamic temporal signals respectively, and the multimodal feature extraction task can be completed on-chip.

[0036] In chip manufacturing, the three-layer stack can be achieved using heterogeneous integration technology. This involves vertically integrating the sensing / processing layer, intelligent decision-making layer, and driving / control layer—each employing different manufacturing materials and processes—using three-dimensional integration technology. To improve overall electrical performance and thermal stability, high thermal conductivity materials can be introduced between the stacked layers for thermal diffusion. Simultaneously, extreme ultraviolet (EUV) lithography is used at critical signal links to precisely define nanoscale wire patterns, thereby improving interconnect accuracy and reducing crosstalk. Within the stacked structure, to achieve high-speed communication between the sensing layer and the decision-making layer, and between the decision-making layer and the control layer, first and second vertical interconnect channels are constructed. The conductive paths of these channels are formed by depositing metal materials through through-silicon vias (TSVs), ensuring stable and reliable data transmission between layers.

[0037] To further enhance the collaborative scheduling capabilities of on-chip resources, a processor core, on-chip memory, and a dedicated accelerator module are integrated into the chip architecture. The processor core is used for overall computational flow scheduling and control logic execution, the on-chip memory is used to cache perceived data, weight parameters, and intermediate computation results, and the dedicated accelerator module is used to accelerate specific data processing paths, such as vectorization operations on specific modalities like images and speech. During chip integration, to avoid resource contention and data bottlenecks, an on-chip interconnect network and a dynamic bus arbitration mechanism can be introduced to ensure independent and efficient data channels between multiple functional units.

[0038] To achieve the above structure, silicon-based 3D packaging technology can be used for stacking processes. This integrates the CMOS-based sensing and processing module, the neuromorphic computing module based on two-dimensional materials, and the power drive module manufactured using GaN technology into a unified functional chip structure. Vertical interconnect paths can be achieved by first pre-reserving vertical vias on each chip wafer, then filling them with conductive materials such as tungsten or copper using deep reactive ion etching and chemical vapor deposition, and finally completing interlayer bonding through reflow soldering. Between the neuromorphic processing module and the sensing interface, to ensure the synchronization of multimodal data and consistency of processing latency, an interface synchronization controller can perform buffering and timing alignment on various data streams and dynamically adjust the sampling accuracy.

[0039] In various tasks such as graphics processing, pressure recognition, or inertial analysis, different sensing units and processing circuit structures can be selected based on the input modal characteristics. For example, in high frame rate video processing, the perception processing layer can deploy a high-concurrency pixel array and its auxiliary analog-to-digital converter; in high-precision pressure sensing tasks, a low-noise charge amplifier and a bandgap reference source can be selected for signal conditioning. In neuromorphic processing structures, an array of neurons with plastic synaptic weights can be constructed using two-dimensional materials such as molybdenum disulfide, and the synaptic current change process can be simulated through a variable resistance structure. The neuronal pulse firing mechanism is implemented through on-chip digital circuitry to achieve event-driven processing logic. In the drive control module, high-voltage wideband modulation is achieved through GaN devices to control the target device's behavior in real time.

[0040] Example Description: In healthcare scenarios, this chip architecture can be used to build surgical-assisted robots with multimodal perception and motion control capabilities. This robot uses visual perception of lesion images and tactile sensing to analyze changes in tissue stiffness, generating real-time control signals for surgical tools to achieve high-precision responses to minimally invasive procedures. In this scenario, the multi-layered heterogeneous modules within the chip architecture efficiently process video and force data while ensuring millisecond-level feedback for motion control.

[0041] In fintech business scenarios, this chip can be integrated into self-service interactive terminals or smart counter devices to identify user behavior patterns and control interactive actions. For example, it can acquire user commands and operation feedback through cameras and tactile sensors to execute risk warning actions or adjust business processes. The chip structure can process image and touch commands simultaneously, enabling real-time risk control decisions and execution, and continuously optimizing response strategies based on historical feedback, thereby improving the decision-making accuracy and response stability of the device in complex scenarios.

[0042] This embodiment integrates sensing, decision-making, and control functions into a single chip structure using a vertical integration approach, and utilizes a vertical interconnect structure to achieve high-speed communication connections. This significantly shortens the data transmission path, improves system response speed, reduces power consumption, and increases integration density. This structure supports multimodal parallel processing capabilities, adapting to the complex and diverse needs of task data. Furthermore, its neuromorphic architecture supports adaptive adjustment and online learning capabilities, further enhancing the system's intelligent response and robustness.

[0043] S20, receive multimodal task data through the perception processing layer, and perform parallel feature extraction on the multimodal task data to generate a multimodal feature vector;

[0044] In this embodiment, the perception processing layer, a first-layer computing unit integrated into the chip, is responsible for collecting task-related raw input data from the external environment and performing preliminary preprocessing and feature encoding operations. The structure of the perception processing layer is derived from a multi-dimensional collaborative design of convolutional neural networks, temporal networks, and feature fusion modules, enabling concurrent processing of multimodal signals such as images, speech, motion, and touch. The data acquisition interfaces deployed in the perception processing layer support level compatibility and protocol decoding with multiple types of sensors, enabling physical interface access including USB, I2C, SPI, and MIPI, ensuring stable data transmission at the chip level.

[0045] Multimodal task data refers to information inputs of different categories simultaneously involved in a task scenario, characterized by cross-modal, unstructured, and spatiotemporally coupled features. Common modalities include visual images, speech signals, ambient sound, motion trajectories, tactile pressure, and electrophysiological signals, which are collected simultaneously by different sensors within the same processing cycle, exhibiting information heterogeneity. Unified processing of multimodal task data is a key step in improving the environmental adaptability and understanding capabilities of intelligent processing systems.

[0046] Parallel feature extraction refers to the construction of specific feature extraction pathways for different modalities within the perceptual processing layer. Each pathway can independently run parallel convolutional units, temporal convolutional units, attention mechanism modules, or frequency domain transformation units. Spatial data can have its local structure extracted through two-dimensional convolutional networks, temporal data can have its temporal dependencies extracted through one-dimensional convolutions or recurrent structures, and tactile data can have its mechanical distribution features extracted using multi-channel filtering networks. The pathways are decoupled, enabling the perceptual processing layer to simultaneously extract features from multiple input modalities within the same time period.

[0047] Generating multimodal feature vectors involves encoding, aligning, and fusing the feature representations of each modality, and outputting the fusion result as a unified low-dimensional or sparse feature vector. This feature vector, through dimensionality reconstruction and semantic compression, preserves the collaborative information between the multimodalities and can be directly recognized by subsequent intelligent processing modules. Fusion methods can include concatenation, attention weighting, tensor compression, etc., and the specific method needs to be dynamically selected based on the spatiotemporal coupling structure of the data modalities.

[0048] In one implementation, the perception processing layer consists of two sets of parallel pathways. The first set of pathways receives RGB image data from the camera, extracts spatial features through a two-dimensional convolutional network, and further enhances the semantic expression of the image through a multi-scale pyramid structure. The second set of pathways receives three-axis time-series data from a motion accelerometer, extracts motion rhythm features using a one-dimensional temporal convolutional structure, and combines a learnable attention mechanism to focus on key change segments.

[0049] The speech signal can also be preprocessed by configuring a standardized input channel, including spectrogram transformation, logarithmic amplitude normalization and high-frequency filtering, and then the processing result is input into the frequency domain convolution module to extract acoustic features.

[0050] Furthermore, in scenarios with tactile sensor input, an adaptive filter bank can be used to model the tactile pressure matrix, dynamically mapping pressure changes at different contact points into a heatmap, which is then input into a sparse convolutional unit to extract tactile feature vectors. The outputs of all pathways are then unified into multimodal feature vectors at the end of the sensory processing layer through feature concatenation and linear transformation.

[0051] In different implementation methods, the number of pathways, the depth of the feature extraction network, and the modal fusion strategy in the perception processing layer can be flexibly adjusted according to chip area limitations, power consumption budget, and response time requirements.

[0052] Example Description: In mobile intelligent medical devices, to achieve real-time health monitoring of elderly individuals in their home environment, the edge chip receives physiological and behavioral data from multiple sensors in parallel through a perception processing layer. These sensors include an electrocardiogram (ECG) acquisition module, a body motion accelerometer, a voice interface, and sleep environment light and temperature sensing modules. The multi-pathway structure in the perception processing layer independently performs feature extraction for each modality: a one-dimensional convolutional network extracts temporal patterns from the ECG, the action pathway captures abnormal nighttime behavioral changes, the voice pathway identifies abnormal breathing sounds, and the environmental pathway assesses sleep disturbance factors. All features are fused into a unified feature vector in the perception processing layer and output to the subsequent decision layer in real time, without relying on cloud processing. This enables edge detection of potential health risks even in environments with offline or weak networks, effectively meeting the intelligent perception needs of medical and health scenarios such as long-term chronic disease management and emergency event early warning.

[0053] In edge devices used in branch intelligent teller systems or mobile sales terminals, to achieve real-time perception of customer behavior and risk situations, the perception processing layer embedded in the chip simultaneously receives image sequences from cameras, voice input, ID card data, and touch operation records. The image processing channel uses convolutional networks to extract facial expression features and suspicious occlusion patterns; the voice processing channel analyzes tone and semantic deviations; the touch processing channel records operation rhythm and frequency; and the ID card recognition channel encodes identity data structure features. All modal data are processed in parallel through their respective channels and then fused into a unified multimodal feature vector, which is directly fed back to the intelligent judgment module at the edge to perform risk situation prediction. This processing mechanism can complete data parsing within milliseconds of user operation, supporting rapid early warning of suspicious behavior in branch environments without a continuous network connection, improving business security and service efficiency.

[0054] This embodiment constructs a multi-path heterogeneous sensing structure, which enables parallel feature extraction of multiple modal data within the chip. This significantly improves the sensing response speed and modal collaborative expression capability, avoiding the latency accumulation and modal fragmentation problems caused by traditional serial processing. As a result, it enhances the chip system's ability to quickly understand the environmental state and perform task-driven linkage processing in complex task scenarios.

[0055] S30, the multimodal feature vector is input into the neuromorphic processing unit of the intelligent decision layer, and the multimodal feature vector is processed based on synaptic weights in the neuromorphic processing unit to obtain the task decision result;

[0056] In this embodiment, a multimodal feature vector refers to a high-dimensional representation formed by extracting and fusing data from multiple modalities through parallel pathways in the perception processing layer. It includes feature dimensions from different sources such as images, speech, actions, text, and environmental sensing. This feature vector needs to be fed into the intelligent decision-making layer to perform cognitive processing operations such as task judgment, state estimation, or generation of behavioral control instructions.

[0057] A neuromorphic processing unit (NPU) is a computational module constructed by mimicking the structure of a biological neural network. It typically includes synaptic weight modules, pulse transmission modules, and neuronal activation structures. This unit uses electrical charge, pulse voltage, or materials that mimic synaptic dynamics to achieve information processing, and features low power consumption, high parallelism, and asynchronous execution.

[0058] The specific operational process includes: First, the multimodal feature vectors are input into a synaptic weight module with an adjustable weighting mechanism to perform weighted fusion of feature channels between different modalities. Then, the fused signal is processed through nonlinear functions, such as impulse response shaping and threshold activation, to generate intermediate neuron responses. Next, the response signal is converted into a pulse stream in the time or amplitude domain, transmitted through the synaptic network within the neuromorphic unit, and stimulates interactions between multiple neurons to form a recognition response to the input task scenario.

[0059] Ultimately, by identifying the activation patterns of certain output pathways within a stable population of output neurons, the conclusions, behavioral categories, or policy decisions for the current task are determined. This output exhibits sparsity, parallelism, and fine-tunability, adapting to subsequent control or feedback mechanisms.

[0060] To enhance robustness, pulse coding rule variation control, adaptive threshold dynamic adjustment mechanism, and external synaptic weight reprogramming capability can be introduced into this processing unit.

[0061] In implementing this processing operation, a synaptic array can be constructed using memristors based on molybdenum disulfide (MoS2) or vanadium oxide materials, allowing the input feature weights to be adjusted via voltage programming; alternatively, predefined weight parameters in the storage unit can be called through a digital control interface to achieve structurally controllable static neural network decision-making.

[0062] The neuron activation module can employ hardware-implemented nonlinear function units, such as comparator amplifier circuits built on CMOS integrated circuits, or perform activation function calculations such as ReLU and Sigmoid by integrating a neural network accelerator kernel. The pulse delivery module can combine temporal coding and event-driven mechanisms to achieve time-domain processing of dynamic inputs, significantly reducing overall energy consumption while ensuring real-time performance.

[0063] In practical applications, the depth and width of the neural network can be adjusted according to the task category. For example, a temporal recursive channel can be introduced for time-series recognition tasks, a spatial attention mechanism can be introduced for image decision-making tasks, and a dedicated sub-channel can be specified in a heterogeneous chip structure to achieve directional computing, thereby improving processing efficiency in specific scenarios.

[0064] Example: In intelligent monitoring devices for chronic diseases in the elderly, when the multimodal feature vector contains information from heart rate change trends, body movement patterns, and voice anomaly detection, it is input into the neuromorphic processing unit for judgment. This processing unit can infer in real time whether the user is in a state of being unable to move after a fall, whether there is apnea or other emergency anomalies, and output the corresponding alarm category and processing priority, which can be directly used by edge devices to trigger call services or execute emergency actions without uploading to the cloud.

[0065] In smart teller terminals, customer facial recognition features, voice tone analysis results, and touch operation data are integrated into a multimodal feature vector and input into a neuromorphic processing unit to perform suspicious behavior identification tasks. When the judgment result indicates abnormal operation patterns, identity verification failure, or a customer with a high risk tendency, the device can directly block the transaction process or notify the back-end for manual intervention. This ability to make real-time decisions at the edge significantly reduces response delays to potential fraud incidents and ensures counter security.

[0066] This embodiment utilizes a neuromorphic processing unit with a heterogeneous neuronal structure and synaptic plasticity to achieve rapid decoding of multimodal feature vectors and task decision generation, overcoming the problem of traditional neural networks' heavy reliance on central processing unit or graphics accelerator resources. Employing a neuromorphic structure can significantly reduce energy consumption and shorten response latency, making it suitable for high-frequency task switching and high-concurrency scenarios. It exhibits extremely high operating efficiency and adaptability in edge devices, effectively improving the autonomy and real-time performance of task execution.

[0067] S40, the task decision result is converted into a drive control signal through the drive control layer, and the target execution device is controlled to perform an action according to the drive control signal;

[0068] In this embodiment, the task decision result is an instruction output generated by the neuromorphic processing unit within the intelligent decision layer based on multimodal feature vectors. This is typically represented by numerical vectors, activation states, or impulse patterns to indicate the target behavior or state switching signal to be executed. The drive control layer is responsible for receiving this task decision result and performing signal conversion and control signal output operations to drive specific physical or logical execution modules to complete the actual actions.

[0069] A signal mapping mechanism is needed between the task decision results and the drive control signals to resolve the task category and decision mode. This mechanism can be implemented based on lookup tables in hardware circuits, programmable logic modules, or dedicated mapping function logic. The mapping logic can encode the high-dimensional decision vector into a sequence of electrical signals with execution meaning, and further modulate it into drive signals that can be resolved by the control interface.

[0070] Drive control signals generally contain two types of information: control fields and execution parameters. Control fields define the action type (e.g., on, off, acceleration, steering), while execution parameters provide specific values ​​(e.g., rotation angle, torque magnitude, duration). This signal can be transmitted through various interfaces, such as SPI and I / O. 2 C, CAN, or PWM, etc., to adapt to the input requirements of different target actuators.

[0071] The target execution device encompasses robotic arms, actuators, sensor arrays, voice broadcasters, image display components, fluid control units, and more. By receiving control signals from the drive control layer, the execution device completes specific actions under the guidance of instructions from the locally embedded control system. The drive control layer can further integrate a feedback receiving module to read back the status of the execution results, providing support for subsequent task judgment and weight adjustment.

[0072] This process can support concurrent triggering of multiple task paths, and can also manage the priority of concurrent control signals through scheduling control strategies to ensure stable execution of action logic under timing, resource and power consumption constraints.

[0073] In practical implementation, the drive control layer can be composed of an integrated microcontroller unit (MCU) or field-programmable gate array (FPGA) to realize real-time mapping and dynamic conversion between task decision results and drive control signals. For example, a state machine-based signal modulation structure can be used to map multi-dimensional task outputs into a control instruction set, and the corresponding GPIO control signals can be generated in the MCU to drive the stepper motor to perform displacement operations.

[0074] For multi-output control task structures, a multi-channel control architecture can be constructed, setting multiple sets of parallel control logic in the FPGA and connecting them to different target execution devices, enabling multiple actions to be executed collaboratively within the same time window. For latency-sensitive execution devices, such as injection systems in medical devices or locking mechanisms in financial terminals, a low-latency interrupt mechanism can be introduced to ensure that high-priority control signals are preempted for execution.

[0075] During signal modulation, the system can switch between PWM modulation, level drive, pulse control, and continuous signal output depending on the controlled object. For example, in applications that drive the opening and closing of mechanical fingers, PWM signals control the opening and closing angle, while continuous level control is used for precise position setting when driving a camera to adjust its direction.

[0076] Example Description: In a home rehabilitation training robot, the intelligent decision-making layer determines the patient's rehabilitation stage based on their movement characteristics and generates a task decision to adjust the assistive force. The drive control layer converts this result into a precise torque control signal, which is output to the motor controller via PWM modulation. This drives the lower limb support mechanism to adjust the assistive force according to the patient's gait changes, improving training comfort and safety while reducing the patient's need for manual intervention.

[0077] In unmanned financial service terminals, the intelligent decision-making layer outputs a security anomaly level judgment based on user facial expressions, operation paths, and voice emotion recognition. The drive control layer converts this judgment result into a control signal to execute the locking mechanism, triggering the counter door lock to close via the GPIO interface. Simultaneously, it calls the voice module to play guiding voice messages to guide the customer through subsequent identity verification steps. This direct pathway from cognitive control to physical response, completed within the edge device, effectively improves the device's risk control capabilities and response efficiency.

[0078] This embodiment establishes a closed-loop path from cognitive judgment to physical execution by setting up a drive control layer to parse and convert task decision results, thereby improving the response speed and decision-making efficiency in the multimodal task execution chain. As a bridge connecting intelligent decision-making and physical execution, the drive control layer features flexible configuration, low latency, and strong adaptability, enabling the entire integrated chip to have higher scenario-based deployment capabilities and dynamic execution efficiency. This mechanism avoids external communication or server response latency, enhancing the system's execution reliability under high real-time requirements.

[0079] S50, based on the task feedback signal returned after the action is performed, adjust the synaptic weights of the neuromorphic processing unit in real time;

[0080] In this embodiment, after an action is performed, the target execution device generates a result during its interaction with the environment. This result can be collected by sensors in the system and form a task feedback signal. The task feedback signal is a complex set of information, typically including whether the action was completed, the quality indicators of completion, environmental state change data, and target response delay. This information can be digitized and transmitted to the intelligent decision-making layer for reinforcement learning.

[0081] The analysis of feedback signals includes multidimensional feature extraction and index normalization, transforming raw sensor data into task completion indicators that can be recognized and utilized by neuromorphic structures. For example, displacement error, time delay, and energy consumption changes can be weighted and combined to form a normalized score. This score reflects the quality of the current action and serves as a reward signal in the synaptic weight update mechanism during reinforcement learning.

[0082] The adjustment of synaptic weights is based on the synaptic connection structure within the current neuromorphic processing unit, with each neuron evolving its state through the transmission of pulse signals. The task completion index generated in the feedback signal is matched with the current neural activity path to determine whether the path generated by the current action is worth strengthening or inhibiting, thus determining the direction of synaptic weight adjustment. Positive reinforcement increases the synaptic weights on the path, making it easier to activate in future decisions; negative reinforcement decreases the synaptic weights on the path, reducing the path's involvement.

[0083] The weight adjustment is performed by the control circuitry, and the adjustment magnitude can be determined based on STDP (Spike-Timing-Dependent Plasticity) or synaptic update matrix rules, through the impulse response relationship within a time window. In the chip architecture, the adjusted synaptic weights are directly written to the corresponding weight registers and stored in on-chip memory for subsequent inference.

[0084] The entire process does not rely on external servers or high-performance processors. It can independently complete the three stages of feedback acquisition, feedback evaluation, and weight update in a closed loop at the edge device, thereby realizing the implementation of embedded reinforcement learning capabilities.

[0085] In practical implementation, the feedback signal acquisition module can be integrated into the execution feedback path of the drive control layer to acquire the response time, accuracy, and target state achievement of the current action. The feedback signal is transmitted to the front-end buffer of the neuromorphic processing structure through the local control bus, and is parsed into standardized task completion indicators by the feedback interpretation module.

[0086] Depending on the architecture of different neuromorphic processing units, various weight adjustment rules can be adopted. For example, in a spiking neural network structure, a selective weight update strategy based on the current activation path can be set, dynamically adjusting the learning rate and update amplitude thresholds to avoid overfitting or response saturation. At the circuit level, weight parameters can be stored in an on-chip digital register array or resistive memory (such as RRAM), supporting fine-grained adjustment.

[0087] For neuromorphic structures with multi-task concurrency, a grouping weight update strategy can be set, with each task path having an independent feedback update cycle and update range. The system can dynamically determine the current task priority through a hardware scheduler, and perform sampling control and update window management on feedback signals to improve multi-task adaptability.

[0088] After the weights are updated, the register state is synchronized to the input path of the neural computing unit to ensure that the latest synaptic connection strength is accurately applied in subsequent inference.

[0089] Example Explanation: In assistive wearable devices for the elderly, after the intelligent control chip guides the patient to complete gait training movements, pressure sensors and accelerometers can detect information such as stride length and stabilization time. Feedback signals convert these indicators into task completion rates. The neuromorphic unit enhances or inhibits the current synaptic pathway, making subsequent control strategies more aligned with the patient's gait rhythm, thereby achieving personalized evolution of the gait reconstruction model and improving rehabilitation training effectiveness.

[0090] In an edge-deployed smart locker system, after a user interacts via facial recognition and voice, the device opens the locker. The system collects user response speed, command execution consistency, and locker door status, generating task feedback signals. If the operation is judged to be highly consistent, the neuromorphic processing unit will increase the weight of the current path; if the operation is judged to be suspicious, the response strength of that path will be weakened to prevent similar strategies from being used again. This gradually builds the optimal anti-fraud decision path, enabling dynamic risk control capabilities to be embedded within the terminal device.

[0091] This embodiment utilizes a dynamic synaptic weight adjustment mechanism based on feedback signals to enable the neuromorphic processing unit to learn and self-optimize after task execution. During multimodal task processing, this mechanism can update the connection structure in real time based on environmental changes and execution results, thereby improving the accuracy and strategy adaptability of inference for the next task. Without cloud synchronization and model retraining, the system can achieve edge-based autonomous learning and dynamic adaptation, significantly improving the chip's intelligent response capabilities and long-term operating efficiency.

[0092] S60, monitor the operating status parameters of the multi-level heterogeneous integrated chip, and dynamically adjust the processing frequency and structural parameters of the multi-level heterogeneous integrated chip based on the operating status parameters.

[0093] In this embodiment, the multi-level heterogeneous integrated chip consists of a perception processing layer, an intelligent decision-making layer, and a drive control layer. These layers are physically coupled through a vertical interconnect structure, forming a hardware unit with functional division of labor and collaborative computing capabilities. During continuous operation of this chip, the load status, power consumption distribution, and temperature gradient of different layers dynamically change with the task type and complexity. Without state awareness and adaptive structural adjustment, problems such as computational bottlenecks, excessive energy consumption, or overload of a certain layer can easily occur.

[0094] Operating status parameters refer to measured values ​​that reflect the current operating performance and load status of the chip. Specifically, these include the operating temperature, power consumption level, task processing latency, cache hit rate, bus congestion rate, and activation frequency of each processing path for each core unit. These parameters can be collected in real time by sensing modules, monitoring probes, and hardware performance counters embedded in each layer, and uploaded to the status awareness module for unified management.

[0095] The processing frequency within the chip represents the clock frequency of the control logic and computing modules in each processing layer. This frequency can be adjusted via an on-chip phase-locked loop or a digital frequency synthesis module. Structural parameters represent the active configuration of the processing units, such as the number of concurrent channels in the sensing processing layer, the activation ratio of neurons in the intelligent decision-making layer, and the switching state of execution units in the drive control layer. These parameters are dynamically set through control logic, a scheduler, or configuration registers. By adaptively adjusting the processing frequency and structural parameters, the current task load and energy efficiency requirements can be effectively matched.

[0096] The aforementioned adjustment mechanism does not rely on external software control and can respond quickly based on operating status parameters through the on-chip control module, achieving dynamic optimization at the hardware level. This mechanism is particularly suitable for scenarios in embodied intelligence systems where task states change frequently, response time limits are strict, and computing resources fluctuate drastically.

[0097] In actual deployment, the operational status parameter monitoring module is physically integrated with the processing core at each level, achieving real-time sampling through devices such as temperature sensors, voltage samplers, power consumption counters, and performance event probes. All monitoring data is aggregated and transmitted to the chip's control and coordination unit, which embeds a finite state machine or a lightweight rule engine to determine whether parameter adjustments are needed based on preset scheduling rules.

[0098] The processing frequency can be adjusted using a hierarchical dynamic voltage-frequency adjustment mechanism (DVFS), which independently sets the frequency and supply voltage for different levels. If the sensing processing layer experiences a rapid temperature rise and a decrease in cache access saturation, its frequency is reduced to alleviate thermal load; if the intelligent decision-making layer detects a significant increase in the number of synaptic activations and an increase in task processing latency, its frequency is increased to enhance processing capabilities.

[0099] Adjustments to structural parameters rely on reconfigurable processing units. For example, the intelligent decision-making layer can enable more neuron arrays or suppress some low-activity regions, while the drive control layer can selectively enable multiple execution control channels to enhance drive responsiveness. All changes in structural states can be transmitted via control logic to the on-chip structural control register, thereby achieving real-time configuration reconfiguration at the cycle level.

[0100] At the end of each cycle, the system evaluates the effectiveness of the current adjustment strategy. If the strategy is found to cause a deterioration in processing latency or an increase in energy consumption, it rolls back to the last stable configuration to ensure system robustness.

[0101] Example Explanation: In a wearable rehabilitation assistive bracelet, a multi-layered heterogeneous chip is responsible for collecting muscle tension and amplitude data, and outputting the next action command based on the historical rehabilitation model. If the patient temporarily makes an excessive amplitude movement, the load and temperature of the sensing and processing layer rise rapidly. The chip immediately reduces the frequency of that layer to avoid thermal damage, while activating more decision-making layer neurons to participate in judging whether the current movement is a normal rehabilitation behavior, dynamically adjusting the strategy generation logic to ensure stable device operation and strategy accuracy.

[0102] In edge risk control terminals, multi-layered heterogeneous chips process user behavior trajectories and image analysis tasks in real time. Under high-concurrency operation, the intelligent decision-making layer frequently calls pattern recognition paths. When the chip detects that its synaptic activation density is approaching its upper limit and is accompanied by an increase in power consumption, it immediately increases the processing frequency of this layer and reduces the activation of invalid paths. At the same time, it shuts down some backup response channels in the driver layer to reduce overall energy consumption and release resources, thereby ensuring stable response and risk decision-making quality in high-density trading environments.

[0103] This embodiment monitors the real-time operating status parameters of a multi-level heterogeneous integrated chip during task execution and dynamically adjusts the processing frequency and structural parameters based on these parameters. This enables the chip to achieve optimal energy efficiency, adaptive scheduling, and high stability under complex tasks. This approach avoids performance bottlenecks or thermal runaway problems caused by fixed configuration of single parameters, improves continuous processing capabilities and execution flexibility in embodied intelligent tasks, strengthens the chip's ability to quickly adapt to dynamic environmental changes, and helps build an efficient, secure, and long-term operational edge intelligence platform.

[0104] This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as elderly care services, fintech, and healthcare. It discloses a multi-level heterogeneous integrated chip task processing method, apparatus, device, and medium, comprising: constructing a multi-level heterogeneous integrated chip, which consists of a perception processing layer, an intelligent decision-making layer, and a drive control layer connected by a vertical interconnect structure; receiving multimodal task data and performing parallel feature extraction to generate multimodal feature vectors; inputting the multimodal feature vectors into a neuromorphic processing unit to determine the task decision result; converting the task decision result into a drive control signal to control the target execution device to perform actions; adjusting synaptic weights based on task feedback signals; monitoring chip operating status parameters and dynamically adjusting the chip's processing frequency and structural parameters based on these parameters. This invention integrates multimodal perception, neuromorphic decision-making, and dynamic feedback optimization mechanisms to achieve closed-loop processing of data perception, intelligent decision-making, and control execution within the chip. Simultaneously, by combining operating status monitoring with dynamic adjustment of the chip's frequency and structure, it improves the real-time performance, computational efficiency, and system adaptability of data processing, meeting the needs of embodied intelligence for high-performance and low-power chips.

[0105] In one embodiment, step S10 above includes:

[0106] S101 divides the chip's hierarchical structure into a perception processing layer, an intelligent decision-making layer, and a drive control layer;

[0107] S102, a nanoscale carbon nanotube field-effect transistor array is configured in the sensing and processing layer;

[0108] S103, a molybdenum disulfide semiconductor neuromorphic processing unit is constructed in the intelligent decision-making layer;

[0109] S104, Integrating a gallium nitride power device module in the drive control layer;

[0110] S105, the perception processing layer, intelligent decision-making layer and drive control layer are vertically stacked using three-dimensional stacking technology;

[0111] S106 forms the first vertical interconnection channel between the stacked perception processing layer and the intelligent decision-making layer;

[0112] S107 forms a second vertical interconnection channel between the stacked intelligent decision-making layer and the drive control layer;

[0113] S108 uses extreme ultraviolet lithography to form nanoscale circuit features on the multi-level heterogeneous integrated chip;

[0114] S109, the conductive connection between the first vertical interconnect channel and the second vertical interconnect channel is achieved through through-silicon via (TSV) technology;

[0115] S110, a processor core, a storage unit, and a dedicated accelerator module are integrated on the multi-level heterogeneous integrated chip.

[0116] In this embodiment, the chip-level structure is a hardware architecture spatial division oriented towards the three dimensions of perception, cognition, and execution in embodied intelligence tasks. The perception processing layer, intelligent decision-making layer, and drive control layer are respectively responsible for multimodal data acquisition and preliminary processing, neural pattern task judgment and strategy output, and the final precise drive control task of the execution device. This structural division is not a planar logical combination, but a functional nesting based on three-dimensional spatial coupling, requiring each layer to have clear process material boundaries, functional density blocks, and interface definitions.

[0117] The deployment of a nanoscale carbon nanotube field-effect transistor array within the sensing and processing layer is based on the excellent electron mobility and extremely small critical size of carbon nanotubes. The carbon nanotube array is arranged in an interleaved pattern within the metal wiring region beneath the logic units of the sensing layer. It is precisely grown using low-temperature chemical vapor deposition and template-guided technology, and combined with analog amplifier circuitry through multilayer metal interconnects to construct a high-parallelism, low-power, multimodal signal acquisition array. This transistor array can achieve bio-responsive current modulation when dealing with analog signals such as vision, touch, and sound, serving as a pre-processed nonlinear activation source to output feature-encoded signals, providing the processing vector foundation for the intelligent decision-making layer.

[0118] A semiconductor neuromorphic processing unit based on molybdenum disulfide (MoS2) is constructed in the intelligent decision-making layer, leveraging MoS2's two-dimensional layered structure, high charge mobility, and strong controllability. MoS2 thin films are deposited on an insulating substrate using micro / nano fabrication processes such as mechanical exfoliation, solution deposition, and plasma etching. A static random access memory array and variable conductivity synaptic nodes are then constructed to achieve a neural-like structural unit with impulse response. This layer simulates synaptic dynamic plasticity and neuronal activation response behavior to achieve temporal window integration of task features and state-encoded reasoning. This type of neuromorphic unit possesses in-memory computing fusion characteristics, significantly reducing data migration overhead and logical redundancy in traditional Von Neumann architectures.

[0119] The drive control layer integrates gallium nitride (GaN) power device modules, achieving precise current pulse output control through a GaNHEMT (high electron mobility transistor) array that operates stably under high frequency and high voltage. In this layer, multi-stage output channels are designed according to the required voltage levels and switching speeds of the controller devices. Insulated gate isolation and integrated inductor matching tuning enhance the drive response rate at high frequencies. GaN's superior breakdown field strength and saturated electron velocity characteristics compared to silicon give it extremely high reliability under high temperature and high load conditions, making it particularly suitable for control tasks in real-time response structures such as micro-actuators, multi-degree-of-freedom robotic arms, and biomimetic tissue systems.

[0120] By employing 3D stacking technology to vertically stack the aforementioned three layers within a single wafer system, not only is planar layout area saved, but the data flow path from perception to decision-making to actuation is also minimized. This stacking process combines bonding interconnects, laser annealing, and interlayer thermal regulation structures to ensure stacking stability and thermal diffusion efficiency. A first vertical interconnect channel is constructed between the perception processing layer and the intelligent decision-making layer to transmit multimodal feature vectors; a second vertical interconnect channel is constructed between the intelligent decision-making layer and the actuation control layer to transmit task decision results to the actuation control signal generation module. These interconnect channels provide high-speed cross-layer data transmission support with minimal latency paths.

[0121] The conductive connections of vertical interconnect channels are achieved using through-silicon via (TSV) technology. The TSV formation process includes reactive ion etching to create high aspect ratio vias, sidewall insulating layer deposition, seed layer deposition, and electroplating filling. The filler metal is typically copper or tungsten, possessing excellent electrical conductivity and mechanical compatibility. After laser annealing to eliminate stress micro-defects, a complete vertical metal path is formed, thereby achieving high-speed, low-latency interconnects between different layers.

[0122] By employing extreme ultraviolet (EUV) lithography to construct nanoscale circuit patterns across the entire chip surface, transistor sizes can be further reduced to below 7nm, achieving higher device density and lower power consumption. With EUV lithography wavelengths controlled at the 13.5nm level, combined with multiple pattern generation techniques and resist mask configurations, sub-10nm wiring accuracy and array density can be achieved. This is a key supporting process ensuring the collaborative construction of multi-material devices and high-performance interconnects.

[0123] Finally, a processor core, memory units, and dedicated accelerator modules are integrated onto this multi-layered heterogeneous integrated chip. The processor core is based on a custom instruction set and can call upon cross-layer computing resources; the memory units consist of distributed SRAM and non-volatile memory, forming a cross-layer data retention mechanism; the dedicated accelerators include vector matrix computation modules, graph neural network inference modules, or modality alignment modules, providing task customization support in a modular configuration. All these modules construct a unified address space and dynamic resource scheduling structure through intra-layer interconnect structures and vertical channel parallel addressing.

[0124] This embodiment constructs a multi-level heterogeneous integrated chip by relying on a three-dimensional stacked architecture to build the perception processing layer, intelligent decision-making layer, and driving control layer. It employs a heterogeneous material system based on carbon nanotube field-effect transistors, molybdenum disulfide neuromorphic units, and gallium nitride power devices, combined with extreme ultraviolet lithography and through-silicon via (TSV) conductive interconnect technology, enabling on-chip heterogeneous fusion of the entire process of perception, cognition, and execution. Vertical interconnect channels facilitate high-speed flow of multimodal feature vectors, task decision results, and control signals, significantly improving the closed-loop response speed from perception to control in embodied intelligence systems and reducing the latency and energy consumption bottlenecks caused by cross-chip communication in the perception-cognition-execution path of traditional systems. This chip configuration exhibits high coordination in terms of structural coupling, functional integration, material selection, and process matching, providing a high-performance hardware support platform for real-time, efficient, and low-power embodied intelligence tasks in edge scenarios.

[0125] In one embodiment, step S20 above includes:

[0126] S201 receives visual sensor data and tactile sensor data through a standardized sensor interface;

[0127] S202, Perform noise reduction and enhancement processing on the visual sensor data to generate processed visual data;

[0128] S203, Perform filtering and calibration processing on the tactile sensor data to generate processed tactile data;

[0129] S204, Construct a parallel convolutional neural network unit array specifically for spatial feature extraction;

[0130] S205, constructs a parallel temporal convolutional network unit array dedicated to temporal feature extraction;

[0131] S206, the processed visual data is distributed to the parallel convolutional neural network unit array;

[0132] S207, the processed tactile data is distributed to the parallel temporal convolutional network unit array;

[0133] S208, extract spatial visual feature vectors through the parallel convolutional neural network unit array;

[0134] S209, extract temporal tactile feature vectors through the parallel temporal convolutional network unit array;

[0135] S210, the spatial visual feature vector and the temporal tactile feature vector are fused to generate a multimodal feature vector.

[0136] In this embodiment, receiving visual and tactile sensor data via a standardized sensor interface serves as the entry interface structure for the entire on-chip sensing process. Essentially, it's a hardware abstraction layer encapsulation based on a multimodal signal transmission protocol. This interface must not only provide signal adaptation capabilities for voltage, current, and level signals, but also support protocol layer parsing capabilities, adapting to different sensor output formats including MIPI CSI-2, USB 3.1, I2C, SPI, or custom LVDS. Its input signals include raw sensor streams such as image frame sequences, capacitive pressure data, piezoresistive arrays, and current / voltage change trends. These are synchronized with the receiving logic and on-chip buffer structure before being sent to subsequent processing units.

[0137] Denoising and enhancement processing of visual sensor data are prerequisites for improving visual data quality. Denoising employs a joint spatial and frequency domain strategy, including Gaussian filtering, bilateral filtering, and wavelet denoising, to suppress interference from ambient light variations, sensor thermal noise, and low-frequency shifts. Enhancement focuses on improving texture contrast and dynamic range compression, with common techniques including histogram equalization, adaptive local contrast enhancement, and edge sharpening to ensure that boundaries, structural textures, and color gradients in image frames retain discriminative features even under low-precision storage conditions.

[0138] Performing filtering and calibration on tactile sensor data is a crucial step in processing low-frequency and easily distorted time series. Filtering employs Kalman filtering or Savitzky-Golay filtering to smooth the raw tactile signal, eliminating high-frequency mechanical oscillations and electromagnetic interference. Calibration incorporates multi-channel on / off state calibration, bias removal, and reference contact baseline mapping to map the raw capacitance, resistance, voltage, or current values ​​to a unified physical unit, such as N / cm². 2 Contact pressure at mN resolution, and eliminate hardware drift between sensor arrays.

[0139] Constructing a parallel convolutional neural network unit array dedicated to spatial feature extraction refers to building a structured visual coding network on-chip. Each convolutional unit consists of a programmable logic block and a dedicated convolutional accelerator, possessing capabilities such as multi-channel convolutional kernel loading, weight storage, sliding window operation, activation function execution, and pooling operations. In this structure, different neuron arrays undertake tasks such as edge extraction, corner detection, and texture pattern encoding of images, and support parallel processing of multiple visual channels (such as RGB or grayscale-gradient composite channels), preserving image spatial information through shared weights and local connectivity mechanisms.

[0140] A dedicated parallel temporal convolutional network array for temporal feature extraction is constructed, specifically for temporal pattern recognition of continuous tactile signal sequences. Unlike visual processing, tactile data exhibits stronger temporal dependence. Employing a 1D convolutional structure and combining a multi-kernel sliding window mechanism, gated activation mechanism, and time-step recursive feedback processing, this array can identify and encode dynamic changes such as contact duration, force change slope, and acceleration oscillations. Each temporal neuron in this array captures short-term contact features through a temporal convolutional kernel, while pooling layers further extract macroscopic patterns of contact events.

[0141] Distributing processed visual data to parallel convolutional neural network unit arrays is a signal routing behavior based on hardware schedulers. On-chip bus or point-to-point DMA mechanism maps image data blocks to different spatial CNN nodes. The allocation strategy can be dynamically scheduled based on channels, regions, frame numbers or the importance of perception tasks, and supports multi-threaded asynchronous execution and output buffer reordering.

[0142] The distribution of the processed tactile data to the parallel temporal convolutional network unit array is also accomplished through an internal scheduling mechanism, involving time slice partitioning, temporal window calibration, and channel load balancing strategies, to ensure that each tactile subsequence can be accurately delivered to its corresponding TD-CNN node, thereby improving on-chip concurrency capabilities.

[0143] The process of extracting spatial visual feature vectors using a parallel convolutional neural network array is essentially a vectorized encoding of key structures in an image frame. The extraction results are output as fixed-length or hierarchical feature maps, which can be encoded into task-interpretable vectors using principal component encoding, fully connected embedding, or attention-weighted projection. The process of extracting temporal tactile feature vectors using a parallel temporal convolutional network array maps multidimensional tactile time series to a low-dimensional perceptual space, preserving its dynamic response and temporal pattern characteristics. The vector results typically include contact event intensity spectra, pressure change trajectory encoding, and touch point displacement trends.

[0144] The fusion of spatial visual feature vectors and temporal tactile feature vectors to generate multimodal feature vectors is a key node in the modality alignment mechanism of the cognitive path in this film. Fusion methods can include concatenation, cross-attention mechanisms, gated coupling layers, or self-attention fusion modules; different methods can be selected according to the task type. The final multimodal feature vector possesses information representation capabilities across time, space, and modality, and has semantic integrity suitable for high-level task reasoning by the neuromorphic processing layer.

[0145] This embodiment constructs standardized acquisition, enhancement, calibration, and preprocessing paths for visual and tactile data, and introduces parallel feature extraction neural network structures for spatial and temporal dimensions respectively. This achieves parallel processing capabilities with clear modal division of labor, compact structure, and independent resources at the perception processing layer. Through customized deployment and input scheduling mechanisms of deep neural network arrays, different modal data structures can be effectively converted into a unified vector representation, significantly reducing modal conversion losses between the perception input layer and the cognitive layer. Finally, a modal fusion mechanism outputs multimodal feature vectors, forming a unified expression structure. This provides accurate, timely, and hierarchically decoupled data support for intelligent decision-making, thereby enhancing the responsiveness and adaptability of the embodied system in complex task environments. This mechanism demonstrates significant advantages in scenarios with limited edge chip resources and low latency tolerance.

[0146] In one embodiment, step S30 above includes:

[0147] S301, an adjustable synaptic weight module, a neuron activation function module, and a pulse transmission module are configured in the neuromorphic processing unit of the intelligent decision-making layer;

[0148] S302, Set the initial weight parameters of the adjustable synaptic weight module;

[0149] S303, input the multimodal feature vector into the neuromorphic processing unit;

[0150] S304, The multimodal feature vector is weighted by the adjustable synaptic weight module to generate a weighted feature vector;

[0151] S305, The weighted feature vector is processed by the neuron activation function module to generate the neuron output signal;

[0152] S306, The output signal of the neuron is transmitted through the pulse transmission module to generate the task decision result.

[0153] In this embodiment, an adjustable synaptic weight module, a neuron activation function module, and a pulse transmission module are configured in the neuromorphic processing unit of the intelligent decision layer, involving the functional construction and functional region division of the heterogeneous on-chip neuromorphic hardware structure. The adjustable synaptic weight module stores and updates the connection weights of the on-chip neural network, typically implemented using a variable resistance unit based on memristors, floating-gate transistors, or ferroelectric capacitors. Each synaptic connection has an adjustable conduction state, performing a weighted summation function during neural signal transmission. The neuron activation function module implements the computational response of nonlinear functions in the neural network, and can construct rectification, threshold functions, or simulate activation function models such as sigmoid, tanh, and ReLU based on analog circuits. The pulse transmission module implements the routing, synchronization, and transmission control of time-discrete pulse signals, and can construct an SNN (spiking neural network) structure using an event-driven mechanism, supporting the time-dependent calculation of presynaptic-postsynaptic activities.

[0154] Setting the initial weight parameters for the adjustable synaptic weight module is a parameter loading operation performed before the neuromorphic system enters task inference. This typically involves programmatically writing the weight matrix obtained during training into the hardware structure, or generating random initial values ​​based on a preset task template and dynamically correcting them. This initialization operation relates to the starting point of synaptic plasticity and has a decisive impact on the task response in the early stages of model inference. The weight parameters can come from external configuration files, non-volatile memory, or on-chip ROM.

[0155] The process of inputting multimodal feature vectors into the neuromorphic processing unit is achieved through an on-chip modal bus or cross-layer interconnect structure, mapping the structured high-dimensional semantic vectors from the perceptual processing layer to the input ports of the neuromorphic processing unit. There is a one-to-one physical mapping relationship between each input feature dimension and the input row of the synaptic matrix. The input signal is transmitted in pulse coding or analog current mode, and the input path is synchronized by a clock mechanism to ensure that the parallel input dimensions are aligned within a unified neural step.

[0156] The core process of neuromorphic computation involves weighting multimodal feature vectors using an adjustable synaptic weight module to generate weighted feature vectors. Each dimension of the feature vector is multiplied by the synaptic matrix to complete the weighting operation. Internally, the module can perform vector-weight multiplication and accumulation in parallel using a horizontal array structure. Its hardware implementation includes an arrayed memristor crossover structure, a current-mode multiplier, and a dual-ended accumulation capacitor array. The dimension of the weighted feature vector matches the number of output neurons and serves as the input for the next activation operation.

[0157] Processing the weighted feature vectors through the neuron activation function module to generate the neuron output signal is a key operation for realizing the nonlinear expressive power of the network. The activation function can be implemented in hardware using a dynamic response circuit constructed with analog voltage thresholding devices, or it can be implemented using digital circuits based on a lookup table mechanism. This module performs bit-by-bit calculations on the weighted results for each neuron, and its output form is either a continuous voltage amplitude (analog neural network) or a time-based pulse firing probability (spiking neural network), depending on the implementation structure.

[0158] The process of transmitting neuron output signals through the pulse delivery module to generate task decision results is the process of sending high-level semantic instructions to the downstream control layer. The pulse delivery module includes an event scheduling controller, a router, and synaptic emission control logic. Its responsibility is to encode, compress, and schedule the activated neural signals on-chip, ultimately outputting discrete action codes, behavior weight distributions, classification results, or decision instructions. In implementation, a time-division multiplexing transmission channel based on a network chip (NoC) architecture or an event broadcast mechanism with configured static path routing can be used.

[0159] Throughout the process, the neuromorphic processing unit, mimicking the structure and temporal behavior of biological neurons, completes end-to-end semantic reasoning without relying on external storage or centralized computation. The physical adjustability of synaptic weights ensures that the model structure is adaptable to environmental changes; the activation mechanism enhances on-chip nonlinear discrimination capabilities; and the pulse path propagation design enables it to have low-power, low-latency data output characteristics.

[0160] This embodiment constructs a neuromorphic processing unit composed of adjustable synaptic weights, nonlinear activation responses, and a temporal pulse propagation mechanism, forming a low-power, high-concurrency, and tunable on-chip inference path within a heterogeneous chip. Multimodal feature vectors are used as input signals, combined with a weight matrix to complete an associative memory-based weighted response, and an activation function generates distinguishable neural outputs, which are then converted into discretized task decision results via an event-driven path. This process possesses strong concurrency, low power consumption, and high real-time performance, enabling it to respond to multi-source perception tasks and perform dynamic strategy selection in embodied intelligence systems. This mechanism avoids the computation-storage bottleneck in traditional von Neumann architectures. Through physically coordinated synaptic connections and task-related parameter plasticity, it achieves unified cognitive representation and on-chip semantic response across modal information, possessing crucial adaptability value for scenarios with extremely limited computing power (such as edge terminals and mobile robots).

[0161] In one embodiment, step S40 above includes:

[0162] S401, monitors the real-time task load of the multi-level heterogeneous integrated chip through the dynamic voltage and frequency adjustment module;

[0163] S402, dynamically set the operating voltage and operating frequency of the drive control layer according to the real-time task load;

[0164] S403, the task decision result is converted into a pulse width modulation signal by the digital signal processing module;

[0165] S404, Generate motor torque control parameters based on the pulse width modulation signal;

[0166] S405, outputs a joint angle control signal according to the motor torque control parameters;

[0167] S406, the joint angle control signal is transmitted to the target actuator through the actuator interface to perform mechanical actions.

[0168] In this embodiment, a dynamic voltage and frequency adjustment module monitors the real-time task load of a multi-level heterogeneous integrated chip, establishing a dynamic sampling and feedback mechanism for on-chip control. The dynamic voltage and frequency adjustment module, through the configuration of on-chip multi-point voltage and current monitoring sensors, power sensing units, and thermistor arrays, collects real-time information on the processing load, power consumption, and temperature rise of each computing module. The task load is determined based on the processing density of multimodal inputs, the activation rate of neuromorphic inference paths, and the frequency of output control commands. The module employs a sliding window-based dynamic load analysis algorithm to extract the first-order transient power consumption curve and periodic task fluctuation characteristics, which serve as the basis for subsequent voltage and frequency adjustment.

[0169] The operating voltage and frequency of the drive control layer are dynamically adjusted based on real-time task load, reflecting the coupling and adjustment mechanism between chip operating parameters and task behavior. This adjustment mechanism typically employs a dynamic power management system (DVFS) composed of a multi-level frequency synthesizer based on a phase-locked loop (PLL) and an adjustable DC-DC converter (DCDC). Different task load levels correspond to different processing rate requirements and energy consumption budgets. This mechanism suppresses overload by reducing the operating frequency and voltage, or increases the frequency to ensure timeliness when responding to sudden tasks. Such strategies can be implemented based on a hardware-software co-engineering architecture, where a low-latency hardware-triggered path ensures rapid response, while software-assisted optimization provides a smooth transition between strategies.

[0170] The digital signal processing (DSP) module converts task decision results into pulse-width modulated (PWM) signals, serving as an intermediate mapping mechanism between neuromorphic inference outputs and specific physical quantities being executed. Task decision results are typically expressed as discrete codes, multi-valued classification results, or continuous behavioral parameter vectors, which the DSP module must convert into corresponding control quantities. This module can consist of a lookup-table matching-based digital converter, digital filters, and a PWM encoder. The core mechanism involves amplitude and period modulation of the output value, resulting in a standard duty cycle signal. This PWM signal not only contains amplitude information but also a time control dimension, used to drive the actuator response curve.

[0171] Generating motor torque control parameters based on pulse width modulation (PWM) signals is a physical process of mapping control electrical signals to mechanical parameters. This mapping can be achieved using a calibrated motor control model, which includes the current-torque function, the PWM duty cycle-drive voltage relationship, and the mechanical load characteristics of the actuator. This process relies on the power stage demodulation circuitry within the motor control chip to convert the PWM signal into a continuous current waveform and adjust the conduction angle of the drive arm, ultimately outputting controllable torque. The control parameters not only affect the magnitude of the torque but also determine the force application timing and damping response, having a decisive impact on high-precision control tasks.

[0172] The joint angle control signal, output based on motor torque control parameters, is part of the operational chain that provides specific control over the robot or embodied actuator. In the motor system, this angle control signal is input as part of a feedback loop to the position controller, such as a PI, PID, or model predictive control (MPC) module. This module outputs a target angle signal based on the target torque and the current joint angle error. This signal can further drive servos, lead screws, hydraulic actuators, or other actuation components to achieve pose adjustment, balance control, or path planning response for multi-joint structures.

[0173] The actuator interface transmits joint angle control signals to the target actuator to perform mechanical actions, completing the final link from on-chip inference to environmental interaction. The actuator interface includes a level conversion module, drive amplifier circuit, and communication protocol adapter, supporting various actuator communication standards such as CAN, I2C, SPI, or RS485. At this stage, the signal needs to undergo electrical isolation, voltage matching, and drive amplitude enhancement to ensure that the control signal can stably drive high-impedance loads or long-distance distributed actuator modules. The target actuator may include a robotic arm, robot end effector, mobile chassis, or medical wearable devices, whose motion control behavior is precisely mapped to on-chip instructions.

[0174] This integrated control chain, from neural inference output to physical action feedback, forms an on-chip to off-chip control closed loop, characterized by low latency, low power consumption, and strong robustness. Through parameterized adjustment of the parallel control chain, it can adapt to high-precision tasks, multi-task collaboration, or sudden anomaly response scenarios, providing stable and efficient control execution capabilities for edge-end embodied intelligent agents.

[0175] This embodiment transforms task decision results into physical control actions, constructing a complete closed-loop control link within a multi-level heterogeneous integrated chip. A dynamic voltage and frequency adjustment module monitors the chip's task load in real time and adjusts the driving control layer's operating parameters, achieving on-demand allocation of computing power and energy consumption. A digital signal processing module precisely maps the inference results into controllable modulation signals, which are then converted into motor torque and joint angle control parameters. Finally, an interface structure drives the target execution device to complete the action response. This process realizes the energy transfer and signal encoding / decoding path from cognitive decision-making to physical manipulation, without relying on an external control platform, possessing autonomous response, real-time regulation, and closed-loop action control capabilities. Through on-chip structure collaboration and signal flow optimization, the execution link response rate and energy efficiency are significantly improved, while enhancing the stability and adaptability of the embodied system in complex environments.

[0176] In one embodiment, step S50 above includes:

[0177] S501, receive a task feedback signal containing information about changes in the environmental state after the action is performed;

[0178] S502, Analyze the task completion index in the task feedback signal;

[0179] S503, determine the direction of synaptic weight adjustment based on the task completion index;

[0180] S504, determine the synaptic weight adjustment amount of the neuromorphic processing unit according to the synaptic weight adjustment direction;

[0181] S505, adjust the synaptic weight parameters of the neuromorphic processing unit according to the synaptic weight adjustment amount;

[0182] S506 stores the adjusted synaptic weight parameters into the weight register.

[0183] In this embodiment, the task feedback signal, which contains information about changes in the environmental state, received after the execution action, constitutes the external environmental state feedback path from the execution device to the neuromorphic inference module. The task feedback signal typically includes multi-dimensional information such as task response latency, execution result accuracy, resource consumption, and environmental interference indicators. It is acquired through an actuator sensor array, such as an attitude sensor, force sensor, contact feedback module, or environmental visual backtracking device. This signal is transmitted to the on-chip processing module via a high-speed interface, and can employ an asynchronous interrupt mechanism or a polling read method to adapt to the stability requirements of edge execution cycles.

[0184] Analyzing task completion metrics in task feedback signals involves abstracting complex, multi-dimensional feedback information into structured metrics that can be used for learning and optimization. Task completion metrics are calculated based on the deviation between the task objective and the actual result. Common forms include target displacement error, action execution accuracy, deviation of resource usage from expectations, or binary success judgment values. This metric can be generated through a rule engine, edge inference module, or reinforcement learning reward function. The completion metric, in the form of a numerical value or probability distribution, is input into the learning adjustment module and serves as the triggering basis for synaptic weight adjustment.

[0185] Determining the direction of synaptic weight adjustment based on task completion metrics is a crucial step in achieving synaptic adaptive learning. In neuromorphic structures, the direction of synaptic weights typically represents the adjustment trend of the current weights in a high-dimensional parameter space. If task completion is high, the system will tend to maintain the original weight direction with minor adjustments; if completion is low, a larger gradient backpropagation adjustment is required. This adjustment direction can be achieved based on the error backpropagation mechanism, or approximated using the local Hebbian or STDP (Spike-Timing-Dependent Plasticity) rules, where the activation sequence between neurons determines the sign and slope of the weight adjustment.

[0186] Determining the synaptic weight adjustment amount of the neuromorphic processing unit based on the direction of synaptic weight adjustment is a quantification process that converts directional regulation into specific numerical changes. The weight adjustment amount can be determined using a fixed step-size strategy, a dynamic step-size strategy, or a gradient magnitude strategy based on an error function. Among them, the dynamic step-size mechanism adaptively adjusts the adjustment magnitude according to the fluctuations of recent feedback signals to prevent weight oscillations or convergence stagnation. This process can be implemented in the local synaptic module by simulating voltage gain changes, charge accumulation processes, or state transitions of on-chip variable resistance elements (such as memristors).

[0187] Adjusting the synaptic weight parameters of a neuromorphic processing unit according to the synaptic weight adjustment amount is a process of physically or logically modifying the weight matrix in a brain-like computing structure. This can be achieved using multi-bit memristor arrays, capacitor array adjustment structures, charge pump variable capacitance mechanisms, or floating-gate storage control techniques. The system uses the adjustment amount as a control signal input, driving the storage structure of the synaptic unit to change its conduction characteristics through analog signals, thereby achieving weight adjustment. Weight parameters are typically represented by analog voltage, conductance values, or digital codes, and in neuromorphic circuits, they couple with the neuronal activation function to influence the network output.

[0188] Storing the adjusted synaptic weight parameters in a weight register is fundamental to maintaining the traceability and transferability of the learned state and preventing transient forgetting. The weight register can employ a non-volatile storage structure, such as MRAM, ReRAM, or Flash, supporting long-term retention after writing. For neuromorphic systems operating in low-power edge scenarios, a fast-write on-chip cache structure can also be configured, synchronously writing to persistent memory during subsequent idle cycles. This type of structure supports parameter recovery after power failure and restart and can be used for weight retrieval in model compression, edge migration, or federated synchronization.

[0189] This complete learning closed-loop path guides the neuromorphic structure to perform synaptic modulation through task feedback signals, forming an autonomous learning and adaptive control mechanism. It establishes a neural circuit-like reinforcement feedback path within the embodied system. The stable structural characteristics of maintaining physical connectivity, adjustable parameters, and a closed-loop path between each module are the underlying guarantee for achieving continuous online optimization, heterogeneous system adaptation, and steady improvement in accuracy.

[0190] This embodiment receives task feedback signals after action execution, analyzes the completion rate indicators in the feedback, and uses them to dynamically adjust the direction and amplitude of synaptic weights in the neuromorphic processing unit, realizing the on-chip inference structure's ability to learn from changes in the external environment. By adjusting the weight parameters and storing them in a weight register, the system possesses memory effects, parameter retention, and continuous optimization capabilities. This approach does not rely on an external training platform or central server, enabling a real-time feedback-driven adaptive learning process at the edge, thereby enhancing the agent's generalization ability and action decision-making efficiency in complex task scenarios, and improving the system's robustness and decision stability in changing environments.

[0191] In one embodiment, step S60 above includes:

[0192] S601 monitors the real-time temperature parameters of the multi-level heterogeneous integrated chip through an integrated temperature sensor;

[0193] S602, collects the real-time power consumption parameters of the multi-level heterogeneous integrated chip through the power consumption monitoring unit;

[0194] S603, the processing load parameters of the multi-level heterogeneous integrated chip are obtained through the processing load analysis module;

[0195] S604 transmits the real-time temperature parameters, real-time power consumption parameters, and processing load parameters to the analysis module via the on-chip bus;

[0196] S605, Determine the temperature overload state and the processing efficiency state in the analysis module;

[0197] S606, When a temperature overload is detected, the processing frequency of the multi-level heterogeneous integrated chip is reduced;

[0198] S607, When a state of decreased processing efficiency is detected, the neural network structure of the neuromorphic processing unit is reconstructed;

[0199] S608, reset the synaptic weight distribution of the neuromorphic processing unit.

[0200] In this embodiment, monitoring the real-time temperature parameters of a multi-layered heterogeneous integrated chip using an integrated temperature sensor is a crucial component of the on-chip environmental state perception mechanism. This temperature sensor is deployed across multiple physical layers within the chip stack structure, particularly covering the thermally sensitive critical area between the sensing and processing layer, the intelligent decision-making layer, and the drive and control layer. A distributed temperature monitoring network is constructed using nanoscale thermocouples, thermal resistance arrays, or integrated micro-MEMS temperature sensors, enabling precise hotspot location and longitudinal heat flow identification. Temperature data from the chip's surface and deeper layers is collected through periodic sampling or a thermal threshold interruption mechanism to facilitate subsequent dynamic adjustment of the logic path.

[0201] Real-time power consumption parameters of multi-layered heterogeneous integrated chips are acquired through a power consumption monitoring unit, which is a necessary foundation for building a full-stack power regulation mechanism. This monitoring unit can acquire real-time voltage, current, and transient power data of each module on the chip based on integrated current sensing resistors, charge counting modules, and sampling circuits in the power regulator feedback loop. The monitoring range covers the analog computing area of ​​the neuromorphic processing module, the accelerator array area, and the high-frequency I / O subsystem, and can capture peak power consumption behavior, abnormal power drift, and uneven energy load. All data is aggregated to the power data bus with low latency, ensuring rapid regulation response.

[0202] The processing load analysis module acquires processing load parameters for multi-level heterogeneous integrated chips, primarily used to assess resource usage and operational pressure within a specific time window. These parameters include thread concurrency, memory utilization, accelerator occupancy time, neuron activation density, and synapse access frequency. This module samples behavior within the runtime cycle using the on-chip Performance Monitor Unit (PMU) and constructs a sliding window averaging model to evaluate inference density and task bottlenecks. This analysis not only supports load balancing and regulation but also provides a basis for structural reconfiguration decisions.

[0203] The transmission of real-time temperature, power consumption, and processing load parameters to the analysis module via the on-chip bus embodies the system-level state aggregation mechanism. The on-chip bus employs high-speed inter-chip interconnect protocols (such as AMBA, AXI, NoC, TSV, etc.) to establish a low-latency communication link from the state-aware unit to the analysis module. To ensure thermal stability and bandwidth load control, this communication link can utilize asynchronous FIFO buffering, polling channel priority, or compression coding mechanisms to reduce on-chip communication interference. Once the state parameters enter the analysis module, state synthesis judgment and control signal generation are immediately initiated.

[0204] The analysis module determines the temperature overload state and processing efficiency state, serving as the logical unit for transitioning from state to behavior. The temperature overload state can be obtained by comparing the current temperature value with a preset thermal threshold, or by predicting critical anomalies through the temperature rise rate, thermal equilibrium time, and local thermal field distribution trends. The processing efficiency state is analyzed based on the inference output rate, module utilization, and neuron redundancy ratio to determine if the current computational efficiency has decreased. This module features a configurable, adjustable rule engine, allowing users to set temperature and efficiency response strategies for different tasks to adapt to high-reliability or high-throughput scenarios.

[0205] Reducing the processing frequency of a multi-layered heterogeneous integrated chip when a temperature overload is detected is a proactive load reduction strategy in chip thermal management. The processing frequency can be controlled via a Dynamic Voltage and Frequency Scaling (DVFS) module. This module receives a frequency reduction command from the analysis module and, through corresponding power management units and clock control circuits, reduces the operating frequency of each functional module on the chip. In specific implementations, the sensing and processing layer, intelligent decision-making layer, and drive control layer can each be configured with independent clock sources and frequency domains, supporting asynchronous or step-by-step frequency reduction mechanisms. This allows for localized frequency reduction when heat dissipation is limited in certain modules, preventing significant overall performance degradation.

[0206] When a decline in processing efficiency is detected, reconstructing the neural network structure of the neuromorphic processing unit is a key path for building an online structural plasticity mechanism. Neural network structure reconstruction typically includes enabling / freezing neuronal channels, remapping network layer connections, and sparsifying or reorganizing synaptic connection weight maps. Specific implementation methods include overloading the structure mapping table, replacing structure descriptors, or performing hardware-level MUX path rewriting. In physical implementation, this function can be carried out using cross-clock domain reconfigurable logic (such as FPGA chips) or a neuromorphic chip architecture with dynamic structural adaptation capabilities (such as Loihi).

[0207] Resetting the synaptic weight distribution of the neuromorphic processing unit is a means to achieve state convergence and behavioral stability after structural reconstruction. This operation clears some of the original synaptic memory values ​​and reinitializes the parameter space based on a preset distribution strategy. Weight resetting methods may include uniform distribution initialization, Gaussian noise perturbation, or loading prior parameters according to a task template. Specific implementation methods involve batch rewriting of the synaptic storage structure, charge release, or regression of the conductivity values ​​of simulated channels to ensure stable operation and initial learnability of inference behavior under the new structure.

[0208] Example: In edge computing scenarios in the healthcare field, taking smart wearable devices assisting rehabilitation patients in completing physical therapy tasks as an example, the devices need to collect, understand, execute, and self-optimize multimodal physiological data in an environment without cloud dependence, limited latency, and controlled energy.

[0209] The smart wearable device incorporates the multi-level heterogeneous integrated chip described in this patent method, whose perception processing layer receives task data from visual and tactile sensors. The visual sensor acquires video of the patient's movements and postures, while the tactile sensor acquires mechanical signals such as muscle vibration frequency and skin pressure. To enhance data quality, visual data undergoes edge-side denoising and brightness enhancement processing before input, while tactile data undergoes multi-level low-pass filtering and signal amplitude normalization calibration to ensure effective resolution of input features.

[0210] The processed data are input into a parallel convolutional neural network array (for extracting spatial motion features) and a parallel temporal convolutional neural network array (for extracting temporal tactile dynamics), respectively. For example, spatial motion features may include limb angles, movement trajectories, rhythmic patterns, etc.; tactile features focus on the temporal trend of muscle tension changes. The resulting multimodal feature vector, fused together, characterizes the global performance of the patient's current motor execution state.

[0211] The multimodal feature vector is input into a neuromorphic processing unit deployed in the intelligent decision-making layer. This unit consists of an adjustable synaptic weight module, a neuronal activation function module, and a pulse transmission module, constructing an inference path similar to a biological neural network. The feature vector is first weighted by synapses, then processed by the activation function to form an output electrical signal, which is finally transmitted to form a clear task decision result, such as "continue to maintain the action," "posture needs to be corrected," or "increase the range of motion."

[0212] The task decision results are transmitted to the drive control layer, where dynamic voltage and frequency adjustments are performed to adapt to the current chip task intensity, avoiding thermal runaway or sudden increases in power consumption due to prolonged high load. This layer converts the decision results into pulse width modulation signals to generate motor torque control commands, and further forms commands to control wearable braces or exoskeleton systems to perform joint adjustment movements, achieving precise rehabilitation assistance control.

[0213] After an action is performed, the device receives task feedback signals from the end of the action execution, such as posture change data, action completion indicators, or abnormal flags. The feedback signals are parsed by the chip to extract indicators such as "action deviation rate" and "goal achievement rate," and the synaptic weight parameters in the neuromorphic processing unit are updated based on these indicators. For example, if the patient's action completion is insufficient, the system will increase the attention weight to specific feature dimensions, improving the model's responsiveness to that dimension, thereby strengthening the activation of related pathways in subsequent inference.

[0214] To ensure continuous and stable system operation, the chip also implements an operational status monitoring and adaptive adjustment mechanism. Its embedded temperature sensor detects the current temperature distribution across each chip layer, the power consumption unit records the energy consumption of local modules, and the load processing module assesses the current network utilization and activation density. In the chip analysis module, if thermal overload is detected, the chip will reduce the local or global processing frequency; if a decrease in processing efficiency is detected, it will reconstruct the neuromorphic network structure, such as reducing redundant paths, restarting synaptic connections, and resetting the weight distribution to restore inference efficiency.

[0215] In edge computing scenarios within the fintech field, consider deploying integrated smart chips on self-service terminals or smart mobile teller machines (such as smart wealth management machines and portable credit review terminals in bank branches) to enable customer identity verification, facial expression and behavior analysis, risk control interaction response, and authorization action control in environments without network connection or in high-risk offline conditions.

[0216] This terminal integrates a multi-level heterogeneous integrated chip structure comprising a perception processing layer, an intelligent decision-making layer, and a drive control layer. Upon receiving a service request from a customer, the terminal acquires facial images and gaze trajectories via a visual sensor connected to the perception processing layer. Simultaneously, it collects hand pressure dynamics and finger vibration data triggered by micro-expressions via tactile sensors (such as a finger pressure touchpad). The visual images are processed using a local denoising and enhancement algorithm to improve the clarity of facial key points, while the tactile data undergoes hardware filtering and channel gain calibration to eliminate low-frequency interference and is uniformly formatted into a processable input tensor.

[0217] These processed multi-source sensory data are fed into parallel convolutional neural network unit arrays and parallel temporal convolutional network unit arrays, respectively, to extract spatial features (such as facial contour shift and eye movement patterns) and temporal features (such as changes in pressing speed and continuous vibration cycle), thereby generating multimodal feature vectors describing customer interaction behavior.

[0218] The feature vector is input to the neuromorphic processing unit of the intelligent decision-making layer, which includes a synaptic weighting module, a nonlinear activation module, and a signal pulse transmission module for information selection. After weighting and activation function processing, the features form multiple output signal channels pointing to "identity risk level," "behavioral consistency score," and "response credibility" in the decision space. Based on internal encoding rules, these signals are integrated into a clear task decision result through a pulse mechanism, such as "passed preliminary verification," "requires secondary verification," or "triggers risk control response."

[0219] The task decision result is transmitted to the drive control layer. This layer first monitors the chip's current load status (such as the number of concurrent processing requests, identifying latency trends, etc.) through a dynamic voltage and frequency adjustment module, and optimizes the operating voltage and frequency settings accordingly to ensure stable operation. Subsequently, the digital signal processing module converts the task result into a pulse width modulation signal, which is used to drive the behavior of various modules in the terminal. For example, when the task result is "verification passed," the control signal generated by the chip will be converted into an authorization action, such as unlocking the digital signature module, or issuing an "open operation" to the drive module controlling the electronic lock door.

[0220] After an action is completed, the system receives execution feedback signals, including feedback indicators such as hatch opening delay, action completion status, and whether the user interacts again. This feedback is sent back to the intelligent decision-making layer, where the task completion calculation module extracts values ​​such as completion rate and operation deviation rate. Based on the results, the weight parameters of synaptic connections in the neuromorphic processing unit are adjusted to strengthen the ability to distinguish effective behavioral features, weaken invalid path connections, and improve the accuracy of subsequent interactions. The new weights are stored in the weight register in real time, ensuring that the device can adapt to the risk identification mode in a short period of time.

[0221] To ensure continuous and stable operation, the chip continuously monitors the temperature parameters, power consumption status, and inference load of each module. When the temperature rises abnormally, the chip automatically reduces the computing frequency to prevent power consumption from exceeding limits. If a decrease in inference efficiency is detected (such as decision delay or low synaptic activation density), the chip automatically performs neural network structure reconstruction operations to suppress redundant channels and redistribute synaptic weights, thereby restoring the computational efficiency of the decision layer.

[0222] Ultimately, this intelligent financial terminal can autonomously perform user behavior recognition, dynamic risk assessment, and action control in high-risk environments such as offline, mobile, and unsupervised financial services, ensuring the privacy and security of local data processing and real-time interaction. It is particularly suitable for the urgent need for edge intelligent processing capabilities in scenarios such as digital banks, mobile financial tellers, and unmanned transaction facilities.

[0223] This embodiment integrates temperature, power consumption, and load monitoring units within the chip and aggregates their status data to an analysis module for judgment, enabling the system to sensitively perceive changes in the external environment and fluctuations in operating load. By dynamically adjusting the processing frequency, the system can reduce energy consumption under thermal overload conditions, preventing device degradation or operational interruption due to heat accumulation. When processing efficiency decreases, the system achieves adaptive structural evolution and inference path reconstruction through neural network structure reconstruction and synaptic weight distribution reset, significantly enhancing the robustness and sustainable operation capability of the neuromorphic system.

[0224] In one embodiment, a multi-level heterogeneous integrated chip task processing device is provided, which corresponds one-to-one with the multi-level heterogeneous integrated chip task processing method described in the above embodiments. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the multi-level heterogeneous integrated chip task processing device of the present invention. The modules include a chip-level construction module 10, a multimodal perception module 20, a neural decision-making module 30, an execution control module 40, a synaptic weight learning module 50, and a running state adaptive module 60. Detailed descriptions of each functional module are as follows:

[0225] Chip-level construction module 10 is used to construct a multi-level heterogeneous integrated chip including a perception processing layer, an intelligent decision-making layer, and a drive control layer connected by a vertical interconnect structure.

[0226] The multimodal perception module 20 is used to receive multimodal task data through the perception processing layer, and to perform parallel feature extraction on the multimodal task data to generate a multimodal feature vector.

[0227] The neural decision module 30 is used to input the multimodal feature vector into the neuromorphic processing unit of the intelligent decision layer, and process the multimodal feature vector based on synaptic weights in the neuromorphic processing unit to obtain the task decision result.

[0228] The execution control module 40 is used to convert the task decision result into a drive control signal through the drive control layer, and control the target execution device to perform actions according to the drive control signal;

[0229] The synaptic weight learning module 50 is used to adjust the synaptic weights of the neuromorphic processing unit in real time based on the task feedback signal returned after the action is performed.

[0230] The adaptive operation status module 60 is used to monitor the operation status parameters of the multi-level heterogeneous integrated chip and dynamically adjust the processing frequency and structural parameters of the multi-level heterogeneous integrated chip based on the operation status parameters.

[0231] In one embodiment, the chip-level construction module 10 is specifically used for:

[0232] The chip's hierarchical structure is divided into a perception and processing layer, an intelligent decision-making layer, and a drive and control layer.

[0233] A nanoscale carbon nanotube field-effect transistor array is configured in the sensing and processing layer;

[0234] A molybdenum disulfide semiconductor neuromorphic processing unit is constructed in the intelligent decision-making layer;

[0235] A gallium nitride power device module is integrated into the drive control layer;

[0236] The perception processing layer, intelligent decision-making layer, and drive control layer are vertically stacked using three-dimensional stacking technology.

[0237] A first vertical interconnection channel is formed between the stacked perception processing layer and the intelligent decision-making layer;

[0238] A second vertical interconnection channel is formed between the stacked intelligent decision-making layer and the drive control layer;

[0239] Nanoscale circuit features are formed on the multi-level heterogeneous integrated chip using extreme ultraviolet lithography.

[0240] Conductive connection between the first vertical interconnect channel and the second vertical interconnect channel is achieved through through-silicon via (TSV) technology.

[0241] The processor core, storage unit and dedicated accelerator module are integrated on the multi-level heterogeneous integrated chip.

[0242] In one embodiment, the multimodal sensing module 20 is specifically used for:

[0243] Receives visual sensor data and tactile sensor data through a standardized sensor interface;

[0244] The visual sensor data is subjected to denoising and enhancement processing to generate processed visual data;

[0245] The tactile sensor data is filtered and calibrated to generate processed tactile data;

[0246] Construct a parallel convolutional neural network unit array specifically for spatial feature extraction;

[0247] Construct a parallel temporal convolutional network unit array specifically for temporal feature extraction;

[0248] The processed visual data is distributed to the parallel convolutional neural network unit array;

[0249] The processed tactile data is distributed to the parallel temporal convolutional network unit array;

[0250] Spatial visual feature vectors are extracted using the parallel convolutional neural network unit array;

[0251] Temporal tactile feature vectors are extracted using the parallel temporal convolutional network unit array;

[0252] The spatial visual feature vector and the temporal tactile feature vector are fused to generate a multimodal feature vector.

[0253] In one embodiment, the neural decision-making module 30 is specifically used for:

[0254] An adjustable synaptic weight module, a neuron activation function module, and a pulse transmission module are configured in the neuromorphic processing unit of the intelligent decision-making layer.

[0255] Set the initial weight parameters of the adjustable synaptic weight module;

[0256] The multimodal feature vector is input into the neuromorphic processing unit;

[0257] The adjustable synaptic weight module is used to weight the multimodal feature vector to generate a weighted feature vector.

[0258] The weighted feature vector is processed by the neuron activation function module to generate the neuron output signal;

[0259] The pulse transmission module transmits the neuron's output signal to generate task decision results.

[0260] In one embodiment, the execution control module 40 is specifically used for:

[0261] The real-time task load of the multi-level heterogeneous integrated chip is monitored through a dynamic voltage and frequency adjustment module.

[0262] The operating voltage and operating frequency of the drive control layer are dynamically set according to the real-time task load.

[0263] The task decision result is converted into a pulse width modulation signal by a digital signal processing module;

[0264] Motor torque control parameters are generated based on the pulse width modulation signal;

[0265] Output a joint angle control signal based on the motor torque control parameters;

[0266] The joint angle control signal is transmitted to the target actuator through the actuator interface to perform mechanical actions.

[0267] In one embodiment, the synaptic weight learning module 50 is specifically used for:

[0268] Receive the task feedback signal containing information about changes in the environmental state after the action is performed;

[0269] Analyze the task completion rate index in the task feedback signal;

[0270] The direction of synaptic weight adjustment is determined based on the task completion index;

[0271] The synaptic weight adjustment amount of the neuromorphic processing unit is determined based on the synaptic weight adjustment direction;

[0272] The synaptic weight parameters of the neuromorphic processing unit are adjusted according to the synaptic weight adjustment amount;

[0273] The adjusted synaptic weight parameters are stored in the weight register.

[0274] In one embodiment, the adaptive running state module 60 is specifically used for:

[0275] The real-time temperature parameters of the multi-level heterogeneous integrated chip are monitored by an integrated temperature sensor.

[0276] The real-time power consumption parameters of the multi-level heterogeneous integrated chip are collected by the power consumption monitoring unit.

[0277] The processing load parameters of the multi-level heterogeneous integrated chip are obtained through the processing load analysis module.

[0278] The real-time temperature parameters, real-time power consumption parameters, and processing load parameters are transmitted to the analysis module via the on-chip bus.

[0279] The analysis module determines the temperature overload state and the processing efficiency state.

[0280] When a temperature overload is detected, the processing frequency of the multi-level heterogeneous integrated chip is reduced.

[0281] When a decrease in processing efficiency is detected, the neural network structure of the neuromorphic processing unit is reconstructed;

[0282] Reset the synaptic weight distribution of the neuromorphic processing unit.

[0283] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external user terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a multi-level heterogeneous integrated chip task processing method on the server side.

[0284] In one embodiment, a computer device is provided, which may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the user side of a multi-level heterogeneous integrated chip task processing method.

[0285] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0286] Construct a multi-level heterogeneous integrated chip that includes a perception processing layer, an intelligent decision-making layer, and a drive control layer connected through a vertical interconnect structure;

[0287] The perception processing layer receives multimodal task data and performs parallel feature extraction on the multimodal task data to generate multimodal feature vectors.

[0288] The multimodal feature vector is input into the neuromorphic processing unit of the intelligent decision layer, and the multimodal feature vector is processed based on synaptic weights in the neuromorphic processing unit to obtain the task decision result;

[0289] The drive control layer converts the task decision result into a drive control signal, and controls the target execution device to perform actions according to the drive control signal;

[0290] Based on the task feedback signal returned after the action is performed, the synaptic weights of the neuromorphic processing unit are adjusted in real time.

[0291] The operating status parameters of the multi-level heterogeneous integrated chip are monitored, and the processing frequency and structural parameters of the multi-level heterogeneous integrated chip are dynamically adjusted based on the operating status parameters.

[0292] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0293] Construct a multi-level heterogeneous integrated chip that includes a perception processing layer, an intelligent decision-making layer, and a drive control layer connected through a vertical interconnect structure;

[0294] The perception processing layer receives multimodal task data and performs parallel feature extraction on the multimodal task data to generate multimodal feature vectors.

[0295] The multimodal feature vector is input into the neuromorphic processing unit of the intelligent decision layer, and the multimodal feature vector is processed based on synaptic weights in the neuromorphic processing unit to obtain the task decision result;

[0296] The drive control layer converts the task decision result into a drive control signal, and controls the target execution device to perform actions according to the drive control signal;

[0297] Based on the task feedback signal returned after the action is performed, the synaptic weights of the neuromorphic processing unit are adjusted in real time.

[0298] The operating status parameters of the multi-level heterogeneous integrated chip are monitored, and the processing frequency and structural parameters of the multi-level heterogeneous integrated chip are dynamically adjusted based on the operating status parameters.

[0299] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0300] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0301] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0302] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A multi-level heterogeneous integrated chip task processing method, characterized in that, Includes the following steps: Construct a multi-level heterogeneous integrated chip that includes a perception processing layer, an intelligent decision-making layer, and a drive control layer connected through a vertical interconnect structure; The perception processing layer receives multimodal task data and performs parallel feature extraction on the multimodal task data to generate multimodal feature vectors. The multimodal feature vector is input into the neuromorphic processing unit of the intelligent decision layer, and the multimodal feature vector is processed based on synaptic weights in the neuromorphic processing unit to obtain the task decision result; The drive control layer converts the task decision result into a drive control signal, and controls the target execution device to perform actions according to the drive control signal; Based on the task feedback signal returned after the action is performed, the synaptic weights of the neuromorphic processing unit are adjusted in real time. The operating status parameters of the multi-level heterogeneous integrated chip are monitored, and the processing frequency and structural parameters of the multi-level heterogeneous integrated chip are dynamically adjusted based on the operating status parameters.

2. The multi-level heterogeneous integrated chip task processing method as described in claim 1, characterized in that, Constructing a multi-level heterogeneous integrated chip comprising a perception processing layer, an intelligent decision-making layer, and a drive control layer connected via a vertical interconnect structure, including: The chip's hierarchical structure is divided into a perception and processing layer, an intelligent decision-making layer, and a drive and control layer. A nanoscale carbon nanotube field-effect transistor array is configured in the sensing and processing layer; A molybdenum disulfide semiconductor neuromorphic processing unit is constructed in the intelligent decision-making layer; A gallium nitride power device module is integrated into the drive control layer; The perception processing layer, intelligent decision-making layer, and drive control layer are vertically stacked using three-dimensional stacking technology. A first vertical interconnection channel is formed between the stacked perception processing layer and the intelligent decision-making layer; A second vertical interconnection channel is formed between the stacked intelligent decision-making layer and the drive control layer; Nanoscale circuit features are formed on the multi-level heterogeneous integrated chip using extreme ultraviolet lithography. Conductive connection between the first vertical interconnect channel and the second vertical interconnect channel is achieved through through-silicon via (TSV) technology. The processor core, storage unit and dedicated accelerator module are integrated on the multi-level heterogeneous integrated chip.

3. The multi-level heterogeneous integrated chip task processing method as described in claim 1, characterized in that, The perception processing layer receives multimodal task data and performs parallel feature extraction on the multimodal task data to generate multimodal feature vectors, including: Receives visual sensor data and tactile sensor data through a standardized sensor interface; The visual sensor data is subjected to denoising and enhancement processing to generate processed visual data; The tactile sensor data is filtered and calibrated to generate processed tactile data; Construct a parallel convolutional neural network unit array specifically for spatial feature extraction; Construct a parallel temporal convolutional network unit array specifically for temporal feature extraction; The processed visual data is distributed to the parallel convolutional neural network unit array; The processed tactile data is distributed to the parallel temporal convolutional network unit array; Spatial visual feature vectors are extracted using the parallel convolutional neural network unit array; Temporal tactile feature vectors are extracted using the parallel temporal convolutional network unit array; The spatial visual feature vector and the temporal tactile feature vector are fused to generate a multimodal feature vector.

4. The multi-level heterogeneous integrated chip task processing method as described in claim 1, characterized in that, The multimodal feature vector is input into the neuromorphic processing unit of the intelligent decision layer. The neuromorphic processing unit processes the multimodal feature vector based on synaptic weights to obtain the task decision result, including: An adjustable synaptic weight module, a neuron activation function module, and a pulse transmission module are configured in the neuromorphic processing unit of the intelligent decision-making layer. Set the initial weight parameters of the adjustable synaptic weight module; The multimodal feature vector is input into the neuromorphic processing unit; The adjustable synaptic weight module is used to weight the multimodal feature vector to generate a weighted feature vector. The weighted feature vector is processed by the neuron activation function module to generate the neuron output signal; The pulse transmission module transmits the neuron's output signal to generate task decision results.

5. The multi-level heterogeneous integrated chip task processing method as described in claim 1, characterized in that, The drive control layer converts the task decision result into a drive control signal, and controls the target execution device to perform actions according to the drive control signal, including: The real-time task load of the multi-level heterogeneous integrated chip is monitored through a dynamic voltage and frequency adjustment module. The operating voltage and operating frequency of the drive control layer are dynamically set according to the real-time task load. The task decision result is converted into a pulse width modulation signal by a digital signal processing module; Motor torque control parameters are generated based on the pulse width modulation signal; Output a joint angle control signal based on the motor torque control parameters; The joint angle control signal is transmitted to the target actuator through the actuator interface to perform mechanical actions.

6. The multi-level heterogeneous integrated chip task processing method as described in claim 1, characterized in that, Based on the task feedback signal returned after the action is performed, the synaptic weights of the neuromorphic processing unit are adjusted in real time, including: Receive the task feedback signal containing information about changes in the environmental state after the action is performed; Analyze the task completion rate index in the task feedback signal; The direction of synaptic weight adjustment is determined based on the task completion index; The synaptic weight adjustment amount of the neuromorphic processing unit is determined based on the synaptic weight adjustment direction; The synaptic weight parameters of the neuromorphic processing unit are adjusted according to the synaptic weight adjustment amount; The adjusted synaptic weight parameters are stored in the weight register.

7. The multi-level heterogeneous integrated chip task processing method as described in claim 1, characterized in that, Monitoring the operating status parameters of the multi-level heterogeneous integrated chip, and dynamically adjusting the processing frequency and structural parameters of the multi-level heterogeneous integrated chip based on the operating status parameters, including: The real-time temperature parameters of the multi-level heterogeneous integrated chip are monitored by an integrated temperature sensor. The real-time power consumption parameters of the multi-level heterogeneous integrated chip are collected by the power consumption monitoring unit. The processing load parameters of the multi-level heterogeneous integrated chip are obtained through the processing load analysis module. The real-time temperature parameters, real-time power consumption parameters, and processing load parameters are transmitted to the analysis module via the on-chip bus. The analysis module determines the temperature overload state and the processing efficiency state. When a temperature overload is detected, the processing frequency of the multi-level heterogeneous integrated chip is reduced. When a decrease in processing efficiency is detected, the neural network structure of the neuromorphic processing unit is reconstructed; Reset the synaptic weight distribution of the neuromorphic processing unit.

8. A multi-level heterogeneous integrated chip task processing device, characterized in that, The multi-level heterogeneous integrated chip task processing device includes: Chip-level building module, used to build a multi-level heterogeneous integrated chip including a perception processing layer, an intelligent decision-making layer and a drive control layer connected by a vertical interconnect structure; The multimodal perception module is used to receive multimodal task data through the perception processing layer, and to perform parallel feature extraction on the multimodal task data to generate multimodal feature vectors. The neural decision-making module is used to input the multimodal feature vector into the neuromorphic processing unit of the intelligent decision-making layer, and process the multimodal feature vector based on synaptic weights in the neuromorphic processing unit to obtain the task decision result; An execution control module is used to convert the task decision result into a drive control signal through the drive control layer, and control the target execution device to perform actions according to the drive control signal; The synaptic weight learning module is used to adjust the synaptic weights of the neuromorphic processing unit in real time based on the task feedback signal returned after the action is performed. An adaptive operation status module is used to monitor the operation status parameters of the multi-level heterogeneous integrated chip and dynamically adjust the processing frequency and structural parameters of the multi-level heterogeneous integrated chip based on the operation status parameters.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a multi-level heterogeneous integrated chip task processing program stored in the memory and executable on the processor. When the multi-level heterogeneous integrated chip task processing program is executed by the processor, it implements the steps of the multi-level heterogeneous integrated chip task processing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a multi-level heterogeneous integrated chip task processing program, which, when executed by a processor, implements the steps of the multi-level heterogeneous integrated chip task processing method as described in any one of claims 1-7.

Images