Low-power-consumption heterogeneous NPU system and corresponding control method and device

Abstract

The invention relates to the technical field of NPUs, and discloses a low-power-consumption heterogeneous NPU system and a corresponding control method and device. A low-power-consumption domain senses environmental parameters in a preset range; judging whether the environmental parameters are matched with preset environmental parameters or not; when the judgment result is no, generating a first trigger instruction to trigger the low-power-consumption domain to execute the operation of sensing the environmental parameters in the preset range; when the judgment result is yes, generating a second trigger instruction; the high-performance domain executes a preset first analysis processing operation on the environment parameter according to the second trigger instruction to obtain an application analysis parameter of the environment parameter, so that the high-performance domain executes a service processing operation matched with the application analysis parameter; and judging whether a newly added second trigger instruction exists in a preset time period, and if not, generating a third trigger instruction to trigger the low-power-consumption domain to execute the operation of sensing the environmental parameters in the preset range. Therefore, the power consumption of the NPU can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of NPU technology, and in particular to a low-power heterogeneous NPU system and a corresponding control method and device. Background Art

[0002] With the widespread application of artificial intelligence technology in embedded terminals, the computing power demand of smart devices (such as smart speakers, security cameras, industrial detection equipment, etc.) for neural network processors (NPUs) continues to grow.

[0003] To meet the computing power requirements of AI tasks, traditional solutions mainly use two types of NPU architectures: Single-core high-performance NPU: Processes complex models by improving single-core computing power (e.g., 3TOPS@int8); Homogeneous multi-core NPU: Improves parallelism by stacking multiple medium-power cores (such as 2T+2T+2T); However, practice has found that for single-core architectures, there are problems with continuous operation of high-computing units, resulting in excessive energy consumption, and an inability to support low-power scenarios. At the same time, there is a certain degree of computing power redundancy when using it to process lightweight tasks. For multi-core architectures, practice has found that when inferring the same model, the multi-core architecture does not have the effect of doubling the speed compared to the single-core architecture. For example, the dual-core 4T (2T+2T) is about 30%~50% faster than the single-core 2T. On this basis, the multi-core architecture increases the chip area, and the power consumption increases significantly when activated at the same time.

[0004] It can be seen that how to reduce the power consumption of NPU is particularly important. Summary of the Invention

[0005] The present invention provides a low-power heterogeneous NPU system and a corresponding control method and device, which can reduce the power consumption of the NPU.

[0006] In order to solve the above technical problems, the first aspect of the present invention discloses a low-power heterogeneous NPU system, which includes at least a low-power domain and a high-performance domain, wherein: The low power consumption domain is used to sense environmental parameters within a preset range; The low-power domain is further configured to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, a first trigger instruction is generated to trigger the low-power domain to perform the operation of sensing the environmental parameters within the preset range; when it is determined that the environmental parameters match the preset environmental parameters, a second trigger instruction is generated; The high-performance domain is configured to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation matching the application parsing parameter; The high-performance domain is also used to determine whether there is a newly added second trigger instruction within the preset time period. When it is determined that there is no newly added second trigger instruction within the preset time period, a third trigger instruction is generated to trigger the low-power domain to perform the operation of perceiving the environmental parameters within the preset range.

[0007] As an optional embodiment, in the first aspect of the present invention, the low power consumption domain includes a continuously powered domain and a selectively powered domain, the continuously powered domain includes at least a sensing unit, and the selectively powered domain includes at least a first processing unit, wherein: The continuously powered domain is used to sense the environmental parameters within the preset range, where the environmental parameters include at least ambient sound parameters and perceived object parameters; The continuously powered domain is further configured to determine whether the ambient sound parameter is within a preset ambient sound parameter range and / or whether the perceived object parameter is within a preset perceived object parameter range, and generate a fourth trigger instruction when it is determined that the ambient sound parameter is within the preset ambient sound parameter range and / or the perceived object parameter is within the preset perceived object parameter range; The selected powered domain is configured to perform a preset second parsing operation on the environmental parameter according to the fourth trigger instruction to obtain environmental characteristic parameters of the environmental parameter, wherein the environmental characteristic parameters include environmental sound characteristic parameters and / or perceived object characteristic parameters; The selection of the powered domain is also used to determine whether the environmental characteristic parameters match the preset environmental characteristic parameters. When it is determined that the environmental characteristic parameters do not match the preset environmental characteristic parameters, the first trigger instruction is generated; when it is determined that the environmental characteristic parameters match the preset environmental characteristic parameters, the second trigger instruction is generated according to the environmental characteristic parameters.

[0008] As an optional embodiment, in the first aspect of the present invention, the high-performance domain includes a high-performance processing domain and a data domain, the high-performance processing domain includes at least a second processing unit, the data computing and processing capability of the second processing unit is higher than that of the first processing unit, and the data domain includes at least DDR memory chips, wherein: The high-performance processing domain is configured to perform a preset first parsing operation on the environment parameter according to the second trigger instruction, obtain an application parsing parameter of the environment parameter, and perform a business processing operation matching the application parsing parameter; The high-performance processing domain is further configured to determine whether a newly added second trigger instruction exists within a preset time period, and when it is determined that no newly added second trigger instruction exists within the preset time period, generate a third trigger instruction to trigger the low-power consumption domain to perform the operation of sensing the environmental parameters within the preset range; The data field is used to store the environmental parameters.

[0009] As an optional embodiment, in the first aspect of the present invention, the ambient sound characteristic parameter includes a keyword characteristic parameter, the perceived object characteristic parameter includes a perceived object type characteristic parameter, and the selecting a powered domain to determine whether the ambient characteristic parameter matches a preset ambient characteristic parameter includes: According to the environmental sound characteristic parameters, determining whether the keyword characteristic parameters match the preset keyword characteristic parameters; According to the perceived object characteristic parameters, determining whether the perceived object type characteristic parameters match preset perceived object type characteristic parameters; When it is determined that the keyword characteristic parameter matches the preset keyword characteristic parameter and / or the perceptual object type characteristic parameter matches the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter matches the preset environmental characteristic parameter; When it is determined that the keyword characteristic parameter does not match the preset keyword characteristic parameter and the perceptual object type characteristic parameter does not match the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter does not match the preset environmental characteristic parameter.

[0010] As an optional embodiment, in the first aspect of the present invention, the data domain is further used to activate the self-refresh mechanism of the DDR memory particle when the low power domain is running; the self-refresh mechanism is used to represent an operation of maintaining data integrity through periodic charge re-storage; The data domain is further configured to exit the self-refresh mechanism and provide data read and write services when the high-performance processing domain is running.

[0011] As an optional implementation, in the first aspect of the present invention, the high-performance processing domain is further used to parse the calculation attribute parameters and real-time attribute parameters of the application parsing parameters; The high-performance processing domain is further configured to decompose the business processing operation matching the application parsing parameter into a computing-intensive business processing task operation and a real-time business processing task operation according to the computing attribute parameter and the real-time attribute parameter; The high-performance processing domain is further used to allocate computationally intensive business processing task operations to the second processing unit; The high-performance processing domain is further used to allocate real-time task business processing task operations to the first processing unit.

[0012] As an optional embodiment, in the first aspect of the present invention, the system further includes a power recovery unit, wherein: The power recovery unit is configured to trigger execution of a preset power recovery operation in response to the third trigger instruction; Furthermore, the preset power recovery operation includes: Cutting off active power supply to the high-performance processing domain; Cutting off active power supply to the selected powered domain; The continuously powered domain is maintained to be powered, so as to continuously sense the environmental parameter within the preset range.

[0013] A second aspect of the present invention discloses a low-power heterogeneous NPU system control method, the method comprising: The method is applied to a low-power heterogeneous NPU system, which includes at least a low-power domain and a high-performance domain. The method includes: Controlling the low power domain to sense environmental parameters within a preset range; controlling the low-power domain to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, controlling the low-power domain to generate a first trigger instruction to trigger the low-power domain to perform the operation of sensing the environmental parameters within the preset range; and when it is determined that the environmental parameters match the preset environmental parameters, controlling the low-power domain to generate a second trigger instruction; controlling the high-performance domain to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation matching the application parsing parameter; The high-performance domain is controlled to determine whether there is a newly added second trigger instruction within the preset time period. When it is determined that there is no newly added second trigger instruction within the preset time period, the high-performance domain is controlled to generate a third trigger instruction to trigger the low-power consumption domain to perform the operation of perceiving the environmental parameters within the preset range.

[0014] As an optional embodiment, in the second aspect of the present invention, the low power consumption domain includes a continuously powered domain and a selectively powered domain, the continuously powered domain includes at least a perception unit, the selectively powered domain includes at least a first processing unit, the environmental parameters include at least ambient sound parameters and perceived object parameters, and controlling the low power consumption domain to perceive environmental parameters within a preset range includes: Controlling the continuously powered domain to sense the environmental parameter within the preset range; Furthermore, controlling the low power consumption domain to determine whether the environmental parameter matches a preset environmental parameter includes: controlling the continuously powered domain to determine whether the ambient sound parameter is within a preset ambient sound parameter range and / or whether the perceived object parameter is within a preset perceived object parameter range; and when it is determined that the ambient sound parameter is within the preset ambient sound parameter range and / or the perceived object parameter is within the preset perceived object parameter range, controlling the continuously powered domain to generate a fourth trigger instruction; controlling the selected powered domain to perform a preset second parsing operation on the environmental parameter according to the fourth trigger instruction to obtain environmental characteristic parameters of the environmental parameter, where the environmental characteristic parameters include environmental sound characteristic parameters and / or perceived object characteristic parameters; controlling the selected powered domain to determine whether the environmental characteristic parameter matches the preset environmental characteristic parameter, and when it is determined that the environmental characteristic parameter does not match the preset environmental characteristic parameter, controlling the selected powered domain to determine that the environmental parameter does not match the preset environmental parameter; When it is determined that the environmental characteristic parameter matches the preset environmental characteristic parameter, controlling the selected powered domain to determine that the environmental parameter matches the preset environmental parameter; Furthermore, the step of controlling the low power consumption domain to generate a first trigger instruction to trigger the low power consumption domain to perform the operation of sensing the environmental parameters within a preset range includes: Controlling the selected powered domain to generate the first trigger instruction to trigger an operation of controlling the continuously powered domain to sense the environmental parameter within the preset range; Furthermore, controlling the low power consumption domain to generate a second trigger instruction includes: The controlling the selected powered domain generates the second trigger instruction according to the environmental characteristic parameter.

[0015] As an optional implementation, in the second aspect of the present invention, the high-performance domain includes a high-performance processing domain and a data domain, the high-performance processing domain includes at least a second processing unit, the data calculation and processing capability of the second processing unit is higher than that of the first processing unit, and the data domain includes at least DDR memory chips. Controlling the high-performance domain to perform a preset first parsing and processing operation on the environmental parameters according to the second trigger instruction to obtain application parsing parameters of the environmental parameters, so that the high-performance domain performs a business processing operation that matches the application parsing parameters, includes: controlling the high-performance processing domain to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance processing domain performs a business processing operation matching the application parsing parameter; Furthermore, controlling the high-performance domain to determine whether there is a newly added second trigger instruction within a preset time period, and when it is determined that there is no newly added second trigger instruction within the preset time period, controlling the high-performance domain to generate a third trigger instruction to trigger the low-power domain to perform the operation of sensing the environmental parameters within the preset range, includes: controlling the high-performance processing domain to determine whether a newly added second trigger instruction exists within a preset time period; and when it is determined that no newly added second trigger instruction exists within the preset time period, controlling the high-performance processing domain to generate a third trigger instruction to trigger the low-power consumption domain to perform the operation of sensing the environmental parameters within the preset range; And, the method further comprises: The data domain is controlled to store the environmental parameters.

[0016] As an optional embodiment, in the second aspect of the present invention, the ambient sound characteristic parameter includes a keyword characteristic parameter, the perceived object characteristic parameter includes a perceived object type characteristic parameter, and the selecting a powered domain to determine whether the ambient characteristic parameter matches a preset ambient characteristic parameter includes: According to the environmental sound characteristic parameters, determining whether the keyword characteristic parameters match the preset keyword characteristic parameters; According to the perceived object characteristic parameters, determining whether the perceived object type characteristic parameters match preset perceived object type characteristic parameters; When it is determined that the keyword characteristic parameter matches the preset keyword characteristic parameter and / or the perceptual object type characteristic parameter matches the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter matches the preset environmental characteristic parameter; When it is determined that the keyword characteristic parameter does not match the preset keyword characteristic parameter and the perceptual object type characteristic parameter does not match the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter does not match the preset environmental characteristic parameter.

[0017] As an optional embodiment, in the second aspect of the present invention, the method further includes: Controlling the data domain to operate in the low-power domain to activate a self-refresh mechanism of the DDR memory particle; the self-refresh mechanism is used to represent an operation of maintaining data integrity through periodic charge restoration; The data domain is controlled to exit the self-refresh mechanism and provide data reading and writing services when the high-performance processing domain is running.

[0018] As an optional embodiment, in the second aspect of the present invention, the method further includes: Controlling the high-performance processing domain to parse the calculation attribute parameters and the real-time attribute parameters of the application parsing parameters; Controlling the high-performance processing domain to decompose the business processing operation matching the application parsing parameter into a computing-intensive business processing task operation and a real-time business processing task operation according to the computing attribute parameter and the real-time attribute parameter; Controlling the high-performance processing domain to allocate computationally intensive business processing task operations to the second processing unit; The high-performance processing domain is controlled to distribute the real-time task business processing task operation to the first processing unit.

[0019] As an optional embodiment, in the second aspect of the present invention, the system further includes a power recovery unit, and the method further includes: controlling the power recovery unit to trigger execution of a preset power recovery operation in response to the third trigger instruction; Furthermore, the preset power recovery operation includes: Cutting off active power supply to the high-performance processing domain; Cutting off active power supply to the selected powered domain; The continuously powered domain is maintained to be powered, so as to continuously sense the environmental parameter within the preset range.

[0020] A third aspect of the present invention discloses another low-power heterogeneous NPU system control device, which is applied to a low-power heterogeneous NPU system. The system includes at least a low-power domain and a high-performance domain. The device includes: A control module, configured to control the low power domain to sense environmental parameters within a preset range; a control module, configured to control the low-power domain to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, control the low-power domain to generate a first trigger instruction to trigger the low-power domain to perform the operation of sensing the environmental parameters within the preset range; and when it is determined that the environmental parameters match the preset environmental parameters, control the low-power domain to generate a second trigger instruction; a control module, configured to control the high-performance domain to perform a preset first parsing operation on the environment parameter according to the second trigger instruction, to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation matching the application parsing parameter; A control module is used to control the high-performance domain to determine whether there is a newly added second trigger instruction within a preset time period. When it is determined that there is no newly added second trigger instruction within the preset time period, the high-performance domain is controlled to generate a third trigger instruction to trigger the low-power consumption domain to perform the operation of perceiving the environmental parameters within the preset range.

[0021] A fourth aspect of the present invention discloses another low-power heterogeneous NPU system control device, the device comprising: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the low-power heterogeneous NPU system control method disclosed in the second aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: In an embodiment of the present invention, the system includes at least a low-power domain and a high-performance domain, the low-power domain senses environmental parameters within a preset range; the low-power domain determines whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, a first trigger instruction is generated to trigger the low-power domain to perform an operation of sensing the environmental parameters within the preset range; when it is determined that the environmental parameters match the preset environmental parameters, a second trigger instruction is generated; the high-performance domain performs a preset first parsing and processing operation on the environmental parameters according to the second trigger instruction to obtain application parsing parameters of the environmental parameters, so that the high-performance domain performs a business processing operation that matches the application parsing parameters; the high-performance domain determines whether there is a new second trigger instruction within a preset time period, and when it is determined that there is no new second trigger instruction within the preset time period, a third trigger instruction is generated to trigger the low-power domain to perform an operation of sensing the environmental parameters within the preset range. It can be seen that the implementation of the embodiment of the present invention can divide the system into at least a low-power domain and a high-performance domain through a heterogeneous design of the NPU system, and creatively proposes a dynamic perception and matching mechanism of environmental parameters. Based on the low-power domain perceiving the environmental parameters within a preset range, when it is judged that the environmental parameters match the preset environmental parameters, the high-performance domain is triggered to perform a preset first parsing and processing operation on the environmental parameters according to the second trigger instruction, and obtain the application parsing parameters of the environmental parameters, so that the high-performance domain executes the business processing operation that matches the application parsing parameters. Through continuous perception and intelligent matching judgment, the high-performance domain can be activated on demand, avoiding the invalid power consumption caused by the continuous operation of the high-performance module in the traditional solution, which can significantly reduce the average power consumption of the system and extend the battery life of the device. At the same time, Through trigger instruction closed-loop control, that is, the first trigger instruction maintains the low-power monitoring state, the second trigger instruction accurately wakes up the high-performance domain, and the third trigger instruction realizes automatic sleep after timeout, constructs an adaptive power consumption control loop, eliminates the need for manual intervention, and improves the system energy efficiency ratio, that is, it can meet the continuous AI computing needs of low-power scenarios, and can shut down the NPU large core in the high-performance domain in low-power scenarios and run the NPU small core in the low-power domain. Compared with the single-core NPU architecture design, the present invention can also meet the needs of simultaneous reasoning of multiple AI models. When the NPU large core is occupied by a model with large computational workload, the NPU small core can independently infer a model with small computational workload and high real-time performance, thereby improving the richness of NPU application scenarios, the flexibility and controllability of NPU applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a schematic diagram of the structure of a low-power heterogeneous NPU system disclosed in an embodiment of the present invention; Figure 2 2 is a schematic structural diagram of another low-power heterogeneous NPU system disclosed in an embodiment of the present invention; Figure 3 This is a structural diagram of another low-power heterogeneous NPU system disclosed in an embodiment of the present invention; Figure 4 This is a flow chart of a low-power heterogeneous NPU system control method disclosed in an embodiment of the present invention; Figure 5 This is a schematic structural diagram of a low-power heterogeneous NPU system control device disclosed in an embodiment of the present invention; Figure 6 This is a structural diagram of another low-power heterogeneous NPU system control device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0026] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different items, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or end comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed therein, or may optionally include other steps or elements inherent to such process, method, product, or end.

[0027] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0028] The present invention discloses a low-power heterogeneous NPU system and a corresponding control method and device. The system can be divided into at least a low-power domain and a high-performance domain through a heterogeneous design of the NPU system, and creatively proposes a dynamic perception and matching mechanism for environmental parameters. The low-power domain perceives environmental parameters within a preset range. When it is judged that the environmental parameters match the preset environmental parameters, the high-performance domain is triggered to perform a preset first parsing and processing operation on the environmental parameters according to a second trigger instruction, and obtain application parsing parameters of the environmental parameters, so that the high-performance domain executes a business processing operation that matches the application parsing parameters. Through continuous perception and intelligent matching judgment, the high-performance domain is activated on demand, avoiding the ineffective power consumption caused by the continuous operation of the high-performance module in the traditional solution, which can significantly reduce the average power consumption of the system and delay Long device life; at the same time, through trigger instruction closed-loop control, that is, the first trigger instruction maintains the low-power monitoring state, the second trigger instruction accurately wakes up the high-performance domain, and the third trigger instruction realizes automatic sleep after timeout, constructs an adaptive power consumption control loop, eliminates the need for manual intervention, and improves the system energy efficiency ratio, that is, it can meet the continuous AI computing needs of low-power scenarios, and can shut down the NPU large core in the high-performance domain in low-power scenarios and run the NPU small core in the low-power domain. Compared with the single-core NPU architecture design, the present invention can also meet the needs of simultaneous reasoning of multiple AI models. When the NPU large core is occupied by a model with a large amount of computation, the NPU small core can independently infer a model with a small amount of computation and high real-time performance, thereby improving the richness of NPU application scenarios, the flexibility and controllability of NPU applications. The following are detailed explanations.

[0029] Example 1 See also Figure 1 , Figure 1 This is a schematic diagram of the structure of a low-power heterogeneous NPU system disclosed in an embodiment of the present invention. Figure 1 The described low-power heterogeneous NPU system can be applied to NPU application devices, and can also be applied to smart devices associated with NPU application devices, including but not limited to one or more of battery devices, cloud devices, edge computing devices, relay devices, base station devices, urban management devices, smart network devices, smart home devices, and AI industrial devices, which are not limited in the embodiments of the present invention. Figure 1 As shown, the low-power heterogeneous NPU system includes at least a low-power domain and a high-performance domain, wherein: The low power domain 10 is used to sense environmental parameters within a preset range; The low power domain 10 is further configured to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, a first trigger instruction is generated to trigger the low power domain 10 to perform an operation of sensing the environmental parameters within the preset range; when it is determined that the environmental parameters match the preset environmental parameters, a second trigger instruction is generated; The high-performance domain 20 is configured to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation that matches the application parsing parameter; The high-performance domain 20 is also used to determine whether there is a new second trigger instruction within the preset time period. When it is determined that there is no new second trigger instruction within the preset time period, a third trigger instruction is generated to trigger the low-power domain 10 to perform the operation of perceiving environmental parameters within the preset range.

[0030] In the embodiment of the present invention, it should be noted that, if it is determined that a newly added second trigger instruction exists within the preset time period, the high-performance domain 20 continues to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation that matches the application parsing parameter; and the high-performance domain 20 determines whether a newly added second trigger instruction exists within the preset time period; In the embodiment of the present invention, optionally, the low power consumption domain may include a CPU with relatively low processing capability, an NPU, a matching SRAM, a sensing unit, etc.; Optionally, the high-performance domain may include CPU, NPU, bus, DDRC, PHY, memory, storage, connection, etc. with relatively high processing capabilities; Further optionally, the above-mentioned perception unit may include a sensing unit and an imaging unit. The sensing unit may be a radio unit, an infrared sensor, etc., and the imaging unit may be a camera, a radar, etc., which may collect audio decibel values, object movement distance data, etc.; Optionally, for environmental perception, the microphone can sample the ambient audio peak value (unit: dB) every 200ms, and the infrared sensor can detect the distance of moving objects within 1 meter (unit: cm) every 500ms. For parameter matching, if the audio peak is less than 30dB and there is no moving object within 1 meter, it is judged as "mismatched" and the first trigger instruction is generated, which restarts the sensing process. If the audio peak is ≥30dB and / or an object is detected within 1 meter → it is determined to be a "match" → a second trigger instruction is generated.

[0031] For high-performance domain response, after receiving the second trigger command, the speech recognition engine (e.g., Mel spectrum analysis) or behavior analysis model (e.g., skeleton key point detection) can be started to output the command text or behavior classification results; For dormancy determination, after the high-performance domain completes processing, if there is no new trigger instruction within a preset period of time, a third trigger instruction is generated, and the low-power domain restarts monitoring.

[0032] During this process, domains and units not involved can be selectively powered.

[0033] It can be seen that the embodiment of the present invention divides the system into at least a low-power domain and a high-performance domain through a heterogeneous design of the NPU system, and creatively proposes a dynamic perception and matching mechanism of environmental parameters. The low-power domain perceives the environmental parameters within a preset range. When it is judged that the environmental parameters match the preset environmental parameters, the high-performance domain is triggered to perform a preset first parsing and processing operation on the environmental parameters according to the second trigger instruction, and obtains the application parsing parameters of the environmental parameters, so that the high-performance domain executes the business processing operation that matches the application parsing parameters. Through continuous perception and intelligent matching judgment, the high-performance domain is activated on demand, avoiding the invalid power consumption caused by the continuous operation of the high-performance module in the traditional solution, which can significantly reduce the average power consumption of the system and extend the battery life of the device. At the same time, Through trigger instruction closed-loop control, that is, the first trigger instruction maintains the low-power monitoring state, the second trigger instruction accurately wakes up the high-performance domain, and the third trigger instruction realizes automatic sleep after timeout, constructs an adaptive power consumption control loop, eliminates the need for manual intervention, and improves the system energy efficiency ratio, that is, it can meet the continuous AI computing needs of low-power scenarios, and can shut down the NPU large core in the high-performance domain in low-power scenarios and run the NPU small core in the low-power domain. Compared with the single-core NPU architecture design, the present invention can also meet the needs of simultaneous reasoning of multiple AI models. When the NPU large core is occupied by a model with large computational workload, the NPU small core can independently infer a model with small computational workload and high real-time performance, thereby improving the richness of NPU application scenarios, the flexibility and controllability of NPU applications.

[0034] In the embodiment of the present invention, as an optional implementation method, Figure 2 As shown, the low power domain 10 includes a continuously powered domain 101 and a selectively powered domain 102. The continuously powered domain 101 includes at least a sensing unit 1011, and the selectively powered domain 102 includes at least a first processing unit 1021, wherein: The continuously powered domain 101 is used to sense environmental parameters within a preset range, where the environmental parameters include at least ambient sound parameters and perceived object parameters; The continuously powered domain 101 is further configured to determine whether the ambient sound parameter is within a preset ambient sound parameter range and / or whether the perceived object parameter is within a preset perceived object parameter range, and generate a fourth trigger instruction when the ambient sound parameter is determined to be within the preset ambient sound parameter range and / or the perceived object parameter is within the preset perceived object parameter range; Selecting a powered domain 102 for performing a preset second parsing operation on the environmental parameter according to the fourth trigger instruction to obtain environmental characteristic parameters of the environmental parameter, where the environmental characteristic parameters include environmental sound characteristic parameters and / or perceived object characteristic parameters; The powered domain 102 is selected and is also used to determine whether the environmental characteristic parameters match the preset environmental characteristic parameters. When it is determined that the environmental characteristic parameters do not match the preset environmental characteristic parameters, a first trigger instruction is generated; when it is determined that the environmental characteristic parameters match the preset environmental characteristic parameters, a second trigger instruction is generated according to the environmental characteristic parameters.

[0035] In the embodiment of the present invention, it is optional to determine whether the audio peak value is ≥30dB or whether the object distance is ≤1 meter; If any of the conditions are met, the fourth trigger instruction is generated, and the MOSFET is turned on to wake up the selected powered domain.

[0036] For feature extraction and matching, you can start after selecting the powered domain: Audio data → Extract MFCC feature vectors; (e.g. dimension: 20) Object data → Extract HOG features; (such as histogram of oriented gradients) For feature comparison: If the cosine similarity between the MFCC and the preset wake-up word feature is less than 0.8 and the HOG non-humanoid classification confidence is less than 70% → generate the first trigger instruction (close this domain); If any condition is met, a second trigger instruction is generated (to wake up the high-performance domain). It can be seen that the implementation of this optional embodiment can achieve hardware-level power consumption isolation and minimize standby energy consumption by further hierarchical design of power consumption domains, with the continuously powered domain (normal power) ensuring basic monitoring functions and the selected powered domain (powered on demand) performing initial screening tasks; through a two-level judgment mechanism, the continuous domain performs threshold judgment (volume / object distance) and the selected domain performs feature matching (keywords / target types), thereby reducing the false wake-up rate and avoiding invalid startup of the high-performance domain.

[0037] In the embodiment of the present invention, as another optional implementation, Figure 2 As shown, the high-performance domain 20 includes a high-performance processing domain 201 and a data domain 202. The high-performance processing domain 201 includes at least a second processing unit 2011. The data operation and processing capability of the second processing unit 2011 is higher than that of the first processing unit 1021. The data domain 202 includes at least a DDR memory chip 2021. The high-performance processing domain 201 is configured to perform a preset first parsing operation on the environment parameter according to the second trigger instruction, obtain an application parsing parameter of the environment parameter, and perform a business processing operation matching the application parsing parameter; The high-performance processing domain 201 is further configured to determine whether a new second trigger instruction exists within a preset time period, and when it is determined that no new second trigger instruction exists within the preset time period, generate a third trigger instruction to trigger the low-power domain 10 to perform an operation of sensing an environmental parameter within a preset range. The data field 202 is used to store environmental parameters.

[0038] In an embodiment of the present invention, optionally, the above-mentioned DDR particles may be LPDDR4X chips with an integrated self-refresh controller; like Figure 3 As shown, Figure 3 This is a structural diagram of another low-power heterogeneous NPU system disclosed in an embodiment of the present invention; In combination with the above content, it can be seen that, optionally, the perception unit may include the above-mentioned sensing unit and the above-mentioned imaging unit. The sensing unit may be a radio unit, an infrared sensor, etc., and the imaging unit may be a camera, a radar, etc., which may collect audio decibel values, object movement distance data, etc. However, in actual application scenarios, it may be adjusted according to the needs of the application scenario, for example Figure 3 The image unit is classified as being in a selected powered domain; Further optional, such as Figure 3 As shown, the first processing unit is Figure 3 The Little NPU in the figure is different from the Big NPU (the second processing unit). Further optional, such as Figure 3 As shown, the continuously powered domain is further limited to include at least LittleCPU, basic judgment unit, SRAM and sensor unit; The selection of the powered domain is further limited to include at least the Little NPU and the graphics unit; The high-performance processing domain is further defined to include at least Big NPU, bus, DDRC, PHY, Storage, and Connection; The data domain at least includes Memory (i.e., DDR memory particles); Optionally, for the data processing flow, a second processing unit can receive environmental parameters (e.g., audio stream / PCM data). For voice scenarios, an RNN-T model can be used to output text instructions, and for visual scenarios, YOLOv3-Tiny can be used to output target coordinates. For sleep monitoring, a hardware timer (60-second countdown) can be enabled. If no new second trigger instruction is given during this period, a third trigger instruction is generated. Optionally, for data maintenance, DDR can be used to continuously store raw sensor data and intermediate features (e.g., MFCC / HOG matrices).

[0039] It can be seen that the implementation of this optional embodiment can achieve division of labor through heterogeneous processing units. The second processing unit (high computing power) processes complex tasks, and the first processing unit (low computing power) processes preliminary screening tasks. The two have complementary capabilities, which can maximize the utilization of computing resources and avoid resource waste caused by high computing power units processing lightweight tasks; through the data domain dormancy protection mechanism, DDR memory particles save environmental parameters during the dormancy period to ensure data integrity and support rapid system wake-up and on-site recovery.

[0040] In an embodiment of the present invention, as another optional implementation, the above-mentioned ambient sound characteristic parameters include keyword characteristic parameters, the perceived object characteristic parameters include perceived object type characteristic parameters, and selecting the powered domain 102 to determine whether the ambient characteristic parameters match the preset ambient characteristic parameters includes: According to the environmental sound characteristic parameters, determining whether the keyword characteristic parameters match the preset keyword characteristic parameters; According to the characteristic parameters of the perceived object, determining whether the characteristic parameters of the perceived object type match the preset characteristic parameters of the perceived object type; When it is determined that the keyword characteristic parameter matches the preset keyword characteristic parameter and / or the perceptual object type characteristic parameter matches the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter matches the preset environmental characteristic parameter; When it is determined that the keyword characteristic parameters do not match the preset keyword characteristic parameters and the perceptual object type characteristic parameters do not match the preset perceptual object type characteristic parameters, it is determined that the environmental characteristic parameters do not match the preset environmental characteristic parameters.

[0041] In an embodiment of the present invention, optionally, for keyword matching, the MFCC feature sequence can be subjected to CTC decoding → a character sequence can be output → an edit distance calculation can be performed with a preset vocabulary (such as "open" / "close"); for matching conditions such as edit distance ≤ 2; for humanoid matching, the HOG feature can be input into an SVM classifier → a binary classification probability value (such as humanoid / non-humanoid) can be output; for matching conditions such as humanoid probability ≥ 70%; Further optionally, for the decision rule, if any one of the matches is successful, it is determined that the whole is matched → triggering the second instruction; if both matches fail, it is determined that there is no match → triggering the first instruction.

[0042] It can be seen that the implementation of this optional embodiment can achieve parallel matching of voice keywords and visual target types through dual-modal independent matching logic, and any match will be triggered, which can enhance scene adaptability and be compatible with the triggering requirements of voice / vision single modality or fusion modality; through feature-level precise judgment, based on the matching of ambient sound feature parameters (such as acoustic features) and perceived object feature parameters (such as shape features), and non-raw data comparison, it can improve recognition robustness and reduce false triggering caused by environmental noise interference.

[0043] In an optional embodiment, the data domain 202 is further used to activate the self-refresh mechanism of the DDR memory chip when the low power domain 10 is running; the self-refresh mechanism is used to represent an operation of maintaining data integrity through periodic charge re-storage; The data domain 202 is also used to exit the self-refresh mechanism and provide data reading and writing services when the high-performance processing domain 201 is running.

[0044] In an embodiment of the present invention, optionally, for the self-refresh mechanism operation, in terms of starting during the sleep period, when the low-power domain is running, the DDR controller can be switched to self-refresh mode (such as refresh cycle: 64ms), and the voltage is reduced to 0.6V to maintain data charge; in terms of exiting the operating period, the high-performance domain activation signal can trigger the DDR to exit self-refresh → voltage returns to 1.1V → switch to normal read and write mode.

[0045] It can be seen that the implementation of this optional embodiment can intelligently switch through the self-refresh mechanism, activate self-refresh (periodic charge restorage) when operating in the low-power domain, and exit self-refresh when operating in the high-performance domain. It can balance the contradiction between data preservation and power consumption, and only maintain data preservation with extremely low power consumption during the sleep period.

[0046] In another optional embodiment, the high-performance processing domain 201 is further used to parse the calculation attribute parameters and real-time attribute parameters of the application parsing parameters; The high-performance processing domain 201 is further used to decompose the business processing operations matching the application parsing parameters into computing-intensive business processing task operations and real-time business processing task operations according to the computing attribute parameters and the real-time attribute parameters; The high-performance processing domain 201 is further used to allocate computationally intensive business processing task operations to the second processing unit 2011; The high-performance processing domain 201 is further used to allocate real-time task business processing task operations to the first processing unit 1021 .

[0047] In the embodiment of the present invention, optionally, for task splitting and allocation, in terms of attribute parsing: Computational attributes: divided by the number of model layers / parameters; (e.g., ResNet34 → computationally intensive) Real-time attributes: divided according to latency requirements; (e.g., voice wake-up response < 200ms → real-time task) In terms of allocation execution: Second processing unit (large core): performs computationally intensive tasks (e.g., 1080P video object detection) Data path: DDR→DMA→NPU private cache; The first processing unit (small core): performs real-time tasks (e.g., wake-up word detection for a 16kHz audio stream). Data path: SRAM directly connected to the processing unit.

[0048] It can be seen that the implementation of this optional embodiment can split tasks according to computing attributes (computing intensity) and real-time attributes (latency requirements) through a task attribute-oriented allocation mechanism, and allocate them to matching processing units, so that the large core processes computing-intensive tasks → ensuring the efficiency of complex model reasoning; the small core processes real-time tasks → ensuring low-latency response (such as voice wake-up); through parallel scheduling of heterogeneous computing power, large and small cores can process different types of tasks at the same time, which can support multi-model parallel reasoning and solve the problem of competition for single NPU core resources.

[0049] In another optional embodiment, as Figure 2 As shown, the system further includes a power recovery unit 30, wherein: The power recovery unit 30 is configured to trigger execution of a preset power recovery operation in response to a third trigger instruction; And, the preset power recovery operations include: Cut off the active power supply to the high-performance processing domain; Cut off the active power supply of the selected powered domain; Maintain continuous power supply to the powered domain to continuously sense environmental parameters within the preset range.

[0050] In the embodiment of the present invention, optionally, for the power recovery operation: A third trigger instruction is received: the power recovery unit is triggered by the high-performance domain hardware interrupt.

[0051] Hierarchical power-off sequence: Step 1: Cut off the 12V power supply of the high-performance domain; Step 2: Disconnect the 3.3V power supply of the selected powered domain; Step 3: Maintain 1.8V power supply to the power supply domain; Status feedback: After each stage of power-off is completed, an ACK signal is sent to the power management IC.

[0052] It can be seen that the implementation of this optional embodiment can shut down in descending order of power consumption level through a hierarchical power recovery sequence: high-performance domain → select the powered domain → only maintain the continuously powered domain to avoid the risk of voltage mutation; optimize the power recovery efficiency and return to the lowest power consumption state as quickly as possible; through continuous monitoring of the normal power domain, after shutting down the high-power module, the continuously powered domain is still running, maintaining basic environmental perception capabilities, and ensuring that there are no monitoring blind spots.

[0053] Example 2 See also Figure 4 , Figure 4This is a flow chart of a low-power heterogeneous NPU system control method disclosed in an embodiment of the present invention. Figure 4 The described low-power heterogeneous NPU system control method can be applied to NPU application devices, and can also be applied to smart devices associated with NPU application devices, such as but not limited to battery devices, cloud devices, edge computing devices, relay devices, base station devices, city management devices, smart network devices, smart home devices, AI industrial devices, etc. The embodiment of the present invention does not limit this. Optionally, the control method can be applied to the low-power heterogeneous NPU system described in Example 1, and the system includes at least a low-power domain and a high-performance domain, such as Figure 4 As shown, the low-power heterogeneous NPU system control method may include the following operations: 401. Control the low power domain to sense environmental parameters within a preset range; 402. Control the low power domain to determine whether the environmental parameters match the preset environmental parameters; if it is determined that the environmental parameters do not match the preset environmental parameters, step 403 is triggered; if it is determined that the environmental parameters match the preset environmental parameters, step 404 is triggered; 403. Control the low power consumption domain to generate a first trigger instruction to trigger the execution of step 401. 404. Control the low power consumption domain to generate a second trigger instruction; 405. Control the high-performance domain to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation that matches the application parsing parameter. 406. Control the high-performance domain to determine whether there is a newly added second trigger instruction within the preset time period; if it is determined that there is no newly added second trigger instruction within the preset time period, step 407 is triggered; 407 : Control the high-performance domain to generate a third trigger instruction to trigger the execution of step 401 .

[0054] In the embodiment of the present invention, it should be noted that, if it is determined that a new second trigger instruction exists within the preset time period, the operations of step 405 and step 406 are executed; In the embodiment of the present invention, it should be noted that the method can be applied to the low-power heterogeneous NPU system described in Example 1. Therefore, for the supplementary explanation of the embodiment of the present invention, please refer to the supplementary explanation of the low-power heterogeneous NPU system in Example 1, and the embodiment of the present invention will not go into details about this.

[0055] It can be seen that the embodiment of the present invention divides the system into at least a low-power domain and a high-performance domain through a heterogeneous design of the NPU system, and creatively proposes a dynamic perception and matching mechanism of environmental parameters. The low-power domain perceives the environmental parameters within a preset range. When it is judged that the environmental parameters match the preset environmental parameters, the high-performance domain is triggered to perform a preset first parsing and processing operation on the environmental parameters according to the second trigger instruction, and obtains the application parsing parameters of the environmental parameters, so that the high-performance domain executes the business processing operation that matches the application parsing parameters. Through continuous perception and intelligent matching judgment, the high-performance domain is activated on demand, avoiding the invalid power consumption caused by the continuous operation of the high-performance module in the traditional solution, which can significantly reduce the average power consumption of the system and extend the battery life of the device. At the same time, Through trigger instruction closed-loop control, that is, the first trigger instruction maintains the low-power monitoring state, the second trigger instruction accurately wakes up the high-performance domain, and the third trigger instruction realizes automatic sleep after timeout, constructs an adaptive power consumption control loop, eliminates the need for manual intervention, and improves the system energy efficiency ratio, that is, it can meet the continuous AI computing needs of low-power scenarios, and can shut down the NPU large core in the high-performance domain in low-power scenarios and run the NPU small core in the low-power domain. Compared with the single-core NPU architecture design, the present invention can also meet the needs of simultaneous reasoning of multiple AI models. When the NPU large core is occupied by a model with large computational workload, the NPU small core can independently infer a model with small computational workload and high real-time performance, thereby improving the richness of NPU application scenarios, the flexibility and controllability of NPU applications.

[0056] In an embodiment of the present invention, as an optional implementation, the above-mentioned low power domain includes a continuously powered domain and a selectively powered domain, the continuously powered domain includes at least a perception unit, the selectively powered domain includes at least a first processing unit, the environmental parameters include at least ambient sound parameters and perceived object parameters, and controlling the low power domain to perceive environmental parameters within a preset range includes: Control the continuously powered domain to sense environmental parameters within a preset range; Optionally, the controlling low power consumption domain determines whether the environmental parameters match the preset environmental parameters, including: controlling the continuously powered domain to determine whether the ambient sound parameter is within a preset ambient sound parameter range and / or whether the perceived object parameter is within a preset perceived object parameter range; and when it is determined that the ambient sound parameter is within the preset ambient sound parameter range and / or the perceived object parameter is within the preset perceived object parameter range, controlling the continuously powered domain to generate a fourth trigger instruction; Controlling the selected powered domain to perform a preset second parsing operation on the environmental parameter according to the fourth trigger instruction to obtain environmental characteristic parameters of the environmental parameter, where the environmental characteristic parameters include environmental sound characteristic parameters and / or perceived object characteristic parameters; The controlled powered domain is selected to determine whether the environmental characteristic parameter matches the preset environmental characteristic parameter. When it is determined that the environmental characteristic parameter does not match the preset environmental characteristic parameter, the controlled powered domain is selected to determine that the environmental parameter does not match the preset environmental parameter. When it is determined that the environmental characteristic parameters match the preset environmental characteristic parameters, the powered domain is controlled to select the environmental parameters to match the preset environmental parameters; Optionally, the above-mentioned controlling the low power consumption domain to generate a first trigger instruction to trigger the low power consumption domain to perform an operation of sensing an environmental parameter within a preset range includes: Controlling the selected powered domain to generate a first trigger instruction to trigger an operation of controlling the continuously powered domain to sense an environmental parameter within a preset range; Optionally, the above-mentioned controlling the low power consumption domain to generate the second trigger instruction includes: The powered domain is controlled to generate a second trigger instruction according to the environmental characteristic parameters.

[0057] It can be seen that the implementation of this optional embodiment can achieve hardware-level power consumption isolation and minimize standby energy consumption by further hierarchical design of power consumption domains, with the continuously powered domain (normal power) ensuring basic monitoring functions and the selected powered domain (powered on demand) performing initial screening tasks; through a two-level judgment mechanism, the continuous domain performs threshold judgment (volume / object distance) and the selected domain performs feature matching (keywords / target types), thereby reducing the false wake-up rate and avoiding invalid startup of the high-performance domain.

[0058] In an embodiment of the present invention, as another optional implementation, the high-performance domain includes a high-performance processing domain and a data domain. The high-performance processing domain includes at least a second processing unit. The data operation and processing capability of the second processing unit is higher than that of the first processing unit. The data domain includes at least DDR memory chips. The high-performance domain is controlled to perform a preset first parsing and processing operation on the environmental parameters according to the second trigger instruction to obtain application parsing parameters of the environmental parameters, so that the high-performance domain performs a business processing operation that matches the application parsing parameters, including: Controlling the high-performance processing domain to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance processing domain performs a business processing operation that matches the application parsing parameter; Optionally, the above-mentioned control of the high-performance domain to determine whether there is a newly added second trigger instruction within a preset time period, when it is determined that there is no newly added second trigger instruction within the preset time period, controls the high-performance domain to generate a third trigger instruction to trigger the low-power domain to perform an operation of sensing environmental parameters within a preset range, including: Controlling the high-performance processing domain to determine whether a new second trigger instruction exists within a preset time period; when it is determined that no new second trigger instruction exists within the preset time period, controlling the high-performance processing domain to generate a third trigger instruction to trigger the low-power domain to perform an operation of sensing environmental parameters within a preset range; Optionally, the method may further include: The control data domain stores environmental parameters.

[0059] It can be seen that the implementation of this optional embodiment can achieve division of labor through heterogeneous processing units. The second processing unit (high computing power) processes complex tasks, and the first processing unit (low computing power) processes preliminary screening tasks. The two have complementary capabilities, which can maximize the utilization of computing resources and avoid resource waste caused by high computing power units processing lightweight tasks; through the data domain dormancy protection mechanism, DDR memory particles save environmental parameters during the dormancy period to ensure data integrity and support rapid system wake-up and on-site recovery.

[0060] In an embodiment of the present invention, as another optional implementation manner, the above-mentioned ambient sound characteristic parameters include keyword characteristic parameters, the perceived object characteristic parameters include perceived object type characteristic parameters, and selecting the powered domain to determine whether the ambient characteristic parameters match the preset ambient characteristic parameters includes: According to the environmental sound characteristic parameters, determining whether the keyword characteristic parameters match the preset keyword characteristic parameters; According to the characteristic parameters of the perceived object, determining whether the characteristic parameters of the perceived object type match the preset characteristic parameters of the perceived object type; When it is determined that the keyword characteristic parameter matches the preset keyword characteristic parameter and / or the perceptual object type characteristic parameter matches the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter matches the preset environmental characteristic parameter; When it is determined that the keyword characteristic parameters do not match the preset keyword characteristic parameters and the perceptual object type characteristic parameters do not match the preset perceptual object type characteristic parameters, it is determined that the environmental characteristic parameters do not match the preset environmental characteristic parameters.

[0061] It can be seen that the implementation of this optional embodiment can achieve parallel matching of voice keywords and visual target types through dual-modal independent matching logic, and any match will be triggered, which can enhance scene adaptability and be compatible with the triggering requirements of voice / vision single modality or fusion modality; through feature-level precise judgment, based on the matching of ambient sound feature parameters (such as acoustic features) and perceived object feature parameters (such as shape features), and non-raw data comparison, it can improve recognition robustness and reduce false triggering caused by environmental noise interference.

[0062] In an optional embodiment, the method further includes: When the control data domain is running in the low-power domain, the self-refresh mechanism of the DDR memory particles is activated; the self-refresh mechanism is used to maintain data integrity through periodic charge recharge; The control data domain exits the self-refresh mechanism and provides data reading and writing services when the high-performance processing domain is running.

[0063] It can be seen that the implementation of this optional embodiment can intelligently switch through the self-refresh mechanism, activate self-refresh (periodic charge restorage) when operating in the low-power domain, and exit self-refresh when operating in the high-performance domain. It can balance the contradiction between data preservation and power consumption, and only maintain data preservation with extremely low power consumption during the sleep period.

[0064] In another optional embodiment, the method further includes: Control the calculation attribute parameters and real-time attribute parameters of the parsing parameters of the high-performance processing domain parsing application; Control the high-performance processing domain to decompose the business processing operations that match the application parsing parameters into computing-intensive business processing task operations and real-time task business processing task operations based on computing attribute parameters and real-time attribute parameters; Controlling the high-performance processing domain to allocate computationally intensive business processing task operations to the second processing unit; The high-performance processing domain is controlled to distribute the real-time task business processing task operation to the first processing unit.

[0065] It can be seen that the implementation of this optional embodiment can split tasks according to computing attributes (computing intensity) and real-time attributes (latency requirements) through a task attribute-oriented allocation mechanism, and allocate them to matching processing units, so that the large core processes computing-intensive tasks → ensuring the efficiency of complex model reasoning; the small core processes real-time tasks → ensuring low-latency response (such as voice wake-up); through parallel scheduling of heterogeneous computing power, large and small cores can process different types of tasks at the same time, which can support multi-model parallel reasoning and solve the problem of competition for single NPU core resources.

[0066] In yet another optional embodiment, the system further includes a power recovery unit, and the method further includes: Controlling the power recovery unit to trigger execution of a preset power recovery operation in response to a third trigger instruction; And, the preset power recovery operations include: Cut off the active power supply to the high-performance processing domain; Cut off the active power supply of the selected powered domain; Maintain continuous power supply to the powered domain to continuously sense environmental parameters within the preset range.

[0067] It can be seen that the implementation of this optional embodiment can shut down in descending order of power consumption level through a hierarchical power recovery sequence: high-performance domain → select the powered domain → only maintain the continuously powered domain to avoid the risk of voltage mutation; optimize the power recovery efficiency and return to the lowest power consumption state as quickly as possible; through continuous monitoring of the normal power domain, after shutting down the high-power module, the continuously powered domain is still running, maintaining basic environmental perception capabilities, and ensuring that there are no monitoring blind spots.

[0068] Example 3 See also Figure 5 , Figure 5 This is a structural diagram of a low-power heterogeneous NPU system control device disclosed in an embodiment of the present invention. The low-power heterogeneous NPU system control device can be applied to NPU application devices, and can also be applied to smart devices associated with NPU application devices, including but not limited to battery devices, cloud devices, edge computing devices, relay devices, base station devices, urban management devices, smart network devices, smart home devices, AI industrial devices, and one or more of these devices, which are not limited in the embodiment of the present invention. The device is applied to a low-power heterogeneous NPU system, which includes at least a low-power domain and a high-performance domain, such as Figure 5 As shown, the device includes: The control module 501 is used to control the low power domain 10 to sense environmental parameters within a preset range; The control module 501 is configured to control the low-power domain 10 to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, control the low-power domain 10 to generate a first trigger instruction to trigger the low-power domain 10 to perform an operation of sensing the environmental parameters within a preset range; when it is determined that the environmental parameters match the preset environmental parameters, control the low-power domain 10 to generate a second trigger instruction; The control module 501 is configured to control the high-performance domain 20 to perform a preset first parsing operation on the environment parameter according to the second trigger instruction, thereby obtaining an application parsing parameter of the environment parameter, so that the high-performance domain 20 performs a business processing operation that matches the application parsing parameter; The control module 501 is used to control the high-performance domain 20 to determine whether there is a new second trigger instruction within the preset time period. When it is determined that there is no new second trigger instruction within the preset time period, the high-performance domain 20 is controlled to generate a third trigger instruction to trigger the low-power domain 10 to perform an operation of perceiving environmental parameters within a preset range.

[0069] In an embodiment of the present invention, optionally, the device can be used to execute the low-power heterogeneous NPU system control method described in Example 2. For the supplementary content of the embodiment of the present invention, reference can be made to the supplementary content in Example 2, and the embodiment of the present invention will not be repeated here.

[0070] It can be seen that the embodiment of the present invention divides the system into at least a low-power domain and a high-performance domain through a heterogeneous design of the NPU system, and creatively proposes a dynamic perception and matching mechanism of environmental parameters. The low-power domain perceives the environmental parameters within a preset range. When it is judged that the environmental parameters match the preset environmental parameters, the high-performance domain is triggered to perform a preset first parsing and processing operation on the environmental parameters according to the second trigger instruction, and obtains the application parsing parameters of the environmental parameters, so that the high-performance domain executes the business processing operation that matches the application parsing parameters. Through continuous perception and intelligent matching judgment, the high-performance domain is activated on demand, avoiding the invalid power consumption caused by the continuous operation of the high-performance module in the traditional solution, which can significantly reduce the average power consumption of the system and extend the battery life of the device. At the same time, Through trigger instruction closed-loop control, that is, the first trigger instruction maintains the low-power monitoring state, the second trigger instruction accurately wakes up the high-performance domain, and the third trigger instruction realizes automatic sleep after timeout, constructs an adaptive power consumption control loop, eliminates the need for manual intervention, and improves the system energy efficiency ratio, that is, it can meet the continuous AI computing needs of low-power scenarios, and can shut down the NPU large core in the high-performance domain in low-power scenarios and run the NPU small core in the low-power domain. Compared with the single-core NPU architecture design, the present invention can also meet the needs of simultaneous reasoning of multiple AI models. When the NPU large core is occupied by a model with large computational workload, the NPU small core can independently infer a model with small computational workload and high real-time performance, thereby improving the richness of NPU application scenarios, the flexibility and controllability of NPU applications.

[0071] Example 4 See also Figure 6 , Figure 6 This is a structural diagram of another low-power heterogeneous NPU system control device disclosed in an embodiment of the present invention. The low-power heterogeneous NPU system control device can be applied to NPU application devices, and can also be applied to smart devices associated with NPU application devices, including but not limited to battery devices, cloud devices, edge computing devices, relay devices, base station devices, city management devices, smart network devices, smart home devices, AI industrial devices, or one or more of these, which are not limited in the embodiment of the present invention. Figure 6 As shown, the low-power heterogeneous NPU system control device may include: The memory 601 stores executable program codes.

[0072] A processor 602 is coupled to the memory 601 .

[0073] The processor 602 calls the executable program code stored in the memory 601 to execute the steps of the low-power heterogeneous NPU system control method described in the second embodiment of the present invention.

[0074] Example 5 An embodiment of the present invention discloses a computer storage medium storing computer instructions. When the computer instructions are called, they are used to execute the steps of the low-power heterogeneous NPU system control method described in the second embodiment of the present invention.

[0075] Example 6 An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps in the low-power heterogeneous NPU system control method described in Example 2.

[0076] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0077] Through the detailed description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0078] Finally, it should be noted that the low-power heterogeneous NPU system and the corresponding control method and device disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A low-power heterogeneous NPU system, characterized in that: The system includes at least a low power consumption domain and a high performance domain, wherein: The low power consumption domain is used to sense environmental parameters within a preset range; The low-power domain is further configured to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, a first trigger instruction is generated to trigger the low-power domain to perform the operation of sensing the environmental parameters within the preset range; when it is determined that the environmental parameters match the preset environmental parameters, a second trigger instruction is generated; The high-performance domain is configured to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation matching the application parsing parameter; The high-performance domain is also used to determine whether there is a newly added second trigger instruction within the preset time period. When it is determined that there is no newly added second trigger instruction within the preset time period, a third trigger instruction is generated to trigger the low-power domain to perform the operation of perceiving the environmental parameters within the preset range.

2. The low-power heterogeneous NPU system according to claim 1, characterized in that: The low power consumption domain includes a continuously powered domain and a selectively powered domain, the continuously powered domain includes at least a sensing unit, and the selectively powered domain includes at least a first processing unit, wherein: The continuously powered domain is used to sense the environmental parameters within the preset range, where the environmental parameters include at least ambient sound parameters and perceived object parameters; The continuously powered domain is further configured to determine whether the ambient sound parameter is within a preset ambient sound parameter range and / or whether the perceived object parameter is within a preset perceived object parameter range, and generate a fourth trigger instruction when it is determined that the ambient sound parameter is within the preset ambient sound parameter range and / or the perceived object parameter is within the preset perceived object parameter range; The selected powered domain is configured to perform a preset second parsing operation on the environmental parameter according to the fourth trigger instruction to obtain environmental characteristic parameters of the environmental parameter, wherein the environmental characteristic parameters include environmental sound characteristic parameters and / or perceived object characteristic parameters; The selection of the powered domain is also used to determine whether the environmental characteristic parameters match the preset environmental characteristic parameters. When it is determined that the environmental characteristic parameters do not match the preset environmental characteristic parameters, the first trigger instruction is generated; when it is determined that the environmental characteristic parameters match the preset environmental characteristic parameters, the second trigger instruction is generated according to the environmental characteristic parameters.

3. The low-power heterogeneous NPU system according to claim 2, characterized in that: The high-performance domain includes a high-performance processing domain and a data domain. The high-performance processing domain includes at least a second processing unit. The data operation and processing capability of the second processing unit is higher than that of the first processing unit. The data domain includes at least DDR memory chips, wherein: The high-performance processing domain is configured to perform a preset first parsing operation on the environment parameter according to the second trigger instruction, obtain an application parsing parameter of the environment parameter, and perform a business processing operation matching the application parsing parameter; The high-performance processing domain is further configured to determine whether a newly added second trigger instruction exists within a preset time period, and when it is determined that no newly added second trigger instruction exists within the preset time period, generate a third trigger instruction to trigger the low-power consumption domain to perform the operation of sensing the environmental parameters within the preset range; The data field is used to store the environmental parameters.

4. The low-power heterogeneous NPU system according to claim 2, characterized in that: The environmental sound characteristic parameters include keyword characteristic parameters, the perceived object characteristic parameters include perceived object type characteristic parameters, and the selecting of the powered domain to determine whether the environmental characteristic parameters match preset environmental characteristic parameters includes: According to the environmental sound characteristic parameters, determining whether the keyword characteristic parameters match the preset keyword characteristic parameters; According to the perceived object characteristic parameters, determining whether the perceived object type characteristic parameters match preset perceived object type characteristic parameters; When it is determined that the keyword characteristic parameter matches the preset keyword characteristic parameter and / or the perceptual object type characteristic parameter matches the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter matches the preset environmental characteristic parameter; When it is determined that the keyword characteristic parameter does not match the preset keyword characteristic parameter and the perceptual object type characteristic parameter does not match the preset perceptual object type characteristic parameter, it is determined that the environmental characteristic parameter does not match the preset environmental characteristic parameter.

5. The low-power heterogeneous NPU system according to claim 3, characterized in that: The data domain is further used to activate the self-refresh mechanism of the DDR memory particle when the low power domain is running; the self-refresh mechanism is used to represent an operation of maintaining data integrity through periodic charge restoration; The data domain is further configured to exit the self-refresh mechanism and provide data read and write services when the high-performance processing domain is running.

6. The low-power heterogeneous NPU system according to claim 3, characterized in that: The high-performance processing domain is further used to parse the calculation attribute parameters and real-time attribute parameters of the application parsing parameters; The high-performance processing domain is further configured to decompose the business processing operation matching the application parsing parameter into a computing-intensive business processing task operation and a real-time business processing task operation according to the computing attribute parameter and the real-time attribute parameter; The high-performance processing domain is further used to allocate computationally intensive business processing task operations to the second processing unit; The high-performance processing domain is further used to allocate real-time task business processing task operations to the first processing unit.

7. The low-power heterogeneous NPU system according to claim 3, characterized in that: The system further comprises a power recovery unit, wherein: The power recovery unit is configured to trigger execution of a preset power recovery operation in response to the third trigger instruction; Furthermore, the preset power recovery operation includes: Cutting off active power supply to the high-performance processing domain; Cutting off active power supply to the selected powered domain; The continuously powered domain is maintained to be powered, so as to continuously sense the environmental parameter within the preset range.

8. A low-power heterogeneous NPU system control method, characterized in that: The method is applied to a low-power heterogeneous NPU system, which includes at least a low-power domain and a high-performance domain. The method includes: Controlling the low power domain to sense environmental parameters within a preset range; controlling the low-power domain to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, controlling the low-power domain to generate a first trigger instruction to trigger the low-power domain to perform the operation of sensing the environmental parameters within the preset range; and when it is determined that the environmental parameters match the preset environmental parameters, controlling the low-power domain to generate a second trigger instruction; controlling the high-performance domain to perform a preset first parsing operation on the environment parameter according to the second trigger instruction to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation matching the application parsing parameter; The high-performance domain is controlled to determine whether there is a newly added second trigger instruction within the preset time period. When it is determined that there is no newly added second trigger instruction within the preset time period, the high-performance domain is controlled to generate a third trigger instruction to trigger the low-power consumption domain to perform the operation of perceiving the environmental parameters within the preset range.

9. A low-power heterogeneous NPU system control device, characterized in that: The device is applied to a low-power heterogeneous NPU system, the system including at least a low-power domain and a high-performance domain, and the device includes: A control module, configured to control the low power domain to sense environmental parameters within a preset range; a control module, configured to control the low-power domain to determine whether the environmental parameters match the preset environmental parameters; when it is determined that the environmental parameters do not match the preset environmental parameters, control the low-power domain to generate a first trigger instruction to trigger the low-power domain to perform the operation of sensing the environmental parameters within the preset range; and when it is determined that the environmental parameters match the preset environmental parameters, control the low-power domain to generate a second trigger instruction; a control module, configured to control the high-performance domain to perform a preset first parsing operation on the environment parameter according to the second trigger instruction, to obtain an application parsing parameter of the environment parameter, so that the high-performance domain performs a business processing operation matching the application parsing parameter; A control module is used to control the high-performance domain to determine whether there is a newly added second trigger instruction within a preset time period. When it is determined that there is no newly added second trigger instruction within the preset time period, the high-performance domain is controlled to generate a third trigger instruction to trigger the low-power consumption domain to perform the operation of perceiving the environmental parameters within the preset range.

10. A low-power heterogeneous NPU system control device, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the low-power heterogeneous NPU system control method according to claim 8.

Images