Frequency modulation method and electronic equipment

Abstract

The present application provides a frequency modulation method and electronic device. The method includes: when it is necessary to run a first task model on a first module, collecting first state information of the electronic device; inputting the first state information and the first task model into a trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a first frequency combination, the first frequency combination including a first operating frequency and a second operating frequency; adjusting the operating frequency of the ADSP module to the first operating frequency, adjusting the operating frequency of the first module to the second operating frequency, and running the first task model on the first module. In this way, the optimal operating frequency combination of the ADSP module and the first module can be determined according to the task model to be run and the real-time state, thereby improving the performance-to-power ratio.

Description

Technical Field

[0001] The present application relates to the field of terminal equipment, and in particular to a frequency modulation method and electronic equipment. Background Art

[0002] At present, electronic devices generally include ADSP (Audio Digital Signal Processor) modules. ENPU (Embedded Neural Processing Unit) is a low-power module mounted on ADSP. In some functions, some task models run on ENPU. ADSP and ENPU each have multiple operating frequencies, and different operating frequency combinations will have different performance and power consumption benefits. In order to meet the real-time and low-power characteristics of the task model algorithm, it is necessary to configure the operating frequency of ADSP and ENPU.

[0003] In the related art, the performance and power consumption benefits of electronic devices are relatively low. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a frequency modulation method and an electronic device, which can improve the energy consumption ratio while ensuring the performance of the model running in the ENPU of the ADSP module on the electronic device.

[0005] In the first aspect, the present application provides a frequency modulation method, which includes: when it is necessary to run a first task model on a first module, collecting first state information of an electronic device; inputting the first state information and the first task model into a trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a first frequency combination, the first frequency combination including a first operating frequency and a second operating frequency; adjusting the operating frequency of the ADSP module to the first operating frequency, adjusting the operating frequency of the first module to the second operating frequency, and running the first task model on the first module. In this way, the optimal operating frequency combination of the ADSP module and the first module can be determined according to the task model to be run and the real-time state, thereby improving the performance-to-power ratio.

[0006] According to the first aspect, the operating frequency of the ADSP module is adjusted to the first operating frequency, the operating frequency of the first module is adjusted to the second operating frequency, and after the first task model is run on the first module, it also includes: collecting a first power consumption value of the electronic device; obtaining first performance data of the first task model; determining a first energy consumption ratio based on the first power consumption value and the first performance data; if the first energy consumption ratio is less than a preset energy consumption ratio threshold, collecting second state information of the electronic device; inputting the second state information and the first task model into a trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a second frequency combination, the second frequency combination including a third operating frequency and a fourth operating frequency; adjusting the operating frequency of the ADSP module to the third operating frequency, adjusting the operating frequency of the first module to the fourth operating frequency, and running the first task model on the first module.

[0007] According to the first aspect, the operating frequency of the ADSP module is adjusted to a third operating frequency, and the operating frequency of the first module is adjusted to a fourth operating frequency. After running the first task model on the first module, it also includes: collecting a second power consumption value of the electronic device; obtaining second performance data of the first task model; determining a second energy consumption ratio based on the second power consumption value and the second performance data; if the second energy consumption ratio is greater than or equal to an energy consumption ratio threshold, maintaining the operating frequency of the ADSP module at the third operating frequency, and maintaining the operating frequency of the first module at the fourth operating frequency.

[0008] According to the first aspect, the operating frequency of the ADSP module is adjusted to the first operating frequency, and the operating frequency of the first module is adjusted to the second operating frequency. After running the first task model on the first module, it also includes: collecting a first power consumption value of the electronic device; obtaining first performance data of the first task model; determining a first energy consumption ratio based on the first power consumption value and the first performance data; if the first energy consumption ratio is greater than or equal to a preset energy consumption ratio threshold, maintaining the operating frequency of the ADSP module at the first operating frequency, and maintaining the operating frequency of the first module at the second operating frequency.

[0009] According to the first aspect, the dynamic frequency modulation model is a deep reinforcement learning network model.

[0010] According to the first aspect, the dynamic frequency modulation model is a multi-layer perceptron MLP network structure.

[0011] According to the first aspect, the method for obtaining the dynamic frequency modulation model includes: miniaturizing the task model that needs to be deployed on the first module to obtain a miniaturized task model; obtaining sample data required for training based on the miniaturized task model; constructing a deep reinforcement learning network model, and setting initial parameter values ​​for the deep reinforcement learning network model; using the sample data to train the deep reinforcement learning network model to obtain a trained deep reinforcement learning network model, and using the trained deep reinforcement learning network model as the dynamic frequency modulation model.

[0012] According to the first aspect, based on the miniaturized task model, sample data required for training is obtained, including: collecting first actual state information, first actual action information and first actual reward information corresponding to the miniaturized task model as first sample data; and / or, using a simulation model, simulating the simulated state information, simulated action information and simulated reward information of the miniaturized task model under different degrees of aging and / or falling damage as second sample data; and / or, collecting second actual state information, second actual action information and second actual reward information under different degrees of aging and / or falling damage as third sample data; the sample data required for training includes any one or more of the first sample data, the second sample data and the third sample data.

[0013] According to the first aspect, the status information includes: performance of the first task model, temperature of the electronic device, aging degree of the electronic device, bandwidth, load of the electronic device, frame rate of the electronic device, and Mac utilization.

[0014] According to the first aspect, the miniaturization operation includes any one or more of the following processes: quantization, structured pruning, unstructured pruning, efficient backbone selection, and equivalent replacement of time-consuming operators.

[0015] According to the first aspect, the first module is an embedded neural processing unit ENPU module.

[0016] According to the first aspect, the first task model is a classification detection model.

[0017] In a second aspect, the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes any one of the frequency modulation methods of the first aspect.

[0018] In a third aspect, the present application provides a computer-readable storage medium, comprising a computer program, which, when executed on an electronic device, enables the electronic device to execute any one of the frequency modulation methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic structural diagram of an illustrative electronic device 100;

[0020] Figure 2 1 is a software structure block diagram of an electronic device 100 according to an embodiment of the present application;

[0021] Figure 3 An exemplary flow chart of a frequency modulation method in an embodiment of the present application is shown as an example;

[0022] Figure 4 Another example flow chart of the frequency modulation method in the embodiment of the present application is shown as an example;

[0023] Figure 5 A schematic diagram of a deep reinforcement learning process is shown as an example;

[0024] Figure 6 The figure is an example diagram of the overall process of obtaining and applying a dynamic frequency modulation model. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0027] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

[0028] This embodiment provides a frequency modulation method, which can be applied to electronic devices such as mobile phones and tablets. Of course, it is not limited to these electronic devices.

[0029] In this embodiment, the structure of the electronic device can be as follows Figure 1 shown.

[0030] Figure 1 FIG. 1 is a schematic diagram of the structure of an electronic device 100 exemplarily shown. It should be understood that Figure 1 The illustrated electronic device 100 is merely one example of an electronic device, and the electronic device 100 may have more or fewer components than those shown in the figure, may combine two or more components, or may have a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0031] See also Figure 1The electronic device 100 may include: a processor 110, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, an indicator 192, a camera 193, etc.

[0032] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0033] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0034] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0035] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0036] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0037] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0038] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0039] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0040] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0041] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0042] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0043] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0044] Of course, in addition to the above components, the electronic device 100 may also include other hardware components, which are not listed here one by one.

[0045] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0046] Figure 2 1 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0047] The layered architecture of the electronic device 100 divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system may include an application layer, an application framework layer, a system library, and a kernel layer.

[0048] The application layer can include a series of application packages.

[0049] like Figure 2 As shown, the application package may include camera, gallery, map, WLAN, music, short message, call, navigation, Bluetooth, video and other applications. Of course, these applications are only exemplary. In other embodiments, the application layer may include Figure 2 The applications not shown in the figure may not include Figure 2 One or more applications shown in .

[0050] like Figure 2 As shown, the application framework layer may include a window manager, a resource manager, a content provider, a view system, a frequency modulation module, and the like.

[0051] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0052] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0053] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0054] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0055] The frequency modulation module is used to execute the frequency modulation method of the embodiment of the present application.

[0056] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0057] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0058] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0059] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0060] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0061] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0062] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0063] A 2D graphics engine is a drawing engine for 2D (two-dimensional) drawing.

[0064] The kernel layer is the layer between hardware and software.

[0065] like Figure 2 As shown, the kernel layer may include display driver, Wi-Fi driver, audio driver, sensor driver, Bluetooth driver, etc.

[0066] Understandably, Figure 2 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer layers than shown, and each layer may include more or fewer components, which is not limited in the present application.

[0067] The frequency modulation method of the embodiment of the present application can be applied to the scenario of switching the model running on the ENPU. When switching the model, the frequency modulation method of the embodiment of the present application can be used to adjust the operating frequencies of the ENPU and ADSP.

[0068] For example, the frequency modulation method of the embodiment of the present application can be applied to the use scenario of air gestures. In the air gesture feature, there are multiple stages, one of which is the classification model, the second stage is the detection model and the key point model. The performance, frame rate and power consumption required by these three models are different. When switching the model running on the ENPU, the operating frequency of the ENPU and ADSP needs to be adjusted.

[0069] The frequency modulation method proposed in the embodiment of the present application is a dynamic frequency modulation method based on reinforcement learning. The method analyzes parameters such as the state during task switching to find the optimal frequency combination for different tasks to obtain the optimal power consumption ratio, that is, the performance per walt is maximized, and the maximum performance benefit is achieved at the minimum power consumption cost, thereby maximizing the utilization of hardware resources.

[0070] The frequency modulation method proposed in the embodiment of the present application is applied to the air gesture solution, which can minimize power consumption while ensuring the model frame rate.

[0071] When configuring the operating frequencies of ADSP and ENPU, although the performance will increase with the increase of frequency, a higher frequency combination does not necessarily lead to a higher power consumption increment; and as the operating frequency increases, the marginal effect of performance growth becomes more and more obvious.

[0072] In addition, the power consumption increment is essentially a noisy black box function, which is related to hardware characteristics, frequency, voltage, task status, model performance, current temperature, hardware aging, etc. of large and small cores (referring to large and small cores in enpu, where the computing power of large cores in enpu is better than that of small cores in enpu). And as the user's usage and hardware aging are different, the mapping relationship between various variables and power consumption increment will also change, and the direction of change is also inconsistent.

[0073] In response to the above problems, the embodiments of the present application are intended to obtain the optimal frequency combination for different tasks deployed on the EPU through deep reinforcement learning under the limited computing resources and network resources of the SOC (here refers to ADSP), so as to achieve the best energy consumption ratio, ensure that energy consumption and reliability are at an acceptable stable value, so that when different tasks are switched, the operating frequency combination of the EPU and ADSP is also dynamically switched. While improving the user experience, it improves resource utilization and achieves the goal of high performance and low power consumption, thereby meeting the efficient collaborative reasoning of different tasks on the EPU (model reasoning refers to completing a task, such as outputting a detection box, key point position, etc.). Compared to experimenting with different tasks to obtain the best performance-to-power ratio (hereinafter referred to as the energy consumption ratio), the embodiments of the present application automatically adjust the frequency through code, which has obvious advantages in accuracy and efficiency.

[0074] Figure 3 FIG. 1 is a flowchart of a frequency modulation method in an embodiment of the present application. Figure 3 In this embodiment, the frequency modulation process may include the following steps:

[0075] S301, when it is necessary to run a first task model on a first module, first state information of an electronic device is collected.

[0076] S302, inputting the first state information and the first task model into a trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a first frequency combination, where the first frequency combination includes a first operating frequency and a second operating frequency.

[0077] S303, adjusting the operating frequency of the ADSP module to the first operating frequency, adjusting the operating frequency of the first module to the second operating frequency, and running the first task model on the first module.

[0078] It can be seen that in the embodiment of the present application, the frequency modulation method is based on a dynamic frequency modulation model, and the training process of the dynamic frequency modulation model will be described in the following content.

[0079] Among them, the first module is a module mounted in the ADSP module, for example, the first module can be the aforementioned ENPU module.

[0080] Among them, the first task model is a model that needs to run on the first module. For example, the first task model can be a classification model, detection model, key point model, etc. in the air gesture scene.

[0081] The first status information is the current status information of the ADSP module collected in real time.

[0082] The input information of the dynamic frequency modulation model includes the first state information and the first task model, and the output information of the dynamic frequency modulation model is the first frequency combination corresponding to the first task model. The frequency combination includes two operating frequencies, one of which is the operating frequency of the ADSP module, and the other is the operating frequency of the first module.

[0083] The dynamic frequency modulation model determines the first frequency combination corresponding to the first task model in real time according to the current state information of the ADSP module and the first task model to be run on the first module. Then, the ADSP module and the first module are adjusted to the corresponding working frequency according to the first frequency combination. After the working frequency is adjusted, the first task model is run on the first module.

[0084] Figure 4 FIG. 1 is another example flow chart of the frequency modulation method in the embodiment of the present application. Figure 4 In this embodiment, the frequency modulation process may include the following steps:

[0085] S401, when it is necessary to run a first task model on a first module, first state information of an electronic device is collected.

[0086] S402, inputting the first state information and the first task model into a trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a first frequency combination, where the first frequency combination includes a first operating frequency and a second operating frequency.

[0087] S403, adjusting the operating frequency of the ADSP module to the first operating frequency, adjusting the operating frequency of the first module to the second operating frequency, and running the first task model on the first module.

[0088] S404: Collect a first power consumption value of the electronic device.

[0089] The first power consumption value is the power consumption of the electronic device when the first task model is run on the first module, when the ADSP module operates at the first operating frequency and the first module operates at the second operating frequency.

[0090] The first power consumption value is a power consumption value collected in real time.

[0091] S405: Acquire first performance data of the first task model.

[0092] The first performance data of the first task model may be read from a log of the electronic device.

[0093] S406: Determine a first energy consumption ratio according to the first power consumption value and the first performance data.

[0094] In one example, the energy consumption ratio may be equal to a ratio of the performance data to the power consumption value.

[0095] The first energy consumption ratio is an energy consumption ratio of the electronic device obtained when the ADSP module operates at a first operating frequency, the first module operates at a second operating frequency, and the first task model is run on the first module.

[0096] S407, determine whether the first energy consumption ratio is less than a preset energy consumption ratio threshold. If the first energy consumption ratio is less than the preset energy consumption ratio threshold, execute step S408; otherwise, if the first energy consumption ratio is greater than or equal to the preset energy consumption ratio threshold, execute step S411.

[0097] S408: Collect second status information of the electronic device.

[0098] S409, inputting the second state information and the first task model into a trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a second frequency combination, where the second frequency combination includes a third operating frequency and a fourth operating frequency.

[0099] S410, adjusting the operating frequency of the ADSP module to a third operating frequency, adjusting the operating frequency of the first module to a fourth operating frequency, and running the first task model on the first module.

[0100] It should be noted that after step S410, the energy consumption ratio of the electronic device obtained when the ADSP module operates at the third operating frequency, the first module operates at the fourth operating frequency, and the first task model is run on the first module must also be calculated (for the calculation steps, please refer to the aforementioned steps S404 to S406), and then the energy consumption ratio is compared with the energy consumption ratio threshold as in S407. If the energy consumption ratio is less than the energy consumption ratio threshold, then continue to redetermine the operating frequency combination of the ADSP module and the first module as in steps S408 to S410, and recalculate the energy consumption ratio; if the energy consumption ratio is greater than or equal to the energy consumption ratio threshold, the operating frequencies of the ADSP module and the first module are maintained at the currently determined operating frequency combination, and the first task model is run on the first module until the first task model is completed.

[0101] S411, maintaining the operating frequency of the ADSP module as the first operating frequency, and maintaining the operating frequency of the first module as the second operating frequency.

[0102] It can be seen that the frequency modulation method of this embodiment can determine the optimal operating frequency combination of the ADSP module and the first module according to the task model to be run and the real-time status, thereby improving the performance-to-power ratio.

[0103] The following is an exemplary description of the training process of the dynamic frequency modulation model.

[0104] The dynamic frequency modulation model in this embodiment is a model based on reinforcement learning.

[0105] Reinforcement learning is a special type of machine learning algorithm that draws on behaviorist psychology. Different from the goals of supervised learning and unsupervised learning, the problem that reinforcement learning algorithms solve is how the intelligent agent (i.e., the entity running the reinforcement learning algorithm) performs actions in the environment to obtain the maximum cumulative reward.

[0106] For example, for an autonomous car, the reinforcement learning algorithm controls the car's movements to ensure safe driving to the destination. For a Go algorithm, the reinforcement learning algorithm must decide how to move based on the current game to win the game. For the first question, the environment is an abstract system composed of parameters such as the vehicle's current driving state (such as speed) and road conditions, and the reward is the expected result, that is, the car drives correctly on the road and reaches the destination without an accident.

[0107] Many control and decision-making problems can be abstracted into this model. Similar to supervised learning, reinforcement learning also has a training process, which requires continuous execution of actions, observation of the effects after execution of actions, and accumulation of experience to form a model. Unlike supervised learning, each action here generally does not have a directly calibrated label value as a supervisory signal. The system only gives feedback to the action executed by the algorithm. This feedback is generally delayed, and the consequences of the current action will not be fully reflected until the future. In addition, the future is also random.

[0108] Deep reinforcement learning (DRL) is a combination of deep learning and reinforcement learning. It integrates the powerful understanding ability of deep learning in visual and other perceptual problems, and the decision-making ability of reinforcement learning, and realizes end-to-end learning. Deep reinforcement learning uses deep learning technology to solve decision-making problems in reinforcement learning and train artificial neural networks to represent strategies π(a|s).

[0109] Figure 5 This is a flowchart of deep reinforcement learning. Figure 5 , using deep learning technology to solve the decision-making problem in reinforcement learning, the process of training artificial neural networks can be represented by the Markov decision process. At each time point, the agent is in a state s of the environment. After the agent takes an action a, it will receive a reward r and transfer to the next state s' according to the state transition function p(s'|s,a) of the environment. The agent's goal is to learn a set of strategies π(a|s) (that is, a set of corresponding relationships from the current state to the action to be taken) to maximize the total reward.

[0110] The state s′ is obtained as shown in the following formula (1):

[0111] s t+1=P(s t ,a t ,θ P ) (1)

[0112] Among them, s t represents the state at time t, s t+1 represents the state at time t+1, a t represents the action at time t, θ P In it, θ represents the strategy and P represents the probability.

[0113] The method of obtaining action a is shown in the following formula (2):

[0114] a t =π(s t ,θ π ) (2)

[0115] Among them, a t represents the action at time t, s t represents the state at time t, θ π Represents the parameters of the policy function.

[0116] The reward r is obtained as shown in the following formula (3):

[0117] r t =R(s t ,a t ,θ R ) (3)

[0118] Among them, s t represents the state at time t, a t represents the action at time t, θ R Represents the parameters of the policy function.

[0119] In this embodiment, the method for obtaining the dynamic frequency modulation model may include the following steps:

[0120] Performing a miniaturization operation on the task model to be deployed on the first module to obtain a miniaturized task model;

[0121] According to the miniaturized task model, obtain the sample data required for training;

[0122] Build a deep reinforcement learning network model and set initial parameter values ​​for the deep reinforcement learning network model;

[0123] The deep reinforcement learning network model is trained using sample data to obtain a trained deep reinforcement learning network model, and the trained deep reinforcement learning network model is used as a dynamic frequency modulation model.

[0124] In one example, obtaining sample data required for training according to the miniaturized task model may include:

[0125] Collecting first actual state information, first actual action information and first actual reward information corresponding to the miniaturized task model as first sample data; and / or,

[0126] Using the simulation model, simulated state information, simulated action information and simulated reward information of the miniaturized task model under different aging degrees and / or falling damage conditions are simulated as second sample data; and / or,

[0127] Collecting second actual state information, second actual action information, and second actual reward information under different actual aging degrees and / or falling damage conditions as third sample data;

[0128] The sample data required for training includes any one or more of the first sample data, the second sample data and the third sample data.

[0129] The simulation model used to obtain the second sample data is a trained simulation model. Using the simulation model to obtain sample data can obtain more data with higher efficiency and speed up the acquisition time of sample data.

[0130] After the initial simulation model is created, the training process of the simulation model may include the following steps:

[0131] Step 1, using this simulation model, simulate the simulated state information, simulated action information and simulated reward information of the miniaturized task model under different aging degrees and / or falling damage conditions; the simulation model used in the first training is the initial simulation model, and the simulation model used in the second and subsequent trainings is the simulation model corrected in the previous training;

[0132] Step 2, collecting second actual state information, second actual action information, and second actual reward information under different actual aging degrees and / or falling damage conditions;

[0133] Step 3, according to the second actual state information and the simulated state information, the second actual action information and the simulated action information, and the second actual reward information and the simulated reward information, the initial simulation model is corrected to obtain a corrected simulation model;

[0134] Determine whether the revised simulation model meets the requirements. If so, use the revised simulation model as the trained simulation model. If not, repeat steps 1, 2, and 3.

[0135] Through correction, the simulation model can be made more accurate, so that higher quality sample data can be obtained using the simulation model.

[0136] The status information may include:

[0137] The performance of the first task model, the temperature of the electronic device, the aging degree of the electronic device, the bandwidth, the load of the electronic device, the frame rate of the electronic device, and the Mac (Multiply-Accumulate) utilization.

[0138] Take the first module as enpu module as an example. Figure 6 This is an example diagram of the overall process of obtaining and applying a dynamic frequency modulation model. Figure 6 In this embodiment, the overall process of obtaining and applying the dynamic frequency modulation model may include the following steps:

[0139] S601, determine the model structure that needs to be deployed on enpu, take some model miniaturization operations, such as quantization, structured pruning, unstructured pruning, efficient backbone selection, time-consuming operator equivalent replacement, etc., train the model to a state that can be deployed on enpu, so as to obtain a miniaturized task model.

[0140] The miniaturized task model is deployed on enpu. The first task model in the aforementioned step S301 is the miniaturized task model.

[0141] For example, the models that need to be deployed on enpu are: Model 1, Model 2, ... Model m, where m is a natural number. Then, each model needs to be miniaturized, and after miniaturization, Model 1, Model 2, ... Model m are miniaturized to obtain miniaturized models Model 1A, Model 2A, ... Model mA, respectively.

[0142] S602, collect sample data for deep reinforcement model training.

[0143] Each set of sample data collected may include state information, action information, and reward information.

[0144] Among them, the status information may include: model performance (model performance corresponding to the combination of enpu operating frequency and adsp operating frequency), temperature of the electronic device, aging degree of the electronic device, bandwidth, load of the electronic device, frame rate of the electronic device, and Mac utilization.

[0145] The action information may include a frequency combination (the working frequency of the enpu and the working frequency of the adsp) and a bandwidth of the electronic device.

[0146] The reward information may be a power consumption ratio.

[0147] In one example, sample data can be collected as follows:

[0148] Method 1:

[0149] Directly collect actual sample data, including actual state information, actual action information, and actual reward information.

[0150] For example, a miniaturized model model 1A is run on the enpu, and then actual state information 1, actual action information 1, and actual reward information 1 of the electronic device are collected to obtain a set of sample data 1;

[0151] Next, stop running the miniaturized model model 1A on the enpu, run the miniaturized model model 2A on the enpu, and then collect the actual state information 2, the actual action information 2 and the actual reward information 2 of the electronic device to obtain a set of sample data 2;

[0152] …

[0153] Method 2:

[0154] The trained simulation model is used to simulate and obtain simulated sample data.

[0155] Using the simulation model to simulate user 1 running the miniaturized model model 1A on enpu, the simulation sample data is obtained, the simulation sample data includes the simulation state information 1, the simulation action information 1 and the simulation reward information 1 of the electronic device, and a set of sample data 1 is obtained;

[0156] Using the simulation model to simulate user 2 running miniaturized model model 1A on enpu, simulated sample data is obtained, the simulated sample data includes simulated state information 2, simulated action information 2 and simulated reward information 2 of the electronic device, and a set of sample data 2 is obtained; using the simulation model to simulate user 1 running miniaturized model model 2A on enpu, simulated sample data is obtained, the simulated sample data includes simulated state information 3, simulated action information 3 and simulated reward information 3 of the electronic device, and a set of sample data 3 is obtained;

[0157] Using the simulation model to simulate the user 2 running the miniaturized model model 2A on the enpu, the simulation sample data is obtained, the simulation sample data includes the simulation state information 4, the simulation action information 4 and the simulation reward information 4 of the electronic device, and a set of sample data 4 is obtained;

[0158] …

[0159] For example, the simulation data may include model performance, voltage, power consumption, temperature, and other data corresponding to the enpu frequency and the adsp frequency.

[0160] Using the simulation model, you can simulate the usage habits of different users to draw the hardware aging curve, and quickly collect the model power consumption corresponding to the enpu and adsp frequencies at different aging levels. When the user is different, the simulation data can include not only the aforementioned status information, but also the aging level, model performance, load and other data.

[0161] Method 3:

[0162] Collect hardware data under different real user usage habits as sample data, organize the data, and build a dictionary (performance, temperature, aging degree, bandwidth, load, frame rate Mac utilization) as the state space for reinforcement learning.

[0163] The hardware sample data collected in method three can also be used to correct the simulation model in method two to obtain a more accurate simulation model.

[0164] The sample data collected in the last step S602 is a collection of the actual sample data collected in the first method and the modified sample data obtained in the second method.

[0165] S603, construct a deep reinforcement learning network, whose structure is a three-layer multi-layer perceptron (MLP) network structure, which is used to map state information to action space. The reward is the energy consumption ratio.

[0166] S604, start training, use the PPO (proximal policy optimization) algorithm, and implement small batch updates in multiple training steps.

[0167] The network structure of the Actor and Critic of the PPO algorithm is an MLP network, and the Actor and Critic share the weights of this MLP network. Among them, the Actor is used to map the state space to the action space, and the Critic is used to map the state space to the Value value. The action space is defined as a dictionary structure a = {adsp freq, enpu freq, model}, corresponding to adsp frequency, enpu frequency, model and other information.

[0168] Deep reinforcement learning methods are divided into value-based, policy-based, and the combination of the two, the ac algorithm. This embodiment uses the combination of the two ac algorithms. Value-based is a learning method for value functions, a method for solving the optimal value function to obtain the optimal policy. The optimal action output by the value function will eventually converge to a certain action, thereby learning a deterministic policy.

[0169] S605, after the model parameters are determined, the trained dynamic frequency modulation model is obtained, and the dynamic frequency modulation model is deployed on the terminal side (that is, deployed in the electronic device). At the same time, the terminal side status information is obtained, such as hardware status, current voltage, temperature and other status information. According to the scheduling situation, when entering different tasks, the frequency combination with the current best power consumption ratio is determined, and the frequency combination is added to the queue.

[0170] S606, determine whether the data in the queue has the best energy consumption ratio and meets the model performance requirements (such as frame rate requirements, etc.). If so, execute step S607, otherwise return to step S605, re-run the dynamic frequency modulation model, and add the data to the queue for re-judgment after obtaining the data.

[0171] Step S607: Under the premise that the model performance is satisfied, the frequency combination with the best energy consumption ratio is taken out as the working frequency combination of adsp and enpu.

[0172] The frequency modulation method proposed in this embodiment analyzes parameters such as the task switching state to find the best frequency combination for different tasks to obtain the best power consumption ratio, that is, the performance per walt is maximized, the maximum performance benefit is achieved with the minimum power consumption cost, and the hardware resources are maximized.

[0173] The frequency modulation method proposed in the embodiment of the present application can minimize power consumption while ensuring the model frame rate.

[0174] An embodiment of the present application also provides an electronic device, which includes a memory and a processor. The memory is coupled to the processor, and the memory stores program instructions. When the program instructions are executed by the processor, the electronic device implements the frequency modulation method executed by the aforementioned electronic device.

[0175] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0176] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the frequency modulation method in the above-mentioned embodiment.

[0177] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the frequency modulation method in the above-mentioned embodiment.

[0178] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the frequency modulation method in the above-mentioned method embodiments.

[0179] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0180] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0181] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0183] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0184] Any content of each embodiment of the present application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of the present application.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0186] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0187] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disk (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0188] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0189] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A frequency modulation method, characterized in that: Applied to an electronic device, the electronic device includes an ADSP module, the ADSP module is mounted with a first module, and the method includes: When it is necessary to run the first task model on the first module, first state information of the electronic device is collected; wherein the first state information is current state information of the ADSP module collected in real time; Inputting the first state information and the first task model into a trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a first frequency combination, where the first frequency combination includes a first operating frequency and a second operating frequency; Adjusting the operating frequency of the ADSP module to the first operating frequency, adjusting the operating frequency of the first module to the second operating frequency, and running the first task model on the first module; Collecting a first power consumption value of the electronic device; Acquire first performance data of the first task model; Determining a first energy consumption ratio according to the first power consumption value and the first performance data; If the first energy consumption ratio is less than a preset energy consumption ratio threshold, collecting second state information of the electronic device; Inputting the second state information and the first task model into the trained dynamic frequency modulation model, and the dynamic frequency modulation model outputs a second frequency combination, wherein the second frequency combination includes a third operating frequency and a fourth operating frequency; The operating frequency of the ADSP module is adjusted to the third operating frequency, the operating frequency of the first module is adjusted to the fourth operating frequency, and the first task model is run on the first module.

2. The method according to claim 1, characterized in that After adjusting the operating frequency of the ADSP module to the third operating frequency, adjusting the operating frequency of the first module to the fourth operating frequency, and running the first task model on the first module, the method further includes: collecting a second power consumption value of the electronic device; Acquire second performance data of the first task model; determining a second energy consumption ratio according to the second power consumption value and the second performance data; If the second energy consumption ratio is greater than or equal to the energy consumption ratio threshold, the operating frequency of the ADSP module is maintained at the third operating frequency, and the operating frequency of the first module is maintained at the fourth operating frequency.

3. The method according to claim 1, characterized in that After adjusting the operating frequency of the ADSP module to the first operating frequency, adjusting the operating frequency of the first module to the second operating frequency, and running the first task model on the first module, the method further includes: Collecting a first power consumption value of the electronic device; Acquire first performance data of the first task model; Determining a first energy consumption ratio according to the first power consumption value and the first performance data; If the first energy consumption ratio is greater than or equal to a preset energy consumption ratio threshold, the operating frequency of the ADSP module is maintained at the first operating frequency, and the operating frequency of the first module is maintained at the second operating frequency.

4. The method according to claim 1, characterized in that: The dynamic frequency modulation model is a deep reinforcement learning network model.

5. The method according to claim 4, characterized in that The dynamic frequency modulation model is a multi-layer perceptron MLP network structure.

6. The method according to claim 1, characterized in that The method for obtaining the dynamic frequency modulation model includes: Performing a miniaturization operation on the task model to be deployed on the first module to obtain a miniaturized task model; According to the miniaturized task model, obtaining sample data required for training; Constructing a deep reinforcement learning network model and setting initial parameter values ​​for the deep reinforcement learning network model; The deep reinforcement learning network model is trained using the sample data to obtain a trained deep reinforcement learning network model, and the trained deep reinforcement learning network model is used as a dynamic frequency modulation model.

7. The method according to claim 6, characterized in that According to the miniaturized task model, sample data required for training is obtained, including: Collecting first actual state information, first actual action information and first actual reward information corresponding to the miniaturized task model as first sample data; and / or, Using the simulation model, simulated state information, simulated action information and simulated reward information of the miniaturized task model under different aging degrees and / or falling damage conditions are simulated as second sample data; and / or, Collecting second actual state information, second actual action information, and second actual reward information under different aging degrees and / or falling damage conditions as third sample data; The sample data required for training includes any one or more of the first sample data, the second sample data and the third sample data.

8. The method according to claim 7, characterized in that Status information includes: The performance of the first task model, the temperature of the electronic device, the aging degree of the electronic device, the bandwidth, the load of the electronic device, the frame rate of the electronic device, and the Mac utilization rate.

9. The method according to claim 6, characterized in that The miniaturization operation includes any one or more of the following processes: Quantization, structured pruning, unstructured pruning, efficient backbone selection, and equivalent replacement of time-consuming operators.

10. The method according to claim 1, characterized in that The first module is an embedded neural processing unit ENPU module.

11. The method according to claim 1, characterized in that: The first task model is a classification detection model.

12. An electronic device, characterized in that: include: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the frequency modulation method according to any one of claims 1 to 11.

13. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device executes the frequency modulation method according to any one of claims 1 to 11.

Images