Frequency modulation method, electronic device, readable storage medium and program product

Abstract

The present application provides a frequency modulation method, an electronic device, a readable storage medium and a program product, and relates to the field of terminals. The method is applied to an electronic device, and the electronic device includes a digital signal processing module, and the method includes: obtaining first algorithm information, and the algorithm corresponding to the first algorithm information is used to process a first preview frame; setting the frequency of the algorithm corresponding to the first algorithm information to a first frequency, and the first frequency is determined based on the performance data and power consumption data of the digital signal processing module when the above algorithm is run at different frequencies; and running the algorithm corresponding to the first algorithm information according to the first frequency to process the first preview frame. The scheme provided in the present application can control the frequency decided by the decision unit in the digital signal processing module, so that the decided frequency can balance the performance and power consumption of the digital signal processing module running the algorithm to process the preview frame.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a frequency modulation method, an electronic device, a readable storage medium and a program product. Background Art

[0002] Digital signal processing (DSP) technology supports single instruction multiple data (SIMD) vector computing, which can realize multi-tasking and improve computing performance. When DSP technology is used for computing the preview stream, a device that implements DSP technology (hereinafter, DSP chip is used as an example) can run multiple algorithms for computing the preview stream. Each algorithm specifies the required frequency to the DSP chip during initialization. The decision unit in the DSP chip will decide a frequency based on the frequency specified by each algorithm, and the computing unit in the DSP chip will process the preview frames in the preview stream based on the decided frequency.

[0003] In the above scheme, the decision unit can be understood as a black box with a certain decision strategy. The frequency decided by the decision unit cannot be controlled by the user side, and the calculation unit will use the above-decided frequency when processing each preview frame in the preview stream subsequently. It is not flexible enough and cannot give full play to the computing performance of the DSP chip. Summary of the invention

[0004] The present application provides a frequency modulation method, electronic device, readable storage medium and program product. The solution provided by the present application can control the frequency determined by the decision unit in the DSP chip on the user side, so that the determined frequency can balance the performance and power consumption of the DSP chip running algorithm to process preview frames.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a frequency modulation method is provided, which is applied to an electronic device, wherein the electronic device includes a digital signal processing module, and the method includes: obtaining first algorithm information, wherein the algorithm corresponding to the first algorithm information is used to process a first preview frame; setting the frequency of the algorithm corresponding to the first algorithm information to a first frequency, wherein the first frequency is determined based on performance data and power consumption data of the digital signal processing module when the above algorithm is run at different frequencies; and running the algorithm corresponding to the first algorithm information according to the first frequency to process the first preview frame.

[0007] Based on the above scheme, the frequency of the algorithm involved in processing the first preview frame can be set to a unified first frequency, so that the frequencies specified by the above algorithms to the decision unit in the digital signal processing module are the same, that is, the above algorithms all specify the first frequency, so that the frequency decided by the decision unit is the set first frequency. It can be seen that the decision unit is no longer a black box, the user side can control the frequency decided, and the first frequency can be determined based on the performance data and power consumption data of the digital signal processing module, so that the set first frequency can balance the performance and power consumption of the digital signal processing module as much as possible, avoiding the set first frequency when the performance of the digital signal processing module is good. The power consumption is too high, or the performance of the set first frequency is too poor when the power consumption of the digital signal processing module is low.

[0008] In a possible implementation manner, obtaining the first algorithm information includes: obtaining the first algorithm information each time a first preview frame is obtained.

[0009] Based on the above scheme, the first algorithm information is obtained once each time a first preview frame is obtained (for example, each preview frame in the preview stream is regarded as the first preview frame), and the running frequency of the algorithm for processing the first preview frame is determined once, so that the running frequency of the algorithm can be different when processing different preview frames. Compared with the scheme of always using one frequency to process preview frames, this scheme is more flexible and can maximize the computing performance of the digital signal processing module.

[0010] In a possible implementation, running the algorithm according to a first frequency to process a first preview frame includes: the algorithm indicates the first frequency to a decision unit in a digital signal processing module; the decision unit determines, based on the indication, that the frequency at which a computing unit in the digital signal processing module runs the algorithm is the first frequency, and indicates the first frequency to the computing unit; and the computing unit runs the algorithm according to the first frequency to process the first preview frame.

[0011] In a possible implementation manner, the first frequency is determined based on the following formula:

[0012]

[0013]

[0014] Wherein, P is the first frequency, i is the identification information of the frequency point, m is the number of algorithms corresponding to the first algorithm information, A is the weight of the power consumption data, B is the weight of the performance data, and P in is the power consumption data of the digital signal processing module when running algorithm n at the frequency corresponding to frequency point i, C in is the performance data of the digital signal processing module when running algorithm n at the frequency corresponding to frequency point i, P target is the performance threshold of the digital signal processing module.

[0015] In a possible implementation, the method further includes: obtaining test information of multiple algorithms, the test information including performance data and power consumption data of the digital signal processing module when the multiple algorithms are run at different frequencies, the multiple algorithms including the algorithm corresponding to the first algorithm information.

[0016] In a possible implementation, the method further includes: obtaining second algorithm information, the algorithm corresponding to the second algorithm information is used to process the second preview frame, and the second algorithm information is the same as the first algorithm information; and the computing unit runs the algorithm corresponding to the second algorithm information according to the first frequency to process the second preview frame.

[0017] Based on the above solution, when the algorithms involved in processing the second preview frame and the first preview frame are the same, the calculation unit in the digital signal processing module can directly run the algorithm involved in the second preview frame according to the first frequency without re-determining the frequency for processing the second preview frame.

[0018] In one possible implementation, the method further includes: obtaining second algorithm information, the algorithm corresponding to the second algorithm information is used to process the second preview frame, and the second algorithm information is different from the first algorithm information; setting the frequency of the algorithm corresponding to the second algorithm information to a second frequency, and the second frequency is determined based on performance data and power consumption data of the digital signal processing module when the algorithm corresponding to the second algorithm information is run at different frequencies; and running the algorithm corresponding to the second algorithm information according to the second frequency to process the second preview frame.

[0019] Based on the above scheme, when the algorithms involved in processing the second preview frame and processing the first preview frame are different, for example, the number of algorithms for processing the second preview frame is different from the number of algorithms for processing the first preview frame, or the number of algorithms for processing the second preview frame is the same as the number of algorithms for processing the first preview frame but the algorithms are different, the second frequency is re-determined and the algorithm for processing the second preview frame is run according to the second frequency.

[0020] In a possible implementation, the above performance data is the time consumed for the algorithm to run.

[0021] In a second aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of processing in the first aspect or any one of the implementations of the first aspect when executing the computer program.

[0022] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the steps of processing in the first aspect or any one of the methods in the first aspect.

[0023] In a fourth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes the steps of processing in the first aspect or any one of the methods in the first aspect.

[0024] Among them, the beneficial effects of the second to fourth aspects can refer to the first aspect and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application;

[0026] Figure 2 It is a schematic diagram of the algorithm provided in the embodiment of the present application projecting frequency to the decision unit;

[0027] Figure 3 It is a schematic diagram showing the power consumption and performance of the DSP chip provided in the embodiment of the present application as a function of frequency;

[0028] Figure 4 is a flow chart of a frequency modulation method provided in an embodiment of the present application;

[0029] Figure 5 It is a schematic diagram of applying the frequency modulation method 400 provided in an embodiment of the present application in a DSP chip;

[0030] Figure 6 is a structural block diagram of a device 600 provided in an embodiment of the present application;

[0031] Figure 7 It is a structural diagram of an electronic device 700 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0033] In the description of this application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in this application is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, in the description of this application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solution of the embodiment of the present application, in the embodiment of the present application, the words "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences.

[0034] The frequency modulation method provided in the embodiments of the present application can be applied to electronic devices, including but not limited to mobile phones, tablet computers, laptop computers, PDAs, mobile Internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), smart homes, etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0035] The following is a schematic diagram of the hardware structure of the electronic device 100 for implementing the above method, for example, Figure 1 A schematic diagram of the hardware structure of the electronic device 100 is shown.

[0036] refer to Figure 1The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0037] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0038] For example, when the electronic device 100 is a mobile phone or a tablet computer, it may include all the components shown in the figure, or may include only some of the components shown in the figure.

[0039] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0040] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0041] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0042] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0043] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0044] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0045] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0046] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0047] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0048] The mobile communication module 150 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the electronic device 100 .

[0049] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.

[0050] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0051] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0052] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0053] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0054] The electronic device 100 can realize the shooting / video recording function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0055] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element (such as an image sensor) through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0056] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0057] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0058] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0059] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0060] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170 and the application processor.

[0061] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0062] The sensor module 180 includes, for example, a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and the like.

[0063] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0064] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0065] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0066] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0067] In actual application scenarios, the operating system of the electronic device 100 may include, but is not limited to, Symbian, Android, Windows, MacOS, iOS, Blackberry, HarmonyOS, etc. This application does not impose any limitation on the operating system of the electronic device.

[0068] Before describing the solutions provided by the embodiments of the present application, the terms involved in the present application are first explained.

[0069] Preview stream: A preview stream refers to a data stream used for preview display, such as a data stream used for camera preview, including multiple preview frames.

[0070] Preview frame: a frame of data in the preview stream, which can be understood as the unit granularity of the preview stream, such as an image frame.

[0071] Figure 2It is a schematic diagram of the algorithm provided in the embodiment of the present application projecting frequency to the decision unit.

[0072] like Figure 2 As shown, the DSP chip includes a decision unit and a computing unit. When the DSP chip is about to process a preview stream, the algorithms involved in the preview stream include, for example, algorithm #1, algorithm #2, ..., algorithm #N. Each algorithm will cast a frequency to the decision unit through a frequency casting interface between the algorithm and the decision unit, that is, algorithm #1 specifies frequency #1 to the decision unit, algorithm #2 specifies frequency #2 to the decision unit, ..., algorithm #N specifies frequency #N to the decision unit. The frequency specified by each algorithm is the frequency required by the algorithm itself, which is not perceived by other algorithms. Therefore, the frequency specified by each algorithm may be different. At this time, the decision unit will decide the frequency of the computing unit running the above-mentioned algorithms #1, algorithm #2, ..., algorithm #N to process the preview stream according to a certain strategy and instruct the computing unit. For example, the decision unit selects the maximum frequency specified by all algorithms and instructs it to the computing unit, or the decision unit calculates the average value of the frequency specified by each algorithm and instructs it to the computing unit. The computing unit runs the above-mentioned algorithm to process the preview stream according to the instruction.

[0073] In the above scheme, the decision strategy of the decision unit is determined by the supplier of the DSP chip. Therefore, the decision unit is a black box for the user. The user side cannot control the frequency of the decision, and the calculation unit will use the above-mentioned frequency when processing each preview frame in the preview stream subsequently. It is not flexible enough and cannot give full play to the computing performance of the DSP chip.

[0074] Figure 3 It is a schematic diagram showing how the power consumption and performance of the DSP chip provided in the embodiment of the present application vary with frequency.

[0075] like Figure 3 As shown in the figure, the performance of the DSP chip when running the algorithm decreases with the increase of frequency. For example, the time it takes for the DSP chip to complete the calculation of the algorithm is in milliamperes (ms). The power consumption of the DSP chip when running the algorithm increases with the increase of frequency. The power consumption is in milliamperes (mA). Specifically, as the frequency increases, the algorithm performance becomes better, but the power consumption is higher. In other words, performance is closely related to power consumption. How to balance the performance and power consumption of the DSP chip is a problem that needs to be considered.

[0076] In view of this, an embodiment of the present application provides a frequency modulation method, so that the user side can control the frequency decided by the decision unit in the DSP chip, so that the decided frequency can balance the performance and power consumption of the DSP chip running the algorithm to process the preview frame, so as to minimize the power consumption of the DSP chip while meeting the performance.

[0077] Figure 4 It is a flowchart of the frequency modulation method provided in an embodiment of the present application. Figure 4The exemplary method 400 may be applied to an electronic device, the electronic device comprising a digital signal processing module, the digital signal processing module comprising a decision unit (eg Figure 4 As shown in the decision unit #2) and the calculation unit, the digital signal processing module such as a DSP chip, the method 400 includes the following steps.

[0078] S401, the algorithm inputs test information to decision unit #1.

[0079] Exemplarily, a single algorithm can obtain test information of the algorithm during offline testing, and the test information includes performance data and power consumption data of the DSP chip running the algorithm at frequencies corresponding to different frequency points. After the test is completed, the test information is input into the decision unit #1 to provide a decision basis for the decision unit. Alternatively, after obtaining the test information, the single algorithm can store the test information in the electronic device in the form of xml data. The decision unit #1 can read the test information of the relevant algorithm in the electronic device when initializing the algorithm information.

[0080] Understandably, Figure 4 The algorithm for inputting test information to the decision unit in the example may be one or more algorithms. For example, there may be N algorithms that can be run in the computing unit in the future. After the N algorithms complete the offline test, the N test information can be input into the decision unit #1, or the N test information can be stored in the electronic device. If the decision unit #1 needs the test information of certain algorithms, it can read the test information of the corresponding algorithms when initializing the algorithm information. This application does not impose any restrictions on this.

[0081] Table 1 below illustrates the test information of a single algorithm, such as algorithm #1.

[0082] Table 1

[0083]

[0084] As shown in Figure 1, the power consumption of the DSP chip running algorithm #1 at frequency 1 (the frequency corresponding to frequency 1 is 576.00 MHz) is P1 mA, and the performance of the DSP chip running algorithm #1 at frequency 1 is C1 ms. Similarly, the frequency points include low frequency (LOW), medium frequency (MID), high frequency (HIGH) and ultra-high frequency (PERFORMANCE).

[0085] S402, the decision unit #1 requests the scheduling unit for first algorithm information, and the algorithm corresponding to the first algorithm information is used to process the first preview frame.

[0086] In a possible implementation, before processing the preview stream or when the preview stream is about to be processed, execution of S402 may be triggered.

[0087] Exemplarily, the algorithm information requested by decision unit #1 may be in the granularity of one preview frame or in the granularity of multiple preview frames, and the present application does not impose any restrictions on this. If the algorithm information requested by decision unit #1 is in the granularity of one preview frame, for example, the first preview frame refers to preview frame #1, then the first algorithm information sent by the scheduling unit is the algorithm information used to process preview frame #1. If the algorithm information requested by decision unit #1 is in the granularity of multiple preview frames, for example, the first preview frame refers to preview frame #1 and preview frame #2, then the first algorithm information sent by the scheduling unit is the algorithm information used to process preview frame #1 and preview frame #2. When the algorithm information requested by decision unit #1 is in the granularity of multiple preview frames, the embodiment of the present application does not impose any restrictions on the number of preview frames related to the algorithm information requested by decision unit #1 each time.

[0088] S403, the scheduling unit feeds back the first algorithm information to the decision unit #1.

[0089] Exemplarily, the scheduling unit feeds back the first algorithm information in response to the request of the decision unit #1, or after the preview stream arrives, the scheduling unit is initialized and can determine the algorithm required for each preview frame in the preview stream. After determining the algorithm required for each preview frame, the scheduling unit can directly send the algorithm information to the decision unit. In this case, S402 is an optional step. The embodiment of the present application does not limit the number of preview frames related to the algorithm information sent by the scheduling unit each time. The first algorithm information includes an algorithm identifier for processing the first preview frame.

[0090] S404, decision unit #1 determines a first frequency, where the first frequency is determined based on performance data and power consumption data of the digital signal processing module when the above algorithm is run at different frequencies.

[0091] In a possible implementation manner, the decision unit #1 may obtain test information of multiple algorithms through S401, or the decision unit #1 may read test information of related algorithms from the algorithm corresponding to the first algorithm information after S403.

[0092] For example, the first algorithm information corresponds to algorithm #1, algorithm #2, and algorithm #3, and the test information of multiple algorithms obtained by decision unit #1 through S401 includes test information #1 of algorithm #1, test information #2 of algorithm #2, and test information #3 of algorithm #3; or decision unit #1 knows after S403 that it needs to obtain test information of algorithm #1, algorithm #2, and algorithm #3, and can read test information #1 of algorithm #1, test information #2 of algorithm #2, and test information #3 of algorithm #3 from the xml data stored in the electronic device; or decision unit #1 knows after S403 that it needs to obtain test information of algorithm #1, algorithm #2, and algorithm #3, and directly reads test information #1 from algorithm #1, reads test information #2 from algorithm #2, and reads test information #3 from algorithm #3.

[0093] Exemplarily, after reading the relevant test information, the first frequency is determined based on the following formula:

[0094] (1)

[0095] (2)

[0096] Wherein, P is the first frequency, i is the identification information of the frequency point, m is the number of algorithms corresponding to the first algorithm information, A is the weight of the power consumption data, B is the weight of the performance data, and P in is the power consumption data of the digital signal processing module when running algorithm n at the frequency corresponding to frequency point i, C in is the performance data of the digital signal processing module when running algorithm n at the frequency corresponding to frequency point i, P target is the performance threshold of the digital signal processing module.

[0097] For example, if it is found through calculation that there are multiple frequency points that satisfy formula (1), then each of the multiple frequency points is substituted into formula (2), and the frequency point that obtains the minimum value after substitution is selected, and the frequency corresponding to the frequency point is the first frequency.

[0098] S405, decision unit #1 sets the frequency of the algorithm corresponding to the first algorithm information to the first frequency.

[0099] Exemplarily, the first algorithm information corresponds to algorithm #1, algorithm #2 and algorithm #3, and the decision unit #1 sets the frequency of algorithm #1, algorithm #2 and algorithm #3 to the first frequency.

[0100] S406, each algorithm corresponding to the first algorithm information indicates the first frequency to the decision unit #2, that is, each algorithm corresponding to the first algorithm information casts a frequency to the decision unit #2, and the cast frequency is the first frequency.

[0101] Exemplarily, the DSP opens a frequency projection interface before each algorithm processes a preview frame. Algorithm #1 projects a frequency of the first frequency to decision unit #2 through the frequency projection interface between algorithm #1 and decision unit #2. Algorithm #2 projects a frequency of the first frequency to decision unit #2 through the frequency projection interface between algorithm #2 and decision unit #2. Algorithm #3 projects a frequency of the first frequency to decision unit #2 through the frequency projection interface between algorithm #3 and decision unit #2.

[0102] S407, the decision unit #2 determines the first frequency according to the frequency of each algorithm corresponding to the first algorithm information.

[0103] Exemplarily, Algorithm #1, Algorithm #2 and Algorithm #3 all vote for the first frequency, so no matter whether the strategy of decision unit #2 is to select the maximum value, the minimum value or the average value, the frequency decided is the first frequency.

[0104] S408, the decision unit #2 indicates the first frequency determined by the decision unit #2 to the calculation unit.

[0105] S409: The calculation unit runs the algorithm corresponding to the first algorithm information based on the first frequency indicated by the decision unit #2 to process the first preview frame.

[0106] Exemplarily, after receiving the instruction, the computing unit runs algorithm #1, algorithm #2, and algorithm #3 according to the first frequency to process the first preview frame.

[0107] Optionally, the method 400 further includes:

[0108] The decision unit #1 obtains second algorithm information, the algorithm corresponding to the second algorithm information is used to process the second preview frame, and the second algorithm information is the same as the first algorithm information; the calculation unit runs the algorithm corresponding to the second algorithm information according to the first frequency to process the second preview frame.

[0109] Exemplarily, the way in which decision unit #1 obtains the second algorithm information can refer to the way in which decision unit #1 obtains the first algorithm information, and no further details are given. The number of preview frames included in the second preview frame can be the same as the number of preview frames included in the first preview frame.

[0110] It can be understood that when the algorithm corresponding to the second algorithm information is exactly the same as the algorithm corresponding to the first algorithm information, that is, the number of algorithms and the algorithms are the same, for example, the first algorithm information corresponds to algorithm #1, algorithm #2 and algorithm #3, and the second algorithm information also corresponds to algorithm #1, algorithm #2 and algorithm #3, then the algorithm corresponding to the second algorithm information is said to be exactly the same as the algorithm corresponding to the first algorithm information. At this time, when the decision unit #1 determines the frequency, it no longer determines it through formula (1) and formula (2) as in S404, but directly adopts the first frequency determined in S404.

[0111] Optionally, the method 400 further includes:

[0112] The decision unit #1 obtains second algorithm information, the algorithm corresponding to the second algorithm information is used to process the second preview frame, and the second algorithm information is different from the first algorithm information; the decision unit #1 sets the frequency of the algorithm corresponding to the second algorithm information to a second frequency, and the second frequency is determined based on the performance data and power consumption data of the digital signal processing module when the algorithm corresponding to the second algorithm information is run at different frequencies; the calculation unit runs the algorithm corresponding to the second algorithm information according to the second frequency to process the second preview frame.

[0113] Exemplarily, when the algorithm corresponding to the second algorithm information is algorithm #1, algorithm #2, and algorithm #4, similar to S404, decision unit #1 needs to calculate and determine the second frequency based on the test information of algorithm #1, algorithm #2, and algorithm #4 in combination with formula (1) and formula (2). For details on how to obtain the test information and how to determine the second frequency, please refer to the relevant description above and will not be repeated. The steps performed after determining the second frequency can refer to S405 to S409 and will not be repeated.

[0114] Based on the above scheme, the scheme of obtaining algorithm information and determining the algorithm's operating frequency based on the number of preview frames included in the first preview frame can achieve different algorithm operating frequencies when processing different batches of preview frames, and a batch of preview frames is the preview frames under the granularity number. Compared with the scheme of always using one frequency to process all preview frames, this scheme is more flexible and can maximize the computing performance of the DSP chip.

[0115] Figure 5 It is a schematic diagram of the application of the frequency modulation method 400 provided in an embodiment of the present application in a DSP chip.

[0116] For example, Figure 5 As shown, when the first preview frame arrives and needs to be processed, the calling module can determine which algorithms are used to process the first preview frame, and the decision unit #1 can request the first algorithm information related to the first preview frame from the calling unit, specifically referring to the relevant descriptions in S402 and S403. Prior to this, each algorithm inputs the test data into the decision unit #1 during the offline test, specifically referring to the relevant description in S401. After the decision unit #1 obtains the first algorithm information, it can know which algorithms will be used to process the first preview frame in the future. For example, the algorithms corresponding to the first algorithm information include algorithm #1, algorithm #2, ..., algorithm #M, and the first frequency is determined based on the performance data and power consumption data corresponding to the algorithm #1, algorithm #2, ..., algorithm #M input in S401. For the specific determination method, refer to the relevant description in S404. After determining the first frequency, decision unit #1 sets the frequency of each algorithm corresponding to the first algorithm information to the first frequency. Subsequently, the frequency of each algorithm voting to decision unit #2 is the first frequency. The decision unit determines that the frequency at which the calculation unit runs the algorithm is the first frequency and indicates the first frequency to the calculation unit. Subsequently, the calculation unit runs algorithm #1, algorithm #2, ..., algorithm #M according to the first frequency to process the first preview frame.

[0117] Based on the method 400 provided in the present application, the user can set the frequency of the algorithm involved in processing the first preview frame to a unified first frequency, so that the frequencies specified by the above algorithms to the decision unit in the digital signal processing module are the same, that is, the above algorithms all specify the first frequency, and the frequency decided by the decision unit is the set first frequency. It can be seen that the decision unit is no longer a black box, the user side can control the frequency decided, and the first frequency can be determined based on the performance data and power consumption data of the digital signal processing module, so that the set first frequency can balance the performance and power consumption of the digital signal processing module as much as possible, avoiding the set first frequency when the performance of the digital signal processing module is good. The power consumption is too high, or the performance of the set first frequency is too poor when the power consumption of the digital signal processing module is low.

[0118] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0119] Figure 6 It is a structural block diagram of a device 600 provided in an embodiment of the present application. For the convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0120] The device 600 is, for example, an electronic device, and the device 600 includes a digital signal processing module, referring to Figure 6 The device 600 includes: an acquisition unit 601, a processing unit 602 and a calculation unit 603.

[0121] Among them, the acquisition unit 601 is used to obtain first algorithm information, and the algorithm corresponding to the first algorithm information is used to process the first preview frame; the processing unit 602 is used to set the frequency of the algorithm corresponding to the first algorithm information to a first frequency, and the first frequency is determined based on the performance data and power consumption data of the digital signal processing module when the above algorithm is run at different frequencies; the calculation unit 603 is used to run the algorithm corresponding to the first algorithm information according to the first frequency to process the first preview frame.

[0122] In a possible implementation manner, the acquisition unit 601 acquires the first algorithm information each time a first preview frame is acquired.

[0123] In one possible implementation, the digital signal processing module includes a decision unit, and the algorithm indicates a first frequency to the decision unit; the decision unit determines that the frequency at which the calculation unit 603 runs the algorithm is the first frequency based on the indication, and indicates the first frequency to the calculation unit 603; the calculation unit 603 runs the algorithm according to the first frequency to process the first preview frame.

[0124] In a possible implementation manner, the first frequency is determined based on the following formula:

[0125]

[0126]

[0127] Wherein, P is the first frequency, i is the identification information of the frequency point, m is the number of algorithms corresponding to the first algorithm information, A is the weight of the power consumption data, B is the weight of the performance data, and P in is the power consumption data of the digital signal processing module when running algorithm n at the frequency corresponding to frequency point i, C in is the performance data of the digital signal processing module when running algorithm n at the frequency corresponding to frequency point i, P target is the performance threshold of the digital signal processing module.

[0128] In a possible implementation, the acquisition unit 601 is also used to obtain test information of multiple algorithms, the test information including performance data and power consumption data of the digital signal processing module when the multiple algorithms are run at different frequencies, and the multiple algorithms include the algorithm corresponding to the first algorithm information.

[0129] In a possible implementation, the acquisition unit 601 is further used to acquire second algorithm information, the algorithm corresponding to the second algorithm information is used to process the second preview frame, and the second algorithm information is the same as the first algorithm information; the calculation unit 603 is specifically used to run the algorithm corresponding to the second algorithm information according to the first frequency to process the second preview frame.

[0130] In one possible implementation, the acquisition unit 601 is also used to acquire second algorithm information, the algorithm corresponding to the second algorithm information is used to process the second preview frame, and the second algorithm information is different from the first algorithm information; the processing unit 602 is used to set the frequency of the algorithm corresponding to the second algorithm information to a second frequency, and the second frequency is determined based on the performance data and power consumption data of the digital signal processing module when the algorithm corresponding to the second algorithm information is run at different frequencies; the calculation unit 603 is specifically used to run the algorithm corresponding to the second algorithm information according to the second frequency to process the second preview frame.

[0131] In a possible implementation, the above performance data is the time consumed for the algorithm to run.

[0132] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions, execution steps, interpretation of terms, associations and technical effects brought about can be found in the method embodiment part and will not be repeated here.

[0133] It can be clearly understood by those skilled in the art that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. For example, a "module" can be a software program, a hardware circuit, or a combination of the two to implement the above-mentioned functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a merged logic circuit, and / or other suitable components that support the described functions.

[0134] Therefore, the modules of each example described in the embodiments of the present application can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0135] In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the above method embodiment, which will not be repeated here.

[0136] It should be understood that the hardware systems, devices and electronic devices in the embodiments of the present application can execute the frequency modulation methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0137] Figure 7 4 is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present application. The electronic device 700 includes a processor 701, such as a DSP chip, and the DSP chip includes a decision unit #2 and a computing unit. Optionally, the electronic device 700 also includes a decision unit #1 and a scheduling unit, and the electronic device 700 can implement the technical solution shown in the method 400.

[0138] Optionally, the electronic device 700 further includes a transceiver 702, and the transceiver 702 is used to accept the control of the processor 701 and to support the electronic device in executing the technical solution shown above.

[0139] Optionally, the electronic device 700 may further include: a storage medium 703 .

[0140] It should be noted that the processor 701 can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGA), programmable logic devices (PLD), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.

[0141] The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium is run on an electronic device, the electronic device executes the method shown above. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) methods. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more media that can be integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0142] The embodiment of the present application also provides a computer program product including computer instructions, which, when executed on an electronic device, enables the electronic device to execute the technical solution shown above.

[0143] The devices, electronic devices, computer storage media, and computer program products provided in the above-mentioned embodiments of the present application are all used to execute the method 400 provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the method 400 provided above, and will not be repeated here.

[0144] It should be understood that the above is only to help those skilled in the art to better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application. According to the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, some steps in each embodiment of the above detection method may be unnecessary, or some steps may be newly added. Or a combination of any two or any multiple embodiments of the above. Such modifications, changes or combined schemes also fall within the scope of the embodiments of the present application.

[0145] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

[0146] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0147] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0148] It should also be understood that in the embodiments of the present application, "pre-configuration", "pre-setting" and "pre-definition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a first device (for example, including a terminal device), and the present application does not limit its specific implementation method.

[0149] It should also be understood that the division of the methods, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various methods, categories, situations and embodiments can be combined without contradiction.

[0150] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A frequency modulation method, characterized in that: Applied to an electronic device, the electronic device includes a digital signal processing module, and the method includes: Acquire first algorithm information, where the algorithm corresponding to the first algorithm information is used to process the first preview frame; Acquire test information of a plurality of algorithms, the test information comprising performance data and power consumption data of the digital signal processing module when the plurality of algorithms are run at different frequencies, the plurality of algorithms including an algorithm corresponding to the first algorithm information; Setting the frequency of the algorithm corresponding to the first algorithm information to a first frequency, wherein the first frequency is determined based on performance data and power consumption data of the digital signal processing module when the algorithm is run at different frequencies; The algorithm indicates the first frequency to a decision unit in the digital signal processing module; The decision unit determines, based on the indication, that the frequency at which the calculation unit in the digital signal processing module runs the algorithm is the first frequency, and indicates the first frequency to the calculation unit; The computing unit runs the algorithm according to the first frequency to process the first preview frame.

2. The method according to claim 1, characterized in that The obtaining of the first algorithm information comprises: The first algorithm information is obtained each time a first preview frame is obtained.

3. The method according to claim 1 or 2, characterized in that: The first frequency is determined based on the following formula: Wherein, P is the first frequency, i is the identification information of the frequency point, m is the number of algorithms corresponding to the first algorithm information, A is the weight of the power consumption data, B is the weight of the performance data, and P in is the power consumption data of the digital signal processing module when the algorithm n is run at the frequency corresponding to the frequency point i, C in is the performance data of the digital signal processing module when the algorithm n is run at the frequency corresponding to the frequency point i, P target is the performance threshold of the digital signal processing module.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquire second algorithm information, where the algorithm corresponding to the second algorithm information is used to process the second preview frame, and the second algorithm information is the same as the first algorithm information; The computing unit runs the algorithm corresponding to the second algorithm information according to the first frequency to process the second preview frame.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquire second algorithm information, where an algorithm corresponding to the second algorithm information is used to process a second preview frame, and the second algorithm information is different from the first algorithm information; Setting the frequency of the algorithm corresponding to the second algorithm information to a second frequency, where the second frequency is determined based on performance data and power consumption data of the digital signal processing module when the algorithm corresponding to the second algorithm information is run at different frequencies; The algorithm corresponding to the second algorithm information is run according to the second frequency to process the second preview frame.

6. The method according to claim 1 or 2, characterized in that: The performance data is the time consumed by the algorithm to run.

7. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 6.

9. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 6.

Images