Algorithm scheduling method and electronic device

Abstract

The present application relates to the field of artificial intelligence, and provides an algorithm scheduling method and an electronic device, which can improve the accuracy of scheduling multiple algorithms. The method includes: obtaining pre-stored algorithm basic information and real-time performance status information of an electronic device, inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into an algorithm scheduling module for processing, obtaining an algorithm scheduling table, and scheduling the algorithms in a running state according to the algorithm scheduling table, wherein the algorithm scheduling table used for algorithm scheduling is dynamically generated according to the pre-stored algorithm basic information and the real-time performance status information of the electronic device, thereby making the generated algorithm scheduling table more consistent with the real-time performance status of the electronic device, avoiding the problem of unprepared algorithm scheduling caused by the mismatch between the pre-stored algorithm scheduling table and the real-time performance status of the electronic device, and improving the accuracy of scheduling multiple algorithms.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to an algorithm scheduling method and an electronic device. Background Art

[0002] When collecting images, electronic devices usually use multiple algorithms to process the images at the same time. For example, when a user opens a camera application (Application, APP) to start collecting portrait images, a focus frame for the face may be displayed in the display interface of the camera app. In this case, the electronic device will call the image detection algorithm to detect the face in the image, and call the image optimization algorithm to process the image to improve the clarity of the displayed image.

[0003] At present, when an electronic device calls multiple algorithms at the same time, in order to avoid resource conflicts caused by multiple algorithms occupying the same computing unit at the same time, an algorithm scheduling table is usually pre-stored in the electronic device. The electronic device runs multiple algorithms at the time indicated in the algorithm scheduling table to avoid resource conflicts caused by different algorithms occupying the same computing unit.

[0004] However, when the electronic device is in use, the state of the electronic device may change dynamically, which may cause the electronic device to schedule multiple algorithms according to a pre-stored algorithm scheduling table, resulting in inaccurate scheduling.

[0005] In view of this, how to improve the accuracy of scheduling multiple algorithms becomes an urgent problem to be solved. Summary of the invention

[0006] The present application provides an algorithm scheduling method and electronic device, which can improve the accuracy of scheduling multiple algorithms.

[0007] In a first aspect, an algorithm scheduling method is provided, comprising: the method is applied to an electronic device, the electronic device comprising an algorithm scheduling module and at least two computing units, the method comprising: obtaining pre-stored algorithm basic information and real-time performance status information of the electronic device; inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module to obtain an algorithm scheduling table, wherein each algorithm indicated by the algorithm scheduling table occupies different computing units in the same frame; scheduling at least two algorithms running in the electronic device according to the algorithm scheduling table to obtain output data of the at least two algorithms.

[0008] The algorithm scheduling method provided in the embodiment of the present application obtains pre-stored algorithm basic information and real-time performance status information of the electronic device, inputs the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module for processing, obtains an algorithm scheduling table, and schedules the algorithms in running state according to the algorithm scheduling table, wherein the algorithm scheduling table used for algorithm scheduling is dynamically generated based on the pre-stored algorithm basic information and the real-time performance status information of the electronic device, thereby making the generated algorithm scheduling table more consistent with the real-time performance status of the electronic device, avoiding the problem of unprepared algorithm scheduling caused by the mismatch between the pre-stored algorithm scheduling table and the real-time performance status of the electronic device, and improving the accuracy of scheduling multiple algorithms.

[0009] It should be understood that the pre-stored algorithm basic information can be stored in Figure 3 The algorithm module shown in the figure can obtain the occupied computing units of each algorithm running in the electronic device, the minimum frame rate of running the algorithm, the time consumption of different computing units, the algorithm power consumption and the algorithm peak memory before the electronic device leaves the factory.

[0010] It should also be understood that the performance status information of the electronic device may be the remaining power of the electronic device at the current moment, the operating temperature, and the load information of each computing unit in the electronic device. The algorithm scheduling module may obtain the remaining power of the electronic device through the power management module 141 in the electronic device, read the temperature data in the temperature sensor 180J to obtain the operating temperature of the electronic device, and read the load information of each computing unit in the electronic device to obtain the performance status information of the electronic device.

[0011] In combination with the first aspect, in certain embodiments of the first aspect, the algorithm scheduling module includes a decision-making sub-module, which inputs pre-stored algorithm basic information and real-time performance status information of the electronic device into the algorithm scheduling module to obtain an algorithm scheduling table, including: inputting pre-stored algorithm basic information and real-time performance status information of the electronic device into the decision-making sub-module to obtain an algorithm scheduling table.

[0012] It should be understood that the algorithm scheduling module can be as follows Figure 5 As shown, it includes a decision submodule and an information decoder. The information decoder can decode the input algorithm basic information and real-time performance status information and convert them into feature vectors. The decision submodule can process the converted feature vectors to obtain an algorithm scheduling table.

[0013] In combination with the first aspect, in certain embodiments of the first aspect, the method also includes: training the initial decision sub-module according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain a trained decision sub-module: inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the trained decision sub-module to obtain the algorithm scheduling table.

[0014] It should be understood that the electronic device can input the sample algorithm basic information and the sample real-time performance status information into the initial decision sub-module to obtain the target algorithm scheduling table, and then compare the target algorithm scheduling table with the sample algorithm scheduling table, and adjust the loss function in the algorithm scheduling module based on the comparison result until the difference between the target algorithm scheduling table and the sample algorithm scheduling table is less than the preset threshold, completing the training of the initial algorithm scheduling module and obtaining the trained algorithm scheduling module.

[0015] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned training of the initial decision sub-module according to the sample algorithm scheduling table and the corresponding sample algorithm basic information, and the sample real-time performance status information to obtain the trained decision sub-module includes: in a first time period, training the initial decision sub-module according to the sample algorithm scheduling table and the corresponding sample algorithm basic information, and the sample real-time performance status information to obtain the trained decision sub-module; in a second time period, inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the trained decision sub-module to obtain the algorithm scheduling table; wherein the first time period and the second time period are different time periods, and the first time period is before the second time period.

[0016] It should be understood that the first period may refer to the period before the electronic device leaves the factory, and the second period may refer to the period when the user uses the electronic device after the electronic device leaves the factory.

[0017] For example, the electronic device can turn on the pre-training switch before leaving the factory, and use the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to train the initial algorithm scheduling module to obtain the trained algorithm scheduling module. Then, the electronic device can turn off the pre-training switch, so that when the electronic device is used by the user after leaving the factory, the computing unit will not be frequently occupied to train the algorithm scheduling module.

[0018] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned training of the initial decision sub-module according to the sample algorithm scheduling table and the corresponding sample algorithm basic information, and the sample real-time performance status information to obtain the trained decision sub-module includes: according to a preset time interval, periodically training the initial decision sub-module according to the sample algorithm scheduling table and the corresponding sample algorithm basic information, and the sample real-time performance status information to obtain the trained decision sub-module.

[0019] It should be understood that as the usage time of electronic devices increases, the hardware performance of electronic devices will also decrease. Therefore, the initial algorithm scheduling module is periodically trained at preset time intervals to obtain a trained algorithm scheduling module. This can make the trained algorithm scheduling module more compatible with the hardware performance of the electronic device, thereby improving the accuracy of the trained algorithm scheduling module.

[0020] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned training of the initial decision sub-module according to the sample algorithm scheduling table and the corresponding sample algorithm basic information, and the sample real-time performance status information to obtain the trained decision sub-module includes: when the electronic device receives a system version update, the initial decision sub-module is trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information, and the sample real-time performance status information to obtain the trained decision sub-module.

[0021] It should be understood that since system version updates may affect the performance of electronic devices, each time the system version is pushed, the initial algorithm scheduling module is trained to obtain a trained algorithm scheduling module, which can make the trained algorithm scheduling module more compatible with the system version of the electronic device, thereby improving the accuracy of the trained algorithm scheduling module.

[0022] In combination with the first aspect, in certain embodiments of the first aspect, the algorithm scheduling module further includes a compensation submodule, and the method further includes: obtaining first data of a first algorithm, the first algorithm being an algorithm for frame skipping among at least two algorithms running in an electronic device, and the first data refers to output data of the first algorithm in a frame before the frame skipping; inputting the first data into the compensation submodule to obtain second data, the second data being output data obtained by simulating the first algorithm running during frame skipping; and obtaining output data of the first algorithm based on the second data.

[0023] It should be understood that at least two algorithms running at the same time need to occupy the same computing unit. At this time, the electronic device can avoid the above-mentioned computing unit conflict problem by skipping frames. For example, if the algorithms A and B running at the same time both need to occupy the CPU in the third frame, in order to avoid the problem of computing unit conflict, algorithm A can be run in the CPU in the third frame, and algorithm B is not run, that is, algorithm B skips the frame in the third frame. However, since algorithm B skips the frame in the third frame, one frame of processing is missing, and there may be a problem of reduced accuracy of algorithm B.

[0024] For example, if an image detection algorithm used to determine the face area in an image skips frames at the third frame, since the image detection algorithm usually processes the previous frame as the input of the current frame, if the output data of the third frame is missing, when the image detection algorithm is run at the fourth frame, since there is no output data of the third frame, the image detection algorithm may use the output data of the second frame for processing to obtain the output data of the fourth frame. This may cause the output of the image detection algorithm running on the fourth frame to be inaccurate, and thus cause the determined face area in the image to be inaccurate.

[0025] The compensation submodule may also be a neural network model. For example, the compensation submodule is a lightweight Transformer model.

[0026] In some possible cases, the compensation submodule may be run in the EAI. Since the EAI is not a computing unit in the electronic device, this can avoid the problem of computing unit conflicts caused by the compensation submodule occupying the computing unit.

[0027] The algorithm scheduling method provided in the embodiment of the present application can simulate the frame skipping of the first algorithm through a compensation submodule to obtain output data of the frame skipping of the first algorithm according to the algorithm scheduling table when there is a first algorithm with frame skipping, and then obtain the output data of the first algorithm according to the simulated output data of the first algorithm after the frame skipping, so that the output data of the first algorithm that runs according to the algorithm scheduling table and has frame skipping is closer to the output data obtained by the full frame running of the first algorithm, thereby improving the accuracy of the first algorithm.

[0028] In combination with the first aspect, in certain embodiments of the first aspect, the electronic device also includes an algorithm module, and the above-mentioned acquisition of the first data of the first algorithm includes: if the decision submodule determines that the first algorithm skips frames, the decision submodule sends a first acquisition request to the algorithm module, and the first acquisition request is used to request the algorithm module to obtain the first data; the algorithm module sends the first data to the decision submodule in response to the first acquisition request; the decision submodule sends the first data to the compensation submodule.

[0029] In combination with the first aspect, in certain embodiments of the first aspect, the electronic device also includes an algorithm module, and the above-mentioned acquisition of the first data of the first algorithm includes: if the decision submodule determines that the first algorithm skips frames, the decision submodule sends first indication information to the compensation submodule, and the first indication information is used to indicate the frame skipping of the first algorithm; the compensation submodule sends a second acquisition request to the algorithm module based on the first indication information, and the second acquisition request is used to request the algorithm module to obtain the first data; the algorithm module sends the first data to the compensation submodule in response to the second acquisition request.

[0030] In combination with the first aspect, in certain embodiments of the first aspect, the method also includes: training an initial compensation submodule according to sample full-frame algorithm data and corresponding sample first data to obtain a trained compensation submodule; and inputting the first data into the trained compensation submodule to obtain second data.

[0031] It should be understood that the compensation submodule is a neural network model, so the electronic device can first train the initial compensation submodule to obtain a trained compensation submodule, and then obtain the second data through the trained compensation submodule.

[0032] The full frame algorithm data may refer to input data and output data of each frame when the first algorithm performs operation on each frame. The sample full frame algorithm data may refer to data obtained when the electronic device simulates running the first algorithm without frame skipping.

[0033] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned training of the initial compensation submodule according to the sample full-frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule includes: training the initial compensation submodule according to the sample full-frame algorithm data and the corresponding sample first data in a third time period to obtain the trained compensation submodule; inputting the first data into the compensation submodule in a fourth time period to obtain the second data; wherein the third time period and the fourth time period are different time periods, and the third time period is before the fourth time period.

[0034] The third period may refer to a period before the electronic device leaves the factory, and the fourth period may refer to a period after the electronic device leaves the factory and is used by the user.

[0035] It should be understood that the third time period may be the same time period as the first time period, or may be a different time period from the first time period, and the embodiment of the present application does not impose any limitation on this.

[0036] If the third time period is the same as the first time period, it means that the training time periods of the decision submodule and the compensation submodule are the same. Since the intermediate data of the decision submodule can be used as the input data of the compensation submodule, training the decision submodule and the compensation submodule at the same time can improve the training accuracy of the decision submodule and the compensation submodule, and improve the accuracy of the decision submodule and the compensation submodule.

[0037] If the third time period and the first time period are two different time periods, it may mean that the decision submodule and the compensation submodule are trained separately, which can improve the flexibility of training the decision submodule and the compensation submodule.

[0038] It should be understood that the fourth time period may be the same time period as the second time period, or may be a time period different from the second time period, and the embodiment of the present application does not limit this.

[0039] Exemplarily, the electronic device can turn on the pre-training switch of the compensation submodule before leaving the factory, and use the sample full frame algorithm data and the corresponding sample first data to train the initial compensation submodule to obtain the trained compensation submodule. Then, the electronic device can turn off the pre-training switch, so that when the electronic device is used by the user after leaving the factory, the computing unit will not be frequently occupied to train the compensation submodule.

[0040] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned training of the initial compensation submodule according to the sample full-frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule includes: according to a preset time interval, periodically training the initial compensation submodule according to the sample full-frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule.

[0041] It should be understood that as the usage time of electronic devices increases, the hardware performance of the electronic devices will also decrease. Therefore, regular training of the initial compensation sub-module to obtain a trained compensation sub-module can make the trained compensation sub-module better match the hardware performance of the electronic device, thereby improving the accuracy of the trained compensation sub-module.

[0042] In combination with the first aspect, in certain embodiments of the first aspect, the above-mentioned training of the initial compensation submodule according to the sample full-frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule includes: when the electronic device receives a system version update, training the initial compensation submodule according to the sample full-frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule.

[0043] It should be understood that since system version updates may affect the performance of electronic devices, each time the system version is pushed, the initial compensation sub-module is trained to obtain a trained compensation sub-module, which can make the trained compensation sub-module more compatible with the system version of the electronic device, thereby improving the accuracy of the trained compensation sub-module.

[0044] In a second aspect, an electronic device is provided, which includes an algorithm scheduling module for executing any one of the methods shown in the first aspect above.

[0045] In combination with the second aspect, in certain embodiments of the second aspect, the electronic device includes an algorithm scheduling module and an algorithm module, the algorithm scheduling module is used to obtain an algorithm scheduling table based on pre-stored algorithm basic information in the algorithm module and real-time performance status information of the electronic device, wherein the algorithms indicated in the algorithm scheduling table occupy different computing units in the same frame.

[0046] In combination with the second aspect, in certain embodiments of the second aspect, the algorithm scheduling module includes a decision submodule, which is used to obtain an algorithm scheduling table based on pre-stored algorithm basic information in the algorithm module and real-time performance status information of the electronic device.

[0047] In combination with the second aspect, in certain embodiments of the second aspect, the algorithm scheduling module also includes a compensation submodule, which is used to obtain second data based on first data of the first algorithm, wherein the first algorithm is a frame skipping algorithm among at least two algorithms running in the electronic device, the first data refers to the output data of the first algorithm in a frame before the frame skipping, and the second data is the output data obtained by simulating the first algorithm running during frame skipping.

[0048] In a third aspect, an algorithm scheduling device is provided, comprising a unit for executing any one of the methods in the first aspect. The device may be a terminal device or a chip in the terminal device. The device may include an input unit and a processing unit.

[0049] When the device is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may also include a memory, which is used to store computer program code. When the processor executes the computer program code stored in the memory, the terminal device executes any one of the methods in the first aspect.

[0050] When the device is a chip in a terminal device, the processing unit may be a processing unit inside the chip, and the input unit may be an output interface, a pin or a circuit, etc.; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the methods in the first aspect.

[0051] In one possible implementation, a memory is used to store computer program codes; a processor executes the computer program codes stored in the memory, and when the computer program codes stored in the memory are executed, the processor is used to perform the following: obtaining pre-stored basic algorithm information and real-time performance status information of the electronic device; inputting the pre-stored basic algorithm information and the real-time performance status information of the electronic device into an algorithm scheduling module to obtain an algorithm scheduling table, wherein the algorithm scheduling table indicates that each algorithm occupies different computing units in the same frame; and scheduling at least two algorithms running in the electronic device according to the algorithm scheduling table to obtain output data of at least two algorithms.

[0052] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an algorithm scheduling device, the algorithm scheduling device executes any one of the algorithm scheduling methods in the first aspect.

[0053] In a fifth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an algorithm scheduling device, the algorithm scheduling device executes any one of the algorithm scheduling methods in the first aspect.

[0054] The algorithm scheduling method and electronic device provided in the embodiments of the present application obtain pre-stored algorithm basic information and real-time performance status information of the electronic device, and then input the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module for processing to obtain an algorithm scheduling table, and then schedule the algorithm in the running state according to the algorithm scheduling table, wherein the algorithm scheduling table used for algorithm scheduling is dynamically generated based on the pre-stored algorithm basic information and the real-time performance status information of the electronic device, so that the generated algorithm scheduling table can be more in line with the real-time performance status of the electronic device, avoiding the problem of unprepared algorithm scheduling caused by the mismatch between the pre-stored algorithm scheduling table and the real-time performance status of the electronic device, and improving the accuracy of scheduling multiple algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of an application scenario of an algorithm scheduling method;

[0056] Figure 2 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0057] Figure 3 is a schematic diagram of a software system for an electronic device applicable to the present application;

[0058] Figure 4 is a schematic diagram of an algorithm scheduling method provided in an embodiment of the present application;

[0059] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0060] Figure 6 is a schematic diagram of an algorithm scheduling method provided in an embodiment of the present application;

[0061] Figure 7 is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application;

[0062] Figure 8 It is a schematic diagram of an algorithm scheduling device provided by the present application;

[0063] Fig. 9 It is a schematic diagram of an electronic device for algorithm scheduling provided by the present application. DETAILED DESCRIPTION

[0064] To facilitate understanding, relevant terms and concepts that may be involved in the embodiments of the present application are first introduced below.

[0065] (1) Artificial intelligence (AI).

[0066] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0067] Machine learning is an important branch of artificial intelligence, and deep learning is an important branch of machine learning. Deep learning refers to the use of multi-layer neural network structures to learn from big data the representation of various things in the real world that can be directly used for computer calculations (for example, things in images, sounds in audio, etc.). In the field of image processing, deep learning has achieved excellent results in object detection, image generation, image segmentation and other problems.

[0068] (2) Deep neural networks.

[0069] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with multiple hidden layers. According to the position of different layers, the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer.

[0070] (3) Loss function

[0071] In the process of training deep neural networks, because we hope that the output of the deep neural network is as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0072] In some scenarios, if the user clicks on the camera application 101 on the main interface 10 of the electronic device, for example, Figure 1 In response to the user clicking the camera application 101, the display interface of the electronic device jumps from the main interface 10 to the camera preview interface 20, for example, Figure 1 As shown in (b) in FIG. 1 . At this time, the preview interface 20 of the camera includes a tracking frame 201, a photo control 202, and a night view control 203. It should be noted that the image optimization algorithms corresponding to different controls are usually different algorithms. For example, the image optimization algorithm corresponding to the photo control 202 may be a photo optimization algorithm, and the image optimization algorithm corresponding to the night view control 203 is a night view optimization algorithm. Usually, the default image optimization algorithm of the electronic device is a photo optimization algorithm. Therefore, when the preview interface 20 of the camera is jumped to in response to the user clicking the camera application 101, the preview interface 20 refers to the preview interface corresponding to the photo control 20, that is, the preview interface for image optimization based on the photo optimization algorithm. The tracking frame 201 that moves with the portrait avatar will also be displayed in the preview interface 20 to facilitate the user to collect the image of the target person. The electronic device can dynamically display the tracking frame 201 based on the image detection algorithm to mark the position of the portrait in the preview interface 20 of the camera.

[0073] In some possible cases, the image captured by the electronic device is a portrait of a person at night. Due to the poor light at night, the clarity of the captured portrait is poor, such as Figure 1 In this case, the user can switch from the photo scene to the night scene to optimize the image captured by the camera using the night scene optimization algorithm to improve the image clarity. Figure 1 As shown in (c) of FIG. 1 , the user clicks the night scene control 203. In response to the user clicking the night scene control 203, the display interface of the electronic device switches from the preview interface 20 of the photo scene to the preview interface 30 of the night scene scene, as shown in FIG. Figure 1 As shown in (d) in .

[0074] When displaying the preview interface 30 of the night scene, the electronic device needs to run at least two algorithms, namely, the image detection algorithm and the night scene optimization algorithm. Since the electronic device runs at least two algorithms, in order to avoid conflicts caused by multiple algorithms occupying the same computing unit at the same time, usually, an algorithm scheduling table of algorithms is pre-stored in the electronic device, and the electronic device can schedule multiple algorithms according to the pre-stored algorithm scheduling table to avoid conflicts in computing units.

[0075] The computing unit may include a central processing unit (CPU), a graphic processing unit (GPU) and a neural processing unit (NPU). Among them, the NPU is a microprocessor specifically used to process artificial neural network calculations and is an important part of artificial intelligence technology. The NPU simulates the working principle of the human brain neural network and uses large-scale parallel processing units and efficient interconnection structures to accelerate complex calculations such as large-scale matrix operations and convolution operations in deep neural networks. In the fields of face recognition, image search, etc., the NPU can accelerate the processing and analysis of image data and improve recognition accuracy and processing speed.

[0076] It is understandable that the scheduling of the computing units in the electronic device is performed according to the preset frame rate. For example, in the 0th frame, the electronic device runs algorithm A in the CPU, runs algorithm B in the GPU, and runs algorithm C in the NPU; in the 1st frame, the electronic device runs algorithm B in the CPU, runs algorithm C in the GPU, and runs algorithm A in the NPU.

[0077] If the electronic device runs algorithm A and algorithm B in the CPU at the same time in the i-th frame, algorithm A and algorithm B need to occupy the CPU at the same time, which will cause a computing unit conflict problem.

[0078] It should be noted that different algorithms can occupy computing units at different frame rates (Frames Per Second, FPS). FPS refers to the number of frames displayed per second, which is an important indicator for measuring the smoothness of video or animation. In electronic devices, the execution frame rate of the algorithm needs to match the refresh rate of the display device to avoid problems such as screen tearing, freezing or delay. For example, algorithm A can be executed by the CPU at a frame rate of 30FPS, that is, the electronic device can occupy the CPU to execute algorithm A in the 1st frame, the 3rd frame, the 5th frame, ..., the 59th frame.

[0079] The pre-stored algorithm scheduling table can be shown in Table 1. As shown in Table 1, the execution frame rate of algorithm A is 60 FPS, and the occupied computing unit is CPU; the execution frame rate of algorithm B is 30 FPS, and the occupied computing unit is CPU; the execution frame rate of algorithm C is 20 FPS, and the occupied computing units are CPU and NPU; the execution frame rate of algorithm D is 10 FPS, and the occupied computing units are CPU and NPU; the execution frame rate of algorithm E is 15 FPS, and the occupied computing units are CPU and NPU; the execution frame rate of algorithm F is 4 FPS, and the occupied computing units are CPU and NPU; the execution frame rate of algorithm G is 12 FPS, and the occupied computing units are CPU and NPU.

[0080] Table 1

[0081]

[0082] Taking the running of algorithm A as an example, the execution frame rate of algorithm A is 60 FPS, and the occupied computing unit is the CPU. That is to say, the electronic device can run algorithm A in the 1st frame, the 2nd frame, the 3rd frame... through the CPU.

[0083] Take the example of running Algorithm B and Algorithm C at the same time. The execution frame rate of Algorithm C is 10 FPS, and the occupied computing units are CPU and NPU. If Algorithm B and Algorithm C are executed at the same time, in the 4th frame, the problem of Algorithm B and Algorithm C occupying the same computing unit may occur. In this case, the electronic device can run Algorithm B on the CPU and Algorithm C on the NPU to avoid conflicts in computing units.

[0084] It is understandable that as electronic devices are used, the performance of the electronic devices will change, and the frame rate of the running algorithm will change. For example, as the electronic device is used, the remaining power of the electronic device becomes low. If the remaining power of the electronic device is less than the preset power threshold, the electronic device will reduce the frame rate of the algorithm, thereby reducing the power consumption of the electronic device and increasing the use time of the electronic device. In this case, the frame rate of each algorithm in the pre-stored algorithm scheduling table is different from the actual running frame rate of the algorithm, so there may be a problem of computing unit conflict according to the pre-stored algorithm scheduling table.

[0085] In this case, generally, multiple algorithm scheduling tables can be pre-stored in the electronic device according to the performance changes of the electronic device, and the multiple algorithm scheduling tables correspond to multiple performance states of the electronic device respectively. For example, two algorithm scheduling tables can be pre-stored in the electronic device, including algorithm scheduling table 1 and algorithm scheduling table 2, wherein algorithm scheduling table 1 is an algorithm scheduling table used when the remaining power of the electronic device is greater than a preset power threshold, and algorithm scheduling table 2 is an algorithm scheduling table used when the remaining power of the electronic device is less than or equal to the preset power threshold.

[0086] It should be noted that the performance of an electronic device is usually related to the remaining power of the electronic device, the operating temperature, and the load conditions of multiple computing units. The above selection of the corresponding algorithm scheduling table according to the remaining power is only an example. In actual use, the correspondence between multiple algorithm scheduling tables and multiple performance states of the electronic device is usually determined based on comprehensive considerations such as the remaining power of the electronic device, the operating temperature, and the load conditions of multiple computing units.

[0087] However, the algorithm scheduling table corresponding to the selection of the electronic device's performance is also a preset algorithm scheduling table, and the algorithm scheduling cannot be dynamically performed based on the actual performance status of the electronic device.

[0088] In view of this, an embodiment of the present application provides an algorithm scheduling method, which can dynamically perform algorithm scheduling based on the actual performance status of the electronic device, including: obtaining pre-stored algorithm basic information and real-time performance status information of the electronic device, inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into an algorithm scheduling module for processing, obtaining an algorithm scheduling table, and scheduling the algorithm in the running state according to the algorithm scheduling table, wherein the algorithm scheduling table used for algorithm scheduling is dynamically generated based on the pre-stored algorithm basic information and the real-time performance status information of the electronic device, so that the generated algorithm scheduling table can be more in line with the real-time performance status of the electronic device, avoiding the problem of unprepared algorithm scheduling due to the mismatch between the pre-stored algorithm scheduling table and the real-time performance status of the electronic device, and improving the accuracy of scheduling multiple algorithms.

[0089] Figure 2 A hardware system of an electronic device suitable for the present application is shown.

[0090] The electronic device 100 can be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc. The embodiment of the present application does not impose any limitation on the specific type of the electronic device 100.

[0091] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0092] It should be noted that Figure 1 The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 1 More or fewer components than those shown, or the electronic device 100 may include Figure 1 Combinations of some of the components shown, or the electronic device 100 may include Figure 1 Subassemblies of some of the components shown. Figure 1 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0093] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Different processing units may be independent devices or integrated devices.

[0094] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0095] In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0096] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal interface or as a data signal interface.

[0097] In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170 and the sensor module 180. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface or a MIPI interface.

[0098] The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0099] The display screen 194 can be used to display images or videos. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED (Micro OLED) or a quantum dot light emitting diode (QLED). In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0100] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0101] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithm optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0102] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red green blue (RGB), YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0103] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0104] NPU is a processor that draws on the structure of biological neural networks, such as the transmission mode between neurons in the human brain to quickly process input information, and can also continuously self-learn. Through NPU, the electronic device 100 can realize intelligent cognition and other functions, such as image recognition, face recognition, voice recognition and text understanding.

[0105] The hardware system of the electronic device 100 is described in detail above. In addition, an operating system is running in the electronic device 100. For example, the iOS operating system developed by Apple, the Android open source operating system developed by Google, the Windows operating system developed by Microsoft, etc. Application programs can be installed and run on the operating system.

[0106] The operating system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture or a cloud architecture. The embodiment of the present application takes the layered architecture as an example to exemplarily describe the software system of the electronic device 100.

[0107] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0108] The layered architecture of electronic devices divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system library, and kernel layer.

[0109] The application layer can include a series of application packages.

[0110] like Figure 3 As shown, the application package may include applications such as camera and algorithm scheduling modules.

[0111] The camera application can be used to capture images or videos and display a preview of the image or video on the display interface of the electronic device. When displaying a preview of an image or video, the electronic device usually optimizes the preview. In some possible cases, when capturing an image or video through the camera application, a tracking frame is also displayed on the preview screen, such as Figure 1 As shown in (b) in the figure, the image optimization processing of the preview image can be realized by running an image optimization algorithm, and the tracking frame can be displayed on the preview image by detecting the object of the preview image through an image detection algorithm, thereby realizing the display of the tracking frame.

[0112] The algorithm scheduling module can be used to obtain the performance status of the electronic device and basic algorithm information, and determine the algorithm scheduling table so that the camera application can call the corresponding algorithm based on the algorithm scheduling table to optimize the image and / or display the tracking frame when capturing images or videos.

[0113] Optionally, the algorithm scheduling module may include a decision submodule and a compensation submodule.

[0114] Among them, the decision submodule can be used to obtain the performance status of the electronic device and basic information of the algorithm, and generate an algorithm scheduling table.

[0115] It should be noted that the decision submodule can be a neural network model, and the performance status of the electronic device and the basic information of the algorithm are used as the input of the neural network model, and the algorithm scheduling table can be regarded as the output of the neural network model.

[0116] It should be noted that, in some possible cases, in order to avoid the problem of computing unit conflicts between multiple algorithms running simultaneously, the above-mentioned computing unit conflicts can be avoided by skipping frames when executing the algorithms. For example, if both algorithms A and B running simultaneously need to occupy the CPU in the third frame, in order to avoid the problem of computing unit conflicts, algorithm A can be run in the CPU in the third frame, and algorithm B is not run, that is, algorithm B skips frames in the third frame. The input data of the compensation submodule is the output data before the algorithm is interrupted, which may refer to the data output by algorithm B in the second frame. The supplementary submodule can simulate the processing of algorithm B in the third frame for the data output from the second frame to obtain the compensated output data, that is, the data output by algorithm B in the second frame is processed by the compensation submodule to simulate the output data of algorithm B in the third frame.

[0117] It can be understood that the compensation submodule can be a neural network model, which takes the output data before the algorithm is interrupted as input and outputs the compensated output data.

[0118] The decision submodule and the compensation submodule may be independent neural network models or two modules in the same neural network model, which is not limited in the embodiments of the present application.

[0119] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0120] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, an algorithm module, etc.

[0121] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0122] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0123] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0124] The algorithm module can store basic algorithm information, and the decision submodule can obtain the basic algorithm information by accessing the algorithm module.

[0125] The basic information of the algorithm can be shown in Table 2, including the computing units occupied by the algorithm, the minimum frame rate of running the algorithm, the time consumption of different computing units, the power consumption of the algorithm, and the peak memory of the algorithm.

[0126] Table 2

[0127]

[0128] For example, as shown in Table 2, algorithm A1 runs in the CPU, and the minimum frame rate for executing algorithm A1 is 30FPS. Since algorithm A1 runs only in the CPU, the time consumed by algorithm A1 in the CPU is X1ms, and the time consumed by algorithm A1 in the GPU and DSP is 0ms. The power consumption of the electronic device when running algorithm A1 is Y1mAh, and the maximum memory required by algorithm A1, which is also the peak memory of algorithm A1, is Z1KB.

[0129] Exemplarily, as shown in Table 2, algorithm A2 runs in the CPU and DSP, and the minimum frame rate for executing algorithm A2 is 20FPS. Since algorithm A2 can run in the CPU and DSP, the time consumption of algorithm A2 in the CPU is X2ms, the time consumption in the DSP is X3ms, and the time consumption of algorithm A2 in the GPU is 0ms. It should be noted that the time consumption of algorithm A2 in the CPU can be the same as or different from the time consumption of algorithm A2 in the DSP, and the embodiment of the present application does not limit this. The power consumption of the electronic device when running algorithm A2 is Y2 mAh, and the maximum memory required by algorithm A2, which is also the peak memory of algorithm A2, is Z2 KB.

[0130] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0131] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0132] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0133] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0134] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, sensor driver, etc.

[0135] It should be noted that the electronic device mentioned in the embodiments of the present application may include more or fewer modules in the above-mentioned electronic device.

[0136] The following is an illustrative description of the workflow of the software system and the hardware system of the electronic device 100 in conjunction with displaying a photo-taking scene.

[0137] When the user performs a touch operation on the touch sensor 180K, the corresponding hardware interrupt is sent to the kernel layer, and the kernel layer processes the touch operation into a raw input event, which includes, for example, information such as the touch coordinates and the timestamp of the touch operation. The raw input event is stored in the kernel layer, and the application framework layer obtains the raw input event from the kernel layer, identifies the control corresponding to the raw input event, and notifies the application (APP) corresponding to the control. For example, the above touch operation is a single-click operation, and the APP corresponding to the above control is a camera APP. After the camera APP is awakened by the single-click operation, the camera driver of the kernel layer can be called through the API, and the camera driver can be used to control the camera 193 to shoot.

[0138] The application scenarios of the embodiments of the present application are briefly described below.

[0139] The algorithm scheduling method of the present application can dynamically schedule at least two algorithms when an electronic device runs at least two algorithms simultaneously, thereby avoiding conflicts caused by at least two algorithms occupying the same computing unit.

[0140] For example, Figure 1 As shown, the algorithm scheduling method of the present application can be applied to the field of image processing. When an electronic device displays a preview screen through a camera application, the electronic device at least runs an image optimization algorithm and an image detection algorithm. By using the algorithm scheduling method of the present application, it is possible to avoid the image optimization algorithm and the image detection algorithm occupying the same computing unit at the same time, thereby avoiding the problem of computing unit conflicts.

[0141] Exemplarily, the algorithm scheduling method provided in the embodiment of the present application can also be applied in the field of speech recognition. When an electronic device recognizes address information through speech, at least a speech recognition algorithm and a natural language processing algorithm are run in the electronic device, wherein the speech recognition algorithm can convert the received speech information into text, and the natural language processing algorithm can determine the information indicated by the converted text. The algorithm scheduling method of the present application can avoid the speech recognition algorithm and the natural language processing algorithm from occupying the same computing unit at the same time, thereby avoiding the problem of computing unit conflict.

[0142] It should be understood that the above is an example of an application scenario and does not limit the application scenario of the present application.

[0143] Combine the following Figures 4 to 7 The algorithm scheduling method provided in the embodiment of the present application is described in detail.

[0144] Figure 4 is a flow chart of the algorithm scheduling method provided in the embodiment of the present application. Figure 4 As shown, the pre-stored algorithm basic information and the performance status information of the electronic device can be obtained, and the pre-stored algorithm basic information and the performance status information of the electronic device are input into the algorithm scheduling module for processing to obtain the algorithm scheduling table. The method includes the following steps:

[0145] S101, obtaining pre-stored algorithm basic information and real-time performance status information of the electronic device.

[0146] Among them, the pre-stored algorithm basic information can be stored in Figure 3 According to the algorithm module shown in , the electronic device can obtain the occupied computing units, the minimum frame rate of the algorithm, the time consumption of different computing units, the algorithm power consumption and the algorithm peak memory of each algorithm running in the electronic device before leaving the factory. For example, the electronic device can obtain the occupied computing units, the minimum frame rate of the algorithm, the time consumption of different computing units, the algorithm power consumption and the algorithm peak memory of each algorithm by testing each algorithm. For another example, the electronic device can also obtain the occupied computing units, the minimum frame rate of the algorithm, the time consumption of different computing units, the algorithm power consumption and the algorithm peak memory of each algorithm through simulation. Then the electronic device can store this information in the algorithm module to obtain the pre-stored algorithm basic information, as shown in Table 2. The algorithm scheduling module can request the algorithm module to obtain the pre-stored algorithm basic information. For example, when the camera application starts to display the preview screen, the camera application sends a request to the algorithm scheduling module to obtain the algorithm scheduling table, and the algorithm scheduling module responds to the request of the algorithm scheduling table and requests the algorithm module to obtain the pre-stored algorithm basic information.

[0147] The performance status information of the electronic device may be the remaining power of the electronic device at the current moment, the operating temperature, and the load information of each computing unit in the electronic device. The algorithm scheduling module may obtain the remaining power of the electronic device through the power management module 141 in the electronic device, read the temperature data in the temperature sensor 180J to obtain the operating temperature of the electronic device, read the load information of each computing unit in the electronic device, and obtain the performance status information of the electronic device. For example, when the camera application starts to display the preview screen, the camera application sends a request to the algorithm scheduling module to obtain the algorithm scheduling table, and the algorithm scheduling module responds to the request of the algorithm scheduling table to obtain the performance status information of the electronic device. In other words, the performance status information of the electronic device obtained by the algorithm scheduling module is the real-time performance status information of the electronic device.

[0148] S102: Input the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module for processing to obtain an algorithm scheduling table.

[0149] The pre-stored algorithm basic information and the performance status information of the electronic device are input into the algorithm scheduling module for processing to obtain an algorithm scheduling table. The algorithm scheduling module may be a neural network model. The pre-stored algorithm basic information and the performance status information of the electronic device are used as input to output an algorithm scheduling table. The algorithm scheduling table may be as shown in Table 3.

[0150] Table 3

[0151]

[0152] As shown in Table 3, in the 0th frame, algorithm A1 occupies the CPU to run, algorithm A2 can occupy the DSP to run, algorithm B does not occupy any computing unit, and algorithm C occupies the GPU to run; in the 1st frame, algorithm A1 does not occupy any computing unit, algorithm A2 can occupy the CPU to run, algorithm B can occupy the DSP to run, and algorithm C can occupy the GPU to run; in the 2nd frame, algorithm A1 occupies the CPU to run, algorithm A2 can occupy the DSP to run, algorithm B occupies the GPU to run, and algorithm C does not occupy any computing unit; in the 3rd frame, algorithm A1 occupies the CPU to run, algorithm A2 does not occupy any computing unit, algorithm B occupies the DSP to run, and algorithm C can occupy the GPU to run.

[0153] It can be seen from the above description that by using the algorithm scheduling table shown in Table 3 for algorithm scheduling, there is no situation where different algorithms occupy the same computing unit in the same frame.

[0154] It should be noted that in some frames, such as the second frame, since algorithm A1 only runs in the CPU, algorithm A1 occupies the CPU. At this time, algorithm A2 can run in the CPU and DSP. Since algorithm A1 has already occupied the CPU, algorithm A2 can choose to occupy the DSP when the CPU is occupied. Similarly, algorithm B can run in the GPU and DSP. Since algorithm A2 has already occupied the DSP, in order to avoid the computing unit conflict caused by occupying the DSP at the same time, algorithm B can occupy the GPU to run.

[0155] The electronic device schedules at least two algorithms running simultaneously according to the algorithm scheduling table to avoid conflicts caused by at least two algorithms occupying the same computing unit.

[0156] S103: Scheduling the algorithms in operation according to the algorithm scheduling table.

[0157] Scheduling the running algorithms according to the algorithm scheduling table may mean that the running algorithms in the electronic device may occupy the corresponding computing units according to the algorithm scheduling table. Since the algorithms in the algorithm scheduling table do not occupy the same computing unit in the same frame, the conflicts of computing units may be avoided when the algorithms occupy the corresponding computing resources according to the algorithm scheduling table.

[0158] The algorithm scheduling method provided in the embodiment of the present application obtains pre-stored algorithm basic information and real-time performance status information of the electronic device, inputs the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module for processing, obtains an algorithm scheduling table, and schedules the algorithms in running state according to the algorithm scheduling table, wherein the algorithm scheduling table used for algorithm scheduling is dynamically generated based on the pre-stored algorithm basic information and the real-time performance status information of the electronic device, thereby making the generated algorithm scheduling table more consistent with the real-time performance status of the electronic device, avoiding the problem of unprepared algorithm scheduling caused by the mismatch between the pre-stored algorithm scheduling table and the real-time performance status of the electronic device, and improving the accuracy of scheduling multiple algorithms.

[0159] It is understandable that the algorithm scheduling module can be a neural network model, and usually an initial neural network model needs to be pre-trained so that the accuracy of the algorithm scheduling table output by the algorithm scheduling module can meet the requirements.

[0160] In some possible cases, the electronic device may train the initial algorithm scheduling module according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain a trained algorithm scheduling module.

[0161] Among them, the electronic device can input the sample algorithm basic information and the sample real-time performance status information into the initial algorithm scheduling module to obtain the target algorithm scheduling table, and then compare the target algorithm scheduling table with the sample algorithm scheduling table, and adjust the loss function in the algorithm scheduling module based on the comparison result until the difference between the target algorithm scheduling table and the sample algorithm scheduling table is less than the preset threshold, completing the training of the initial algorithm scheduling module and obtaining the trained algorithm scheduling module.

[0162] Exemplarily, the algorithm scheduling module can be as follows Figure 5 As shown, it includes a decision submodule and an information decoder.

[0163] Among them, the information decoder can decode the input algorithm basic information and real-time performance status information and convert them into feature vectors.

[0164] The decision submodule can process the feature vector obtained by the conversion to obtain an algorithm scheduling table.

[0165] The initial training of the algorithm scheduling module may be completed before the electronic device leaves the factory, or may be performed during the use of the electronic device, and the embodiments of the present application do not impose any limitation on this.

[0166] For example, the electronic device can turn on the pre-training switch before leaving the factory, and use the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to train the initial algorithm scheduling module to obtain the trained algorithm scheduling module. Then, the electronic device can turn off the pre-training switch, so that when the electronic device is used by the user after leaving the factory, the computing unit will not be frequently occupied to train the algorithm scheduling module.

[0167] For another example, during the user's use, the electronic device can also turn on the pre-training switch at a preset interval, and use the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to train the initial algorithm scheduling module to obtain the trained algorithm scheduling module. It is understandable that as the use time of the electronic device increases, the hardware performance of the electronic device will also decrease. Therefore, regularly training the initial algorithm scheduling module to obtain the trained algorithm scheduling module can make the trained algorithm scheduling module more compatible with the hardware performance of the electronic device, thereby improving the accuracy of the trained algorithm scheduling module.

[0168] For another example, the electronic device may also turn on the pre-training switch each time the system version update information is pushed, and use the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to train the initial algorithm scheduling module to obtain the trained algorithm scheduling module. Since the system version update may affect the performance of the electronic device, each time the system version is pushed, the initial algorithm scheduling module is trained to obtain the trained algorithm scheduling module, which can make the trained algorithm scheduling module more compatible with the system version of the electronic device, thereby improving the accuracy of the trained algorithm scheduling module.

[0169] In some possible cases, at least two algorithms running at the same time need to occupy the same computing unit. At this time, the electronic device can avoid the above computing unit conflict problem by skipping frames. For example, if the algorithms A and B running at the same time both need to occupy the CPU in the third frame, in order to avoid the computing unit conflict problem, algorithm A can be run in the CPU in the third frame, and algorithm B is not run, that is, algorithm B skips the frame in the third frame. However, since algorithm B skips the frame in the third frame, one frame of processing is missing, and there may be a problem of reduced accuracy of algorithm B.

[0170] For example, if an image detection algorithm used to determine the face area in an image skips frames at the third frame, since the image detection algorithm usually processes the previous frame as the input of the current frame, if the output data of the third frame is missing, when the image detection algorithm is run at the fourth frame, since there is no output data of the third frame, the image detection algorithm may use the output data of the second frame for processing to obtain the output data of the fourth frame. This may cause the output of the image detection algorithm running on the fourth frame to be inaccurate, and thus cause the determined face area in the image to be inaccurate.

[0171] The electronic device can compensate the frame skipping algorithm through the compensation submodule in the algorithm scheduling module. For example, Figure 6 As shown, the algorithm scheduling module may further include a compensation submodule, and the compensation submodule may use the output data before the algorithm is interrupted as the input data of the compensation submodule, and simulate the output data before the algorithm is interrupted by the supplementary submodule to simulate the output data after the algorithm is compensated. For example, the compensation submodule may use the output data before the algorithm B is interrupted, that is, the data output by the algorithm B in the second frame, as the input data of the compensation submodule, and simulate the processing of the algorithm B in the third frame on the data output from the second frame by the compensation submodule to obtain the output data after compensation, that is, simulate the output data of the algorithm B in the third frame.

[0172] The output data before the algorithm is interrupted can be the intermediate data generated by the decision submodule when outputting the algorithm scheduling table. The decision submodule can send the generated intermediate data, that is, the output data before the algorithm is interrupted, to the compensation submodule, so that the compensation submodule performs compensation processing based on the output data before the algorithm is interrupted, and simulates the output data of the algorithm when it is interrupted. Figure 7 Let me explain in detail.

[0173] Figure 7 A flowchart of another algorithm scheduling method provided in an embodiment of the present application is provided. The method can be applied in the following situations: Figure 6 In the electronic device shown, Figure 7 As shown, the method includes:

[0174] S201. Obtain pre-stored algorithm basic information and real-time performance status information of the electronic device.

[0175] S202: input the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the decision submodule in the algorithm scheduling module for processing, and obtain the first data corresponding to the first algorithm and the algorithm scheduling table.

[0176] In order to avoid computing unit conflicts that occur when multiple algorithms are running simultaneously, the algorithm scheduling table may indicate that some of the algorithms perform frame skipping. For example, algorithm A1 shown in Table 3 skips frame 1.

[0177] The first algorithm may refer to an algorithm that needs to be adjusted, which is determined by the decision submodule in the process of obtaining the algorithm scheduling table, such as the algorithm A1 shown in Table 3.

[0178] The first data refers to the output data of the first algorithm before the frame skipping, for example, the output data of the algorithm A1 at the 0th frame as shown in Table 3.

[0179] It should be noted that the first algorithm may be a plurality of algorithms, for example, Algorithm A1, Algorithm A2, Algorithm B, and Algorithm C as described in Table 3. Different algorithms have different frame skipping numbers. For example, the frame skipping number of Algorithm A1 is the 1st frame, the frame skipping number of Algorithm A2 is the 3rd frame, the frame skipping number of Algorithm B is the 0th frame, and the frame skipping number of Algorithm C is the 2nd frame.

[0180] The decision submodule can determine the first algorithm corresponding to the frame number according to different frame numbers. For example, the first algorithm at frame 0 is algorithm B, the first algorithm at frame 1 is algorithm A1, the first algorithm at frame 2 is algorithm C, and the first algorithm at frame 3 is algorithm A2.

[0181] The algorithm module requests to obtain the output data of the algorithm A1 at the 0th frame, and sends the output data of the algorithm A1 at the 0th frame as the first data to the compensation submodule.

[0182] S203: Send the first data to the compensation submodule.

[0183] The first data corresponding to the first algorithm may be obtained from an algorithm module, and the algorithm module may monitor the running process of each algorithm and obtain the intermediate data of each algorithm.

[0184] For example, if the decision submodule determines that the first algorithm in the first frame is algorithm A1, the decision submodule can request the algorithm module to obtain the output data of algorithm A1 in the 0th frame, and send the output data of algorithm A1 in the 0th frame as the first data to the compensation submodule.

[0185] For another example, if the decision submodule determines that the first algorithm at the first frame is algorithm A1, the decision submodule can send indication information that the first algorithm is algorithm A1 to the compensation submodule. Based on the above indication information, the compensation submodule requests the algorithm module to obtain the output data of algorithm A1 at the 0th frame.

[0186] S204: The compensation submodule generates second data based on the first data.

[0187] The compensation submodule can also be a neural network model. For example, the compensation submodule is a lightweight Transformer model. The Transformer model is a deep learning model based on the self-attention mechanism. It uses the self-attention mechanism to capture the dependencies between the elements in the input sequence, thereby realizing the modeling of sequence data. The lightweight Transformer model refers to a type of model that is optimized based on the traditional Transformer structure to reduce the amount of calculation, model size and improve processing efficiency. The design goal of this type of model is mainly to solve the traditional Transformer model in terms of parameter quantity and computational complexity, and can be applied to electronic devices with limited computing resources.

[0188] In order to avoid conflicts in computing units caused by the compensation submodule occupying computing units, the compensation submodule can be run in computing units that process other tasks. For example, if the multiple algorithms to be scheduled are algorithms related to image processing, the compensation submodule can be run in EAI, where EAI can be a Qualcomm TM A computing unit for audio processing is provided, for example, EA1 may refer to a Hexagon ENPU, in which case the compensation submodule will not occupy the computing unit occupied by image processing, and thus the problem of computing unit conflict caused by the compensation submodule will not occur.

[0189] The first data can be used as input data of the compensation submodule, and after being processed by the compensation submodule, the second data is obtained, that is, the data generated by simulating the processing of the first algorithm during frame skipping. Based on this, the electronic device can use the second data as input data of the first algorithm after frame skipping to improve the accuracy of the first algorithm.

[0190] Exemplarily, if the decision submodule determines that the first algorithm at the first frame is algorithm A1, the decision submodule can request the algorithm module to obtain the output data of algorithm A1 at the 0th frame, and send the output data of algorithm A1 at the 0th frame as the first data to the compensation submodule. The compensation submodule can use the output data of algorithm A1 at the 0th frame as input to simulate the output data of the first algorithm at the 1st frame, that is, the second data. Then the second data is used as the input data of algorithm A1 at the 2nd frame to improve the accuracy of algorithm A1.

[0191] It is understandable that the compensation submodule is a neural network model, so the electronic device can first train the initial compensation submodule to obtain the trained compensation submodule, and then obtain the second data through the trained compensation submodule.

[0192] The electronic device can obtain the sample second data by inputting the sample first data into the initial compensation submodule training, and then compare the sample second data with the second data corresponding to the first data in the sample full-frame algorithm data, and adjust the loss function in the compensation submodule until the difference between the sample second data and the second data corresponding to the first data in the sample full-frame algorithm data is less than a preset threshold, thereby obtaining the trained compensation submodule. The full-frame algorithm data may refer to the input data and output data of each frame when the first algorithm is operated in each frame. The sample full-frame algorithm data may be the data obtained when the electronic device simulates the operation of the first algorithm without frame skipping.

[0193] For example, the electronic device can turn on the pre-training switch of the compensation submodule before leaving the factory, and use the sample full frame algorithm data and the corresponding sample first data to train the initial compensation submodule to obtain the trained compensation submodule. Then, the electronic device can turn off the pre-training switch, so that when the electronic device is used by the user after leaving the factory, the computing unit will not be frequently occupied to train the compensation submodule.

[0194] For another example, the electronic device can also turn on the pre-training switch of the compensation submodule at a preset interval during the user's use, and use the sample full-frame algorithm data and the corresponding sample first data to train the initial algorithm scheduling module to obtain the trained compensation submodule. It is understandable that as the use time of the electronic device increases, the hardware performance of the electronic device will also decrease. Therefore, regularly training the initial compensation submodule to obtain the trained compensation submodule can make the trained compensation submodule more compatible with the hardware performance of the electronic device, thereby improving the accuracy of the trained compensation submodule.

[0195] For another example, the electronic device may also turn on the pre-training switch of the compensation submodule each time the system version update information is pushed, and use the sample full frame algorithm data and the corresponding sample first data to train the initial compensation submodule to obtain the trained compensation submodule. Since the system version update may affect the performance of the electronic device, each time the system version is pushed, the initial compensation submodule is trained to obtain the trained compensation submodule, which can make the trained compensation submodule more compatible with the system version of the electronic device, thereby improving the accuracy of the trained compensation submodule.

[0196] S205. Schedule the algorithm in operation according to the algorithm scheduling table, and obtain output data of the first algorithm based on the second data.

[0197] The second data obtained by simulating the frame skipping state of the first algorithm through the compensation submodule is used as input data of a frame after the frame skipping of the first algorithm, which can improve the accuracy of the first algorithm.

[0198] It can be understood that there may be multiple frame skips in the first algorithm. The compensation submodule can simulate each frame skip respectively to obtain the second data corresponding to each frame skip, thereby realizing compensation for multiple frame skips, so that the first algorithm can obtain the output data of the first algorithm based on the second data of each frame after compensation, which is equivalent to simulating the output data obtained by running the first algorithm in full frame, and can further improve the accuracy of the first algorithm.

[0199] The algorithm scheduling method provided in the embodiment of the present application can simulate the frame skipping of the first algorithm through a compensation submodule to obtain output data of the frame skipping of the first algorithm, and then obtain the output data of the first algorithm based on the simulated output data of the first algorithm after the frame skipping, so that the output data of the first algorithm that runs according to the algorithm scheduling table and has frame skipping is closer to the output data obtained by the full frame running of the first algorithm, thereby improving the accuracy of the first algorithm.

[0200] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0201] Combination of the above Figures 1 to 7 , describes in detail the algorithm scheduling method of the embodiment of the present application, and will be combined with Figure 8 and Fig. 9 , describes the device embodiment of the present application in detail. It should be understood that the algorithm scheduling device in the embodiment of the present application can execute the various algorithm scheduling methods in the aforementioned embodiment of the present application, that is, the specific working process of the following various products can refer to the corresponding process in the aforementioned method embodiment.

[0202] Figure 8 It is a schematic diagram of an algorithm scheduling device provided in an embodiment of the present application.

[0203] It should be understood that the algorithm scheduling device 600 can execute Figures 4 to 7 The algorithm scheduling method shown; the algorithm scheduling device 600 includes: an acquisition unit 610 and a processing unit 620.

[0204] In one example, the acquisition unit 610 is used to acquire pre-stored algorithm basic information and real-time performance status information of the electronic device; input the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module to obtain an algorithm scheduling table, wherein the algorithm scheduling table indicates that each algorithm occupies different computing units in the same frame; and schedule at least two algorithms running in the electronic device according to the algorithm scheduling table to obtain output data of at least two algorithms.

[0205] It should be noted that the algorithm scheduling device 600 is implemented in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0206] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0207] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0208] Fig. 9 A structural schematic diagram of an electronic device provided by the present application is shown. Fig. 9 The dotted line in the figure indicates that the unit or the module is optional. The electronic device 700 can be used to implement the algorithm scheduling method described in the above method embodiment.

[0209] The electronic device 700 includes one or more processors 701, which can support the algorithm scheduling method in the method embodiment of the electronic device 700. The processor 701 can be a general-purpose processor or a special-purpose processor. For example, the processor 701 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0210] The processor 701 may be used to control the electronic device 700, execute software programs, and process data of the software programs. The electronic device 700 may also include a communication unit 705 to implement input (reception) and output (transmission) of signals.

[0211] For example, the electronic device 700 may be a chip, the communication unit 705 may be an input and / or output circuit of the chip, or the communication unit 705 may be a communication interface of the chip, and the chip may be a component of a terminal device or other electronic devices.

[0212] For another example, the electronic device 700 may be a terminal device, and the communication unit 705 may be a transceiver of the terminal device, or the communication unit 705 may be a transceiver circuit of the terminal device.

[0213] The electronic device 700 may include one or more memories 702 on which a program 704 is stored. The program 704 can be executed by the processor 701 to generate instructions 703, so that the processor 701 executes the algorithm scheduling method described in the above method embodiment according to the instructions 703.

[0214] Optionally, data may be stored in the memory 702. Optionally, the processor 701 may read data stored in the memory 702. The data may be stored at the same storage address as the program 704, or may be stored at a different storage address than the program 704.

[0215] The processor 701 and the memory 702 may be provided separately or integrated together; for example, integrated on a system on chip (SOC) of a terminal device.

[0216] Exemplarily, the memory 702 can be used to store the related program 704 of the algorithm scheduling method provided in the embodiment of the present application, and the processor 701 can be used to call the related program 704 of the algorithm scheduling method stored in the memory 702 when performing image restoration on the terminal device, and execute the algorithm scheduling method of the embodiment of the present application; for example, obtaining pre-stored algorithm basic information and real-time performance status information of the electronic device; inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module to obtain an algorithm scheduling table, wherein the algorithm scheduling table indicates that the various algorithms occupy different computing units in the same frame; scheduling at least two algorithms running in the electronic device according to the algorithm scheduling table to obtain output data of at least two algorithms.

[0217] The present application also provides a computer program product, which, when executed by the processor 701, implements the algorithm scheduling method described in any method embodiment of the present application.

[0218] The computer program product may be stored in the memory 702 , for example, a program 704 , which is converted into an executable target file that can be executed by the processor 701 after preprocessing, compiling, assembling, and linking.

[0219] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the algorithm scheduling method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0220] The computer-readable storage medium is, for example, a memory 702. The memory 702 may be a volatile memory or a nonvolatile memory, or the memory 702 may include both a volatile memory and a nonvolatile memory. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0221] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0222] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0223] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0224] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0225] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0226] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0227] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0228] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An algorithm scheduling method, characterized in that: The method is applied to an electronic device, the electronic device comprising an algorithm scheduling module and at least two computing units, and the method comprises: Acquiring pre-stored algorithm basic information and real-time performance status information of the electronic device; Inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the algorithm scheduling module to obtain an algorithm scheduling table, wherein the algorithms indicated in the algorithm scheduling table occupy different computing units in the same frame; Scheduling at least two algorithms running in the electronic device according to the algorithm scheduling table to obtain output data of the at least two algorithms; The algorithm scheduling module includes a decision submodule, and the pre-stored algorithm basic information and the real-time performance status information of the electronic device are input into the algorithm scheduling module to obtain an algorithm scheduling table, including: Inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the decision submodule to obtain an algorithm scheduling table; The algorithm scheduling module includes a compensation submodule, and the method further includes: Acquire first data of a first algorithm, where the first algorithm is a frame skipping algorithm among at least two algorithms running in the electronic device, and the first data refers to output data of the first algorithm in a frame before the frame skipping; Inputting the first data into the compensation submodule to obtain second data, where the second data is output data obtained by simulating the operation of the first algorithm during frame skipping; The output data of the first algorithm is obtained according to the second data.

2. The method according to claim 1, characterized in that The method further comprises: The initial decision submodule is trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain a trained decision submodule; The step of inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the decision submodule to obtain an algorithm scheduling table includes: The pre-stored algorithm basic information and the real-time performance status information of the electronic device are input into the trained decision submodule to obtain an algorithm scheduling table.

3. The method according to claim 2, characterized in that The initial decision submodule is trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain the trained decision submodule, including: In the first period, the initial decision submodule is trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain a trained decision submodule; The step of inputting the pre-stored algorithm basic information and the real-time performance status information of the electronic device into the trained decision submodule to obtain an algorithm scheduling table includes: In the second period, the pre-stored algorithm basic information and the real-time performance status information of the electronic device are input into the trained decision submodule to obtain an algorithm scheduling table; The first time period and the second time period are different time periods, and the first time period is before the second time period.

4. The method according to claim 2, characterized in that: The initial decision submodule is trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain the trained decision submodule, including: At preset time intervals, the initial decision submodule is periodically trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain a trained decision submodule.

5. The method according to claim 2, characterized in that: The initial decision submodule is trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain the trained decision submodule, including: When the electronic device receives a system version update, the initial decision submodule is trained according to the sample algorithm scheduling table and the corresponding sample algorithm basic information and sample real-time performance status information to obtain a trained decision submodule.

6. The method according to claim 1, characterized in that The electronic device further includes an algorithm module, and the acquiring the first data of the first algorithm includes: If the decision submodule determines that the first algorithm skips frames, the decision submodule sends a first acquisition request to the algorithm module, where the first acquisition request is used to request the algorithm module to acquire the first data; The algorithm module sends the first data to the decision submodule in response to the first acquisition request; The decision submodule sends the first data to the compensation submodule.

7. The method according to claim 1, characterized in that The electronic device further includes an algorithm module, and the acquiring the first data of the first algorithm includes: If the decision submodule determines that the first algorithm skips frames, the decision submodule sends first indication information to the compensation submodule, where the first indication information is used to indicate the frame skipping of the first algorithm; The compensation submodule sends a second acquisition request to the algorithm module based on the first indication information, where the second acquisition request is used to request the algorithm module to acquire the first data; The algorithm module sends the first data to the compensation submodule in response to the second acquisition request.

8. The method according to claim 1, characterized in that The method further comprises: The initial compensation submodule is trained according to the sample full frame algorithm data and the corresponding sample first data to obtain a trained compensation submodule; The step of inputting the first data into the compensation submodule to obtain second data includes: The first data is input into the trained compensation submodule to obtain second data.

9. The method according to claim 8, characterized in that The initial compensation submodule is trained according to the sample full frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule, including: In a third period, the initial compensation submodule is trained according to the sample full frame algorithm data and the corresponding sample first data to obtain a trained compensation submodule; The step of inputting the first data into the compensation submodule to obtain second data includes: Inputting the first data into the compensation submodule in a fourth time period to obtain second data; The third time period and the fourth time period are different time periods, and the third time period is before the fourth time period.

10. The method according to claim 8, characterized in that The initial compensation submodule is trained according to the sample full frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule, including: At preset time intervals, the initial compensation submodule is periodically trained according to the sample full-frame algorithm data and the corresponding sample first data to obtain a trained compensation submodule.

11. The method according to claim 8, characterized in that The initial compensation submodule is trained according to the sample full frame algorithm data and the corresponding sample first data to obtain the trained compensation submodule, including: When the electronic device receives a system version update, the initial compensation submodule is trained according to the sample full-frame algorithm data and the corresponding sample first data to obtain a trained compensation submodule.

12. An electronic device, characterized in that: The electronic device comprises an algorithm scheduling module for executing the method as claimed in any one of claims 1 to 11.

13. The electronic device according to claim 12, characterized in that: The electronic device includes an algorithm scheduling module and an algorithm module, wherein the algorithm scheduling module is used to obtain an algorithm scheduling table based on pre-stored algorithm basic information in the algorithm module and real-time performance status information of the electronic device, wherein the algorithms indicated in the algorithm scheduling table occupy different computing units in the same frame.

14. The electronic device according to claim 13, characterized in that: The algorithm scheduling module includes a decision submodule, and the decision submodule is used to obtain an algorithm scheduling table based on pre-stored algorithm basic information in the algorithm module and real-time performance status information of the electronic device.

15. The electronic device according to claim 13, characterized in that: The algorithm scheduling module also includes a compensation submodule, which is used to obtain second data based on first data of a first algorithm, wherein the first algorithm is a frame skipping algorithm among at least two algorithms running in the electronic device, the first data refers to output data of the first algorithm in a frame before the frame skipping, and the second data is output data obtained by simulating the operation of the first algorithm during frame skipping.

16. An algorithm scheduling device, characterized in that: The algorithm scheduling device includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the algorithm scheduling device executes the algorithm scheduling method described in any one of claims 1 to 11.

17. A chip, characterized in that: The method comprises a processor, and when the processor executes instructions, the processor executes the algorithm scheduling method according to any one of claims 1 to 11.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the algorithm scheduling method according to any one of claims 1 to 11.

Images