Computing power distribution method, computing power distribution system and electronic equipment

Abstract

The invention relates to a computing power distribution method, a computing power distribution system and electronic equipment. The computing power distribution method for image transformation comprises the following steps: acquiring real-time state information of to-be-processed data and a plurality of computing resources; based on the real-time state information of the plurality of computing resources, disassembling an image change algorithm corresponding to the to-be-processed data to generate a computing power disassembling result; generating a computing power distribution scheme based on the computing power disassembly result, the image information of the to-be-processed data and the real-time state information of the plurality of computing resources; and performing image transformation processing on the to-be-processed data based on the computing power distribution scheme.

Description

Technical Field

[0001] The present disclosure relates to the technical field of computing power load scheduling, and in particular to a computing power distribution method, a computing power distribution system, and an electronic device. Background Art

[0002] There are many steps in the camera pipeline that require image transformation processing on the images collected by the Image Signal Processor (ISP). These image transformation processes require the use of a variety of image-related algorithms for calculation.

[0003] At present, system-level chips (SOCs) usually use dedicated hardware image processing modules for image transformation processing. If there is no dedicated hardware image transformation unit, the graphics processing unit (GPU) in the system-level chip can be used for image transformation processing. However, the resources of the graphics processor are limited, and in some complex scenes, the graphics processor load may be too high, causing the system or camera to freeze. Therefore, it is necessary to reasonably allocate the computing power resources of the system-level chip to improve the smoothness of device operation. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a computing power allocation method, a computing power allocation system, and an electronic device to reasonably allocate computing power resources and improve performance such as fluency.

[0005] The technical solution of the present invention is achieved in this way:

[0006] The present disclosed embodiment discloses a computing power allocation method, a computing power allocation method for image transformation, comprising: obtaining real-time status information of data to be processed and multiple computing resources; based on the real-time status information of the multiple computing resources, disassembling the image change algorithm corresponding to the data to be processed to generate a computing power disassembly result; based on the computing power disassembly result, the image information of the data to be processed and the real-time status information of the multiple computing resources, generating a computing power allocation plan; based on the computing power allocation plan, performing image transformation processing on the data to be processed.

[0007] In the above scheme, the real-time status information includes: computing power and computing priority; based on the real-time status information of multiple computing resources, the image change algorithm corresponding to the data to be processed is disassembled to generate the computing power disassembly result, including: disassembling the scene of the image change algorithm into multiple basic scenes; wherein each of the basic scenes corresponds to multiple basic operation algorithms; based on the scene of the image change algorithm and the computing power and the computing priority of multiple computing resources, one basic operation algorithm is selected from the multiple basic operation algorithms corresponding to each basic scene; the selected basic operation algorithms are sorted according to the amount of operation to generate the computing power disassembly result.

[0008] In the above scheme, the real-time status information also includes: power consumption; based on the computing power decomposition result, the image information of the data to be processed and the real-time status information of multiple computing resources, the computing power allocation plan is generated, including: based on the image information of the data to be processed and the computing power decomposition result, determining multiple image processing tasks for performing the image change processing on the data to be processed; based on the computing power, the computing priority and the power consumption of multiple computing resources, selecting the computing resource combination of each image processing task to generate the computing power allocation plan.

[0009] In the above scheme, based on the computing power allocation scheme, the image transformation processing is performed on the data to be processed, including: based on the computing power allocation scheme, distributing the image processing task to the corresponding computing resources; judging whether the computing resources can meet the computing power requirements of the image processing task; if not, replacing the computing resources or waiting according to a preset replacement strategy.

[0010] In the above scheme, based on the computing power allocation scheme, the image processing tasks are distributed to the corresponding computing resources, including: sorting multiple image processing tasks according to task priority and dependency; based on the task priority and the computing power and power consumption of multiple computing resources, each image processing task is allocated to the corresponding computing resource.

[0011] In the above scheme, judging whether the computing resources can meet the computing power requirements of the image processing task includes: judging whether the load of the computing resources is greater than the estimated load of the data to be processed; if the load of the computing resources is greater than the estimated load of the data to be processed, the computing resources cannot meet the computing power requirements of the image processing task; wherein the estimated load is determined based on the computing power allocation scheme; or, judging whether the power consumption of the computing resources is greater than a preset power consumption threshold; if the power consumption of the computing resources is greater than the preset power consumption threshold, the computing resources cannot meet the computing power requirements of the image processing task.

[0012] The disclosed embodiment also provides a computing power allocation system, a hardware computing power unit, comprising a plurality of computing resources; a hardware support capability management module, configured to obtain real-time status information of the plurality of computing resources; a scene decision and reuse management module, configured to receive the data to be processed and the real-time status information of the plurality of computing resources; based on the real-time status information of the plurality of computing resources, the image change algorithm corresponding to the data to be processed is disassembled to generate a computing power disassembly result; and, based on the computing power disassembly result, the image information of the data to be processed and the real-time status information of the plurality of computing resources, a computing power allocation plan is generated; the hardware computing power unit is configured to receive and, based on the computing power allocation plan, perform image transformation processing on the data to be processed.

[0013] The above scheme also includes: a distributed parallel computing processing module, which is configured to distribute the image processing task to the corresponding computing resources based on the computing power allocation scheme; and to determine whether the computing resources can meet the computing power requirements of the image processing task; if not, replace the computing resources or wait according to a preset replacement strategy.

[0014] In the above scheme, the distributed parallel computing processing module includes: a load judgment submodule and a power consumption judgment submodule; the load judgment submodule is configured to judge whether the load of the computing resource is greater than the estimated load of the data to be processed; wherein the estimated load is determined based on the computing power allocation scheme; the power consumption judgment submodule is configured to judge whether the power consumption of the computing resource is greater than a preset power consumption threshold.

[0015] In the above scheme, the scene decision and reuse management module includes: a scene decomposition submodule, configured to decompose the scene of the image change algorithm into multiple basic scenes; wherein each of the basic scenes has corresponding multiple basic operation algorithms; a computing power decomposition submodule, configured to select one basic operation algorithm from the multiple basic operation algorithms corresponding to each basic scene based on the scene of the image change algorithm and the computing power and the computing priority of the multiple computing resources; and, sort the selected basic operation algorithms according to the amount of calculation to generate the computing power decomposition result; a computing power evaluation submodule, configured to determine multiple image processing tasks for performing the image change processing on the data to be processed based on the image information of the data to be processed and the computing power decomposition result; and, based on the computing power, the computing priority and the power consumption of the multiple computing resources, select the computing resource combination of each image processing task to generate the computing power allocation plan.

[0016] An embodiment of the present disclosure also provides an electronic device, comprising: a memory configured to store executable instructions; and a processor configured to implement the computing power allocation method described in any of the above schemes of the claims when executing the executable instructions stored in the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the process of computing power allocation method provided in the embodiment of the present disclosure Figure 1 ;

[0018] Figure 2 A schematic diagram of the structure of the computing power allocation system provided in the embodiment of the present disclosure Figure 1 ;

[0019] Figure 3 A schematic diagram of the decomposition of a computing scenario provided in an embodiment of the present disclosure;

[0020] Figure 4 Schematic diagram of the process of computing power allocation method provided in the embodiment of the present disclosure Figure 2 ;

[0021] Figure 5 A schematic diagram of a computing power allocation solution provided in an embodiment of the present disclosure;

[0022] Figure 6 Schematic diagram of the process of computing power allocation method provided in the embodiment of the present disclosure Figure 3 ;

[0023] Figure 7 A schematic diagram of the structure of the computing power allocation system provided in the embodiment of the present disclosure Figure 2 ;

[0024] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present disclosure. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0026] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0027] If similar descriptions of "first / second" appear in the application documents, the following description is added. In the following description, the terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0029] Figure 1 It is a flowchart of an optional computing power allocation method provided in an embodiment of the present disclosure. Figure 2 is a schematic diagram of the structure of an optional computing power allocation system 100 provided in an embodiment of the present disclosure. Figure 1 The computing power allocation method shown can be achieved by Figure 2 The computing power allocation system 100 shown is implemented and will be described in conjunction with each step.

[0030] S101. Obtaining real-time status information of data to be processed and multiple computing resources.

[0031] In the embodiments of the present disclosure, reference Figure 2 , the computing power allocation system 100 may be a system-level chip. The computing power allocation system 100 may include a hardware computing power unit 10. A plurality of computing resources may be set in the hardware computing power unit 10. The computing resources may include a graphics processor, a central processing unit (CPU), a digital signal processor (DSP), a visual processing unit (VPU), a neural network processor (NPU), etc.

[0032] In the embodiments of the present disclosure, reference Figure 2, the hardware support capability management module 20 can obtain the real-time status information of multiple computing resources from the hardware computing power unit 10. The real-time status information of computing resources may include computing power, power consumption and computing priority. The hardware support capability management module 20 can be directly obtained in real time from the hardware computing power unit 10. The hardware support capability management module 20 can also obtain the initial computing power information of each computing power unit, and then update the real-time status information in real time based on the tasks and initial computing power information assigned to each computing resource during the processing process. For example, computing resources such as graphics processors and central processing units will have a default initial computing power value. Then, as the processing task proceeds, the computing power value consumed by the current task, at this time, the difference between the initial computing power value and the consumed computing power value is the support capability of the computing resource.

[0033] It should be noted that load refers to the workload of computing resources or the number of tasks being processed. Power consumption refers to the amount of electricity consumed by computing resources when they are running. Computing power refers to the ability of computing resources to perform computing tasks. The units of measurement for computing power include Million Instructions Per Second (MIPS) and Floating Point Operations Per Second (FLOPS).

[0034] In the embodiments of the present disclosure, reference Figure 2 , the power consumption of each computing resource can be more accurately evaluated by constructing a model that combines computing power and power consumption. For example, in a graphics processor, the relationship between computing power and power consumption can be estimated by the frequency of the graphics processor. Thus, a computing power consumption model related to the frequency of the graphics processor can be constructed. Furthermore, by running various image transformation processing algorithms on the graphics processor, the support capabilities of the graphics processor for the image transformation processing algorithms at different frequencies (such as computing power utilization, idle state, operation time and power consumption, etc.) are counted. Then, the constructed computing power consumption model and support capabilities can be saved in a database. The hardware support capability management module 20 can call the computing power consumption model and support capabilities saved in the database to evaluate the power consumption and computing power of graphics processors of different frequencies. The models of computing resources such as central processing units, digital signal processors, visual processing units, and neural network processors can be understood with reference to graphics processors, which will not be repeated here.

[0035] In the embodiments of the present disclosure, reference Figure 2, the computing priority of each computing resource can be achieved by setting a priority weight for each computing resource according to different computing scenarios. Graphics processors are more suitable for parallel computing of big data. Central processing units are more suitable for general computing. Digital signal processors are more suitable for tasks that require complex mathematical operations and signal processing. Visual processing units are more suitable for scenarios such as video processing and accelerated computing. Neural network processors are suitable for neural network operations. The priority weights of multiple computing resources can be set with different priority weights according to their applicable and inapplicable computing scenarios, and saved in the database. For example, a graphics processor can have a higher priority weight for parallel computing of big data. Then, the hardware support capability management module 20 can call the priority weights in the database to obtain the computing priorities of multiple computing resources.

[0036] S102: Based on the real-time status information of multiple computing resources, the image change algorithm corresponding to the data to be processed is decomposed to generate a computing power decomposition result.

[0037] In the embodiments of the present disclosure, reference Figure 2 , the image transformation processing process covers many complex image processing scenarios, and the computing power consumption model and support capabilities are models generated based on running different image transformation processing algorithms. In order to accurately evaluate the computing power required for image change processing of the data to be processed, the scene decision and reuse management module 30 can refine the complex scenes of image transformation processing into several basic scenes. Then, a corresponding basic operation algorithm is selected for each basic scene, and the selected multiple basic operation algorithms are sorted according to the calculation order of multiple basic scenes, that is, the computing power decomposition result is generated. The calculation order of multiple basic scenes can be shown in Table 1 below:

[0038] Basic scene types Basic scene calculation order crop and resize crop_resize crop, flip and resize crop_resize_flip Crop, rotate, and resize crop_resize_rotate Crop, resize, flip, and rotate crop_resize_rotate_flip

[0039] Table 1

[0040] It should be noted that the basic scenes in Table 1 include rotation (Rotate), flip (Flip), crop (Crop) and scaling (Resize). Cropping is to crop a part of the image area, which is usually used to adjust the size of the image or remove unnecessary parts. Scaling is to change the size of the image, which can enlarge or reduce the image. Rotation is to rotate the image at a certain angle, which can achieve the flipping of the image or adjust the direction. Flipping is to flip the image horizontally or vertically, which can achieve a mirror effect or adjust the direction of the image. The order of priority in the basic scene calculation order is the basic scene calculation order. For example, "crop_resize" means cropping first and then scaling.

[0041] It should also be noted that the basic scenario may also include other scenarios, and the calculation order of the basic scenarios may also be other sorting methods, which are not limited here.

[0042] S103: Generate a computing power allocation plan based on the computing power decomposition result, the image information of the data to be processed, and the real-time status information of multiple computing resources.

[0043] In the embodiments of the present disclosure, reference Figure 2 , the image information of the data to be processed may include key information such as the size and magnification of the processed image. The scene decision and reuse management module 30 generates a computing power allocation plan based on the computing power decomposition results, the image information of the data to be processed, and the real-time status information of multiple computing resources. For example, the scene decision and reuse management module 30 can obtain the computing priority and computing power of multiple computing resources and the size of the image, and allocate tasks for each basic scene in the computing power decomposition results, that is, select the computing resources for the tasks of each basic scene, and generate a computing power allocation plan.

[0044] The following uses the computing power allocation of various basic scenarios as an example to explain the principle of the computing power allocation solution:

[0045] Cropping mainly involves selecting a subset of an image, which usually does not require a large amount of computing resources, and the central processing unit can handle such tasks efficiently. If multiple cropping operations need to be processed in parallel, the parallel computing capabilities of the graphics processor can complete it more efficiently. However, the image signal processor that captures the image usually comes with the function of cropping operations and does not need to occupy computing resources. Therefore, when the scene decision and reuse management module 30 allocates the cropping scene for computing power allocation, first, the task of cropping the scene is assigned to the image signal processor. If the image signal processor is occupied, the priority of the central processing unit and the image processor is determined based on whether there are parallel processing operations.

[0046] Flipping and rotating scenes involve a simple rearrangement of image data. The graphics processor has a computing priority for processing large-sized images. The priority of the neural network processor and the central processing unit is increased to have a computing priority for processing small-sized images. Therefore, the scene decision and reuse management module 30 can choose to use the graphics processor, the neural network processor or the central processing unit for processing according to the size of the processed image.

[0047] Scaling a scene involves changing the size of an image, and usually requires interpolation of image data. If there is a computing resource dedicated to scaling a scene among multiple computing resources, the scene decision and reuse management module 30 can arrange the task of scaling the scene in the computing resource. If the computing resource is occupied or there is no computing resource dedicated to scaling a scene, the scene decision and reuse management module 30 can allocate the task of scaling the scene according to the computing priority order of the neural network processor, the graphics processor, and the digital signal processor.

[0048] S104: Based on the computing power allocation plan, image transformation processing is performed on the data to be processed.

[0049] In the embodiments of the present disclosure, reference Figure 2 The distributed parallel computing processing module 40 can perform image transformation processing on the data to be processed based on the computing power allocation scheme. For example, the distributed parallel computing processing module 40 can use a task queue to manage the tasks to be processed. The task queue is sorted according to priority, dependency, etc., and dynamically scheduled according to the availability of the hardware computing unit 10 and the task priority to ensure the orderly execution of the tasks.

[0050] It can be understood that the image change algorithm corresponding to the data to be processed is disassembled in the embodiment of the present disclosure. In this way, the present disclosure can distribute the task of image change processing of the data to be processed to multiple different computing resources according to the computing power disassembly result after the image change algorithm is disassembled. Therefore, compared with the solution of using only one computing resource for processing in the related art, the embodiment of the present disclosure can distribute the load generated by the data to be processed to multiple computing resources, avoiding overloading of a single computing resource and causing jamming. At the same time, multiple computing resources can process tasks in parallel, which can reduce the time required to complete the task, improve the overall utilization efficiency of computing resources, and provide customers with a smoother and more efficient image processing experience.

[0051] In some embodiments of the present disclosure, Figure 3 S201~S203 in the implementation Figure 1 S102 in the embodiment will be described in combination with each step.

[0052] S201, decomposing the scene of the image change algorithm into multiple basic scenes; wherein each basic scene corresponds to multiple basic operation algorithms.

[0053] Figure 4 is an optional image processing schematic diagram provided by the embodiment of the present disclosure. Figure 4 The image change processing of the camera involves zooming in on the scene to perform an example of scene decomposition:

[0054] In the embodiments of the present disclosure, reference Figure 2The image transformation processing scenes may include camera application scenes such as face detection scenes, video recording scenes, and photo shooting scenes. The image transformation processing scenes may also include system scenes such as game screen rendering scenes. The scene decision and reuse management module 30 may decompose scenes such as face detection scenes, video recording scenes, photo shooting scenes, and game screen rendering scenes into multiple basic scenes, and select a basic operation algorithm corresponding to each basic scene.

[0055] For example, for some complex computing scenarios in the camera scene (such as the combination of rotation, flipping and zooming), zooming is not a basic scenario when performing computing power decomposition. Therefore, zooming can be first decomposed into cropping and scaling. The scene decision and reuse management module 30 can be based on Figure 4 As shown, the zoom scene from image A to image C is split, that is, image A is first cropped to form image B, and then image B is scaled (Resized) to form image C.

[0056] S202: Based on the scenario of the image change algorithm and the computing power and computing priority of multiple computing resources, a basic operation algorithm is selected from multiple basic operation algorithms corresponding to each basic scenario.

[0057] In the embodiments of the present disclosure, reference Figure 2 , each basic scene has corresponding multiple basic operation algorithms. For example, the basic operation algorithms for the zoom scene include bilinear interpolation algorithm (Bilinear Interpolation), bicubic interpolation algorithm (Bicubic Interpolation) and super-resolution algorithm based on neural network. The power consumption and operation time of the super-resolution algorithm, bicubic interpolation algorithm and bilinear interpolation algorithm decrease in turn. In the face detection data stream scenario, you can choose the bilinear interpolation algorithm with smaller power consumption and operation time. In the ultra-clear mode application scenario of the camera, you can choose the super-resolution algorithm. In the general preview and recording scenes of the camera, you can choose the cubic interpolation algorithm. The algorithm selection for the remaining basic scenes can be understood by referring to the zoom scene, which will not be repeated here.

[0058] S203. Sort the selected multiple basic computing algorithms according to the amount of computing, and generate a computing power decomposition result.

[0059] In the present disclosure, Figure 2As shown in Table 1, since the amount of calculation for rotation and flipping is significantly reduced after cropping, the computing power in this scenario is decomposed as follows: crop first, then rotate and flip, and finally scale. Then, a combination of basic computing algorithms corresponding to cropping, rotation, flipping, and scaling is obtained, that is, the computing power decomposition result is obtained. In other words, the computing power decomposition result generated by the embodiment of the present disclosure can reduce the amount of calculation, further reduce the time required to complete the task, and improve the overall utilization efficiency of computing resources.

[0060] In some embodiments of the present disclosure, Figure 6 S301 to S302 shown are implemented Figure 1 S103 in.

[0061] S301. Determine a plurality of image processing tasks for performing image change processing on the data to be processed based on the image information of the data to be processed and the computing power decomposition result.

[0062] S302: Based on the computing power, computing priority and power consumption of multiple computing resources, a computing resource combination for each image processing task is selected to generate a computing power allocation plan.

[0063] Figure 5 is a structural diagram of an optional method for generating a computing power allocation solution provided in an embodiment of the present disclosure. It should be noted that: Figure 5 The percentage in is used to indicate the computing power utilization of the corresponding computing resources.

[0064] In the embodiments of the present disclosure, reference Figure 2 The scene decision and reuse management module 30 can determine multiple image processing tasks for image change processing of the data to be processed based on the image information of the data to be processed and the computing power decomposition result. For example, the image of the data to be processed is a small-sized image. The computing power decomposition scenarios of the data to be processed include cropping, flipping and scaling.

[0065] Furthermore, the scene decision and reuse management module 30 can select a computing resource combination for each image processing task based on the computing power, computing priority and power consumption of multiple computing resources, and generate a computing power allocation plan. For example, the scene decision and reuse management module 30 can evaluate the computing power required for each image processing task based on the computing power consumption model. Then, based on the support capabilities of multiple computing resources, the computing resources for each image processing task are determined. The image processing tasks of the cropped scene can be divided into Figure 5 The combination 1 shown is processed by a digital signal processor. The image processing task of the flip scene can be performed according to Figure 5 The combination 2 shown is processed by the central processing unit. The image processing task of scaling can be performed according to Figure 5 The combination 3 shown is processed by a graphics processor.

[0066] In the embodiments of the present disclosure, reference Figure 2 The scene decision and reuse management module 30 can also split the calculations in each image processing task based on the computing power, computing priority and power consumption of multiple computing resources, obtain the computing resource combination, and generate a computing power allocation plan. For example, when processing YUV data, "Y" represents luminance information, while "U" and "V" represent chrominance information. Figure 5 If the GPU is overloaded during YUV data processing, you can Figure 5 The combination 4 shown in the figure processes YUV data by means of a graphics processor and a digital signal processor. The scene decision and multiplexing management module 30 can allocate the Y channel of the YUV data to the digital signal processor for calculation. As for the UV channel, a graphics processor is selected for processing. Then, after the data of the Y channel and the UV channel are processed, the results are merged to form the final image processing output. In this way, the embodiment of the present disclosure can further split the image transformation processing of the data to be processed, thereby further distributing the load and improving the overall utilization efficiency of computing resources.

[0067] In some embodiments of the present disclosure, S401 to S402 can also be used to implement Figure 1 S104 in.

[0068] S401. Distribute image processing tasks to corresponding computing resources based on a computing power allocation plan.

[0069] S402: Determine whether the computing resources can meet the computing power requirements of the image processing task; if not, replace the computing resources according to a preset replacement strategy or wait.

[0070] In the embodiments of the present disclosure, reference Figure 2 , the distributed parallel computing processing module 40 can distribute each image processing task to the corresponding computing resource according to the computing power allocation plan. Then, the distributed parallel computing processing module 40 can continuously obtain the real-time status information such as power consumption, priority and load provided by the hardware support capability module 20, and determine whether the support capability provided by each computing resource in the computing power allocation plan can meet the actual computing power requirements. In the event that any computing resource is not satisfied, the distributed parallel computing processing module 40 can wait or replace the computing resource for the image processing task corresponding to the computing resource. For example, the distributed parallel computing processing module 40 can detect the power consumption and load of each computing resource. If any one of the power consumption and load of each computing resource exceeds a preset threshold (for example, 95%), it will wait or replace the computing resource for processing.

[0071] In some embodiments of the present disclosure, S401 may also be implemented through S501 to S502, which will be described in conjunction with each step.

[0072] S501 , sorting a plurality of image processing tasks according to task priorities and dependencies.

[0073] S502: Allocate each image processing task to a corresponding computing resource based on the task priority and the computing power and power consumption of the plurality of computing resources.

[0074] In the embodiments of the present disclosure, reference Figure 2 In the case where there are multiple complex scenes in the data to be processed, the same basic scene will include multiple image processing tasks. Therefore, the distributed parallel computing processing module 40 can use the task queue to manage the image processing tasks to be processed. The task queue can be sorted according to priority and dependency. For example, the priority of the face detection scene is greater than the priority of the photographed image transformation data. The distributed parallel computing processing module 40 can set the image processing tasks corresponding to the face detection scene in the order of the task queue before the image processing tasks of the preview and recording scenes.

[0075] In some embodiments of the present disclosure, S402 may also be implemented through S601 or S602, which will be described in conjunction with each step.

[0076] S601. Determine whether the load of the computing resources is greater than the estimated load of the data to be processed; if the load of the computing resources is greater than the estimated load of the data to be processed, the computing resources cannot meet the computing power requirements of the image processing task; wherein the estimated load is determined based on the computing power allocation plan.

[0077] In the embodiments of the present disclosure, reference Figure 2 , the distributed parallel computing processing module 40 can obtain the current load of each computing resource. The distributed parallel computing processing module 40 can set an upper threshold of the workload for computing resources such as the central processing unit, the graphics processing unit, and the digital signal processor. If the estimated load of the image processing task exceeds the upper threshold of the corresponding computing resource, it is necessary to reallocate the computing resource or wait.

[0078] S602: Determine whether the power consumption of the computing resource is greater than a preset power consumption threshold; if the power consumption of the computing resource is greater than the preset power consumption threshold, the computing resource cannot meet the computing power requirement of the image processing task.

[0079] In the embodiments of the present disclosure, reference Figure 2 The distributed parallel computing processing module 40 can detect the power consumption of each computing resource. If the power consumption of any computing resource exceeds a preset threshold, the image processing task of the computing resource can be allocated to one or more other computing resources with lower power consumption.

[0080] That is to say, reference Figure 2 If the distributed parallel computing processing module 40 determines that either the power consumption or the load of the computing resources is greater than the corresponding preset value, it means that the computing resources cannot meet the computing power requirements of the image processing task.

[0081] Figure 7 is a schematic diagram of the structure of an optional computing power allocation system 100 provided in an embodiment of the present disclosure. It should be noted that: Figure 7 The computing resources of the hardware computing unit 10 shown include an image processor 11, a central processing unit 12, and a digital signal processor 13. The hardware computing unit 10 may also include other types of computing resources, which are not limited here.

[0082] In the embodiments of the present disclosure, reference Figure 7 The computing power allocation system 100 includes a hardware support capability management module 20. The hardware support capability management module 10 is configured to obtain real-time status information of multiple computing resources. For example, the hardware support capability management module 10 can query the support capabilities of multiple computing resources for image processing transformation. The hardware support capability management module 10 can query the support specifications of multiple computing resources, that is, the support capabilities of multiple computing resources for image processing.

[0083] In the embodiments of the present disclosure, reference Figure 7 The computing power allocation system 100 includes a scene decision and reuse management module 30. The scene decision and reuse management module 30 is configured to receive real-time status information of the data to be processed and multiple computing resources. The scene decision and reuse management module 30 is configured to disassemble the image change algorithm corresponding to the data to be processed based on the real-time status information of multiple computing resources, and generate a computing power disassembly result. For example, the scene decision and reuse management module 30 gives priority to using a computing method with a higher comprehensive priority based on real-time status information such as priority weight coefficients of different computing resources, combined with computing power requirements and power consumption requirements, so as to obtain a better allocation plan.

[0084] In the embodiments of the present disclosure, reference Figure 7 The scene decision and reuse management module 30 is configured to generate a computing power allocation plan based on the computing power decomposition results, the image information of the data to be processed, and the real-time status information of multiple computing resources. In this way, the present disclosure can allocate computing power based on the computing power decomposition results. Thus, the load generated by the data to be processed can be distributed to multiple computing resources to avoid overloading a single computing resource. At the same time, multiple computing resources can process tasks in parallel, which can reduce the time required to complete tasks, improve the overall utilization efficiency of computing resources, and provide customers with a smoother and more efficient image processing experience.

[0085] In the embodiments of the present disclosure, reference Figure 7 The hardware computing unit 10 is configured to receive and perform image transformation processing on the data to be processed based on the computing power allocation plan.

[0086] In some embodiments of the present disclosure, reference Figure 7 The computing power allocation system also includes a distributed parallel computing processing module 40. The distributed parallel computing processing module 40 is configured to distribute the image processing task to the corresponding computing resources based on the computing power allocation scheme. The distributed parallel computing processing module 40 is configured to determine whether the computing resources can meet the computing power requirements of the image processing task; if not, the computing resources are replaced or waited according to a preset replacement strategy.

[0087] In some embodiments of the present disclosure, reference Figure 7 The distributed parallel computing processing module 40 includes a load judgment submodule 41 and a power consumption judgment submodule 42 .

[0088] The load judgment submodule 41 is configured to judge whether the load of the computing resource is greater than the estimated load of the data to be processed. The estimated load is determined based on the computing power allocation scheme. In other words, the distributed parallel computing processing module 40 can avoid overloading a single computing resource during the actual calculation according to the computing power allocation scheme based on the load priority principle.

[0089] In the embodiments of the present disclosure, reference Figure 7 The power consumption judgment submodule 42 is configured to judge whether the power consumption of the computing resource is greater than a preset power consumption threshold. For example, if the power consumption of any computing resource is greater than the preset power consumption threshold, it means that the computing resource cannot support the image processing task being processed. In other words, the distributed parallel computing processing module 40 can avoid excessive power consumption of a single computing resource and affect the performance of the computing resource in the process of actual calculation according to the computing power allocation plan based on the load priority principle.

[0090] In some embodiments of the present disclosure, reference Figure 7 The scene decision and reuse management module 30 includes a scene disassembly submodule 31, a computing power disassembly submodule 32, a computing power evaluation submodule 33 and a load evaluation submodule 34. The scene disassembly submodule 31 is configured to disassemble the scene of the image change algorithm into multiple basic scenes; wherein each basic scene has corresponding multiple basic operation algorithms.

[0091] In the embodiments of the present disclosure, reference Figure 7The computing power disassembly submodule 32 is configured to select a basic operation algorithm from multiple basic operation algorithms corresponding to each basic scenario based on the scene of the image change algorithm and the computing power and computing priority of multiple computing resources; and to sort the selected basic operation algorithms according to the amount of operation to generate a computing power disassembly result. In other words, the computing power disassembly submodule 32 can combine the computing power disassembly result and the load of multiple computing resources to evaluate the computing resource support capacity that can be used, and further subdivide and disassemble the computing power demand.

[0092] In the embodiments of the present disclosure, reference Figure 7 The load evaluation submodule 34 is configured to obtain the load conditions of the current multiple computing resources before starting the image transformation processing task, so as to accurately evaluate the currently available idle hardware computing resources. Thus, in the process of generating the computing power allocation plan, it is ensured that the computing plan in the computing power decomposition result can be supported and the computing resources can be reasonably allocated.

[0093] In the embodiments of the present disclosure, reference Figure 7 , the computing power evaluation submodule 33 is configured to determine multiple image processing tasks for image change processing of the data to be processed based on the image information of the data to be processed and the computing power decomposition results; and, based on the computing power, computing priority and power consumption of multiple computing resources, select the computing resource combination of each image processing task to generate a computing power allocation plan. In other words, the computing power evaluation submodule 33 can comprehensively evaluate the computing power required for the image transformation scene in combination with the characteristics of the image processing transformation algorithm and key information such as the size and magnification of the processed image. Thereby, it ensures that the selected computing resources can meet the performance requirements in a specific scenario, while avoiding waste of resources, reducing processing power consumption, and providing a smoother and more efficient image processing experience.

[0094] Figure 8 An optional structural diagram of an electronic device 200 provided in an embodiment of the present application, refer to Figure 8, the hardware entity of the electronic device 200 includes: a processor 201, a memory 202 and a bus 203. The processor 201 and the memory 202 are connected to the bus 203 through an interface. The processor 201 generally controls the overall operation of the electronic device 200. The bus 203 enables the electronic device 200 to communicate with other devices or equipment through a network. The memory 202 is configured to store instructions and applications executable by the processor 101, and can also cache data to be processed or processed by the processor 201 and each module in the electronic device 200 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). When the processor 201 is configured to execute the executable instructions stored in the memory 202, the computing power allocation method in the above embodiment is implemented.

[0095] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0096] The serial numbers of the embodiments of the present disclosure are for description only and do not represent the advantages and disadvantages of the embodiments. The methods disclosed in the several method embodiments provided by the present disclosure can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several product embodiments provided by the present disclosure can be arbitrarily combined without conflict to obtain new product embodiments. The features disclosed in the several method or device embodiments provided by the present disclosure can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0097] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for allocating computing power for image transformation, comprising: Obtain real-time status information of data to be processed and multiple computing resources; Based on the real-time status information of the plurality of computing resources, the image change algorithm corresponding to the data to be processed is disassembled to generate a computing power disassembly result; Generate a computing power allocation plan based on the computing power decomposition result, the image information of the data to be processed and the real-time status information of the plurality of computing resources; Based on the computing power allocation scheme, image transformation processing is performed on the data to be processed.

2. The computing power allocation method according to claim 1, wherein: The real-time status information includes: computing power and computing priority; Based on the real-time status information of the plurality of computing resources, the image change algorithm corresponding to the data to be processed is decomposed to generate the computing power decomposition result, including: Decomposing the scene of the image change algorithm into a plurality of basic scenes; wherein each of the basic scenes corresponds to a plurality of basic operation algorithms; Based on the scenario of the image change algorithm and the computing power and the computing priority of the plurality of computing resources, selecting one of the plurality of basic computing algorithms corresponding to each of the basic scenarios; The selected basic computing algorithms are sorted according to the amount of computing, and the computing power decomposition result is generated.

3. The computing power allocation method according to claim 2, wherein: The real-time status information also includes: power consumption; Based on the computing power decomposition result, the image information of the data to be processed and the real-time status information of the plurality of computing resources, generating the computing power allocation plan includes: Based on the image information of the data to be processed and the computing power decomposition result, determine a plurality of image processing tasks for performing the image change processing on the data to be processed; Based on the computing power, computing priority and power consumption of the plurality of computing resources, a computing resource combination for each image processing task is selected to generate the computing power allocation plan.

4. The computing power allocation method according to claim 3, wherein: Based on the computing power allocation scheme, the image transformation processing is performed on the data to be processed, including: Based on the computing power allocation scheme, the image processing task is distributed to the corresponding computing resources; Determine whether the computing resources can meet the computing power requirements of the image processing task; if not, replace the computing resources according to a preset replacement strategy or wait.

5. The computing power allocation method according to claim 4, wherein: Based on the computing power allocation scheme, distributing the image processing task to the corresponding computing resources includes: Sorting the plurality of image processing tasks according to task priorities and dependencies; Based on the task priority and the computing power and the power consumption of the plurality of computing resources, each of the image processing tasks is allocated to the corresponding computing resource.

6. The computing power allocation method according to claim 4, wherein: Determining whether the computing resources can meet the computing power requirements of the image processing task includes: Determine whether the load of the computing resource is greater than the estimated load of the data to be processed; if the load of the computing resource is greater than the estimated load of the data to be processed, the computing resource cannot meet the computing power requirement of the image processing task; wherein the estimated load is determined based on the computing power allocation scheme; or, Determine whether the power consumption of the computing resource is greater than a preset power consumption threshold; if the power consumption of the computing resource is greater than the preset power consumption threshold, the computing resource cannot meet the computing power requirement of the image processing task.

7. A computing power allocation system, comprising: Hardware computing unit, including multiple computing resources; A hardware support capability management module is configured to obtain real-time status information of a plurality of computing resources; The scene decision and reuse management module is configured to receive the data to be processed and the real-time status information of the plurality of computing resources; based on the real-time status information of the plurality of computing resources, the image change algorithm corresponding to the data to be processed is disassembled to generate a computing power disassembly result; And, generating a computing power allocation plan based on the computing power disassembly result, the image information of the data to be processed and the real-time status information of the plurality of computing resources; The hardware computing power unit is configured to receive and perform image transformation processing on the data to be processed based on the computing power allocation plan.

8. The computing power allocation system according to claim 7, wherein: Also includes: The distributed parallel computing processing module is configured to distribute the image processing task to the corresponding computing resources based on the computing power allocation plan; and determine whether the computing resources can meet the computing power requirements of the image processing task; if not, replace the computing resources or wait according to a preset replacement strategy.

9. The computing power allocation system according to claim 8, wherein: The distributed parallel computing processing module includes: a load determination submodule and a power consumption determination submodule; The load judgment submodule is configured to judge whether the load of the computing resource is greater than the estimated load of the data to be processed; wherein the estimated load is determined based on the computing power allocation scheme; The power consumption determination submodule is configured to determine whether the power consumption of the computing resource is greater than a preset power consumption threshold.

10. The computing power allocation system according to claim 7, wherein: The scenario decision and reuse management module includes: A scene decomposition submodule, configured to decompose the scene of the image change algorithm into a plurality of basic scenes; wherein each of the basic scenes has a corresponding plurality of basic operation algorithms; The computing power decomposition submodule is configured to select one basic operation algorithm from the multiple basic operation algorithms corresponding to each basic scenario based on the scenario of the image change algorithm and the computing power and the computing priority of the multiple computing resources; and sort the selected basic operation algorithms according to the amount of operation to generate the computing power decomposition result; The computing power evaluation submodule is configured to determine multiple image processing tasks for performing the image change processing on the data to be processed based on the image information of the data to be processed and the computing power decomposition result; and, based on the computing power, computing priority and power consumption of multiple computing resources, select a computing resource combination for each image processing task to generate the computing power allocation plan.

11. An electronic device, wherein: include: a memory configured to store executable instructions; The processor is configured to implement the computing power allocation method described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.

Images