Object processing method and device, electronic device, and storage medium
By allocating appropriate target computing engines for each type of processing algorithm, the problems of uneven resource allocation and limitations in the prior art are solved, the utilization efficiency and accuracy of computing resources are improved, and the processing efficiency and performance are improved.
Patent Information
- Application Number
- CN202111434793.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-11-29
AI Technical Summary
In the prior art, the uneven allocation of computing resources of artificial intelligence algorithms leads to waste of resources and poor accuracy, and each DSP can only perform one task, which has certain limitations.
By obtaining the attribute information of multiple types of processing algorithms, the target computing engine is allocated to each type of processing algorithm from multiple computing engines and dedicated computing engines in the engine resource pool to achieve resource integration and coordination and avoid resource allocation. Both and limitations.
It improves the allocation balance and accuracy of computing resources, reduces the computing resource requirements during processing, reduces power consumption, and improves the processing efficiency and computing performance of the network model.
Smart Images

Figure CN114090262B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to an object processing method, an object processing device, an electronic device, and a computer-readable storage medium. Background Art
[0002] With the widespread application of artificial intelligence technology, in order to achieve good results, software algorithms require the integration of multiple algorithms to complete each application scenario.
[0003] In the related art, artificial intelligence algorithms are generally calculated through artificial intelligence acceleration circuits and DSPs (Digital signal processing), and other algorithms are processed through their respective corresponding DSPs.
[0004] In the above method, there is a waste of resources during the execution of the algorithm, resulting in uneven resource allocation. In addition, each DSP can only perform one task, which has certain limitations and poor accuracy.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0006] The purpose of the present disclosure is to provide an object processing method and device, an electronic device, and a storage medium, thereby overcoming the problem of uneven resource allocation caused by the limitations and defects of related technologies at least to a certain extent.
[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.
[0008] According to one aspect of the present disclosure, there is provided an object processing method, comprising: obtaining multiple types of processing algorithms included in processing operations on an object to be processed; allocating a target computing engine for each type of processing algorithm from candidate computing engines based on attribute information of the multiple types of processing algorithms; the candidate computing engines include multiple computing engines in an engine resource pool and dedicated computing engines; based on the target computing engine, performing a corresponding processing algorithm on the object to be processed to execute a processing operation, and obtaining an operation result corresponding to the object to be processed.
[0009] According to one aspect of the present disclosure, there is provided an object processing device, comprising: an algorithm acquisition module, used to acquire multiple types of processing algorithms included in processing operations on an object to be processed; a computing engine allocation module, used to allocate a target computing engine for each type of processing algorithm from a candidate computing engine based on attribute information of the multiple types of processing algorithms; the candidate computing engines include multiple computing engines in an engine resource pool and a dedicated computing engine; a processing operation execution module, used to perform a processing operation on the object to be processed based on the target computing engine using the corresponding processing algorithm to obtain an operation result corresponding to the object to be processed.
[0010] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned object processing methods by executing the executable instructions.
[0011] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the object processing method described in any one of the above is implemented.
[0012] In the object processing method, object processing device, electronic device and computer-readable storage medium provided in the embodiment of the present disclosure, a target computing engine is allocated to each type of processing algorithm through the attribute information of multiple types of processing algorithms; the computing engine includes multiple computing engines and a dedicated computing engine in the engine resource pool; and the processing operation is performed based on the target computing engine. On the one hand, since a suitable target computing engine can be allocated to each type of processing algorithm from the candidate computing engines composed of multiple computing engines and a dedicated computing engine in the engine resource pool according to the attribute information of multiple types of processing algorithms, since the engines corresponding to all types of processing algorithms are integrated, and then a suitable target computing engine is selected from the integrated candidate computing engines, the limitation that only one computing engine can be used for processing operations in the related art is avoided, and the target computing engine can be accurately allocated to improve accuracy. On the other hand, all computing engines can be coordinated uniformly, avoiding the singleness of each computing engine performing a task, improving the balance of resource allocation, realizing resource sharing of multiple computing engines, and then being able to quickly and accurately operate the object to be processed according to the target computing engine, and reducing the computing resources required in the processing process, reducing power consumption, and improving the processing efficiency and computing performance of the network model.
[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0015] Figure 1 A schematic diagram is shown of an application scenario to which the object processing method or object processing device according to an embodiment of the present disclosure can be applied.
[0016] Figure 2 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown.
[0017] Figure 3 A schematic diagram of an architecture suitable for implementing related technologies is shown.
[0018] Figure 4 A schematic diagram schematically illustrates an object processing method in an embodiment of the present disclosure.
[0019] Figure 5 The schematic diagram of the architecture in the embodiment of the present disclosure is schematically shown.
[0020] Figure 6 The flowchart of allocating a target computing engine in an embodiment of the present disclosure is schematically shown.
[0021] Figure 7 The flowchart of allocating a target computing engine to a first type of processing algorithm in an embodiment of the present disclosure is schematically shown.
[0022] Figure 8 The application diagram of the target computing engine in the embodiment of the present disclosure is schematically shown.
[0023] Fig. 9 The overall process diagram of allocating computing engines in the embodiment of the present disclosure is schematically shown.
[0024] Fig.10 A block diagram of an object processing device in an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0025] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.
[0026] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0027] The present disclosure provides an object processing method. Figure 1 A schematic diagram is shown of an application scenario to which the object processing method or object processing device according to an embodiment of the present disclosure can be applied.
[0028] refer to Figure 1 As shown in , the client 101 can be various types of devices with computing functions, such as smart phones, tablet computers, desktop computers, vehicle-mounted devices, wearable devices, etc. The object to be processed 102 can be a video, an image, or data, etc. The client 101 can obtain multiple types of processing algorithms included in the processing operation of the object to be processed; through the attribute information of the multiple types of processing algorithms, a target computing engine is assigned to each type of processing algorithm from the selected computing engines; and then based on the target computing engine, a corresponding operation is performed on the object to be processed 102 to perform the processing operation, and the operation result corresponding to the object to be processed 102 is obtained.
[0029] It should be noted that the object processing method provided in the embodiment of the present disclosure can be completely executed by the client. Accordingly, the object processing device can be set in the client.
[0030] Figure 2Schematic diagram of an electronic device suitable for implementing an exemplary embodiment of the present disclosure is shown. The terminal of the present disclosure may be configured as follows Figure 2 The form of the electronic device shown, however, needs to be explained. Figure 2 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0031] The electronic device of the present disclosure includes at least a processor and a memory, wherein the memory is used to store one or more programs. When the one or more programs are executed by the processor, the processor can implement the method of the exemplary embodiment of the present disclosure.
[0032] Specifically, Figure 2 As shown, the electronic device 200 may include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a button 294, and a Subscriber Identification Module (SIM) card interface 295, etc. The sensor module 280 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor, etc.
[0033] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0034] The processor 210 may include one or more processing units, for example: the processor 210 may include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor and / or a neural network processor (Neural-etwork Processing Unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. In addition, a memory can also be set in the processor 210 for storing instructions and data. The model training method in this exemplary embodiment can be executed by an application processor, a graphics processor or an image signal processor, and when the method involves neural network-related processing, it can be executed by an NPU.
[0035] The internal memory 221 can be used to store computer executable program codes, which include instructions. The internal memory 221 can include a program storage area and a data storage area. The external memory interface 222 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200.
[0036] The communication function of the mobile terminal 200 can be implemented by a mobile communication module, an antenna 1, a wireless communication module, an antenna 2, a modulation and demodulation processor, and a baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module can provide 2G, 3G, 4G, 5G and other mobile communication solutions applied to the mobile terminal 200. The wireless communication module can provide wireless communication solutions such as wireless LAN, Bluetooth, and near field communication applied to the mobile terminal 200.
[0037] The display screen is used to implement display functions, such as displaying user interfaces, images, and videos. The camera module is used to implement shooting functions, such as shooting images and videos. The audio module is used to implement audio functions, such as playing audio and collecting voice. The power module is used to implement power management functions, such as charging the battery, powering the device, and monitoring the battery status.
[0038] The present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device.
[0039] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.
[0040] Computer-readable storage media can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0041] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments.
[0042] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0043] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0044] refer to Figure 3The architecture diagram shown in Figure A in the figure, where the total acceleration unit of the artificial intelligence algorithm is the general NPU (Neural Network Processing Unit, embedded neural network processor), as shown in the Generic NPU in the figure. It mainly completes the acceleration implementation of the artificial intelligence network. The artificial intelligence acceleration circuit module is a dedicated acceleration module for artificial intelligence operators, which mainly accelerates relatively fixed operators in the algorithm network, such as convolution and pooling operators. In order to adapt to different network algorithms and maintain sufficient flexibility, the Generic NPU adds a DSP module to handle tasks in the artificial intelligence algorithm network that are not suitable for hardware acceleration. For traditional image processing algorithms and audio processing algorithms, due to the diversity of their algorithms and their wide variety, DSP is generally used to complete them, among which Video DSP completes visual algorithm processing and Audio DSP is used to complete audio data processing. Refer to Figure 3 As shown in Figure B, the processing algorithm includes multiple types of processing algorithms. When distributing tasks, the overall computing engine is allocated according to the multiple types of processing algorithms, for example, only the DSP computing engine is allocated, or only the artificial intelligence acceleration circuit computing engine is allocated.
[0045] Regarding the architecture in related technologies, the algorithm cannot share resources during execution, resulting in a certain degree of resource waste. For example, if the algorithm that currently needs to be processed by the Video DSP is relatively complex, but the DSP inside the AudioDSP and Generic NPU is in a light-loaded operation scenario, the hardware resources of each algorithm cannot be shared. Therefore, from an overall perspective, the separate engine design has a certain degree of uneven resource allocation. The embedded neural network processor NPU is mainly used for hardware acceleration of artificial intelligence network algorithms to process massive multimedia data such as videos and images.
[0046] In order to solve the above technical problems, an object processing method is provided in the embodiment of the present disclosure. Figure 4 The object processing method in the embodiment of the present disclosure is described in detail.
[0047] In step S410, multiple types of processing algorithms included in the processing operation on the object to be processed are obtained.
[0048] In the embodiments of the present disclosure, it can be specifically applied to the application process of the neural network model. The object to be processed can be a video, an image, or other types of objects, etc., which are specifically determined according to the application scenario. The processing operation can be determined according to the actual application scenario. For example, when the actual application scenario is a recognition scenario, the processing operation can be a recognition operation; when the actual application scenario is a classification scenario, the processing operation can be a classification operation, etc.
[0049] In the process of performing a processing operation on an object to be processed, multiple types of processing algorithms may be included. That is, multiple types of processing algorithms are combined to complete the processing operation. For example, in the process of identifying an object to be processed, processing algorithms of types such as artificial intelligence algorithms, image processing algorithms, and audio processing algorithms may be included but are not limited to. Among them, the artificial intelligence algorithm can be identified as a first type of processing algorithm, and the image processing algorithm and the audio processing algorithm can be identified as a second type of processing algorithm.
[0050] In step S420, a target computing engine is allocated to each type of processing algorithm from candidate computing engines through the attribute information of the multiple types of processing algorithms; the candidate computing engines include multiple computing engines in the engine resource pool and dedicated computing engines.
[0051] In the disclosed embodiment, after obtaining multiple types of processing algorithms, the multiple types of processing algorithms can be segmented according to the operator type to be divided into fixed algorithms and non-fixed algorithms. The operator type can be a fixed operator or a non-fixed operator. The fixed algorithm can be an algorithm that can use hardware acceleration, such as a fixed operator in an artificial intelligence algorithm, convolution, pooling, etc. Non-fixed algorithms refer to algorithms that cannot use hardware acceleration, including non-fixed operators in artificial intelligence algorithms and all second-type processing algorithms, such as non-linear activation functions in artificial intelligence algorithms, audio algorithms, visual algorithms, etc.
[0052] When the processing algorithm is an artificial intelligence algorithm, it can be divided into multiple operators. The multiple operators may include fixed operators or reference type operators (i.e., non-fixed operators). For example, when the processing algorithm is an artificial intelligence algorithm, the artificial intelligence algorithm can be divided into fixed operators, such as convolution, pooling, activation, etc. However, the operators used by different artificial intelligence algorithms may also be slightly different. For example, the activation function may use the simpler Relu, or the nonlinear sigmoid, or tanh function.
[0053] After the artificial intelligence algorithm is divided into multiple operators, similarities of the multiple processing algorithms can be extracted, and specifically, non-fixed operators used to represent reference type operators in the multiple processing algorithms can be extracted.
[0054] The attribute information is used to describe the state of the operators contained in each type of processing algorithm. For the first type of processing algorithm, its attribute information can be determined according to the complexity. The complexity is determined by the operators contained in the first type of processing algorithm, and can be specifically determined according to the number of reference type operators contained in the first type of processing algorithm. The reference type operator can be an operator that cannot be accelerated using dedicated hardware, that is, a variable non-fixed operator, such as a non-linear activation function or other operator that cannot be accelerated using dedicated hardware of an artificial intelligence acceleration circuit module. Moreover, the complexity is positively correlated with the number of reference type operators contained in the first type of processing algorithm. That is, the more reference type operators are included, the greater the complexity; the fewer reference type operators are included, the smaller the complexity. For the second type of processing algorithm, its attribute information is determined according to its corresponding operator type, for example, its attribute information is that the complexity remains unchanged.
[0055] After obtaining the attribute information of multiple types of processing algorithms, a corresponding target computing engine may be allocated to each type of processing algorithm from among the candidate computing engines consisting of multiple computing engines in the engine resource pool and dedicated computing engines according to the attribute information.
[0056] In the disclosed embodiment, different DSP engine resources can be brought together for unified management to form a DSP engine resource pool. The engine resource pool can be used to process algorithms that cannot use hardware acceleration in visual algorithms, audio algorithms, and artificial intelligence algorithms. In addition, the dedicated computing engine represented by the hardware circuit that can use hardware acceleration in the artificial intelligence algorithm can be processed separately and used together with the DSP engine resource pool as an engine resource on the bus.
[0057] refer to Figure 5The structure of the engine resources shown in the figure, the engine resource pool 501 includes multiple different types of DSP engine resources (i.e., multiple DSPs), and the number of DSP computing engines can be set according to actual needs. For example, if the number of second-type processing algorithms is large, the number of computing engines is large. The engine resource pool 501 can be connected to the dedicated computing engine 502 to facilitate intercommunication. In addition, the engine resource pool 501 and the dedicated computing resources 502 can be connected to the OCM (on chip memory) 503 to store the processing results of the computing engine to the on-chip memory through the bus. In addition, the required data can also be obtained from the on-chip memory. Adding a direct path between the DSP resource pool and the artificial intelligence acceleration circuit module hardware acceleration unit to control the dedicated computing engine and multiple computing engines in the engine resource pool to communicate directly can enable the intermediate processing results to be processed online, avoiding multiple transfers of data between the computing unit and the storage unit. By providing a direct path, the fusion of some operators is possible. For example, if the result of AI algorithm calculation needs to be post-processed in the DSP pool, it can be sent to the DSP pool for direct calculation through a direct path, without the need for the AI acceleration circuit module to store the result in the on-chip memory OCM through the bus, and then the DSP obtains the data from the OCM. Based on this, the delay time is reduced, the system power consumption is reduced, and the system performance is improved.
[0058] for Figure 5 As for the structure of the engine resources shown in , the engine resource pool 501 can be used to process non-fixed algorithms, and the dedicated computing engine 502 is an artificial intelligence acceleration circuit, which can be used to process fixed algorithms in artificial intelligence algorithms. By dividing the first type of processing algorithm and using a dedicated circuit to accelerate the fixed algorithm, the DSP computing engines in the system are unified and coordinated, avoiding the singleness of each DSP executing tasks, making the overall hardware resource allocation more balanced.
[0059] Based on this, the computing engines to be selected include multiple computing engines and dedicated computing engines in the engine resource pool. Furthermore, a suitable target computing engine can be allocated for each type of processing algorithm from the multiple computing engines and dedicated computing engines included in the computing engines to be selected. It can be seen that it is not that a fixed computing engine is directly used as the target computing engine for each type of processing algorithm, but that multiple computing engines and dedicated computing engines are used as engine resources to determine the target computing engine corresponding to each type of processing algorithm from all engine resources. By selecting from all engine resources based on the overall engine design, it is possible to avoid the situation where only the corresponding engine is used to process a task, avoid limitations and singularity, and be able to coordinate uniformly to achieve sharing of engine resources, thereby making the overall hardware resource allocation more balanced and reasonable.
[0060] Figure 6 FIG. 1 shows a flow chart of assigning a target computing engine. Figure 6 As shown in , it mainly includes the following steps:
[0061] In step S610, the target computing engine is allocated to the first type of processing algorithm according to the complexity of the first type of processing algorithm.
[0062] In the disclosed embodiment, the corresponding target computing engine may be determined according to the number of reference type operators included in the first type of processing algorithm. It should be noted that since the first type of processing algorithm is segmented, the corresponding target computing engine may be at least one computing engine.
[0063] Figure 7 A flowchart for assigning a target computing engine to a first type of processing algorithm is schematically shown in FIG. Figure 7 As shown in , it mainly includes steps S710 to S730, wherein:
[0064] In step S710, if the complexity level is the first level, the dedicated computing engine is used as the target engine;
[0065] In step S720, if the complexity level is the second level, the dedicated computing engine and one computing engine in the engine resource pool are used as the target engine;
[0066] In step S730, if the complexity level is the third level, the dedicated computing engine and multiple computing engines in the engine resource pool are used as the target engines.
[0067] In the embodiment of the present disclosure, the order of complexity from large to small is: third degree, second degree and first degree. If no reference type operator is included, the target computing engine is a dedicated computing engine; if the number of reference type operators included is small, the target computing engine is a dedicated computing engine and a computing engine in the engine resource pool; if the number of reference type operators included is large, the target computing engine is a dedicated computing engine and multiple computing engines in the engine resource pool.
[0068] refer to Figure 8 As shown in Figure A, for AI algorithms with moderate complexity, that is, in addition to the hardened operator part in the AI network algorithm, there are still a small number of algorithms that cannot be accelerated by the dedicated hardware of the artificial intelligence acceleration circuit module, so a DSP computing engine is allocated to execute. Figure 8 As shown in Figure B, for a relatively simple AI algorithm, the operators in it can already be covered by the hardware acceleration unit, and there is no need to allocate a DSP computing engine for it. Figure 8 As shown in Figure C, for AI algorithms that are relatively complex, in addition to algorithms that can be accelerated using dedicated circuits, there are also a large number of operators that are not suitable for direct implementation by hardware circuits, such as nonlinear activation functions. Therefore, two DSP computing engines and a neural network dedicated accelerator (dedicated computing engine) are allocated to perform operations specifically for AI algorithms.
[0069] In step S620, a computing engine in the engine resource pool is used as the target computing engine corresponding to the second type of processing algorithm.
[0070] In the embodiment of the present disclosure, after assigning a target computing engine to a first type of processing algorithm, all remaining engines in the candidate computing engines may be used as target computing engines corresponding to a second type of processing algorithm. Figure 8 As shown in Figure A, for AI algorithms with moderate complexity, the remaining two DSP computing engines in the engine resource pool are allocated to the second type of processing algorithm as the target computing engines. Figure 8 As shown in Figure B, for a relatively simple AI algorithm, all DSP computing engines in the engine resource pool are assigned to the second type of processing algorithm as the target computing engine. Figure 8 As shown in Figure C, for a more complex AI algorithm, the remaining DSP computing engine in the engine resource pool is allocated to the second type of processing algorithm as the target computing engine. It should be noted that the combination of computing engines is not limited to Figure 8 The method shown in the figure is determined according to the actual situation. For example, the multi-core artificial intelligence acceleration circuit module can be used as a dedicated computing engine to form a candidate computing engine with the multi-core DSP computing engine, and a target computing engine suitable for each type of processing algorithm can be selected from them.
[0071] In the disclosed embodiment, multiple DSP computing engines and dedicated computing engines in the engine resource pool can be integrated into a whole, and the first type of processing algorithm can be segmented by operators, so as to select target computing engines that match the first type of processing algorithm and the second type of processing algorithm from the candidate computing resources according to the types of operators contained in the first type of processing algorithm. By taking multiple computing engines and dedicated computing engines in the engine resource pool as candidate computing engines, a target computing engine suitable for each type of processing algorithm is selected from the candidate computing engines. All computing engines can be integrated to avoid the situation in the related art where only the corresponding engine is used to process a task, avoid limitations and singularity, and can be unified and coordinated, so that the overall hardware resource allocation is more balanced and reasonable.
[0072] Next, continue to refer to Figure 4 As shown in , in step S430, a corresponding processing algorithm is performed on the object to be processed based on the target computing engine to execute a processing operation, and an operation result corresponding to the object to be processed is obtained.
[0073] In the embodiment of the present disclosure, after the target computing engine is selected, the corresponding processing algorithm can be performed on the object to be processed according to the target computing engine, thereby executing the processing operation associated with the application scenario and obtaining the operation result of the object to be processed. For example, the artificial intelligence algorithm is processed by the dedicated computing engine, and the audio algorithm and image algorithm are processed by the DSP computing engine to obtain the recognition result of the object to be processed.
[0074] Fig. 9 The overall flow chart of the distribution calculation engine is schematically shown in Fig. 9 As shown in , it mainly includes the following steps:
[0075] In step S910, multiple types of processing algorithms included in the processing operation are obtained, which specifically includes step S911, obtaining a first type of processing algorithm. In addition, step S912 may also be included, obtaining a second type of processing algorithm.
[0076] In step S920, the first type of processing algorithm is segmented to obtain multiple operators;
[0077] In step S930, a target computing engine is allocated to the first type of processing algorithm according to the attribute information corresponding to the plurality of operators;
[0078] In step S940, a target computing engine is allocated to the second type of processing algorithm in combination with the target computing engine corresponding to the first type of processing algorithm.
[0079] For example, the video processing scenario is used as an example. In the process of video processing, the entire processing operation is divided into traditional processing algorithms and algorithm processing based on artificial intelligence. Traditional processing algorithms include audio algorithms and image algorithms. Among them, the artificial intelligence algorithm is split to obtain fixed operators, such as convolution, pooling, activation, etc. However, the operators used by different artificial intelligence algorithms will also be somewhat different. For example, the activation function can use a relatively simple Relu, or a nonlinear sigmoid, or a tanh function. Based on this, according to the number of reference type operators contained in the first type of processing algorithm, the fixed operator can be hardware accelerated by a dedicated computing engine, and the reference type operator contained in the first type of processing algorithm can be hardware accelerated using one or more computing engines in the engine resource pool. That is, the target computing engine of the first type of processing algorithm is determined in combination with the dedicated computing engine and the computing engine in the engine resource pool. And the remaining computing engines in the engine resource pool are used as the target computing engines of the second type of processing algorithm.
[0080] On this basis, the fixed algorithm can be hardware accelerated by the dedicated computing engine corresponding to the first type of processing algorithm, and the non-fixed algorithm can be hardware accelerated by multiple computing engines in the engine resource pool. The second type of processing algorithm can be processed by multiple computing engines in the engine resource pool corresponding to the second type of processing algorithm.
[0081] The technical solution in the embodiment of the present disclosure can perform corresponding operations on the object to be processed based on the calculation results after obtaining the calculation results. The corresponding operations are determined according to the application scenario. For example, the object to be processed can be identified based on the calculation results.
[0082] The present disclosure provides an object processing device, referring to Fig.10 As shown in , the object processing device 1000 may include:
[0083] An algorithm acquisition module 1001 is used to acquire multiple types of processing algorithms included in the processing operation on the object to be processed;
[0084] A computing engine allocation module 1002, configured to allocate a target computing engine for each type of processing algorithm from candidate computing engines according to the attribute information of the multiple types of processing algorithms; the candidate computing engines include multiple computing engines in an engine resource pool and dedicated computing engines;
[0085] The processing operation execution module 1003 is used to perform a corresponding processing algorithm on the object to be processed based on the target computing engine to execute a processing operation and obtain an operation result corresponding to the object to be processed.
[0086] In an exemplary embodiment of the present disclosure, the computing engine allocation module includes: a first allocation module, which is used to allocate the target computing engine to the first type of processing algorithm according to the complexity of the first type of processing algorithm; and a second allocation module, which is used to use the computing engine in the engine resource pool as the target computing engine corresponding to the second type of processing algorithm.
[0087] In an exemplary embodiment of the present disclosure, the first allocation module includes: a first degree module, which is used to use the dedicated computing engine as the target engine if the complexity level is the first degree; a second degree module, which is used to use the dedicated computing engine and a computing engine in the engine resource pool as the target engine if the complexity level is the second degree; and a third degree module, which is used to use the dedicated computing engine and multiple computing engines in the engine resource pool as the target engine if the complexity level is the third degree.
[0088] In an exemplary embodiment of the present disclosure, the apparatus further includes: a complexity determination module, configured to determine the complexity of the first type of processing algorithm according to a reference type operator included in the first type of processing algorithm.
[0089] In an exemplary embodiment of the present disclosure, the apparatus further includes: an algorithm division module, configured to divide the multiple types of processing algorithms into fixed algorithms and non-fixed algorithms according to operator types.
[0090] In an exemplary embodiment of the present disclosure, the algorithm partitioning module is configured to: determine the fixed operators in the first type of processing algorithm as the fixed algorithm, and determine the non-fixed operators in the first type of processing algorithm and the operators corresponding to the second type of processing algorithm as non-fixed algorithms.
[0091] In an exemplary embodiment of the present disclosure, the device further includes: a communication control module, configured to control the dedicated computing engine to perform direct communication with a plurality of computing engines in the engine resource pool through a direct path.
[0092] It should be noted that the specific details of each module in the above-mentioned object processing device have been described in detail in the corresponding object processing method, so they will not be repeated here.
[0093] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0094] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0095] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0096] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered merely as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.
[0097] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An object processing method, characterized in that: include: Obtain multiple types of processing algorithms included in the processing operation on the object to be processed; Allocating a target computing engine for each type of processing algorithm from the candidate computing engines according to the attribute information of the multiple types of processing algorithms; The computing engines to be selected include multiple computing engines in an engine resource pool and a dedicated computing engine; Based on the target computing engine, a corresponding processing algorithm is performed on the object to be processed to execute a processing operation, and an operation result corresponding to the object to be processed is obtained; The method of allocating a target computing engine to each type of processing algorithm from the candidate computing engines according to the attribute information of the multiple types of processing algorithms includes: Allocating the target computing engine to the first type of processing algorithm according to the complexity of the first type of processing algorithm; the complexity is positively correlated with the number of reference type operators included in the first type of processing algorithm; the reference type operators are operators that cannot be accelerated using dedicated hardware; The remaining computing engines in the engine resource pool are used as the target computing engines corresponding to the second type of processing algorithms.
2. The object processing method according to claim 1, characterized in that: The allocating the target computing engine to the first type of processing algorithm according to the complexity of the first type of processing algorithm comprises: If the complexity level is the first level, using the dedicated computing engine as the target computing engine; If the complexity level is the second level, using the dedicated computing engine and a computing engine in the engine resource pool as the target computing engine; If the complexity level is the third level, the dedicated computing engine and a plurality of computing engines in the engine resource pool are used as the target computing engines.
3. The object processing method according to claim 1, characterized in that: The method further comprises: The multiple types of processing algorithms are divided into fixed algorithms and non-fixed algorithms according to operator types.
4. The object processing method according to claim 3, characterized in that: The dividing the multiple types of processing algorithms into fixed algorithms and non-fixed algorithms according to the operator type includes: The fixed operators in the first type of processing algorithm are determined as the fixed algorithms, and the non-fixed operators in the first type of processing algorithm and the operators corresponding to the second type of processing algorithm are determined as the non-fixed algorithms.
5. The object processing method according to claim 1, characterized in that: The method further comprises: The dedicated computing engine is controlled to communicate directly with multiple computing engines in the engine resource pool through a direct path.
6. An object processing device, characterized in that: include: An algorithm acquisition module is used to acquire multiple types of processing algorithms included in the processing operation of the object to be processed; A computing engine allocation module, configured to allocate a target computing engine to each type of processing algorithm from the candidate computing engines according to the attribute information of the multiple types of processing algorithms; The computing engines to be selected include multiple computing engines in an engine resource pool and a dedicated computing engine; A processing operation execution module, used to perform a corresponding processing algorithm on the object to be processed based on the target computing engine to execute a processing operation, and obtain an operation result corresponding to the object to be processed; The method of allocating a target computing engine to each type of processing algorithm from the candidate computing engines according to the attribute information of the multiple types of processing algorithms includes: Allocating the target computing engine to the first type of processing algorithm according to the complexity of the first type of processing algorithm; the complexity is positively correlated with the number of reference type operators included in the first type of processing algorithm; the reference type operators are operators that cannot be accelerated using dedicated hardware; The remaining computing engines in the engine resource pool are used as the target computing engines corresponding to the second type of processing algorithms.
7. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the object processing method according to any one of claims 1 to 5 by executing the executable instructions.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the object processing method according to any one of claims 1 to 5 is implemented.
