Data processing method and electronic device

Abstract

The embodiment of the present application provides a data processing method and electronic device, including: in response to a received computing instruction, obtaining the running status information of multiple processors in a heterogeneous system based on a unified interface, the multiple processors include multiple different types of processors, and the unified interface can shield the type information of the processors; determining an execution processor from the multiple processors based at least on the running status information, the running status information is used to characterize the computing space occupancy of each processor in the multiple processors. The data processing method of the embodiment of the present application can realize computing task allocation in a heterogeneous system.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of smart devices, and in particular to an electronic device. Background Art

[0002] Usually, the hardware architecture for implementing AI is a combination of multiple processors, that is, the hardware architecture for implementing AI computing constitutes a heterogeneous system, and the implementation of AI requires the realization of software-level computing output based on the heterogeneous system. However, in the prior art, in heterogeneous systems, the allocation of computing tasks is fixed, that is, there is actually no strict task allocation in heterogeneous systems, and computing tasks are arbitrarily allocated to each processor. In this way, it is impossible to fully utilize all computing resources in electronic devices, and it is impossible to enable the device to achieve the effect of maximum computing power.

[0003] Application Contents

[0004] The embodiments of the present application provide a data processing method and an electronic device capable of implementing computing task allocation in a heterogeneous system.

[0005] In order to solve the above technical problems, the present application provides a data processing method, including:

[0006] In response to the received computing instruction, obtaining, based on a unified interface, operating state information of a plurality of processors in a heterogeneous system, the plurality of processors including a plurality of processors of different types, the unified interface being capable of shielding type information of the processors;

[0007] An execution processor is determined from the multiple processors based at least on the running status information, wherein the running status information is used to characterize the computing space occupancy of each processor in the multiple processors.

[0008] Preferably, the computing space occupancy includes occupancy rate.

[0009] The obtaining of the operation status information of the plurality of processors in the heterogeneous system based on the unified interface includes: obtaining the processor occupancy rate of each of the plurality of processors based on at least the unified interface;

[0010] Determining an execution processor from the multiple processors at least based on the running status includes: determining an execution processor whose processor occupancy meets a specific threshold range from the multiple processors at least based on the processor occupancy.

[0011] Preferably, at least some of the multiple processors are loaded with different computing models.

[0012] The step of determining an execution processor from the plurality of processors at least based on the running status information comprises:

[0013] Determine a candidate processor that meets a specific requirement from the plurality of processors based on the operation status information;

[0014] Determining data characteristics of the data to be processed based on the computing task information to be processed;

[0015] The execution processor is determined from the candidate processors based on the data features, wherein the data features required by the computing model loaded by the execution processor in the process of executing data processing match the data features of the data to be processed.

[0016] Preferably, at least some of the multiple processors are loaded with the same computing model.

[0017] The step of determining an execution processor from the plurality of processors at least based on the running status information comprises:

[0018] Determine a candidate processor that meets a specific requirement from the plurality of processors based on the operation status information;

[0019] The execution processor is determined based on the priorities of the plurality of processors.

[0020] As a preference, it also includes:

[0021] Processing a plurality of said computing models for assisting in executing the computing tasks to be processed;

[0022] The processed multiple computing models are loaded into the multiple processors respectively to assist the multiple processors in performing corresponding computing tasks.

[0023] Preferably, at least one of the computing models is loaded in each of the multiple processors, and the computing models loaded in each processor are the same or different.

[0024] Preferably, the plurality of computing models are implemented based on an open computing language.

[0025] The processing of the plurality of computing models for assisting in executing the computing tasks to be processed, and loading the processed plurality of computing models into the plurality of processors respectively, comprises:

[0026] Dynamically compile the plurality of computing models implemented based on the open computing language, and load the compiled plurality of computing models into the plurality of processors respectively.

[0027] Preferably, the multiple processors include a first processor, which is a single-chip processor, and the single-chip processor integrates an image receiving module for receiving captured images, a convolutional neural network model for performing image recognition on the images, and a microprocessor for processing and outputting image recognition results.

[0028] Preferably, the multiple processors of different types include at least a central processing unit, a graphics processing unit and a configurable processor.

[0029] Another embodiment of the present invention also provides an electronic device, including:

[0030] A first processor, in response to a received computing instruction, obtains running status information of multiple second processors in a heterogeneous system based on a unified interface, wherein the multiple second processors include multiple processors of different types, and the unified interface is capable of shielding the type information of the processors; and determines an execution processor from the multiple second processors based at least on the running status information, wherein the running status information is used to characterize the computing space occupancy of each second processor in the multiple second processors.

[0031] Based on the disclosure of the above embodiments, it can be known that the beneficial effects of the embodiments of the present application include the ability to reasonably allocate computing tasks to be processed based on the operating status of each processor in a heterogeneous system, thereby maximizing the use of the computing space of each processor in the heterogeneous system and improving the overall data computing capability and efficiency of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 4 is a flow chart of a data method in an embodiment of the present invention.

[0033] Figure 2 FIG. 4 is a flow chart of a data method in another embodiment of the present invention.

[0034] Figure 3 FIG. 4 is a flow chart of a data method in another embodiment of the present invention.

[0035] Figure 4 FIG. 4 is a flow chart of a data method in another embodiment of the present invention.

[0036] Figure 5 It is an application diagram of the method in the embodiment of the present invention.

[0037] Figure 6 FIG. 4 is a structural block diagram of a first processor in an embodiment of the present invention.

[0038] Figure 7 4 is a structural block diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] Below, specific embodiments of the present application are described in detail with reference to the accompanying drawings, but are not intended to limit the present application.

[0040] It should be understood that various modifications may be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope and spirit of the present disclosure will occur to those skilled in the art.

[0041] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0042] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0043] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0044] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0045] Specific embodiments of the present disclosure are described hereinafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant details. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of ways with substantially any suitable detailed structure.

[0046] This specification may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present disclosure.

[0047] Below, embodiments of the present application are described in detail with reference to the accompanying drawings.

[0048] like Figure 1 As shown, an embodiment of the present invention provides a data processing method, including:

[0049] In response to the received computing instruction, obtaining the operation status information of the plurality of processors in the heterogeneous system based on the unified interface, the plurality of processors including a plurality of processors of different types, and the unified interface can shield the type information of the processors;

[0050] An execution processor is determined from a plurality of processors based at least on the operation status information, where the operation status information is used to characterize the computing space occupation of each of the plurality of processors.

[0051] For example, a heterogeneous system includes multiple processors, and the types of the multiple processors are at least partially different, such as the first processor, the second processor, and the third processor are of different types, and are even processors prepared by different manufacturers, and the number of the first processor, the second processor, and the third processor is not unique, and can be only one or more. In order to achieve language unification in a heterogeneous system, so that data can be exchanged between each processor or between each processor and a heterogeneous computing platform, a heterogeneous system computing program can be installed in one of the processors, such as a central processing unit (CPU) through a software development kit (SDK), which can support the use of, for example, an open programming language (Open Computing Language, Opencl) to create an environment in which the language can be used universally for the heterogeneous system, thereby supporting data exchange between each processor and between each processor and a heterogeneous computing platform, and the heterogeneous computing platform can be considered to be formed by a module in the CPU in which the above-mentioned heterogeneous system computing program is installed, or the heterogeneous computing platform can also be understood as a main control module, which is independent of the computing module in the CPU, and the main control module can also obtain the running status of the computing module in the CPU and assign tasks to it. Furthermore, when the user inputs a computing task, the CPU will initiate a computing instruction, and the heterogeneous computing platform responds to the computing instruction and obtains the running status information of multiple processors in the heterogeneous system through a unified interface, that is, a set of interfaces with unified specifications. Based on the unified interface, the heterogeneous computing platform can shield the actual identity of each processor, so that all processors can be directly used and controlled for the heterogeneous computing platform, and at the same time, the unified data specifications for interaction can be ensured based on the unified interface. In practical applications, the unified interface can be a virtual interface, or a physical interface of each processor, or a bus interface. After the heterogeneous computing platform obtains the running status information of each processor, which is equivalent to the CPU obtaining the running status information of each processor, the heterogeneous computing platform in the CPU determines the execution processor from the multiple processors based on at least the running status information, and the running status information is used to characterize the computing space occupancy of each processor in the multiple processors, so that each processor can be determined based on the occupancy to continue to execute the computing task to be processed, thereby determining the processor with sufficient computing space to process the computing task from the multiple processors, avoiding the phenomenon that only one processor is used intensively and the remaining processors are idle in the prior art.

[0052] Based on the disclosure of the above embodiments, it can be known that the beneficial effects of the embodiments of the present application include the ability to reasonably allocate computing tasks to be processed based on the operating status of each processor in a heterogeneous system, thereby maximizing the use of the computing space of each processor in the heterogeneous system and improving the overall data computing capability and efficiency of the electronic device.

[0053] Furthermore, in this embodiment, the calculation space occupancy includes occupancy rate, that is, processor occupancy rate.

[0054] Obtain the operating status information of multiple processors in a heterogeneous system based on a unified interface, including:

[0055] obtaining a processor occupancy rate of each processor of the plurality of processors based at least on the unified interface;

[0056] Determining an execution processor from a plurality of processors based at least on the running state includes:

[0057] An execution processor whose processor occupancy rate meets a specific threshold range is determined from among the plurality of processors based at least on the processor occupancy rate.

[0058] For example, the heterogeneous computing platform obtains the processor occupancy rate of each processor in the heterogeneous system through the same interface, and then judges the operation and usage of multiple processors based on the processor occupancy rate of each processor and the specific threshold value that measures the availability of the processor through the processor occupancy rate, and finally determines the execution processor whose processor usage rate meets the conditions. For example, the processor occupancy rates of multiple processors are 60%, 70%, and 30% respectively, and the specific threshold value is 0-40%, then at this time, the processor with a processor occupancy rate of 30% can be determined as the execution processor, which is used to execute the computing task to be processed.

[0059] Furthermore, at least one computing model is loaded in each of the multiple processors in this embodiment, and the computing models loaded in each processor may be the same or different.

[0060] Specifically, Figure 2 As shown, when at least some of the multiple processors in this embodiment are loaded with different computing models,

[0061] Determining an execution processor from a plurality of processors based at least on the operation status information includes:

[0062] Determine a candidate processor that meets specific requirements from a plurality of processors based on the operation status information;

[0063] Determine data characteristics of the data to be processed based on the computing task information to be processed;

[0064] An execution processor is determined from candidate processors based on data features, wherein data features required by a computing model loaded by the execution processor in a data processing process match data features of the data to be processed.

[0065] Specifically, each processor is loaded with a computing model, such as an AI model, etc., but of course it is not limited to this. When at least some of the multiple processors are loaded with different computing models, when determining the execution processor based on the running state, it is necessary to first determine the candidate processor based on the running state information, for example, determine the processor whose processor occupancy rate is within 0-40% based on the running state information, and define the multiple processors as candidate processors. Then, the data features of the data to be processed are determined based on the computing task information to be processed. For example, if the task to be processed is to perform face recognition on image data, and the data to be processed is image data containing face images, then the data feature can be considered as a face feature. If the task to be processed is to perform voiceprint recognition on audio data, and the data to be processed is audio data containing human voice data, then the data feature can be considered as a human voice feature. After the data feature is determined, it can be matched with the feature data required for data derivation by the computing model loaded in each candidate processor based on the data feature, and the candidate processor where the matching computing model is located is defined as the execution processor. For example, if the computing model is a convolutional neural network model, it needs to receive matching feature data when performing data derivation. The feature data is obtained by preprocessing the data to be processed by the module in the processor for preprocessing data. If a computing model is a model for face recognition or voiceprint recognition, the feature data it receives must be face feature data or voice feature data. Therefore, the feature data required by the computing model in the data derivation process will match the data features of the data to be processed obtained above. At this time, the heterogeneous computing platform can determine that the candidate processor with the computing model can process the data to be processed and can become the execution processor.

[0066] Furthermore, if Figure 3 As shown, when at least some of the multiple processors in this embodiment load the same computing model,

[0067] Determining an execution processor from a plurality of processors based at least on the operation status information includes:

[0068] Determine a candidate processor that meets specific requirements from a plurality of processors based on the operation status information;

[0069] An execution processor is determined based on the priorities of the plurality of processors.

[0070] For example, firstly, candidate processors are determined based on the running status information, for example, processors with a processor occupancy rate within 0-40% are determined based on the running status information, and the multiple processors are defined as candidate processors. Then, the final execution processor is determined based on the priority of each processor. For example, the multiple different types of processors in this embodiment include at least a central processing unit (CPU), a graphics processing unit (GPU), and a configurable field-programmable gate array (FPGA). Then the priority from high to low is configurable processor, graphics processing unit, central processing unit.

[0071] In addition, preferably, multiple priority orders may be pre-stored in the electronic device, and the multiple priorities respectively match multiple different computing data to be processed, or match multiple different computing task types to be processed. When determining the execution processor, the matching priority can be determined based on the computing data to be processed or the computing task type, and the execution processor is determined from the candidate processors by the matching priority. For example, the principle of some priorities is that the configurable processor is more suitable for AI computing, followed by the graphics processor, and finally the central processing unit. And some priority principles are that the graphics processor is more suitable for image processing, followed by the configurable processor, and finally the central processing unit.

[0072] Furthermore, if Figure 4 As shown, in this embodiment, when loading the calculation model into multiple processors, it includes:

[0073] Processing a plurality of computing models for assisting in executing the computing tasks to be processed;

[0074] The processed multiple computing models are loaded into multiple processors respectively to assist the multiple processors in executing corresponding computing tasks.

[0075] That is, the heterogeneous computing platform needs to process multiple computing models used to assist in executing the tasks to be processed in advance, and each processor can only load and run the processed computing models.

[0076] Specifically, Figure 5 As shown, the multiple computing models in this embodiment are all convolutional neural network models implemented based on an open computing language (equivalent to Figure 5 cnn.cl in );

[0077] Processing a plurality of computing models for assisting in executing the computing tasks to be processed, and loading the processed computing models into a plurality of processors respectively, including:

[0078] Dynamically compile multiple computing models based on an open computing language, and load the compiled multiple computing models into multiple processors respectively.

[0079] For example, the multiple computing models are all convolutional neural network models, and are implemented based on open programming languages ​​in heterogeneous systems. In order for each computing model to be able to run normally after being loaded into each processor, it is necessary to process each computing model, specifically, dynamically compile the multiple computing models to obtain executable programs for the computing models (equivalent to Figure 5 Finally, the executable program is loaded into the processor so that the processor can run it directly.

[0080] Specifically, continue to combine Figure 5 When the method in this embodiment is actually applied, the specific process may include:

[0081] The user starts the target application, such as an AI app, and inputs a computing instruction based on the application. After the application obtains the computing instruction input by the user, the CPU equipped with the Opencl program initiates the computing instruction, receives the computing data to be processed, and generates a computing task. The heterogeneous computing platform obtains the processor occupancy of each processor in the heterogeneous system, including the CPU occupancy of the CPU. The heterogeneous computing platform in the CPU determines the processor whose value is within the threshold range of 0-40% as a candidate processor based on the processor occupancy of each processor. Next, the heterogeneous computing platform determines the data feature of the data to be processed, for example, the data feature is a face data feature. At this time, the heterogeneous computing platform matches the computing model that matches the feature data required in the data derivation process from the candidate processor based on the data feature, specifically determining that the candidate processor has a computing model for face recognition, and then the candidate processor with the computing model can be determined as the execution processor. Alternatively, the execution processor is determined from the candidate processor based on the priority level; or the matching priority is determined based on the data feature, and the execution processor is determined from the candidate processor based on the matching priority. If it is determined that the candidate processors include multiple processors that all have face recognition models, the above-mentioned priority method can also be used to assist in further determining the execution processor.

[0082] Furthermore, if Figure 6As shown, in order to cope with the current strong demand for graphics processing, this embodiment also proposes a processor with higher graphics processing efficiency, which can be applied to the heterogeneous system of this embodiment, as the first processor, the first processor is preferably a single-chip processor, and the single chip is integrated with an image receiving module for receiving the collected image, a convolutional neural network model for image recognition, and a microprocessor for processing and outputting the image recognition result. Therefore, based on the first processor in this embodiment, it can independently complete all operations of receiving the collected image from the image acquisition module, such as the camera, performing a series of image processing on the image, and controlling the output of the processed image, without the need to set up multiple electronic devices at the same time as in the prior art to cooperate to complete the above operations together, which not only reduces the setting of electronic devices, reduces costs, saves installation space, but also greatly improves the efficiency of graphics processing and meets the user's demand for graphics processing.

[0083] In a specific implementation, the single-chip processor may be a configurable processor, which realizes the formation of an image receiving module, a graphics processing model, and a microprocessor by changing programming and changing the connection relationship between internal transistors, wherein the graphics processing model is a convolutional neural network model, such as an AI model. When the processor processes graphics, it includes:

[0084] The collected image data is read from the camera, and then the image is preprocessed, including filtering, noise reduction, and white balance processing. The preprocessed image is then transmitted to the graphics processing model for further recognition processing, such as face recognition processing, etc. After the processing is completed, the graphics processing model transmits the recognized and processed data to the microprocessor (MCU), based on which the microprocessor outputs the image processing result as a local response result or sends it to other network devices that have established a communication connection with the electronic device.

[0085] like Figure 7 As shown, another embodiment of the present invention also discloses an electronic device, including:

[0086] A processing module is integrated in a processor. The processing module is used to respond to received computing instructions and obtain operating status information of multiple processors in a heterogeneous system based on a unified interface. The multiple processors include multiple processors of different types. The unified interface can shield the type information of the processors; and determine an execution processor from the multiple processors based at least on the operating status information. The operating status information is used to characterize the computing space occupancy of each processor in the multiple processors.

[0087] For example, a heterogeneous system includes multiple processors, and the types of the multiple processors are at least partially different, such as the first processor, the second processor, and the third processor are of different types, and are even processors prepared by different manufacturers, and the number of the first processor, the second processor, and the third processor is not unique, and can be only one or more. In order to achieve language unification in a heterogeneous system, so that data can be exchanged between each processor or between each processor and a heterogeneous computing platform, a heterogeneous system computing program can be installed in one of the processors, such as a central processing unit (CPU), in advance, which can support the use of, for example, an open programming language (Open Computing Language, Opencl) to create an environment in which the language can be used universally for the heterogeneous system, thereby supporting data exchange between each processor and between each processor and a heterogeneous computing platform, and the heterogeneous computing platform can be considered to be formed by a module in the CPU in which the above-mentioned heterogeneous system computing program is installed, or the heterogeneous computing platform can also be understood as a main control module, which is independent of the computing module in the CPU, and the main control module can also obtain the running status of the computing module in the CPU and assign tasks to it. Furthermore, when the user inputs a computing task, the CPU will initiate a computing instruction, and the heterogeneous computing platform responds to the computing instruction and obtains the running status information of multiple processors in the heterogeneous system through a unified interface, that is, a set of interfaces with unified specifications. Based on the unified interface, the heterogeneous computing platform can shield the actual identity of each processor, so that all processors can be directly used and controlled for the heterogeneous computing platform, and at the same time, the unified data specifications for interaction can be ensured based on the unified interface. In practical applications, the unified interface can be a virtual interface, or a physical interface of each processor, or a bus interface. After the heterogeneous computing platform obtains the running status information of each processor, it determines the execution processor from the multiple processors at least based on the running status information. The running status information is used to characterize the computing space occupancy of each processor in the multiple processors, so that the occupancy is determined based on the occupancy to determine whether each processor can continue to execute the computing task to be processed, thereby determining the processor with sufficient computing space to process the computing task from the multiple processors, avoiding the phenomenon in the prior art that only one processor is used intensively, while the remaining processors are in an idle state.

[0088] Based on the disclosure of the above embodiments, it can be known that the beneficial effects of the embodiments of the present application include the ability to reasonably allocate computing tasks to be processed based on the operating status of each processor in a heterogeneous system, thereby maximizing the use of the computing space of each processor in the heterogeneous system and improving the overall data computing capability and efficiency of the electronic device.

[0089] Furthermore, in this embodiment, the calculation space occupancy includes occupancy rate, that is, processor occupancy rate.

[0090] Obtain the operating status information of multiple processors in a heterogeneous system based on a unified interface, including:

[0091] obtaining a processor occupancy rate of each processor of the plurality of processors based at least on the unified interface;

[0092] Determining an execution processor from a plurality of processors based at least on the running state includes:

[0093] An execution processor whose processor occupancy rate meets a specific threshold range is determined from among the plurality of processors based at least on the processor occupancy rate.

[0094] For example, the heterogeneous computing platform obtains the processor occupancy rate of each processor in the heterogeneous system through the same interface, and then judges the operation and usage of multiple processors based on the processor occupancy rate of each processor and the specific threshold value that measures the availability of the processor through the processor occupancy rate, and finally determines the execution processor whose processor usage rate meets the conditions. For example, the processor occupancy rates of multiple processors are 60%, 70%, and 30% respectively, and the specific threshold value is 0-40%, then at this time, the processor with a processor occupancy rate of 30% can be determined as the execution processor, which is used to execute the computing task to be processed.

[0095] Furthermore, at least one computing model is loaded in each of the multiple processors in this embodiment, and the computing models loaded in each processor may be the same or different.

[0096] Specifically, when at least some of the multiple processors in this embodiment load different computing models,

[0097] Determining an execution processor from a plurality of processors based at least on the operation status information includes:

[0098] Determine a candidate processor that meets specific requirements from a plurality of processors based on the operation status information;

[0099] Determine data characteristics of the data to be processed based on the computing task information to be processed;

[0100] An execution processor is determined from candidate processors based on data features, wherein data features required by a computing model loaded by the execution processor in a data processing process match data features of the data to be processed.

[0101] Specifically, each processor is loaded with a computing model, such as an AI model, etc., but of course it is not limited to this. When at least some of the multiple processors are loaded with different computing models, when determining the execution processor based on the running state, it is necessary to first determine the candidate processor based on the running state information, for example, determine the processor whose processor occupancy rate is within 0-40% based on the running state information, and define the multiple processors as candidate processors. Then, the data features of the data to be processed are determined based on the computing task information to be processed. For example, if the task to be processed is to perform face recognition on image data, and the data to be processed is image data containing face images, then the data feature can be considered as a face feature. If the task to be processed is to perform voiceprint recognition on audio data, and the data to be processed is audio data containing human voice data, then the data feature can be considered as a human voice feature. After the data feature is determined, it can be matched with the feature data required for data derivation by the computing model loaded in each candidate processor based on the data feature, and the candidate processor where the matching computing model is located is defined as the execution processor. For example, if the computing model is a convolutional neural network model, it needs to receive matching feature data when performing data derivation. The feature data is obtained by preprocessing the data to be processed by the module in the processor for preprocessing data. If a computing model is a model for face recognition or voiceprint recognition, the feature data it receives must be face feature data or voice feature data. Therefore, the feature data required by the computing model in the data derivation process will match the data features of the data to be processed obtained above. At this time, the heterogeneous computing platform can determine that the candidate processor with the computing model can process the data to be processed and can become the execution processor.

[0102] Furthermore, when at least some of the multiple processors in this embodiment load the same computing model,

[0103] Determining an execution processor from a plurality of processors based at least on the operation status information includes:

[0104] Determine a candidate processor that meets specific requirements from a plurality of processors based on the operation status information;

[0105] An execution processor is determined based on the priorities of the plurality of processors.

[0106] For example, firstly, candidate processors are determined based on the running status information, for example, processors with a processor occupancy rate within 0-40% are determined based on the running status information, and the multiple processors are defined as candidate processors. Then, the final execution processor is determined based on the priority of each processor. For example, the multiple different types of processors in this embodiment include at least a central processing unit, a graphics processing unit, and a configurable processor. Then the priorities from high to low are configurable processor, graphics processing unit, central processing unit.

[0107] In addition, preferably, multiple priority orders may be pre-stored in the electronic device, and the multiple priorities respectively match multiple different computing data to be processed, or match multiple different computing task types to be processed. When determining the execution processor, the matching priority can be determined based on the computing data to be processed or the computing task type, and the execution processor is determined from the candidate processors by the matching priority. For example, the principle of some priorities is that the configurable processor is more suitable for AI computing, followed by the graphics processor, and finally the central processing unit. And some priority principles are that the graphics processor is more suitable for image processing, followed by the configurable processor, and finally the central processing unit.

[0108] Furthermore, in this embodiment, when the computing model is loaded into multiple processors, it includes:

[0109] Processing a plurality of computing models for assisting in executing the computing tasks to be processed;

[0110] The processed multiple computing models are loaded into multiple processors respectively to assist the multiple processors in executing corresponding computing tasks.

[0111] That is, the heterogeneous computing platform needs to process multiple computing models used to assist in executing the tasks to be processed in advance, and each processor can only load and run the processed computing models.

[0112] Specifically, the multiple computing models in this embodiment are all convolutional neural network models implemented based on an open computing language;

[0113] Processing a plurality of computing models for assisting in executing the computing tasks to be processed, and loading the processed computing models into a plurality of processors respectively, including:

[0114] Dynamically compile multiple computing models based on an open computing language, and load the compiled multiple computing models into multiple processors respectively.

[0115] For example, the multiple computing models are all convolutional neural network models, and are implemented based on an open programming language in the heterogeneous system. In order for each computing model to be able to run normally after being loaded into each processor, each computing model needs to be processed. Specifically, the multiple computing models are dynamically compiled to obtain executable programs for the computing models of each processor, and finally the executable program is loaded into the processor so that the processor can run directly.

[0116] Specifically, when the method in this embodiment is actually applied, the specific process may include:

[0117] The user starts the target application, such as an AI app, and inputs a computing instruction based on the application. After the application obtains the computing instruction input by the user, the CPU equipped with the Opencl program initiates the computing instruction, receives the computing data to be processed, and generates a computing task. The heterogeneous computing platform obtains the processor occupancy of each processor in the heterogeneous system, including the CPU occupancy of the CPU. The heterogeneous computing platform in the CPU determines the processor whose value is within the threshold range of 0-40% as a candidate processor based on the processor occupancy of each processor. Next, the heterogeneous computing platform determines the data feature of the data to be processed, for example, the data feature is a face data feature. At this time, the heterogeneous computing platform matches the computing model that matches the feature data required in the data derivation process from the candidate processor based on the data feature, specifically determining that the candidate processor has a computing model for face recognition, and then the candidate processor with the computing model can be determined as the execution processor. Alternatively, the execution processor is determined from the candidate processor based on the priority level; or the matching priority is determined based on the data feature, and the execution processor is determined from the candidate processor based on the matching priority. If it is determined that the candidate processors include multiple processors that all have face recognition models, the above-mentioned priority method can also be used to assist in further determining the execution processor.

[0118] Furthermore, in order to cope with the current strong demand for graphics processing, this embodiment also proposes a processor with higher graphics processing efficiency, which can be applied to the heterogeneous system of this embodiment, as the first processor, the first processor is preferably a single-chip processor, the single chip is integrated with an image receiving module for receiving the collected image, a convolutional neural network model for image recognition, and a microprocessor for processing and outputting the image recognition result. Therefore, based on the first processor in this embodiment, it can independently complete all operations of receiving the collected image from the image acquisition module, such as the camera, performing a series of image processing on the image, and controlling the output of the processed image, without the need to set up multiple electronic devices at the same time to cooperate to complete the above operations as in the prior art, which not only reduces the setting of electronic devices, reduces costs, saves installation space, but also greatly improves the efficiency of graphics processing and meets the user's demand for graphics processing.

[0119] In a specific implementation, the single-chip processor may be a configurable processor, which realizes the formation of an image receiving module, a graphics processing model, and a microprocessor by changing programming and changing the connection relationship between internal transistors, wherein the graphics processing model is a convolutional neural network model, such as an AI model. When the processor processes graphics, it includes:

[0120] The collected image data is read from the camera, and then the image is preprocessed, including filtering, noise reduction, and white balance processing. The preprocessed image is then transmitted to the graphics processing model for further recognition processing, such as face recognition processing, etc. After the processing is completed, the graphics processing model transmits the recognized and processed data to the microprocessing model, and the microprocessing model outputs the image processing result as a local response result based on the recognized and processed image data, or sends it to other network devices that have established a communication connection with the electronic device.

[0121] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A data processing method, comprising: In response to the received computing instruction, obtaining, based on a unified interface, operating state information of a plurality of processors in a heterogeneous system, the plurality of processors including a plurality of processors of different types, the unified interface being capable of shielding type information of the processors; Determining an execution processor from the plurality of processors based at least on the operation status information, wherein the operation status information is used to characterize the computing space occupation of each processor in the plurality of processors; At least some of the multiple processors are loaded with different computing models, The step of determining an execution processor from the plurality of processors at least based on the running status information comprises: Determine, based on the operation status information, a candidate processor from the plurality of processors, at least one of which has a processor usage rate that satisfies a condition; Determine data features of the data to be processed based on the information of the computing task to be processed, wherein the data features are obtained by preprocessing the computing task to be processed by a module in the processor for preprocessing data; Determine the execution processor from the candidate processors based on the data features, wherein the data features required by the computing model loaded by the execution processor in the process of executing data processing match the data features of the data to be processed; or at least some of the plurality of processors are loaded with the same computing model, The step of determining an execution processor from the plurality of processors at least based on the running status information comprises: Determine, based on the running status information, a candidate processor that meets a specific requirement from the multiple processors, wherein the specific requirement includes that a processor occupancy rate meets a requirement; The execution processor is determined based on the priorities of the plurality of processors.

2. The method according to claim 1, wherein: The computing space occupancy includes occupancy rate, The obtaining of the operating status information of multiple processors in the heterogeneous system based on the unified interface includes: Obtaining a processor occupancy rate of each processor of the plurality of processors based at least on a unified interface; The step of determining an execution processor from the plurality of processors at least based on the running status information comprises: An execution processor whose processor occupancy rate satisfies a specific threshold range is determined from the plurality of processors at least based on the processor occupancy rate.

3. The method according to claim 1, wherein: Also includes: Processing a plurality of said computing models for assisting in executing the computing tasks to be processed; The processed multiple computing models are loaded into the multiple processors respectively to assist the multiple processors in performing corresponding computing tasks.

4. The method according to claim 3, wherein: At least one of the computing models is loaded in each of the multiple processors, and the computing models loaded in each processor are the same or different.

5. The method according to claim 3, wherein: The plurality of computing models are implemented based on an open computing language, The processing of the plurality of computing models for assisting in executing the computing tasks to be processed, and loading the processed plurality of computing models into the plurality of processors respectively, comprises: Dynamically compile the plurality of computing models implemented based on the open computing language, and load the compiled plurality of computing models into the plurality of processors respectively.

6. The method according to claim 1, wherein: The multiple processors include a first processor, which is a single-chip processor. The single-chip processor integrates an image receiving module for receiving a captured image, a convolutional neural network model for performing image recognition on the image, and a microprocessor for outputting an image recognition result.

7. The method according to claim 1, wherein: The multiple processors of different types include at least a central processing unit, a graphics processing unit and a configurable processor.

8. An electronic device, comprising: A processing module, which is integrated in a processor, the processing module is used to respond to a received computing instruction, obtain running status information of multiple processors in a heterogeneous system based on a unified interface, the multiple processors include multiple processors of different types, and the unified interface can shield the type information of the processors; and determine an execution processor from the multiple processors based at least on the running status information, the running status information is used to characterize the computing space occupancy of each processor in the multiple processors; At least some of the multiple processors are loaded with different computing models, The step of determining an execution processor from the plurality of processors at least based on the running status information comprises: Determine, based on the operation status information, a candidate processor from the plurality of processors, at least one of which has a processor usage rate that satisfies a condition; Determine data features of the data to be processed based on the information of the computing task to be processed, wherein the data features are obtained by preprocessing the computing task to be processed by a module in the processor for preprocessing data; Determine the execution processor from the candidate processors based on the data features, wherein the data features required by the computing model loaded by the execution processor in the process of executing data processing match the data features of the data to be processed; or at least some of the plurality of processors are loaded with the same computing model, The step of determining an execution processor from the plurality of processors at least based on the running status information comprises: Determine, based on the running status information, a candidate processor that meets a specific requirement from the multiple processors, wherein the specific requirement includes that a processor occupancy rate meets a requirement; The execution processor is determined based on the priorities of the plurality of processors.

Images