A method and system for intelligent allocation of TPU resources for deep learning

Abstract

This application discloses a method and system for intelligent allocation of TPU resources for deep learning. The method includes: acquiring the data to be processed for deep learning and the configuration parameters of the TPU; extracting processing target features and temporal correlation features from the data to be processed; splitting the data to be processed according to the configuration parameters, the results of the processing target feature extraction, and the results of the temporal correlation feature extraction; determining multiple candidate TPU sub-resources corresponding to each data block to be processed based on a preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed; determining the current capability parameters corresponding to each candidate TPU sub-resource; determining the matching degree score between each data block to be processed and each candidate TPU sub-resource according to the attribute parameters and the current capability parameters; and determining the target TPU sub-resource corresponding to each data block to be processed. This method can improve the rationality of TPU resource allocation and the accuracy of the allocation results.

Description

Technical Field

[0001] This application belongs to the field of electronic digital data processing technology, specifically relating to a method and system for intelligent allocation of TPU resources for deep learning. Background Technology

[0002] With the development of vehicle intelligence, the functional complexity and perception and decision-making accuracy of driver assistance systems are constantly improving. As an important technology for driver assistance systems, deep learning is also experiencing an exponential increase in the amount of data to be processed. To ensure the processing speed and load balance of vehicle TPU resources for the data to be processed for deep learning input to the driver assistance system, dynamic allocation of TPU resources has become a key research direction in the driver assistance industry.

[0003] In existing technologies, TPU resource allocation methods mainly involve pre-setting a TPU resource allocation order and allocating TPU sub-resources to the data to be processed sequentially according to the allocation order and the receiving order of the data to be processed, so as to perform complete data processing based on the allocated TPU sub-resources. However, different input data have different sizes, and different TPU sub-resources have different data processing capabilities, resulting in different load states for each TPU sub-resource at different times. The existing TPU resource allocation method can lead to TPU resource load imbalance, with some TPU sub-resources processing data for a long time while others remain idle for a long time, reducing the overall data processing speed of the system. Therefore, existing technologies suffer from unreasonable TPU resource allocation methods and inaccurate allocation results. Summary of the Invention

[0004] The purpose of this application is to provide a TPU resource intelligent allocation method and system for deep learning, which solves the problems of unreasonable TPU resource allocation methods and inaccurate allocation results in the prior art. By configuring parameters, extracting the target features of the data to be processed, and extracting the temporal correlation features, the data to be processed is split into data blocks. Then, TPU resources are allocated to each data block according to the attributes of the split data blocks and the current capacity of the TPU sub-resources. This achieves the purpose of splitting the data to be processed based on TPU configuration parameters and the features of the data to be processed, and allocating resources to the data blocks according to the real-time capacity of the TPU resources. This ensures TPU resource load balancing and improves the rationality of the TPU resource allocation method and the accuracy of the allocation results.

[0005] In a first aspect, embodiments of this application provide a method for intelligent allocation of TPU resources for deep learning, the method comprising:

[0006] Obtain the data to be processed for deep learning and the configuration parameters of the TPU. Perform target feature extraction and temporal correlation feature extraction on the data to be processed to obtain the target feature extraction results and temporal correlation feature extraction results, respectively.

[0007] Based on the configuration parameters, the results of target feature extraction, and the results of temporal correlation feature extraction, the data to be processed is split into multiple data blocks to be processed.

[0008] Based on the preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed, multiple candidate TPU sub-resources corresponding to each data block to be processed are determined.

[0009] Determine the current capability parameters corresponding to each candidate TPU sub-resource, determine the matching score between each data block to be processed and each candidate TPU sub-resource based on the attribute parameters and the current capability parameters, and determine the target TPU sub-resource corresponding to each data block to be processed based on the matching score, thus obtaining the TPU resource allocation result.

[0010] Furthermore, the attribute parameters include data type and data size; the current capability parameters include the first operator precision parameter, the current achievable processing time, and the current cache balance.

[0011] The matching score between each data block to be processed and each candidate TPU sub-resource is determined based on attribute parameters and current capability parameters, including:

[0012] Determine the second operator precision parameter corresponding to the data type, and determine the operator precision difference between each data block to be processed and each candidate TPU sub-resource based on the first operator precision parameter and the second operator precision parameter;

[0013] Determine the estimated processing time of data blocks and the cache requirement of data blocks corresponding to the data size. Calculate the ratio of the estimated processing time of data blocks to the current available processing time, and the ratio of the cache requirement of data blocks to the current cache availability.

[0014] The matching score between each data block to be processed and each candidate TPU sub-resource is determined based on the operator precision difference, time consumption ratio, and cache size ratio.

[0015] Furthermore, the current capability parameters also include the single-pipeline capability parameters and the number of parallelizable pipelines for each candidate TPU sub-resource;

[0016] Determine the estimated processing time for data blocks corresponding to the data size, including:

[0017] The processing complexity of each data block to be processed is determined based on the data type and data volume, and the data processing efficiency of each candidate TPU sub-resource is determined based on the single pipeline capability parameters and the number of parallel pipeline stages.

[0018] The estimated processing time for each data block is determined based on the data size, processing complexity, and data processing efficiency.

[0019] Furthermore, based on the matching score, the target TPU sub-resources corresponding to each data block to be processed are determined, including:

[0020] Based on the preset matching score threshold and matching score, multiple candidate TPU sub-resources are filtered for available resources to obtain multiple available TPU sub-resources corresponding to each data block to be processed.

[0021] Based on the data type, multiple data blocks to be processed are grouped by data type to obtain multiple data type groups. Based on the estimated processing time of the data blocks, the baseline processing time of each data type group and the estimated processing time of multiple groups are determined.

[0022] Determine the relationship between the baseline processing time of each group and the estimated processing time of each group. Based on the relationship, determine the target available TPU sub-resource combination corresponding to each data type group, and obtain the target TPU sub-resource corresponding to each data block to be processed.

[0023] Furthermore, based on the estimated processing time of data blocks, the baseline processing time for each data type group and the estimated processing time for multiple groups are determined, including:

[0024] Based on the estimated processing time of the data block, determine the average processing time for each data block to be processed based on each available TPU sub-resource. Based on the average processing time, determine the group benchmark processing time corresponding to each data type group.

[0025] Randomly combine multiple available TPU sub-resources corresponding to multiple unprocessed data blocks in each data type group to obtain multiple available TPU sub-resource combinations corresponding to each data type group;

[0026] Based on the estimated processing time of data blocks, the estimated processing time corresponding to each available TPU sub-resource combination is calculated to obtain the estimated processing time of multiple groups.

[0027] Furthermore, the configuration parameters include the resource processing type of each TPU sub-resource in the TPU; the processing target feature extraction results include the target type and average number of targets in the data to be processed; and the temporal correlation feature extraction results include the temporal similarity of the data to be processed.

[0028] The data to be processed is split according to the configuration parameters, the results of target feature extraction, and the results of temporal correlation feature extraction, including:

[0029] Identify multiple available resource processing types corresponding to the target type, and perform a processing type consistency check on each available resource processing type and the resource processing type;

[0030] The operator fit between the TPU and the data to be processed is determined based on the consistency verification results of the processing type and the number of types of multiple available resource processing types. The operator fit, average number of targets and temporal similarity are normalized respectively.

[0031] The dynamic splitting factor of the data to be processed is determined based on the normalization result, and the data to be processed is split according to the dynamic splitting factor.

[0032] Furthermore, based on the preset TPU sub-resource and attribute parameter range lookup table and the attribute parameters of each data block to be processed, multiple candidate TPU sub-resources corresponding to each data block to be processed are determined, including:

[0033] Based on the attribute parameters of each data block to be processed, determine the current data modality label, current data dimension label, and TPU requirement label corresponding to each data block to be processed. Based on the data parameter range in the preset TPU sub-resource and attribute parameter range comparison table, determine multiple processable data modality labels, multiple processable data dimension labels, and multiple available function labels corresponding to each TPU sub-resource.

[0034] When the TPU requirement label is a modal fusion label, determine whether there is a modal transformation function label among multiple available function labels, and match the current data modal label and the current data dimension label with each processable data modal label and each processable data dimension label respectively;

[0035] Based on the judgment and matching results, multiple candidate TPU sub-resources corresponding to each data block to be processed are determined from multiple TPU sub-resources.

[0036] Secondly, embodiments of this application provide a TPU resource intelligent allocation system for deep learning, the system comprising:

[0037] The feature extraction module is used to obtain the data to be processed for deep learning and the configuration parameters of the TPU, and to perform target feature extraction and temporal correlation feature extraction on the data to be processed to obtain the target feature extraction result and the temporal correlation feature extraction result, respectively.

[0038] The data splitting module is used to split the data to be processed into multiple data blocks based on configuration parameters, the results of processing target feature extraction, and the results of time-series correlation feature extraction.

[0039] The candidate resource determination module is used to determine multiple candidate TPU sub-resources corresponding to each data block to be processed based on a preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed.

[0040] The resource allocation module is used to determine the current capability parameters corresponding to each candidate TPU sub-resource, determine the matching score between each data block to be processed and each candidate TPU sub-resource based on the attribute parameters and the current capability parameters, and determine the target TPU sub-resource corresponding to each data block to be processed based on the matching score, thus obtaining the TPU resource allocation result.

[0041] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0042] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0043] Fifthly, embodiments of this application also provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the method described in the first aspect.

[0044] In this embodiment, the data to be processed for deep learning and the configuration parameters of the TPU are obtained. The data to be processed is subjected to target feature extraction and temporal correlation feature extraction to obtain target feature extraction results and temporal correlation feature extraction results, respectively. Based on the configuration parameters, target feature extraction results, and temporal correlation feature extraction results, the data to be processed is split into multiple data blocks to be processed. Based on a preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed, multiple candidate TPU sub-resources corresponding to each data block to be processed are determined. The current capability parameters corresponding to each candidate TPU sub-resource are determined. Based on the attribute parameters and current capability parameters, the matching score between each data block to be processed and each candidate TPU sub-resource is determined. Based on the matching score, the target TPU sub-resource corresponding to each data block to be processed is determined, resulting in a TPU resource allocation result. The aforementioned intelligent TPU resource allocation method for deep learning solves the problems of unreasonable TPU resource allocation and inaccurate allocation results in existing technologies. By using configuration parameters, the results of target feature extraction and time-series correlation feature extraction of the data to be processed, the data to be processed is split into segments. Then, based on the attributes of the split data blocks and the current capacity of the TPU sub-resources, TPU resources are allocated to each data block. This achieves the goal of splitting the data to be processed based on TPU configuration parameters and the characteristics of the data to be processed, and allocating resources to the data blocks based on the real-time capacity of the TPU resources. This ensures TPU resource load balancing and improves the rationality of the TPU resource allocation method and the accuracy of the allocation results. Attached Figure Description

[0045] Figure 1 This is a flowchart of a TPU resource intelligent allocation method for deep learning provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the structure of the driver assistance system provided in the embodiments of this application;

[0047] Figure 3 This is a flowchart of determining the matching score provided in an embodiment of this application;

[0048] Figure 4 This is a flowchart illustrating the determination of a target TPU sub-resource provided in an embodiment of this application;

[0049] Figure 5 This is a structural block diagram of a TPU resource intelligent allocation system for deep learning provided in an embodiment of this application;

[0050] Figure 6 This is a structural block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application are described in detail below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0052] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0053] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0054] Firstly, this solution can be used in scenarios where TPU resources are allocated to process data received by an assisted driving system, particularly when the allocation is based on the current processing capacity or load of the vehicle's TPU resources. By configuring parameters, extracting the target features of the data to be processed, and extracting time-series correlation features, the data to be processed is split into smaller blocks. Then, based on the attributes of the split data blocks and the current capacity of the TPU sub-resources, TPU resources are allocated to each data block. This achieves the goal of splitting the data based on TPU configuration parameters and the characteristics of the data to be processed, and allocating resources to the data blocks based on the real-time capacity of the TPU resources. This ensures TPU resource load balancing and improves the rationality of the TPU resource allocation method and the accuracy of the allocation results. Based on the above use case, it is understandable that the execution entity for each step in this solution can be a vehicle control terminal with functions such as receiving data uploaded by the assisted driving system, splitting the data, and allocating TPU resources.

[0055] The following description, in conjunction with the accompanying drawings, details a method and system for intelligent allocation of TPU resources for deep learning provided in this application, through specific embodiments and application scenarios.

[0056] Figure 1 This is a flowchart illustrating a method for intelligent TPU resource allocation for deep learning, provided in an embodiment of this application. Figure 1 As shown, the specific steps include the following:

[0057] S101, obtain the data to be processed for deep learning and the configuration parameters of the TPU, and perform target feature extraction and temporal correlation feature extraction on the data to be processed to obtain the target feature extraction result and temporal correlation feature extraction result respectively.

[0058] The data to be processed can be the raw data received through the data interaction interface that needs to be processed. In this solution, the data to be processed is mainly multimodal data collected in real time during vehicle stationary or driving processes, received through the interface that interacts with the driver assistance system, used for driver assistance decision-making. Examples include image data, radar data, and time-series trajectory data. The TPU configuration parameters can be data describing the functions, specifications, and memory characteristics of the vehicle's TPU. In this solution, the TPU can be a TPU (Tensor Processing Unit) chip, and the TPU configuration parameters can be the inherent TPU attribute parameters of the TPU chip, which are the basis for resource allocation, including single pipeline processing capability, number of parallel pipelines, TPU sub-resource support set, maximum data processing length, and reconfigurable unit type. The TPU chip can include multiple pre-divided TPU resource modules, each TPU resource module corresponding to a TPU sub-resource, and each TPU sub-resource is adapted to a data processing type, a set of parallel pipelines, and single pipeline processing capability parameters. Target feature extraction can be an operation that identifies the data characteristics of target data that may be processed within the data to be processed. For example, if the input data is image data, entities such as people and objects in the image data may be identified and processed during assisted driving. The processing target features of the input data can be obtained by performing image recognition to extract the number of entities in the image data, the proportion of each entity's corresponding area to the entire image area, and the type of the entity (which corresponds to the image type). Temporal correlation feature extraction can be an operation to identify the degree of correlation between multiple data points in the time dimension. For example, by identifying the generation order, periodicity, and continuity of multiple data points, the degree of correlation between multiple data points in the time dimension can be determined, resulting in temporal correlation feature extraction. The processing target feature extraction result indicates the processing difficulty of the data, while the temporal correlation feature extraction result indicates the strength of the correlation between the data in the time dimension.

[0059] In one embodiment, vehicle driving data can be collected in real time by onboard sensors in the driver assistance system, and the TPU configuration parameters can be read through a communication interface with the TPU control module. Target data in the input data is identified using technologies such as image recognition, speech recognition, semantic recognition, and radar point cloud recognition. The characteristics of the target data in the input data are extracted to obtain the target feature extraction result. Multiple input data within adjacent preset time windows are determined based on the timestamps corresponding to each data point in the input data, and the similarity between these multiple input data within adjacent preset time windows is calculated to obtain the temporal correlation feature extraction result. For example, if the input data is video data, two consecutive video frames can be selected, and the ORB algorithm can be used to calculate the inter-frame feature point matching degree or similarity to obtain the temporal correlation feature extraction result.

[0060] S102, based on the configuration parameters, the results of the target feature extraction, and the results of the temporal correlation feature extraction, the data to be processed is split into multiple data blocks to be processed.

[0061] The data block to be processed can be a data sequence that can be independently computed by TPU sub-resources and has complete processing requirements, obtained by splitting the data to be processed according to the TPU processing capacity, the complexity of the data to be processed, and the degree of temporal correlation. The TPU sub-resources can be reconfigurable computing units in the TPU chip, such as 4D modules, 1D modules, and Recut modules.

[0062] In one embodiment, multiple TPU capability parameters in the configuration parameters can be weighted to obtain a TPU capability evaluation value. The TPU capability evaluation value, the target feature extraction result, and the time-series correlation feature extraction result are normalized. The normalized parameters are compared, and the product of the largest normalized result and the data length of the data to be processed is calculated. The product is used as the length of the data block to be processed. The data to be processed is split according to the length of the data block to be processed to obtain multiple data blocks to be processed.

[0063] In one embodiment, the configuration parameters include the resource processing type of each TPU sub-resource in the TPU; the processing target feature extraction result includes the target type and average number of targets in the data to be processed; the temporal correlation feature extraction result includes the temporal similarity of the data to be processed; and the data to be processed is split according to the configuration parameters, the processing target feature extraction result, and the temporal correlation feature extraction result, including: determining multiple available resource processing types corresponding to the target type, and performing a processing type consistency check on each available resource processing type and the resource processing type; determining the operator fit between the TPU and the data to be processed according to the processing type consistency check result and the number of types of multiple available resource processing types, and normalizing the operator fit, average number of targets, and temporal similarity respectively; determining the dynamic splitting factor of the data to be processed according to the normalization result, and splitting the data to be processed according to the dynamic splitting factor.

[0064] In this context, TPU sub-resources can be reconfigurable units within the TPU with independent computing capabilities. Examples include a 4D module for image or video 4D tensor computation, a 1D module for text or radar time-series 1D tensor computation, and a Recut module for multimodal cross-dimensional transformation. Resource processing type can be an inherent computing capability identifier of the TPU sub-resource, corresponding to the data modalities and core operators supported by each module of the TPU chip. For example, the resource processing type of the 4D module is image or video modality, and the corresponding core operators are convolution or pooling operators. The target type of the processing objective can be the category of entities to be identified or processed in the data to be processed. For example, the target type of image data processing objective can be image types of entities such as vehicles, pedestrians, traffic lights, and lane lines; the target type of radar data can be millimeter-wave data types of entities such as obstacles. The average number of targets can be the average number of entities in a single frame or a single window. The average number of targets reflects data complexity; a larger average number of targets indicates more complex input data, requiring finer decomposition. The temporal similarity of the data to be processed can be used to describe the continuity of the data in the time dimension. The available resource processing type can be the processing type of a TPU sub-resource capable of processing this target data type. Operator fit can be an indicator used to quantify the degree of matching between the TPU sub-resource and the operator requirements of the data to be processed. The dynamic splitting factor can be a parameter used to describe the granularity of splitting the data to be processed.

[0065] In one embodiment, the target type of the processing objective can be determined based on the data source or data format of the data to be processed. Multiple available resource processing types with the data processing functions required for that target type can be identified. It is then determined whether each available resource processing type matches the processing type of the resource processing type, and the number of resource processing types that match the available resource processing types is determined. The ratio of the number of matching types to the number of available resource processing types is calculated, and this ratio is used as the operator fit. The operator fit, average target quantity, and temporal similarity are normalized separately, and a weighted sum of the normalized results for each parameter is calculated based on preset weights. This weighted sum is used as a dynamic splitting factor for the data to be processed, and the data is split according to the dynamic splitting factor.

[0066] This solution determines the operator fit between the TPU and the data to be processed by verifying the consistency of processing types of available resources and processing types. Based on the operator fit, average number of targets, and temporal similarity, a dynamic splitting factor is determined for data splitting. This achieves the goal of splitting data by combining TPU resource processing capacity, data complexity, and continuity, improving the rationality and accuracy of data splitting, and helping to improve the real-time performance and accuracy of the target detection task in the assisted driving system.

[0067] S103, based on the preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed, determine multiple candidate TPU sub-resources corresponding to each data block to be processed.

[0068] The preset TPU sub-resource and attribute parameter range mapping table can be a pre-set table used to map each TPU sub-resource to the range of data attribute parameters it can process. For example, a TPU sub-resource might be the 4D module in a TPU chip, with a corresponding data attribute parameter range of image and video modalities, requiring convolution or pooling operator processing, and a data length range of 512×512~2048×2048 pixels. Candidate TPU sub-resources can be TPU sub-resources capable of processing a specific data block.

[0069] In one embodiment, the attribute parameters of each data block to be processed can be matched with a preset TPU sub-resource and attribute parameter range lookup table to determine the attribute parameter range to which the attribute parameters of each data block to be processed belong, and the TPU sub-resource corresponding to the parameter range, so as to obtain multiple candidate TPU sub-resources corresponding to each data block to be processed.

[0070] In one embodiment, based on a preset TPU sub-resource and attribute parameter range lookup table and the attribute parameters of each data block to be processed, multiple candidate TPU sub-resources corresponding to each data block to be processed are determined, including: determining the current data modality label, current data dimension label, and TPU requirement label corresponding to each data block to be processed according to the attribute parameters of each data block to be processed; and determining multiple processable data modality labels, multiple processable data dimension labels, and multiple available function labels corresponding to each TPU sub-resource according to the data parameter ranges in the preset TPU sub-resource and attribute parameter range lookup table; when the TPU requirement label is a modality fusion label, determining whether there is a modality conversion function label among the multiple available function labels, and matching the current data modality label and the current data dimension label with each processable data modality label and each processable data dimension label respectively; and determining multiple candidate TPU sub-resources corresponding to each data block to be processed from the multiple TPU sub-resources based on the determination result and the matching result.

[0071] The current data modality label can be an identifier describing the data type in each data block to be processed. The current data dimension label can be an identifier describing the structural size of the data block to be processed. The current data dimension label can be determined by parsing the mathematical storage format and dimension parameters of the data block to be processed. The TPU requirement label can be an identifier describing the computing power of the TPU sub-resources required for the data block to be processed. The processable data modality label can be an identifier of the data type that each TPU sub-resource can process. The processable data dimension label can be an identifier of the mathematical computation dimension that each TPU sub-resource can support. The available function label can be an identifier of the core computing functions or special processing capabilities that each TPU sub-resource can provide. The processable data modality label, processable data dimension label, and available function label reflect the inherent capabilities of each TPU sub-resource. The modality fusion label can be an identifier used to mark the fusion processing of multiple modal data. The modality transformation function label can be an identifier representing the TPU sub-resource's ability to perform dimension transformation and format unification of different modal data. For example, an identifier for converting a 4D image tensor into a 1D feature vector. The modality conversion function can unify the dimensions of data from multiple different modalities for use in multimodal data fusion processing.

[0072] In one embodiment, the data type, storage format, and computational requirements of each data block to be processed can be parsed, and a current data modality label, a current data dimension label, and a TPU requirement label can be generated based on the data type, storage format, and computational requirements, respectively. The attribute parameter range corresponding to each TPU sub-resource is read according to a preset TPU sub-resource and attribute parameter range lookup table, and the processable data modality label, processable data dimension label, and available function label of each TPU sub-resource are determined based on the data type, storage format, and computational requirements corresponding to the attribute parameter range. If the TPU requirement label of a data block to be processed includes a modality fusion label, it can be determined whether the available function label of the TPU sub-resource has a modality conversion function. If it does, the matching operation of data modality label and dimension label continues. TPU sub-resources whose current data modality label matches the processable data modality label and whose current data dimension label matches the processable data dimension label are selected as candidate TPU sub-resources corresponding to the data block to be processed; otherwise, it is directly determined that there are no candidate TPU sub-resources corresponding to the data block to be processed.

[0073] This solution determines data requirement tags and available function tags through attribute parameters. When the TPU requirement tag is a modal fusion tag, it determines whether a modal conversion function tag exists among multiple available function tags and performs tag matching. Based on the judgment and matching results, it identifies multiple candidate TPU sub-resources corresponding to each data block to be processed. This achieves the goal of matching TPU sub-resources based on the tagging of processing requirements and processing capabilities, and provides TPU sub-resource filtering for data processing requirements related to modal fusion. This avoids data processing failures or delays caused by TPU function mismatches, improving the efficiency and accuracy of candidate TPU sub-resource filtering.

[0074] S104, determine the current capability parameters corresponding to each candidate TPU sub-resource, determine the matching score between each data block to be processed and each candidate TPU sub-resource based on the attribute parameters and the current capability parameters, and determine the target TPU sub-resource corresponding to each data block to be processed based on the matching score, and obtain the TPU resource allocation result.

[0075] The current capability parameters can be data used to quantify the available capabilities of each candidate TPU sub-resource in its current running state. Examples include: the amount of data currently loaded and executing for each candidate TPU sub-resource, the proportion of idle computing power to total computing power, and the available space in the on-chip cache. The matching score can be a parameter used to evaluate whether a candidate TPU sub-resource can cache and process a specific data block when assigned to it, and the degree of processing suitability. The target TPU sub-resource can be the TPU sub-resource that can process each data block most efficiently while ensuring load balancing among multiple candidate TPU sub-resources.

[0076] In one embodiment, parameters such as the current processing data volume and currently occupied on-chip memory of each TPU sub-resource can be read in real time through the control interface of each TPU sub-resource to obtain the current capability parameters. Based on the current processing data volume of each candidate TPU sub-resource in the current capability parameters, the data block processing wait time is estimated. Based on the occupied on-chip memory of each candidate TPU sub-resource in the current capability parameters, the size of the data that can be covered is determined. Based on the required processing efficiency in the attribute parameters, the data block processing wait time corresponding to each data block is determined, and the time deviation between the data block processing wait time and the data block processing wait time is calculated. The data volume deviation between the size of the data that can be covered and the size of the data block to be processed in the attribute parameters is also calculated. The time deviation and data volume deviation are normalized, and the normalized sum of the time deviation and data volume deviation is calculated to obtain the matching degree score between each data block to be processed and each candidate TPU sub-resource. The candidate TPU sub-resource with the highest matching degree score with each data block to be processed is selected as the target TPU sub-resource, and the target TPU sub-resources of each data block to be processed are integrated to obtain the TPU resource allocation result.

[0077] Figure 2 This is a schematic diagram of the structure of the driver assistance system provided in an embodiment of this application. Figure 2 As shown in the diagram, the system includes an input module, a sensing and computing module, and an output module. The input module acquires the data to be processed for deep learning, including a millimeter-wave radar module that collects radar data such as distance, speed, and angle of obstacles around the vehicle. This is a crucial source of data for processing in assisted driving scenarios, providing raw input for subsequent radar data processing and fusion decision-making. The camera module collects image data of the vehicle's surrounding environment to identify targets such as pedestrians, vehicles, and traffic signs, and is the core module for acquiring visual perception data. The sensing and computing module is the perception and data processing module in the assisted driving system. It includes a TPU core layer composed of a CPU, TPU, DSP, and peripheral TPU interfaces; a driver + BSP support package; a Linux operating system; a deep learning acceleration SDK (convolution / pooling / ReLU); and a data fusion SDK module. The output module outputs the results processed by the sensing and computing module to the central control screen for intuitive presentation to the driver.

[0078] The TPU core layer provides TPU support for the upper-layer "driver + BSP support package" and various SDKs, serving as the TPU carrier for software function implementation. The CPU is responsible for overall system control and logical operations. The TPU is specifically designed to accelerate artificial intelligence tasks, significantly improving the running efficiency of deep learning algorithms. The DSP excels in digital signal processing and can efficiently process signals from sensors such as radar. The peripheral TPU interface is used to connect various input / output devices. The TPU (Tension Processing Unit) chip is the object used for TPU resource allocation in this solution. The vehicle control end needs to dynamically allocate TPU resources in the TPU chip based on the data modality, dimension, and TPU requirements of the data to be processed, in order to meet the real-time and computing power requirements of the assisted driving tasks. For example, when processing image-type data, it will be allocated to the 4D module of the TPU; when processing radar time-series data, it will be allocated to the 1D module. The driver + BSP support package is a collection of device drivers and board-level support packages. The driver enables the operating system to communicate with the TPU device; the BSP support package provides low-level support for the TPU platform, including initialization, configuration, and other functions, acting as a bridge between the operating system and the TPU. The Linux operating system provides fundamental services such as task scheduling, memory management, and file systems, offering a runtime environment for various SDKs and applications at the upper layers, ensuring system stability and scalability. The Deep Acceleration SDK (Convolution / Pooling / ReLU) is a software development kit for accelerating deep neural networks, containing accelerated implementations of deep learning operators such as convolution, pooling, and ReLU. It is used to accelerate computationally intensive operations in deep learning models using TPUs, significantly improving the processing speed of AI tasks such as image recognition and object detection. The allocation of TPU resources in this solution must be based on the operator requirements of the Deep Acceleration SDK to ensure the completeness of operator support for TPU resources. The Data Fusion SDK includes a radar data processing SDK for specifically processing radar data and an image data processing SDK for specifically processing image data.

[0079] The technical solution provided in this application embodiment obtains the data to be processed for deep learning and the configuration parameters of the TPU; performs target feature extraction and temporal correlation feature extraction on the data to be processed to obtain the target feature extraction result and the temporal correlation feature extraction result, respectively; splits the data to be processed into multiple data blocks to be processed based on the configuration parameters, the target feature extraction result, and the temporal correlation feature extraction result; determines multiple candidate TPU sub-resources corresponding to each data block to be processed based on a preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed; determines the current capability parameters corresponding to each candidate TPU sub-resource; determines the matching degree score between each data block to be processed and each candidate TPU sub-resource based on the attribute parameters and the current capability parameters; and determines the target TPU sub-resource corresponding to each data block to be processed based on the matching degree score, thus obtaining the TPU resource allocation result. The aforementioned intelligent TPU resource allocation method for deep learning solves the problems of unreasonable TPU resource allocation and inaccurate allocation results in existing technologies. By using configuration parameters, the results of target feature extraction and time-series correlation feature extraction of the data to be processed, the data to be processed is split into segments. Then, based on the attributes of the split data blocks and the current capacity of the TPU sub-resources, TPU resources are allocated to each data block. This achieves the goal of splitting the data to be processed based on TPU configuration parameters and the characteristics of the data to be processed, and allocating resources to the data blocks based on the real-time capacity of the TPU resources. This ensures TPU resource load balancing and improves the rationality of the TPU resource allocation method and the accuracy of the allocation results.

[0080] Figure 3 This is a flowchart illustrating the determination of a matching score provided in an embodiment of this application. For example... Figure 3 As shown, the attribute parameters include data type and data size, and the current capability parameters include the first operator precision parameter, the current bearable time, and the current cache balance. Specifically, the steps are as follows:

[0081] S301, determine the second operator precision parameter corresponding to the data type, and determine the operator precision difference between each data block to be processed and each candidate TPU sub-resource based on the first operator precision parameter and the second operator precision parameter.

[0082] The first operator precision parameter can be the operator precision specification actually supported by each candidate TPU sub-resource. The current allowable processing time can be the maximum permitted processing time that each candidate TPU sub-resource can allocate to the data block to be processed without affecting other data processing tasks. The current allowable processing time is jointly determined by the real-time load and task priority of each candidate TPU sub-resource. The current cache availability can be the real-time available on-chip cache space of each candidate TPU sub-resource. The second operator precision parameter can be the minimum precision standard required by the data type of each data block to be processed. The operator precision difference can be an indicator used to quantify the deviation between the first operator precision of each candidate TPU sub-resource and the second operator precision of the data block to be processed.

[0083] In one embodiment, a second operator precision parameter corresponding to the data type of each data block to be processed can be determined according to a preset assisted driving safety standard, and the difference between the first operator precision parameter and the second operator precision parameter can be calculated to obtain the operator precision difference between each data block to be processed and each candidate TPU sub-resource.

[0084] S302, determine the estimated processing time of the data block and the cache requirement of the data block corresponding to the data size, and calculate the ratio of the estimated processing time of the data block to the current available processing time, and the ratio of the cache requirement of the data block to the current cache availability.

[0085] The estimated processing time for a data block can be determined based on the historical processing performance data of the TPU sub-resource and the size of the data block to be processed, representing the average processing time of the data block. The data block cache requirement can be the storage space required when the data block to be processed is loaded into the on-chip cache of the TPU sub-resource with the precision of the second operator. For example, a radar data block with 143 points × 4 features (572 data points in total), processed with FP16 precision, requires 572 × 2 bytes = 1144 bytes ≈ 1.1KB of cache; processed with INT8 precision, it requires 572 × 1 byte ≈ 0.56KB. The time consumption ratio can be the ratio of the estimated processing time of a data block to the current processing time that the candidate TPU sub-resource can handle. The time consumption ratio can be used to quantify the pressure on each candidate TPU sub-resource when processing the corresponding data block. The cache size ratio can be the ratio of the data block cache requirement to the current cache availability of each candidate TPU sub-resource. The cache size ratio can be used to quantify the TPU cache occupancy pressure.

[0086] In one embodiment, the estimated processing time of a data block corresponding to the data size can be determined based on the historical processing time of each candidate TPU sub-resource and the data size of the database to be processed. The data block cache requirement for the data block to be processed can be determined based on the second operator precision parameter and the data size. The ratio of the estimated processing time of the data block to the current available processing time is calculated to obtain the time percentage, and the ratio of the data block cache requirement to the current cache availability is calculated to obtain the cache percentage.

[0087] In one embodiment, the current capability parameters also include the single-pipeline capability parameters and the number of parallelizable pipeline stages for each candidate TPU sub-resource; determining the estimated processing time of data blocks corresponding to the data size includes: determining the processing complexity of each data block to be processed based on the data type and data size, and determining the data processing efficiency of each candidate TPU sub-resource based on the single-pipeline capability parameters and the number of parallelizable pipeline stages; and determining the estimated processing time of data blocks corresponding to each data block to be processed based on the data size, processing complexity, and data processing efficiency.

[0088] The processing complexity can be a metric for the difficulty of processing, quantified based on the data type and size of each data block to be processed. The single pipeline capability parameter can be the core processing capability metric of a single pipeline in a TPU sub-resource, i.e., the smallest parallel computing unit. For example, in an assisted driving scenario, a single pipeline in the 4D module can process convolution operations of 1 million pixels per second. The number of parallelizable pipelines can be the number of pipelines that a TPU sub-resource supports running simultaneously. The number of parallelizable pipelines reflects the parallel computing capability of the TPU; the more pipelines, the higher the parallelism and the stronger the processing efficiency. For example, the 4D module supports a fixed 8 parallel pipelines, while the 1D module supports 4-8 dynamic pipelines. Data processing efficiency can be data used to quantify the processing speed of each candidate TPU sub-resource for processing the corresponding data block to be processed. The estimated processing time for a data block can be the theoretical processing time for processing the data block to be processed, calculated based on the data size, processing complexity, and data processing efficiency of each candidate TPU sub-resource.

[0089] In one embodiment, a complexity scoring lookup table can be pre-defined. Based on the data type, the data dimensions and operator requirements of each data block to be processed are determined. The data dimensions, operator requirements, and data size are matched against the pre-defined complexity scoring lookup table to determine the complexity score corresponding to each data dimension, operator requirement, and data size. The complexity scores are summed to determine the processing complexity of each data block to be processed. The data processing efficiency of each candidate TPU sub-resource is determined by multiplying the single pipeline capability parameter by the number of parallelizable pipeline stages. The weighted data size of the data block to be processed is obtained by multiplying the data size by the processing complexity. The ratio of the weighted data size to the data processing efficiency is then calculated to obtain the estimated processing time for each data block to be processed.

[0090] This solution determines the processing complexity of each data block to be processed, and determines the data processing efficiency of each candidate TPU sub-resource based on the single pipeline capability parameters and the number of parallel pipeline stages. It also estimates the processing time of each data block based on the processing complexity, data volume, and data processing efficiency. This achieves the goal of estimating the processing time of each data block to be processed by combining the TPU parallel processing capability and data block complexity, thereby improving the accuracy of processing time assessment and facilitating the rationality of subsequent TPU sub-resource allocation.

[0091] The technical solution provided in this application determines the matching score between each data block to be processed and each candidate TPU sub-resource by calculating the difference in operator precision, the proportion of processing time, and the proportion of data cache between each data block to be processed and each candidate TPU sub-resource. This can improve the accuracy of the matching score calculation results and help ensure the load balance of TPU sub-resources.

[0092] Figure 4 This is a flowchart illustrating the determination of a target TPU sub-resource provided in an embodiment of this application. For example... Figure 4 As shown, the specific steps include the following:

[0093] S401, based on the preset matching score threshold and matching score, select available resources for multiple candidate TPU sub-resources to obtain multiple available TPU sub-resources corresponding to each data block to be processed.

[0094] The preset matching score threshold can be a pre-set minimum matching score when TPU sub-resources are available. Available TPU sub-resources can be those TPU sub-resources that, in the current state, can process the data block to be processed and whose processing capacity matches the data to be processed with a matching score higher than the minimum matching score among multiple candidate TPU sub-resources.

[0095] In one embodiment, the matching scores can be compared with a preset matching score threshold, and available TPU sub-resources that are greater than the preset matching score threshold from multiple candidate TPU sub-resources can be filtered to obtain multiple available TPU sub-resources corresponding to each data block to be processed.

[0096] S402, group multiple data blocks to be processed according to data type to obtain multiple data type groups, and determine the group baseline processing time and multiple group estimated processing time according to the estimated processing time of the data blocks.

[0097] The baseline processing time for a group can be the theoretical reference value for the processing time of multiple data blocks of the same data type. The estimated processing time for a group can be the estimated processing time of the entire group when multiple data blocks of the same data type are processed using different TPU sub-resource allocation strategies.

[0098] In one embodiment, multiple data blocks to be processed can be grouped by data type, with data blocks of the same data type grouped together to obtain multiple data type groups. The median of multiple estimated processing times corresponding to each data block is identified to determine the baseline estimated processing time for each data block. The median of the baseline estimated processing times for multiple data blocks within the same data type group is then identified to obtain the baseline processing time for each data type group. Multiple available TPU sub-resources corresponding to each data block within the same data type group are randomly allocated to obtain multiple sets of available TPU sub-resource allocation combinations corresponding to the same data type group. The median of the estimated processing time corresponding to each set of available TPU sub-resource allocation combinations is then identified to obtain the estimated processing time for multiple groups.

[0099] In one embodiment, a first average processing time for processing each data block to be processed based on the estimated processing time of the data block can be determined, using each available TPU sub-resource. The average of multiple first average processing times corresponding to each data type group is then calculated to obtain the group-based baseline processing time corresponding to each data type group. Since available TPU sub-resources can process the corresponding data blocks to be processed, multiple available TPU sub-resources corresponding to each data block to be processed can be randomly allocated, and the resource allocation results for each data block to be processed can be integrated to obtain multiple combinations of available TPU sub-resources corresponding to each data type group. The estimated processing time corresponding to each combination of available TPU sub-resources can be calculated based on the estimated processing time of the data block to obtain multiple group-based estimated processing times.

[0100] This solution determines the baseline processing time for each data type group by using the average processing time of each available TPU sub-resource to process each data block. It then randomly combines multiple available TPU sub-resources corresponding to multiple data blocks in each data type group, and calculates the estimated processing time for multiple groups based on the estimated processing time of the data blocks. This improves the accuracy of the baseline processing time calculation for groups and the comprehensiveness of the estimated processing time assessment for groups, which is beneficial to improving the accuracy of subsequent TPU sub-resource allocation results.

[0101] S403, determine the relationship between the baseline processing time of each group and the estimated processing time of each group, and determine the target available TPU sub-resource combination corresponding to each data type group based on the relationship, so as to obtain the target TPU sub-resource corresponding to each data block to be processed.

[0102] The target available TPU sub-resource combination can be an available TPU sub-resource combination whose estimated processing time for the group is less than or equal to the group's baseline processing time.

[0103] In one embodiment, the relationship between the baseline processing time of a group and the estimated processing time of each group can be compared. The available TPU sub-resources with the estimated processing time of the group being less than or equal to the baseline processing time of the group are used as the target available TPU sub-resources corresponding to each data type group, thereby obtaining the target TPU sub-resources corresponding to each data block to be processed.

[0104] The technical solution provided in this application uses a preset matching score threshold and matching score to filter available resources from multiple candidate TPU sub-resources. It determines the group baseline processing time and multiple group estimated processing times based on the estimated processing time of data blocks. Based on the processing time, it determines the target TPU sub-resources corresponding to each data block to be processed. This achieves the goal of aligning the processing time granularity of TPU sub-resources with processing time, thereby improving the rationality of TPU resource allocation.

[0105] Figure 5 This is a structural block diagram of a TPU resource intelligent allocation system for deep learning provided in an embodiment of this application. Figure 5 As shown, it specifically includes the following:

[0106] The feature extraction module 501 is used to obtain the data to be processed for deep learning and the configuration parameters of the TPU, and to perform processing target feature extraction and temporal correlation feature extraction on the data to be processed to obtain the processing target feature extraction result and the temporal correlation feature extraction result, respectively.

[0107] The data splitting module 502 is used to split the data to be processed according to the configuration parameters, the processing target feature extraction results and the time-series correlation feature extraction results to obtain multiple data blocks to be processed.

[0108] The candidate resource determination module 503 is used to determine multiple candidate TPU sub-resources corresponding to each data block to be processed based on a preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed.

[0109] The resource allocation module 504 is used to determine the current capability parameters corresponding to each candidate TPU sub-resource, determine the matching score between each data block to be processed and each candidate TPU sub-resource based on the attribute parameters and the current capability parameters, and determine the target TPU sub-resource corresponding to each data block to be processed based on the matching score, thereby obtaining the TPU resource allocation result.

[0110] Furthermore, the attribute parameters include data type and data size; the current capability parameters include the first operator precision parameter, the current achievable processing time, and the current cache balance.

[0111] Resource allocation module 504, specifically used for:

[0112] Determine the second operator precision parameter corresponding to the data type, and determine the operator precision difference between each data block to be processed and each candidate TPU sub-resource based on the first operator precision parameter and the second operator precision parameter;

[0113] Determine the estimated processing time of data blocks and the cache requirement of data blocks corresponding to the data size. Calculate the ratio of the estimated processing time of data blocks to the current available processing time, and the ratio of the cache requirement of data blocks to the current cache availability.

[0114] The matching score between each data block to be processed and each candidate TPU sub-resource is determined based on the operator precision difference, time consumption ratio, and cache size ratio.

[0115] Furthermore, the current capability parameters also include the single-pipeline capability parameters and the number of parallelizable pipelines for each candidate TPU sub-resource;

[0116] Resource allocation module 504, specifically used for:

[0117] The processing complexity of each data block to be processed is determined based on the data type and data volume, and the data processing efficiency of each candidate TPU sub-resource is determined based on the single pipeline capability parameters and the number of parallel pipeline stages.

[0118] The estimated processing time for each data block is determined based on the data size, processing complexity, and data processing efficiency.

[0119] Furthermore, the resource allocation module 504 is specifically used for:

[0120] Based on the preset matching score threshold and matching score, multiple candidate TPU sub-resources are filtered for available resources to obtain multiple available TPU sub-resources corresponding to each data block to be processed.

[0121] Based on the data type, multiple data blocks to be processed are grouped by data type to obtain multiple data type groups. Based on the estimated processing time of the data blocks, the baseline processing time of each data type group and the estimated processing time of multiple groups are determined.

[0122] Determine the relationship between the baseline processing time of each group and the estimated processing time of each group. Based on the relationship, determine the target available TPU sub-resource combination corresponding to each data type group, and obtain the target TPU sub-resource corresponding to each data block to be processed.

[0123] Furthermore, the resource allocation module 504 is specifically used for:

[0124] Based on the estimated processing time of the data block, determine the average processing time for each data block to be processed based on each available TPU sub-resource. Based on the average processing time, determine the group benchmark processing time corresponding to each data type group.

[0125] Randomly combine multiple available TPU sub-resources corresponding to multiple unprocessed data blocks in each data type group to obtain multiple available TPU sub-resource combinations corresponding to each data type group;

[0126] Based on the estimated processing time of data blocks, the estimated processing time corresponding to each available TPU sub-resource combination is calculated to obtain the estimated processing time of multiple groups.

[0127] Furthermore, the configuration parameters include the resource processing type of each TPU sub-resource in the TPU; the processing target feature extraction results include the target type and average number of targets in the data to be processed; and the temporal correlation feature extraction results include the temporal similarity of the data to be processed.

[0128] Data splitting module 502 is specifically used for:

[0129] Identify multiple available resource processing types corresponding to the target type, and perform a processing type consistency check on each available resource processing type and the resource processing type;

[0130] The operator fit between the TPU and the data to be processed is determined based on the consistency verification results of the processing type and the number of types of multiple available resource processing types. The operator fit, average number of targets and temporal similarity are normalized respectively.

[0131] The dynamic splitting factor of the data to be processed is determined based on the normalization result, and the data to be processed is split according to the dynamic splitting factor.

[0132] Furthermore, the candidate resource determination module 503 is specifically used for:

[0133] Based on the attribute parameters of each data block to be processed, determine the current data modality label, current data dimension label, and TPU requirement label corresponding to each data block to be processed. Based on the data parameter range in the preset TPU sub-resource and attribute parameter range comparison table, determine multiple processable data modality labels, multiple processable data dimension labels, and multiple available function labels corresponding to each TPU sub-resource.

[0134] When the TPU requirement label is a modal fusion label, determine whether there is a modal transformation function label among multiple available function labels, and match the current data modal label and the current data dimension label with each processable data modal label and each processable data dimension label respectively;

[0135] Based on the judgment and matching results, multiple candidate TPU sub-resources corresponding to each data block to be processed are determined from multiple TPU sub-resources.

[0136] The technical solution provided in this application includes a feature extraction module for obtaining the data to be processed for deep learning and the configuration parameters of the TPU, and performing target feature extraction and temporal correlation feature extraction on the data to be processed to obtain target feature extraction results and temporal correlation feature extraction results, respectively; a data splitting module for splitting the data to be processed according to the configuration parameters, the target feature extraction results, and the temporal correlation feature extraction results to obtain multiple data blocks to be processed; a candidate resource determination module for determining multiple candidate TPU sub-resources corresponding to each data block to be processed based on a preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed; and a resource allocation module for determining the current capability parameters corresponding to each candidate TPU sub-resource, determining the matching degree score between each data block to be processed and each candidate TPU sub-resource according to the attribute parameters and the current capability parameters, and determining the target TPU sub-resource corresponding to each data block to be processed according to the matching degree score to obtain the TPU resource allocation result. The aforementioned intelligent TPU resource allocation system for deep learning solves the problems of unreasonable TPU resource allocation methods and inaccurate allocation results in existing technologies. It splits the data to be processed based on configuration parameters, the results of target feature extraction, and the results of temporal correlation feature extraction. Then, it allocates TPU resources to each data block based on the attributes of the split data blocks and the current capacity of the TPU sub-resources. This achieves the goal of splitting the data based on TPU configuration parameters and the characteristics of the data to be processed, and allocating resources to the data blocks based on the real-time capacity of the TPU resources. This ensures balanced TPU resource load and improves the rationality of the TPU resource allocation method and the accuracy of the allocation results.

[0137] The TPU resource intelligent allocation system for deep learning, as described in this application embodiment, can be configured in a device, or as a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0138] The TPU resource intelligent allocation system for deep learning in this application embodiment can be an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0139] The TPU resource intelligent allocation system for deep learning provided in this application embodiment can realize the various processes implemented in the above method embodiments. To avoid repetition, it will not be described again here.

[0140] like Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a program or instructions stored in the memory 602 and executable on the processor 601. When the program or instructions are executed by the processor 601, they implement the various processes of the above-described embodiment of a TPU resource intelligent allocation method for deep learning and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0141] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0142] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of the intelligent allocation method for TPU resources for deep learning and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0143] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0144] This application also provides a program product including program code. When the program product is run on a computer device, the program code causes the computer device to perform the steps of the methods described above according to various exemplary embodiments of this application. For example, the computer device can execute a TPU resource intelligent allocation method for deep learning described in an embodiment of this application. The program product can be implemented using any combination of one or more readable media.

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element. Furthermore, it should be noted that the scope of the methods and systems in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus the necessary general-purpose TPU platform. Of course, they can also be implemented by TPU, but in many cases the former is a better implementation. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0147] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0148] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method for intelligent allocation of TPU resources for deep learning, characterized in that, The method includes: The system obtains the data to be processed for deep learning and the configuration parameters of the TPU. It performs target feature extraction and temporal correlation feature extraction on the data to be processed to obtain the target feature extraction results and temporal correlation feature extraction results, respectively. The data to be processed includes multimodal data for assisted driving decision-making that is collected in real time during vehicle stationary or driving process and received through the interface that interacts with the assisted driving system. The data to be processed is split into multiple data blocks based on the configuration parameters, the processing target feature extraction results and the temporal correlation feature extraction results. Based on the preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each data block to be processed, multiple candidate TPU sub-resources corresponding to each data block to be processed are determined. The attribute parameters include data type and data size. Determine the current capability parameters corresponding to each of the candidate TPU sub-resources, the current capability parameters including the first operator precision parameter, the current bearable time, and the current cache balance; A second operator precision parameter corresponding to the data type is determined. Based on the first operator precision parameter and the second operator precision parameter, the operator precision difference between each data block to be processed and each candidate TPU sub-resource is determined. The estimated processing time of the data block and the data block cache requirement corresponding to the data size are determined. The time ratio between the estimated processing time of the data block and the current bearable time, and the cache ratio between the data block cache requirement and the current cache balance are calculated. Based on the operator precision difference, the time ratio, and the cache ratio, the matching score between each data block to be processed and each candidate TPU sub-resource is determined. Based on the matching score, the target TPU sub-resource corresponding to each data block to be processed is determined, and the TPU resource allocation result is obtained.

2. The intelligent TPU resource allocation method for deep learning according to claim 1, characterized in that, The current capability parameters also include the single pipeline capability parameters and the number of parallel pipelines for each candidate TPU sub-resource; Determining the estimated processing time for data blocks corresponding to the data size includes: The processing complexity of each data block to be processed is determined based on the data type and the data volume, and the data processing efficiency of each candidate TPU sub-resource is determined based on the single pipeline capability parameter and the number of parallelizable pipeline stages. Based on the data volume, processing complexity, and data processing efficiency, the estimated processing time for each data block to be processed is determined.

3. The intelligent TPU resource allocation method for deep learning according to claim 1, characterized in that, The step of determining the target TPU sub-resource corresponding to each of the data blocks to be processed based on the matching score includes: Based on a preset matching score threshold and the matching score, the available resources of the multiple candidate TPU sub-resources are filtered to obtain multiple available TPU sub-resources corresponding to each of the data blocks to be processed. Based on the data type, the multiple data blocks to be processed are grouped by data type to obtain multiple data type groups. Based on the estimated processing time of the data blocks, the baseline processing time of each data type group and the estimated processing time of multiple groups are determined. Determine the relationship between the baseline processing time of the group and the estimated processing time of each group, and determine the target available TPU sub-resource combination corresponding to each data type group based on the relationship, so as to obtain the target TPU sub-resource corresponding to each data block to be processed.

4. The intelligent TPU resource allocation method for deep learning according to claim 3, characterized in that, The step of determining the group baseline processing time and multiple group estimated processing times corresponding to each data type group based on the estimated processing time of the data block includes: Based on the estimated processing time of the data block, determine the average processing time for processing each of the data blocks to be processed based on each of the available TPU sub-resources, and determine the group benchmark processing time corresponding to each of the data types based on the average processing time; Randomly combine multiple available TPU sub-resources corresponding to multiple data blocks to be processed in each of the data type groups to obtain multiple available TPU sub-resource combinations corresponding to each of the data type groups; Based on the estimated processing time of the data blocks, the estimated processing time corresponding to each of the available TPU sub-resource combinations is calculated to obtain the estimated processing time of multiple groups.

5. The intelligent TPU resource allocation method for deep learning according to claim 1, characterized in that, The configuration parameters include the resource processing type of each TPU sub-resource in the TPU; the processing target feature extraction results include the target type and average number of targets in the data to be processed; The temporal correlation feature extraction result includes the temporal similarity of the data to be processed; The step of splitting the data to be processed based on the configuration parameters, the target feature extraction result, and the temporal correlation feature extraction result includes: Multiple available resource processing types corresponding to the target type are determined, and a processing type consistency check is performed on each available resource processing type and each resource processing type. The operator fit between the TPU and the data to be processed is determined based on the consistency verification result of the processing type and the number of types of the multiple available resource processing types. The operator fit, the average target number and the temporal similarity are normalized respectively. The dynamic splitting factor of the data to be processed is determined based on the normalization result, and the data to be processed is split according to the dynamic splitting factor.

6. The intelligent TPU resource allocation method for deep learning according to claim 1, characterized in that, The step of determining multiple candidate TPU sub-resources corresponding to each of the data blocks to be processed, based on a preset TPU sub-resource and attribute parameter range lookup table and the attribute parameters of each data block to be processed, includes: Based on the attribute parameters of each data block to be processed, determine the current data modality label, current data dimension label, and TPU requirement label corresponding to each data block to be processed. Based on the data parameter range in the preset TPU sub-resource and attribute parameter range comparison table, determine multiple processable data modality labels, multiple processable data dimension labels, and multiple available function labels corresponding to each TPU sub-resource. When the TPU demand tag is a modal fusion tag, determine whether there is a modal conversion function tag among the multiple available function tags, and match the current data modal tag and the current data dimension tag with each of the processable data modal tags and each of the processable data dimension tags respectively; Based on the judgment and matching results, multiple candidate TPU sub-resources corresponding to each of the data blocks to be processed are determined from multiple TPU sub-resources.

7. A TPU resource intelligent allocation system for deep learning, characterized in that, The system includes: The feature extraction module is used to obtain the data to be processed for deep learning and the configuration parameters of the TPU, and to perform target feature extraction and temporal correlation feature extraction on the data to be processed to obtain the target feature extraction result and the temporal correlation feature extraction result, respectively. The data to be processed includes multimodal data for assisted driving decision-making that is received in real time during vehicle stationary or driving through an interface that interacts with the assisted driving system. The data splitting module is used to split the data to be processed into multiple data blocks according to the configuration parameters, the processing target feature extraction result and the time-series correlation feature extraction result. The candidate resource determination module is used to determine multiple candidate TPU sub-resources corresponding to each of the data blocks to be processed based on a preset TPU sub-resource and attribute parameter range comparison table and the attribute parameters of each of the data blocks to be processed. The attribute parameters include data type and data size. The resource allocation module is used to determine the current capability parameters corresponding to each candidate TPU sub-resource. The current capability parameters include a first operator precision parameter, the current bearable time, and the current cache balance. It also determines a second operator precision parameter corresponding to the data type, determines the operator precision difference between each data block to be processed and each candidate TPU sub-resource based on the first and second operator precision parameters, determines the estimated processing time and cache requirement of the data block corresponding to the data size, calculates the time ratio between the estimated processing time and the current bearable time, and the cache ratio between the cache requirement and the current cache balance, determines the matching score between each data block to be processed and each candidate TPU sub-resource based on the operator precision difference, the time ratio, and the cache ratio, determines the matching score between each data block to be processed and each candidate TPU sub-resource based on the attribute parameters and the current capability parameters, and determines the target TPU sub-resource corresponding to each data block to be processed based on the matching score, thus obtaining the TPU resource allocation result.

8. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and running on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the TPU resource intelligent allocation method for deep learning as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the TPU resource intelligent allocation method for deep learning as described in any one of claims 1-6.

Images