Method and device for determining data shape range of operator, equipment and storage medium
By generating and parsing performance tracking files during the training process of deep learning models, and obtaining and updating the data shape range of operators in real time, the problem of being unable to accurately analyze operator accuracy in the existing technology is solved, and more accurate and targeted Analysis results.
Patent Information
- Application Number
- CN202411508000.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2025-02-18
Smart Images
Figure CN119476529A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for determining the data shape range of an operator. Background Art
[0002] During deep learning model training, the data shape of operators is a key factor affecting their accuracy and performance. Because model training typically uses different input data at different training steps, the data shape of operators often changes dynamically. Currently, during model initialization, forward and backward hook functions are registered for each operator. During model training, these hook functions are called to dump the operator's input data to local storage. After training is complete, by parsing this stored data, we can determine the data shape range of each operator throughout model training, allowing us to analyze and optimize operator accuracy.
[0003] Model structures are composed of multiple operators executed in a specific order. These operators may have different data shape ranges when called in different layers or modules of the model. However, the above solution can only obtain the data shape range of each operator within the global scope of the entire model. It cannot obtain the data shape range of the operator at a specific call point (such as a certain layer of the model), making it impossible to perform more precise and targeted analysis of operator accuracy. Summary of the Invention
[0004] The present invention provides a method, apparatus, device and storage medium for determining the data shape range of an operator, so as to solve the defect in the related art that the data shape range of an operator at a specific call point cannot be obtained.
[0005] The present invention provides a method for determining a data shape range of an operator, comprising: Training the model based on the training data of the current step, and generating a performance tracking file of the current step during the training process; Parsing the performance tracking file of the current step to obtain the operator name, call location, and data shape of each operator at each call location of each operator in the current step; Based on the operator name and calling position of each operator, the data shape of each operator at each calling position of the current step is merged with the data shape range of each operator at each calling position of the previous step to obtain the data shape range of each operator at each calling position of the current step.
[0006] According to a method for determining a data shape range of an operator provided by the present invention, generating a performance tracking file of the current step during training includes: During model training, obtain the data shape, operator name, and calling location of each operator when it is called; The data shape, operator name, and calling location of each operator when it is called are written as event data into the performance tracking file of the current step.
[0007] According to a method for determining the data shape range of an operator provided by the present invention, parsing the performance tracking file of the current step to obtain the operator name, call location, and data shape of each operator at each call location of each operator in the current step includes: Traversing each event data in the performance tracking file of the current step; When the event type of the traversed event data is an operator call, the event data is parsed to obtain the operator name, call location, and data shape of each operator at each call location of the operator corresponding to the event data.
[0008] According to a method for determining a data shape range of an operator provided by the present invention, based on the operator name and call position of each operator, the data shape of each operator at each call position in the current step is correspondingly merged with the data shape range of each operator at each call position in the previous step to obtain the data shape range of each operator at each call position in the current step, including: Based on the operator name and any call location of any operator, query the data shape range of any operator at any call location in the previous step from a preset dictionary, wherein the preset dictionary includes the operator name, call location, and data shape range of each operator at each call location of each operator in the previous step; The data shape of any operator in the current step at any call position is combined with the data shape range of any operator in the previous step at any call position to obtain the data shape range of any operator in the current step at any call position.
[0009] According to a method for determining a data shape range of an operator provided by the present invention, the step of obtaining the data shape range of any operator at any call position in the current step further includes: The data shape range of any operator at any call position in the previous step in the preset dictionary is updated to the data shape range of any operator at any call position in the current step.
[0010] According to a method for determining a data shape range of an operator provided by the present invention, generating a performance tracking file of the current step during training includes: Based on the training progress, generating a performance tracking file of the current step during the training process; The parsing of the performance tracking file of the current step includes: Parsing the performance tracking file of the current step based on the parsing process; The training process and the parsing process are two independent processes working in parallel.
[0011] According to a method for determining a data shape range of an operator provided by the present invention, the step of obtaining the operator name, call location, and data shape of each operator at each call location of each operator in the current step further includes: Deletes the performance trace file for the current step.
[0012] The present invention also provides a device for determining a data shape range of an operator, comprising: A generating unit, configured to train the model based on the training data of the current step and generate a performance tracking file of the current step during the training process; a parsing unit, configured to parse the performance tracing file of the current step to obtain the operator name, call location, and data shape of each operator at each call location of each operator in the current step, wherein the attribute information includes the operator name and the operator call location; The merging unit is used to merge the data shape of each operator at each call position of the current step with the data shape range of each operator at each call position of the previous step based on the operator name and call position of each operator, so as to obtain the data shape range of each operator at each call position of the current step.
[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining the data shape range of the operator as described above is implemented.
[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data shape range determination method of any of the operators described above.
[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the data shape range determination method of any of the operators described above.
[0016] The method, apparatus, device and storage medium for determining the data shape range of an operator provided by the present invention trains the model based on the training data of the current step and synchronously generates a performance tracking file of the current step during the training process. As a result, the generated performance tracking file can be immediately parsed while the model is being trained, thereby being able to obtain the operator name, all call locations and the data shape of each operator at each call location of each operator in the current step in real time. Furthermore, by merging the data shape of each operator at each call location in the current step with the data shape range of each operator at each call location in the previous step, the data shape range of each operator at each call location can be updated and determined in real time, thereby providing a solid foundation for subsequent precision analysis, and being able to perform precision analysis on the data shape range of operators at different call locations, making the analysis results more accurate, effective and targeted. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0018] Figure 1 1 is a flow chart of a method for determining a data shape range of an operator provided by the present invention; Figure 2 This is a schematic diagram of the overall architecture for determining the dynamic shape range of the model operator provided by the present invention; Figure 3 This is a schematic diagram of the trace file parsing process provided by the present invention; Figure 4 It is a schematic diagram of real-time merging of dynamic shape ranges provided by the present invention; Figure 5 It is a structural diagram of the data shape range determination device of the operator provided by the present invention; Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0019] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0020] During deep learning model training, different input data is typically used in different training steps, so the data shape of operators often changes dynamically. Here, during model training, each training step refers to a complete parameter update using a portion of the training data. Obtaining the data shape range of operators is crucial for analyzing and aligning operator accuracy during model training. It not only helps understand the operational characteristics and accuracy of operators, but also improves cross-platform compatibility and provides strong support for model optimization.
[0021] Specifically, the input and output data shapes of an operator directly determine the dimensionality and scale of its operations. Obtaining the data shape range of an operator helps understand its operational characteristics under different inputs, allowing for an accurate assessment of its accuracy. During model training, obtaining the data shape range of an operator and combining it with accuracy analysis results allows for targeted adjustments and optimization of the model. For example, for operators with low accuracy, you can try replacing them with more efficient implementations or adjusting their parameter settings to improve the overall model performance. Furthermore, operators may exhibit different accuracy characteristics on different hardware platforms (such as chips) due to different implementations. By obtaining the data shape range of an operator and comparing it across different platforms, you can ensure that the operator's operational accuracy remains consistent across different platforms, thereby improving the model's cross-platform compatibility and reducing the cost of adapting the model when deployed on different hardware platforms.
[0022] To obtain the data shape range of an operator, a common technical solution is to register forward and backward hook functions for each operator during model initialization. These hook functions are frequently called during model training. Their primary task is to dump the operator's input tensor data to local storage in real time. This allows, after model training is complete, to parse this stored data to obtain the data shape range of each operator throughout the entire model training process, allowing for analysis and optimization.
[0023] However, the above solution obtains the data shape range of each operator in the global scope of the entire model, which cannot be accurately obtained to the data shape range of the operator at a specific call, which limits the accuracy and effectiveness of the analysis of operator precision. Moreover, since the above solution needs to record the input tensor data of each operator during the model training process, this will cause a large amount of intermediate data to accumulate in the local storage. After the training is completed, an offline program is used to parse these massive data to generate the data shape range of the operator. This process is not only time-consuming and labor-intensive, but also limited by the capacity and performance of the local storage. In addition, due to the huge amount of data that needs to be recorded, the data will not be dumped and recorded for each training step. Instead, a small number of training samples will be sampled for model training, resulting in the data shape range of the operator being obtained. It may not be comprehensive and accurate, thereby limiting the comprehensiveness and accuracy of the operator precision analysis.
[0024] In this regard, the present invention provides a method for determining the data shape range of an operator, which can obtain the data shape range of each operator at each model call during the model training process in real time, without the need to record a large amount of intermediate data, without causing pressure on local storage, and without affecting the performance of model training, thereby overcoming the above-mentioned defects.
[0025] It should be noted that in deep learning models, operators are units that perform specific mathematical or logical operations. These operators receive input data and generate output data. The data shape range of an operator refers to the range of variation in the shape of the input data received by the operator and the output data generated (i.e., the number of elements in each dimension of the data) during model training or inference. Considering that the output data shape is often determined by both the input data shape and the operator's operation rules, when the input data shape range is determined, the output data shape range is also subject to corresponding constraints. Therefore, the present invention is primarily directed to determining the input data shape range of an operator.
[0026] The technical solution provided by the present invention is mainly divided into two processes: (1) Train the model to generate a performance trace file: The model is trained using a given training dataset. During the training process, the model can use the Application Programming Interface (API) provided by the deep learning framework to generate a performance trace file. For example, the model can use the torch.profiler.profile API to generate a profiler timeline trace file during training. Here, torch.profiler.profile is a performance analysis tool provided by the deep learning framework PyTorch, and the profiler timeline trace file is a performance trace file generated using the performance analysis tool. In the process of generating the trace file, each operator will write the shape information of the input data as event data to the trace file when it is called, so that the data shape information of the operator can be obtained in real time when the file is parsed later.
[0027] (2) Real-time file parsing to generate dynamic shape ranges: While the model is being trained, the generated performance trace files (such as profiler timeline trace files) are parsed to obtain the data shape of each operator at different call locations in real time, so that the data shape range of each operator at different call locations can be updated. Here, the call location refers to the specific location or level where the operator is called in the model, which is used to indicate at which layer or module of the model the operator is called. Preferably, after the file parsing is completed, the parsed trace files can be deleted, thereby freeing up local storage space and avoiding putting pressure on the local storage.
[0028] It is understandable that the above two processes can be implemented through two processes, namely the training process and the parsing process. The training process is used to generate a performance tracking file for each training step during the model training process. The file records the input data shape of each operator at each call location. The parsing process is another independent process that works in parallel with the training process. It is used to parse each generated performance tracking file in real time and extract the input data shape of each operator at each call location. The shape information is then merged into a preset dictionary. The dictionary records the range of changes in the input data shape of each operator at different call locations, thereby achieving an update of the data shape range of each operator at each call location, thereby providing a solid data foundation for subsequent operator precision analysis, model optimization, etc. The technical solution and specific implementation steps provided by the present invention will be introduced in detail below.
[0029] Figure 1 : is a flow chart of the method for determining the data shape range of the operator provided by the present invention, such as Figure 1 As shown, the method includes: Step 110: Train the model based on the training data of the current step, and generate a performance tracking file of the current step during the training process.
[0030] It's important to note that the current step refers to a specific phase or iteration in the model training process. During deep learning or machine learning model training, the entire training dataset is typically divided into multiple batches or steps for processing. Each step represents the model processing a portion of the data and completing a parameter update. Therefore, the current step refers to the training phase currently in progress. The training data for the current step refers to the portion of data that the model will process during the current training phase (i.e., the current step). This data is typically partitioned from the entire training dataset and used to train the model in the current step.
[0031] Specifically, after obtaining the training data for the current step from the training dataset, you can first perform necessary preprocessing operations on the training data, such as data cleaning, normalization, and enhancement, to ensure that the data meets the model input requirements. Then, you can input the preprocessed data into the model to train the model. During the training process, the model uses the performance analysis tool API provided by the framework itself to generate performance trace files while training, thus obtaining the performance trace file for the current step.
[0032] Here, the performance trace file for the current step refers to the file generated during the current training phase (i.e., the current step) and records various performance metrics and operator information during model training. These performance metrics can include loss, accuracy, and recall, while operator information can include the operator name, call location, and input data shape. The operator name is a string or symbol that identifies and distinguishes the operator. For example, in deep learning, common operator names include Linear, Conv, and MaxPool. The call location refers to the specific location or level in the model where the operator is called, for example, in which layer or module of the model the operator is called. The input data shape of an operator refers to the dimensions and size of the dataset or data tensor it receives as input.
[0033] It is understood that in order to generate a performance trace file using the API provided by the performance analysis tool, the relevant parameters of the performance analysis tool can be configured before executing step 110. For example, when the model generates a profiler timeline trace file during training using the torch.profiler.profile API, the performance analysis tool torch.profiler.profile can be initialized first and the profiler-related configuration information for generating the trace file can be configured. The specific configuration information is shown in Table 1 below: Table 1 Meaning of profiler configuration parameters
[0034] For example, in the configuration parameters of Table 1 above, when wait=1 is set, it means that the first step is ignored. This is because in general, when training just starts (i.e., the first step or the first few steps), the model training speed is very slow. At this time, it will be cold-started slightly and wait for one step before starting to generate the trace file. warmup=1 means that the performance trace event of the first step is not added to the trace file because the first step will be ignored. repeat=1000 means that 1000 trace files will be generated during the entire training process. record_shapes=True means that the input data shape of the operator is recorded in the trace file. with_stack=True means that the call stack of the operator will be recorded, that is, the location where the operator is called will be recorded.
[0035] After completing the configuration of the relevant parameter information of the performance analysis tool through the scripting language, you can run the script during the model training phase to generate a trace file corresponding to each training step under the path specified by the parameter on_trace_ready. This file records the input data shape of each operator at each call location. It should be understood that compared with the traditional technology of using hook functions to copy the input data of the operator from the hardware platform to the local storage, the embodiment of the present invention generates a performance trace file by utilizing the torch.profiler.profile API. It does not need to copy the input data of the operator to the local storage, so the generated performance trace file is very small and does not take up a lot of storage space.
[0036] Step 120 , parsing the performance tracking file of the current step, obtaining the operator name, calling location, and data shape of each operator at each calling location of each operator in the current step.
[0037] Specifically, the performance trace file records the operator name, call location, and data shape of each operator at each call location during the model training process. After generating the performance trace file for the current step, parsing the file can extract useful information (i.e., the operator name, call location, and data shape of each operator at each call location) for subsequent analysis and processing. Specifically, the performance trace file can be parsed through the following steps: First, open and read the performance trace file. The file can be stored in text format (such as JSON, XML, etc.) or binary format. The specific file format depends on the deep learning framework and tools used. Depending on the file format, its data structure can be parsed to extract useful information. For example, if the performance trace file is in JSON format, JSON parsing techniques can be used to parse the file. This process involves traversing each section of the file, identifying and extracting operator-related data such as operator name, operator call location, and input data shape. Finally, the extracted operator information (including operator name, call location, and data shape) is stored in an appropriate data structure (such as a pre-set dictionary or database in memory) for subsequent analysis and processing.
[0038] It is understandable that by obtaining information such as the operator name, call location, and data shape of each operator, the data shape of each operator at each call location can be understood, so that the dynamic shape range of each operator can be updated based on this information later.
[0039] Step 130: Based on the operator name and calling position of each operator, the data shape of each operator at each calling position of the current step is merged with the data shape range of each operator at each calling position of the previous step to obtain the data shape range of each operator at each calling position of the current step.
[0040] Specifically, after obtaining the operator name, call location, and data shape at each call location of each operator in the current step, the data shape of each operator at each call location in the current step can be merged with the data shape range of each operator at each call location obtained in the previous step according to the operator name and each call location of each operator to obtain the data shape range of each operator at each call location in the current step.
[0041] Here, since the embodiments of the present invention are primarily directed to determining the shape range of the input data of an operator, for a particular operator, the data shape range of the operator at each call position refers to the shape variation range of the input data that the operator can process when called at different layers of the model. The previous step refers to the step immediately preceding the current step in the model training process, and the data shape range of each operator in the previous step at each call position refers to the shape range of the input data that each operator can process when called at different layers of the model in the step preceding the current step in model training. This range is obtained by merging the data shapes of each operator in the previous step at each call position with the data shape range of the step before the previous step.
[0042] It should be understood that the purpose of merging is to obtain the data shape range of each operator at each call position in the current step. Specifically, the merging can be performed in a shape matching and expansion-based manner. First, the data shape of each operator at each call position in the current step is compared with the data shape range of the corresponding operator at the corresponding call position in the previous step. For any operator, if the data shape of the current step at a certain call position is within the data shape range of the corresponding call position in the previous step, then the range is directly used as the data shape range of the current step at the call position; if the data shape range of the current step at a certain call position exceeds the data shape range of the previous step at the corresponding call position, then the data shape range of the corresponding call position of the previous step is expanded to include the data shape of the current step.
[0043] Figure 2 This is a schematic diagram of the overall architecture for determining the data shape range of the operator provided by the present invention, such as Figure 2 As shown in the figure, assuming a convolutional model (such as a face recognition model), the entire training dataset can be divided into 1000 steps for training. During the training process, a performance trace file is generated for each step. This file records the input data shape of each operator at each call location. While the model is training, the parsing process parses each generated performance trace file, obtains the input data shape of each operator at each call location in each performance trace file in real time, and merges it into the preset dictionary.
[0044] like Figure 2As shown, assume the model has six layers, labeled 1, 2, 3, 4, 5, and 6. These labels can be thought of as the IDs (identifications) of each layer. In each training step, the model structure processes the input data. The first layer first processes it using the Linear operator, followed by the Conv operator (convolutional operator) in the second layer. The third layer then processes it using the Linear operator again. The fourth and fifth layers then sequentially process it using the MaxPool operator and the Conv operator. Finally, the sixth layer uses the Softmax operator (activation operator) to calculate the probability of a face. The final probability is the result of a serial execution of the input data, starting with the output of the first operator, Linear, which is then fed into the second operator, Conv, for further calculation. This continues in sequence, yielding the final face probability. During this process, it is possible that the input data shapes of the same operator at different call locations are different. For example, the input data shapes of the Linear operator called in the first layer and the Linear operator called in the third layer are different because their input data shapes have changed through the calculation of the intermediate Conv operator. It should be understood that in traditional technology, the input data of the operator is dumped to the local storage by calling a hook function. It does not specify the specific location of a particular operator call, but instead merges the input data shapes of all call locations in all steps of the operator as a whole. This results in a global data shape range for the operator, and it is impossible to perform precision analysis on the data shape range of the operator at a specific call location. In this regard, the embodiments of the present invention can obtain the input data shape of each operator at each call location of the model in real time during the model training process, and merge the input data on the dynamic dimension of the input data to obtain the input data shape range of each operator at each call location. The dynamic dimension here refers to the dimension in which the shape of the input data changes during the model training process.
[0045] Considering that the operator name and call location do not change in each step, in order to accurately obtain the input data shape range of each operator at different call locations, the information obtained by combining the operator name and call location can be used as the key (Key), and the operator data shape corresponding to this combination of information can be used as the value (Value) to construct a preset dictionary to store the data shape range of each operator at each call location. Specifically, when the current step is step1, the model is trained using the training data of step1, and the trace file of step1 is generated through the training process. During the file generation process, the torch.profiler.profile API will add the operator name, call location (i.e., the ID of the model layer), and the input data shape corresponding to the operator as event data to the trace file when each operator is called. Subsequently, after the trace file of step1 is generated, it can be parsed through the parsing process to extract information such as the operator name, call location, and input data shape of each operator.
[0046] like Figure 2 As shown, step:{1} indicates that the input data shape of the operator is determined after step 1 is executed. Since step 1 is the first step, the value of the input data shape in each dimension for each operator is fixed at this time and is not a dynamic range. For step 1, after obtaining the input data shape of each operator at each call location, this information can be saved in a preset dictionary in the form of a key-value pair, where the key consists of the operator name and call location, and the value is the input data shape of the operator. For example, taking the Linear operator called in the first layer of the model as an example, the operator name of the operator can be expressed as Linear, and the operator call location is the identifier 1 of the model layer. Assuming that the input data shape of the operator includes 4 dimensions, it can be expressed as [4096, 1024, 1024, 320].
[0047] When the current step is step 2, the model is trained using the training data of step 2, and the trace file of step 2 is generated through the training process. This part of the process is the same as step 1 and will not be repeated here. It should be understood that while the parsing process is parsing the trace file of step 1, the training process can work in parallel to generate the trace file of step 2. After the trace file of step 2 is generated, the file can be parsed by the parsing process to extract the operator name, call location, and input data shape of each operator in the step 2 training phase, and this information is merged into the preset dictionary. Specifically, after extracting the operator name, call location, and input data shape of each operator in step 2, the combination of the operator name and call location of each operator can be used as a search identifier to query the input data shape of step 1 corresponding to the search identifier in the preset dictionary, so that the input data shape of step 1 corresponding to the search identifier and the input data shape of step 2 can be merged to obtain the data shape range of the operator.
[0048] like Figure 2 As shown in the example, step:{1,2} represents the data shape range obtained by merging the data shape of the operator in step 2 with the data shape of step 1 after step 2 is executed. Similarly, taking the Linear operator called in the first layer of the model as an example, assuming that the input data shape of the operator obtained in step 2 is [4096, 1024, 2048, 320], since the input data shape of the operator obtained in step 1 is [4096, 1024, 1024, 320], then after merging steps 1 and 2, the input data shape range of the Linear operator when called in the first layer of the model can be expressed as [4096, 1024, (1024, 2048), 320]. Here, for the Linear operator, when it is called in the first layer of the model, the third dimension in the input data shape is a dynamic dimension, which is represented by brackets, with the minimum and maximum values of the dimension separated by commas in the brackets. It should be understood that the input data shape range [4096, 1024, (1024, 2048), 320] indicates that when the Linear operator is called in the first layer of the model, the number of elements in the first dimension of the input data is fixed (i.e., 4096) and the number of elements in the second dimension is also fixed (i.e., 1024). The number of elements in the third dimension can change dynamically, with a minimum of 1024 elements supported and a maximum of 2048 elements supported. Of course, the input data shape range may change with the increase in subsequent training steps. This explanation only focuses on the data shape range obtained in step 2.
[0049] like Figure 2 As shown, when the current step is step 3, the model is trained using the training data of step 3, and the trace file of step 3 is generated through the training process. Subsequently, the file is parsed through the parsing process to extract the operator name, call location, input data shape and other information of each operator in the step 3 training phase, and merge this information into the preset dictionary. In other words, the input data shape of each operator at each call location obtained in step 3 is merged with the input data shape range of each operator at each call location obtained in step 2, so as to obtain the input data shape range of each operator at each call location in the current step 3, which is expressed as Figure 2 , as shown in step:{1,2,3}. This is repeated until step 1000 is completed. The input data shape of each operator at each call location in this step can be parsed and obtained. This can then be combined to obtain the final input data shape range for each operator at each call location.
[0050] The method provided by the embodiment of the present invention trains the model based on the training data of the current step and synchronously generates the performance tracking file of the current step during the training process. As a result, the generated performance tracking file can be immediately parsed while the model is being trained, so that the operator name, call location, and data shape of each operator at each call location of each operator in the current step can be obtained in real time. Furthermore, by merging the data shape of each operator at each call location in the current step with the data shape range of each operator at each call location in the previous step, the data shape range of each operator at each call location can be updated and determined in real time, thereby providing a solid foundation for subsequent precision analysis. It is possible to perform precision analysis on the data shape range of operators at different call locations, making the analysis results more accurate, effective, and targeted.
[0051] Based on the above embodiment, in step 110, generating the performance tracking file of the current step during the training process includes: Step 111: During the model training process, obtain the data shape, operator name, and calling location of each operator when it is called; Step 112: Write the data shape, operator name, and calling location of each operator when it is called as event data into the performance tracking file of the current step.
[0052] Specifically, during model training, you can use the performance analysis tools provided by the model framework to monitor the model training process and record information about each operator call, including the operator name, call location, and input data shape. This information can then be written as an event data item to the performance tracking file for the current step for subsequent parsing and extraction.
[0053] For example, in the PyTorch model framework, the torch.profiler.profile API can be used to easily generate trace files required for performance analysis. The torch.profiler.profile API can monitor the model training process and automatically record information such as the operator name, call location, and input data shape when each operator is called. For each operator, after obtaining its call name, call location, and input data shape, this information can be written as event data to a trace file in a specified path.
[0054] It's understood that event data refers to all relevant information recorded during performance profiling. For example, a model can generate a profiler timeline trace file for each step during training using the torch.profiler.profile API. This file is in JSON format. In this trace file, event data refers to a specific activity or operation recorded during program execution. This event data contains a variety of information about program execution and is an important basis for performance analysis and optimization. Specifically, in the trace file, each operator call generates an event data entry. Each event data entry can include the event type, timestamp, operator name, operator call location, input data shape, and other additional information (such as memory usage). Here, the event type refers to the type or category of the event and can be used to distinguish different types of events, such as operator calls, memory allocations, and thread switches. Using event types, you can quickly locate events of interest in performance trace files for further analysis and processing.
[0055] Based on any of the above embodiments, step 120 specifically includes: Step 121, traversing each event data in the performance tracking file of the current step; Step 122: When the event type of the traversed event data is an operator call, the event data is parsed to obtain the operator name, call location, and data shape of the operator at the call location of the operator corresponding to the event data.
[0056] It should be noted that, in addition to recording event data corresponding to operator calls, trace files also contain other types of event data (such as memory allocation and thread switching). Therefore, after generating the performance trace file for the current step, to extract the required information from it, you can traverse each event data in the file to locate event data with operator call type.
[0057] Specifically, after generating the performance trace file for the current step, each event data item can be read from the file. After reading the event data, the event type of the event data item can be identified by examining the event type field (e.g., event_type) of the event data item. When the event type of the read event data item is an operator call, it indicates that the event was generated by an operator call in the model. In this case, the event data item can be parsed to extract the operator name, call location, and data shape of the operator. This information is typically stored in specific fields of the event data item, such as op_name, op_position, and input_shapes. By reading the values of these fields, the name, call location, and data shape of the operator corresponding to the event data item can be obtained. Finally, the extracted operator name, call location, and data shape can be converted into an appropriate data structure (e.g., a dictionary, object, etc.) and stored for subsequent analysis.
[0058] Figure 3 This is a flow chart of the trace file analysis provided by the present invention, such as Figure 3 As shown, for the performance trace file generated in each step, the parsing process can parse the trace file to generate an Event Dict. The Event Dict here is a data structure used to store event information parsed from the trace file. These events are the calls of operators during the model training process, and each operator call contains some key information, such as the operator name, call location, and input data shape. Therefore, the Event Dict can be regarded as a dictionary (or hash table), whose Key is a unique identifier composed of the operator name and call location, and the Value is a data structure containing the input data shape when the operator is called (that is, the input data shape of the operator at the corresponding call location).
[0059] Specifically, when parsing the trace file, a unique identifier (a combination of the operator name and the call location) must be constructed for each operator call. This identifier is used to find or store information related to the operator call in the Event Dict. At the same time, the extracted input data shape must be associated with this identifier. Once the unique identifier is constructed and associated with the input data shape, this information can be stored in the Event Dict. This allows quick access to the input data shape of a particular operator call by searching the Event Dict. Figure 3 The {input_shapes} shown in the figure represents the input data shape associated with the operator name and call location. It should be understood that because model training is dynamic, new trace files are continuously generated. Therefore, the parsing process reads these new files in real time and updates the Event Dict. To save storage space and avoid unnecessary file accumulation, the trace file for each step can be deleted after it is fully parsed and its information is stored in the Event Dict. This prevents trace files from continuously accumulating in local storage.
[0060] Based on any of the above embodiments, step 130 specifically includes: Step 131: Based on the operator name and any call location of any operator, query the data shape range of any operator at any call location in the previous step from a preset dictionary, wherein the preset dictionary includes the operator name, call location, and data shape range of each operator at each call location in the previous step; It should be noted that the preset dictionary refers to a data structure used to store the data shape range of an operator in each training step. For example, the preset dictionary can be named MergedDataRange dictionary, where the key is a unique identifier consisting of the operator name and call location, and the value is the data shape range of the operator in the previous training step. The purpose of the preset dictionary is to record and update the dynamic data shape range of each operator at different call locations in real time during the model training process. This allows the preset dictionary to be queried to obtain the data shape range information of the operator at different call locations for further analysis and processing.
[0061] Specifically, for any operator, it can be called only at a certain layer of the model, or it can be called at different layers of the model, so the calling location corresponding to the operator can be one or more. When the operator is called at multiple layers of the model, before merging the data shape ranges and generating the dynamic data shape range of the operator, it is necessary to first build a unique query key based on the operator name and any calling location of the operator, which is used to find the data shape range of the operator at the calling location in the preset dictionary. If the corresponding entry is found, it means that the data shape range of the operator at the corresponding calling location in the previous step has been obtained; if not found, it means that the operator is called for the first time in the training process, and an empty data shape range can be initialized at this time.
[0062] Step 132: Merge the data shape of any operator at any call position in the current step with the data shape range of any operator at any call position in the previous step to obtain the data shape range of any operator at any call position in the current step.
[0063] Specifically, after finding the data shape range of the operator at the corresponding call position in the previous step from the preset dictionary, the data shape of the operator at the call position in the current step can be merged with the data shape range of the operator at the corresponding call position in the previous step to obtain the data shape range of the operator at the call position in the current step. For example, for the Linear operator, the current step is step2. By parsing the trace file of step2, we can get that the input data shape of the operator when it is called in the first layer of the model is [4096, 1024, 2048, 320]. Using the operator name and calling location (that is, the ID of the model layer), we can find from the MergedDataRange dictionary that the input data shape of the operator when it is called in the first layer of the model in the previous step stpe1 is [4096, 1024, 1024, 320]. Therefore, after merging the data shapes of step1 and stpe2, we can get that the data shape range when the operator is called in the first layer of the model in step2 is [4096, 1024, (1024, 2048), 320].
[0064] Based on any of the above embodiments, after step 132, the method further includes: The data shape range of any operator at any call position in the previous step in the preset dictionary is updated to the data shape range of any operator at any call position in the current step.
[0065] Specifically, after merging to obtain the data shape range for each operator at each call position in the current step, the merged data shape range can be stored back in the preset dictionary for continued use in subsequent training steps. Specifically, for any operator, after merging to obtain the data shape range for each call position of the operator in the current step, the data shape range for each call position in the MergedDataRange dictionary can be updated accordingly based on the data shape range. That is, the data shape range for the operator at a certain call position stored in the dictionary is updated to the data shape range for the operator at the call position obtained by the current merger.
[0066] Figure 4 This is a schematic diagram of the real-time merging of dynamic shape ranges provided by the present invention, such as Figure 4 As shown, after obtaining the data shape of each operator at each call location in step 1, we can store it in the preset dictionary MergedDataRange. After obtaining the data shape of each operator at each call location in step 2, we can merge it with the corresponding data shape of each operator at each call location in the dictionary to obtain the data shape range of each operator at each call location in step 2, and update it to the MergedDataRange dictionary. Similarly, after parsing and obtaining the input data shape of each operator at each call location in each step, we merge the input data shapes of these operators into the dictionary. Figure 4 The {shape_range} in the __name__ indicates the input data shape range of each operator merged in each step at each call location.
[0067] In the embodiment of the present invention, by updating the preset dictionary in real time, it can be ensured that the data shape range queried at any time is the latest, which is crucial for dynamically tracking and monitoring data shape changes during model training.
[0068] Based on any of the above embodiments, after step 120, the method further includes: deleting the performance tracking file of the current step.
[0069] Specifically, after parsing the performance tracking file generated by each step, the performance tracking file for that step can be deleted to save storage space and avoid unnecessary file accumulation, while also achieving the goals of real-time parsing and updating the data shape range. Specifically, deleting the performance tracking file for a particular step can be achieved by calling an API or command of the file system, which is not specifically limited in the embodiments of the present invention.
[0070] Based on any of the above embodiments, an embodiment of the present invention provides a method for determining the dynamic shape range of a model operator. This method can capture the input data shape of each operator at each model call in real time during model training, and merge them on the corresponding data shape dimension to obtain the input data shape range of each operator at each call. At the same time, it does not affect the performance of model training and can also avoid saving a large number of performance tracking files. The method includes: Step S1: Train the model to generate a profiler timeline trace file
[0071] This can be achieved through the following two steps: Step S11, initialize the performance analysis tool torch.profiler.profile and configure the profiler related configuration information for generating the trace file; Step S12: During the model training phase, as the number of training steps increases, a profiler timeline trace file is continuously generated. This file records the input data shape of each operator at each call.
[0072] Step S2: The parsing process generates dynamic shape ranges in real time
[0073] This can be achieved through the following three steps: Step S21: Parse and obtain the input data shape of a certain step: parse the trace file exported by a certain step to generate an Event Dict, and perform parsing and mapping based on the operator name and call location to obtain the input data shape corresponding to the operator name and call location; Step S22, real-time merging of dynamic shape ranges: After each parsing process obtains the input data shape of a trace file, the parsing process merges these input data shapes into the MergedDataRange dictionary; Step S23, real-time deletion of historical trace files: After each trace file is parsed, the trace file is deleted, so that trace files will not continue to accumulate in the local storage, and the purpose of real-time parsing and real-time updating of data shape range can be achieved.
[0074] The method provided by the embodiments of the present invention can obtain the data shape range of each operator at each call in real time, update the data shape range in real time while training the model, and delete historical trace files, without putting pressure on local storage. Furthermore, because historical trace files are deleted, the present invention does not limit the number of training steps, allowing the model to be trained using a more comprehensive and complete training dataset, resulting in more accurate data shape ranges.
[0075] Based on any of the above embodiments, Figure 5 Schematic diagram of the structure of the data shape range determination device of the operator provided by the present invention, such as Figure 5 As shown, the device includes: A generating unit 510 is configured to train a model based on the training data of the current step and generate a performance tracking file of the current step during the training process; A parsing unit 520 is configured to parse the performance tracking file of the current step to obtain the operator name, call location, and data shape of each operator at each call location of each operator in the current step; The merging unit 530 is used to merge the data shape of each operator at each call position in the current step with the data shape range of each operator at each call position in the previous step based on the operator name and call position of each operator, so as to obtain the data shape range of each operator at each call position in the current step.
[0076] The device provided by the embodiment of the present invention trains the model based on the training data of the current step and synchronously generates the performance tracking file of the current step during the training process. As a result, the generated performance tracking file can be immediately parsed while the model is being trained, so that the operator name, call location, and data shape of each operator at each call location of each operator in the current step can be obtained in real time. Furthermore, by merging the data shape of each operator at each call location in the current step with the data shape range of each operator at each call location in the previous step, the data shape range of each operator at each call location can be updated and determined in real time, thereby providing a solid foundation for subsequent precision analysis. It is possible to perform precision analysis on the data shape range of operators at different call locations, making the analysis results more accurate, effective, and targeted.
[0077] Based on any of the above embodiments, the generating unit 510 is specifically configured to: During model training, obtain the data shape, operator name, and calling location of each operator when it is called; The data shape, operator name, and calling location of each operator when it is called are written as event data into the performance tracking file of the current step.
[0078] Based on any of the above embodiments, the parsing unit 520 is specifically configured to: Traversing each event data in the performance tracking file of the current step; When the event type of the traversed event data is an operator call, the event data is parsed to obtain the operator name, call location, and data shape of each operator at each call location of the operator corresponding to the event data.
[0079] Based on any of the above embodiments, the merging unit 530 is specifically configured to: Based on the operator name and any call location of any operator, query the data shape range of any operator at any call location in the previous step from a preset dictionary, wherein the preset dictionary includes the operator name, call location, and data shape range of each operator at each call location of each operator in the previous step; The data shape of any operator in the current step at any call position is combined with the data shape range of any operator in the previous step at any call position to obtain the data shape range of any operator in the current step at any call position.
[0080] Based on any of the foregoing embodiments, the merging unit 530 is further configured to: The data shape range of any operator at any call position in the previous step in the preset dictionary is updated to the data shape range of any operator at any call position in the current step.
[0081] Based on any of the above embodiments, the generating unit 510 is specifically configured to: generate a performance tracking file of the current step during the training process based on the training progress; The parsing unit 520 is specifically configured to: parse the performance tracking file of the current step based on the parsing process; The training process and the parsing process are two independent processes working in parallel.
[0082] Based on any of the above embodiments, the device further includes a deleting unit, which is used to delete the performance tracking file of the current step.
[0083] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call logic instructions in the memory 630 to execute a method for determining the data shape range of an operator, the method comprising: training a model based on training data of a current step, and generating a performance tracking file of the current step during the training process; parsing the performance tracking file of the current step to obtain the operator name, call location, and data shape of each operator at each call location of each operator in the current step; and based on the operator name and call location of each operator, merging the data shape of each operator at each call location of the current step with the data shape range of each operator at each call location of the previous step to obtain the data shape range of each operator at each call location of the current step.
[0084] In addition, the logic instructions in the aforementioned memory 630 can be implemented in the form of a software functional unit and, when sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the relevant art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0085] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data shape range determination method of the operator provided by the above-mentioned methods, the method including: training the model based on the training data of the current step, and generating a performance tracking file of the current step during the training process; parsing the performance tracking file of the current step to obtain the operator name, call location and data shape of each operator at each call location of the current step; based on the operator name and call location of each operator, merging the data shape of each operator at each call location of the current step with the data shape range of each operator at each call location of the previous step to obtain the data shape range of each operator at each call location of the current step.
[0086] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for determining the data shape range of operators provided by the above-mentioned methods, the method comprising: training a model based on the training data of the current step, and generating a performance tracking file of the current step during the training process; parsing the performance tracking file of the current step to obtain the operator name, call location and data shape of each operator at each call location of the current step; based on the operator name and call location of each operator, merging the data shape of each operator at each call location of the current step with the data shape range of each operator at each call location of the previous step to obtain the data shape range of each operator at each call location of the current step.
[0087] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.
[0088] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.
[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A method for determining a data shape range of an operator, characterized in that: include: Training the model based on the training data of the current step, and generating a performance tracking file of the current step during the training process; Parsing the performance tracking file of the current step to obtain the operator name and call location of each operator in the current step and the data shape of each operator at each call location; Based on the operator name and calling position of each operator, the data shape of each operator at each calling position of the current step is correspondingly merged with the data shape range of each operator at each calling position of the previous step to obtain the data shape range of each operator at each calling position of the current step.
2. The method for determining the data shape range of an operator according to claim 1, characterized in that: Generating the performance tracking file of the current step during the training process includes: During model training, obtain the data shape, operator name, and calling location of each operator when it is called; The data shape, operator name and calling location of each operator when it is called are written as event data into the performance tracking file of the current step.
3. The method for determining the data shape range of an operator according to claim 1, characterized in that: The parsing of the performance tracking file of the current step to obtain the operator name, call location and data shape of each operator at each call location of each operator of the current step includes: Traversing each event data in the performance tracking file of the current step; When the event type of the traversed event data is an operator call, the event data is parsed to obtain the operator name, the call location, and the data shape of the operator at the call location of the operator corresponding to the event data.
4. The method for determining the data shape range of an operator according to claim 1, characterized in that: Based on the operator name and call position of each operator, the data shape of each operator at each call position of the current step is correspondingly merged with the data shape range of each operator at each call position of the previous step to obtain the data shape range of each operator at each call position of the current step, including: Based on the operator name and any calling position of any operator, query the data shape range of any operator at any calling position in the previous step from a preset dictionary, wherein the preset dictionary includes the operator name, calling position and data shape range of each operator at each calling position of each operator in the previous step; The data shape of any operator in the current step at any calling position is combined with the data shape range of any operator in the previous step at any calling position to obtain the data shape range of any operator in the current step at any calling position.
5. The method for determining the data shape range of an operator according to claim 4, characterized in that: The step of obtaining the data shape range of any operator at any calling position in the current step further includes: The data shape range of any operator at any calling position in the previous step in the preset dictionary is updated to the data shape range of any operator at any calling position in the current step.
6. The method for determining the data shape range of an operator according to any one of claims 1 to 5, characterized in that: Generating the performance tracking file of the current step during the training process includes: Based on the training progress, generating a performance tracking file of the current step during the training process; The parsing of the performance tracking file of the current step includes: Based on the parsing process, parsing the performance tracking file of the current step; The training process and the parsing process are two independent processes working in parallel.
7. The method for determining the data shape range of an operator according to any one of claims 1 to 5, characterized in that: The obtaining of the operator name, the calling position and the data shape of each operator at each calling position of each operator in the current step further includes: Deletes the performance trace file for the current step.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining the data shape range of the operator according to any one of claims 1 to 7 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining the data shape range of the operator according to any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining the data shape range of the operator according to any one of claims 1 to 7 is implemented.