Deep learning operator positioning method, electronic equipment, storage medium and program product

By converting the deep learning framework data into intermediate representation information and generating intermediate representation diagrams, the adaptation problem between deep learning frameworks is solved, fast and accurate operator positioning is achieved, and model training efficiency is improved.

CN118940862APending Publication Date: 2024-11-12INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411131878.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing deep learning frameworks have problems of waste of resources and low model positioning efficiency in model adaptation and training, especially due to the complexity of the model and changes in parameters, which lead to difficulty in operator positioning, which affects training efficiency.

Method used

By receiving deep learning framework data and positioning requirements instructions, converting them into intermediate representation information, conducting index reconstruction to generate intermediate representation diagrams, querying the target operator location based on positioning requirements and extracting text information, eliminating framework differences, and achieving fast and accurate calculations Subposition.

Benefits of technology

The efficiency of model training is improved, the dependence on different deep learning frameworks is removed, and the characteristics of operators are clearly displayed through the intermediate representation graph to achieve fast and accurate operator positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940862A_ABST
    Figure CN118940862A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a deep learning operator positioning method, electronic equipment, a storage medium and a program product, and relates to a computer system and deep learning, and the method comprises the steps: receiving deep learning framework data and a positioning demand instruction; converting the deep learning framework data into intermediate representation information; performing index reconstruction on the intermediate representation information to generate an intermediate representation graph; searching the intermediate representation graph according to the positioning demand instruction, and determining the position of a target operator; and extracting a target operator at the target operator position to generate text information. According to the method, the difference between the frames is eliminated through conversion into the intermediate representation information, and the specific characteristics of the operator are displayed based on reconstruction of the intermediate representation graph, so that the target operator can be quickly positioned from the intermediate representation graph, text information can be generated, model information can be quickly determined, and the training efficiency can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer systems and deep learning technologies, and in particular to a deep learning operator positioning method, an electronic device, a storage medium and a program product. Background Art

[0002] The current mainstream deep learning frameworks include tensorflow, pytorch, paddlepaddle, onnx, etc. These frameworks lack a unified universal model structure. When adapting the models of these frameworks on deep learning hardware, different methods need to be selected according to different frameworks, which causes a lot of resource waste. In addition, in the process of deep learning, the model may cause errors in model training due to problems such as changes in accuracy; or when some operators need to be modified to improve the model, it is difficult to accurately locate the operators due to the scale of the model and the complexity of the parameters, resulting in low model training efficiency. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a deep learning operator positioning method, an electronic device, a storage medium and a program product that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problems, an embodiment of the present invention discloses a deep learning operator positioning method, including:

[0005] Receive deep learning framework data and positioning requirement instructions;

[0006] Converting the deep learning framework data into intermediate representation information;

[0007] Reconstructing the index of the intermediate representation information to generate an intermediate representation graph;

[0008] According to the positioning requirement instruction, the intermediate representation graph is searched to determine the target operator position;

[0009] The target operator is extracted at the target operator position to generate text information.

[0010] Optionally, the step of reconstructing the index of the intermediate representation information to generate an intermediate representation graph includes:

[0011] Regularizing and extracting the intermediate representation information to determine an operator index;

[0012] The intermediate representation graph is generated according to the operator index.

[0013] Optionally, the step of performing regularization extraction on the intermediate representation information and determining the operator index includes:

[0014] Regularizing and extracting the intermediate representation information to determine the operator structure and dependency relationship;

[0015] The operator structure and the dependency are combined to generate the operator index.

[0016] Optionally, the step of combining the operator structure with the dependency relationship to generate the operator index includes:

[0017] Encoding the operators in the operator structure according to a preset rule to obtain operator codes;

[0018] According to the dependency relationship, the operator codes are associated to generate the operator index.

[0019] Optionally, the step of searching the intermediate representation graph according to the positioning requirement instruction to determine the position of the target operator includes:

[0020] In the operator index, determining a target cutoff position;

[0021] Truncating the intermediate representation graph based on the target truncation position to generate the at least two sub-intermediate representation graphs;

[0022] Determine the positioning type according to the positioning requirement instruction;

[0023] For any of the sub-intermediate representation graphs, searching the sub-intermediate representation graph for an operator matching the positioning type;

[0024] Verifying the operator that matches the positioning type;

[0025] In response to the verification being passed, determining the operator matching the positioning type as the target operator;

[0026] In response to the verification failing, repeatedly performing the step of searching the sub-intermediate representation graph for an operator matching the positioning type as the target operator;

[0027] The position corresponding to the target operator is determined as the target operator position.

[0028] Optionally, when the positioning type is a fault type, the step of verifying the operator matching the positioning type includes:

[0029] Setting the operator matching the positioning type to zero;

[0030] According to the set operator matching the positioning type, the neural network model corresponding to the intermediate representation graph is run to generate running output data;

[0031] Verification is performed based on the operation output data.

[0032] Optionally, the method comprises:

[0033] Reading the text information;

[0034] Based on the target operator in the text information, deep learning model compilation information is generated.

[0035] In the second aspect of the present invention, an embodiment of the present invention discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the deep learning operator positioning method as described above.

[0036] In a third aspect of the present invention, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the deep learning operator positioning method as described above are implemented.

[0037] In a fourth aspect of the present invention, an embodiment of the present invention discloses a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the deep learning operator positioning method as described above.

[0038] The embodiments of the present invention include the following advantages:

[0039] The embodiment of the present invention receives deep learning framework data and positioning requirement instructions; converts the deep learning framework data into intermediate representation information; indexes and reconstructs the intermediate representation information to generate an intermediate representation graph; searches the intermediate representation graph according to the positioning requirement instructions to determine the target operator position; extracts the target operator at the target operator position to generate text information. The differences between frameworks are eliminated by converting to intermediate representation information, and the dependence on the framework is eliminated; based on the reconstruction of the intermediate representation graph, the specific features of the operator are clearly displayed through the intermediate representation graph, so that the position of the target operator can be quickly located from the intermediate representation graph, and text information is generated, achieving fast and accurate operator positioning, thereby improving the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of the steps of an embodiment of a deep learning operator positioning method of the present invention;

[0041] Figure 2 is a flowchart of another embodiment of a deep learning operator positioning method of the present invention;

[0042] Figure 3 is a schematic diagram of operator positioning of another embodiment of a deep learning operator positioning method of the present invention;

[0043] Figure 4 is a flowchart of an example of a deep learning operator positioning method of the present invention;

[0044] Figure 5 It is a flowchart of debugging steps of an example of a deep learning operator positioning method of the present invention;

[0045] Figure 6 It is a flow chart of the intermediate representation graph truncation steps of an example of a deep learning operator positioning method of the present invention;

[0046] Figure 7 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0047] Figure 8 is a structural block diagram of a storage medium provided by an embodiment of the present invention;

[0048] Fig. 9 It is a structural block diagram of a computer program product provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Reference Figure 1 , shows a flowchart of a deep learning operator positioning method embodiment of the present invention, and the deep learning operator positioning method may specifically include the following steps:

[0051] Step 101, receiving deep learning framework data and positioning requirement instructions;

[0052] When debugging and reasoning a deep learning neural network model, when certain operators need to be positioned, the deep learning framework data corresponding to the neural network model and the corresponding positioning requirement instructions can be received.

[0053] Step 102, converting the deep learning framework data into intermediate representation information;

[0054] The obtained deep learning framework data is then converted into intermediate representation information, and the deep learning framework is represented in a unified way through the intermediate representation information, thereby getting rid of the dependence on the framework, making the embodiment of the present invention universal, and subsequent editing and segmentation of the model can be performed based on the intermediate representation.

[0055] Step 103, reconstructing the index of the intermediate representation information to generate an intermediate representation graph;

[0056] When the model data is displayed in a unified form of intermediate representation information, the operators in the neural network model and the characteristics of these operators can be seen based on the intermediate representation information; the index relationship of the neural network model can be reconstructed based on the characteristics of these operators to generate an intermediate representation graph represented in the intermediate representation format.

[0057] Step 104: Search the intermediate representation graph according to the positioning requirement instruction to determine the target operator position.

[0058] Finally, according to the positioning requirement instruction, the corresponding search is performed in the intermediate representation graph based on the reconstructed index to find the target operator position of the target operator. Among them, the target operator can be the symbol or data structure of the specific operation defined in the execution function, algorithm or expression in the deep learning framework data that matches the positioning requirement instruction.

[0059] Step 105: extract the target operator at the target operator position to generate text information.

[0060] At the target operator position, the target operator is extracted. The target operator can be data such as a symbol or data structure of a specific operation defined in an execution function, algorithm or expression that matches the positioning requirement instruction in the deep learning framework data.

[0061] Save the target operator in text and generate text information.

[0062] The embodiment of the present invention receives deep learning framework data and positioning requirement instructions; converts the deep learning framework data into intermediate representation information; indexes and reconstructs the intermediate representation information to generate an intermediate representation graph; searches the intermediate representation graph according to the positioning requirement instructions to determine the target operator position; extracts the target operator at the target operator position to generate text information. The differences between frameworks are eliminated by converting to intermediate representation information, and the dependence on the framework is eliminated; based on the reconstruction of the intermediate representation graph, the specific features of the operator are clearly displayed through the intermediate representation graph, so that the position of the target operator can be quickly located from the intermediate representation graph, achieving fast and accurate operator positioning, thereby improving the efficiency of model training.

[0063] Reference Figure 2 , shows a flowchart of another deep learning operator positioning method embodiment of the present invention, and the deep learning operator positioning method may specifically include the following steps:

[0064] Step 201, receiving deep learning framework data and positioning requirement instructions;

[0065] When it is necessary to locate certain operators in the neural network model, the corresponding deep learning framework data and positioning requirement instructions can be received. The deep learning framework data is the deep learning framework used by the neural network model and the parameters trained in the deep learning framework. The positioning requirement instruction is used to determine the type or feature corresponding to the operator that needs to be located.

[0066] Step 202, converting the deep learning framework data into intermediate representation information;

[0067] The deep learning framework data can be first converted into intermediate representation information in a unified form. Intermediate representation information refers to deep learning framework data represented in an intermediate representation format. The intermediate representation format refers to a representation between source code and target code, which is used for conversion and optimization between different stages of compilation. The intermediate representation format is a bridge between the front end and back end of the compiler, enabling the compiler to be designed and implemented independently of a specific source language and target machine. The intermediate representation format may include three-address code, abstract syntax tree (AST), static single assignment form (SSA), and the like. For example, the three-address code is a low-level intermediate representation generated by the front end of the compiler, which is convenient for simple optimization and conversion; while the abstract syntax tree is a high-level intermediate representation that can more accurately represent the structure and semantics of the source code, facilitating complex analysis and optimization. The embodiment of the present invention does not specifically limit the specific representation form of the intermediate representation format.

[0068] Specifically, the step of converting the deep learning framework data into intermediate representation information includes: compiling the deep learning framework data into the intermediate representation information through a preset compiler.

[0069] In practical applications, models of different frameworks can be converted into a unified format intermediate representation through a deep learning compiler. Using a deep learning compiler, that is, a preset compiler reads the deep learning framework data and compiles the corresponding neural network model into intermediate representation information. By converting the deep learning model into an intermediate representation of the deep learning compiler, the dependence on the framework type can be eliminated, so that the embodiments of the invention can be applied to various different deep learning frameworks.

[0070] Further, after the intermediate representation information is generated, the intermediate representation information can be converted into text file format information, so that relevant technicians can visualize and parameterize the structure of the neural network model based on the text file format information, so as to facilitate relevant technicians to perform relevant processing on the neural network model. The data in the intermediate representation information can be converted into information in text file format. The text file format can be TXT (format name) format. That is, the intermediate representation information is converted into information containing basic text without any additional formatting or image elements. The method for converting the intermediate representation information into text file format information can be determined based on the intermediate representation format of the intermediate representation information. The conversion process can be: according to the format of IR (intermediate representation information), traverse its structure; for each element in IR (such as node, instruction, etc.), convert it into text form to form a serialized IR; use the file operation API (application programming interface) provided by the programming language to write the serialized IR elements into a txt (text file format) file; in order to make the output txt file easy to read and understand, text format and indentation can be added. After traversing the entire IR structure and writing all the content, close the file handle to complete the write operation.

[0071] Furthermore, the text file format information can be stored so that when it is not the first time processing, the stored text file format information can be directly read to process the entire neural network model without converting the information format, thereby improving the efficiency of model training.

[0072] Step 203, reconstructing the index of the intermediate representation information to generate an intermediate representation graph;

[0073] The operator features in the intermediate representation information are indexed and reconstructed to visualize the structure of the neural network model, making it easier to understand any section of the neural network model. An intermediate representation graph is generated based on the index-reconstructed data. The intermediate representation graph is a model structure graph formed in the intermediate representation format after index reconstruction.

[0074] In an optional embodiment of the embodiment of the present invention, the step of reconstructing the index of the intermediate representation information to generate an intermediate representation graph includes:

[0075] Sub-step S2031, performing regularization extraction on the intermediate representation information to determine an operator index;

[0076] Regularized extraction can be performed on the intermediate representation information to identify the model's operators, corresponding relationships, etc., and determine the operator index. Regularized extraction is the use of regularization technology to extract model features in the context of machine learning. Regularization can help reduce the complexity of the model by constraining the model's parameters, thereby improving its generalization ability.

[0077] Specifically, the step of performing regularization extraction on the intermediate representation information and determining the operator index includes: performing regularization extraction on the intermediate representation information to determine the operator structure and dependency relationship; combining the operator structure and the dependency relationship to generate the operator index.

[0078] The intermediate representation information can be regularized and extracted to determine the operator structure of each operator in the model and the dependencies between operators. The operator structures are then combined according to the dependencies to generate the corresponding operator index.

[0079] Furthermore, the step of combining the operator structure and the dependency relationship to generate the operator index includes: encoding the operators in the operator structure according to preset rules to obtain operator codes; and associating the operator codes according to the dependency relationship to generate the operator index.

[0080] In practical applications, the operators in the operator structure can be encoded according to the preset rules to obtain the operator code, so as to represent the operator in the form of data. Then, according to the dependency relationship between operators, the operator codes are associated to form an index relationship and generate the operator index.

[0081] The preset rule may be input and determined based on the pre-setting by relevant technical personnel according to the requirements before processing, or may be a reference to an existing numbering rule, which is not limited in the embodiment of the present invention.

[0082] Sub-step S2032: generating the intermediate representation graph according to the operator index.

[0083] Generate the corresponding intermediate representation graph according to the index relationship in the operator index.

[0084] Step 204, querying the intermediate representation graph to search and determine the target operator position according to the positioning requirement instruction;

[0085] After the intermediate representation graph is obtained, the intermediate representation graph is searched according to the positioning requirement instruction, the operator matching the positioning requirement instruction is determined, and the target operator position of the target operator is located.

[0086] In an optional embodiment of the present invention, the step of searching the intermediate representation graph according to the positioning requirement instruction to locate the target operator includes:

[0087] Step S2041, truncating the intermediate representation graph to generate at least two sub-intermediate representation graphs;

[0088] First, the intermediate representation graph may be truncated into at least two parts, that is, at least two sub-intermediate representation graphs are generated.

[0089] Specifically, the step of truncating the intermediate representation graph to generate at least two sub-intermediate representation graphs includes: determining a target truncation position in the operator index; and truncating the intermediate representation graph based on the target truncation position to generate the at least two sub-intermediate representation graphs.

[0090] In the actual truncation process, firstly, the target truncation position can be determined in the operator index based on the required truncation part, and the target truncation position is the position used for segmenting the model later. Then, the intermediate representation graph is truncated from the target truncation position, so that the intermediate representation graph is truncated into at least two parts, that is, at least two sub-intermediate representation graphs.

[0091] Further, the step of determining a target truncation position in the operator index includes: acquiring a truncation quantity; and locating a target truncation position corresponding to the truncation quantity in the operator index.

[0092] In practical applications, the number of truncation can be obtained, and the number of truncation can be determined according to the size of the model. The larger the model, the more truncation is required. The determination of the number of truncation can be determined by threshold judgment. When the number of operators exceeds a certain threshold, it needs to be truncated into three or more sub-models. When the number of model operators is less than the threshold, it only needs to be truncated into two sub-models.

[0093] After determining the number of truncation, locate the target truncation position corresponding to the number of truncation in the operator index. For example, if the number of truncation is 2, it means that it needs to be truncated into two sub-models, and the middle position of the operator index can be determined as the target truncation position. If the number of truncation is 3, it means that it needs to be truncated into three sub-models, and the positions of one-third and two-thirds of the operator index can be determined as the target truncation positions.

[0094] Then the operator index is truncated at the target truncation position to generate the sub-intermediate representation graph.

[0095] Step S2042, determining the positioning type according to the positioning requirement instruction;

[0096] At the same time, it is also necessary to determine the positioning type according to the instructions in the positioning requirement instruction.

[0097] Step S2043: For any of the sub-intermediate representation graphs, search according to the positioning type to locate the target operator.

[0098] In each obtained sub-intermediate representation graph, according to the positioning type, a feature search is performed in a single sub-intermediate representation graph to determine whether there is an operator that meets the requirements, thereby locating the target operator.

[0099] Specifically, the step of searching for any of the sub-intermediate representation graphs according to the positioning type to locate the target operator includes: searching the sub-intermediate representation graph for an operator matching the positioning type as the target operator.

[0100] The operator that matches the positioning type can be searched in the sub-intermediate representation graph as the target operator. When the operator is still not found in all the sub-intermediate representation graphs of the same level, all the sub-intermediate representation graphs can be truncated again, and the operator that matches the positioning type can be searched from the new sub-intermediate representation graphs until the target operator is obtained.

[0101] For example, see Figure 3 , using the binary search method, for a model with 100 operators, we first truncate the model at 0-50 to verify its mixed precision reasoning results. If there is a problem, we truncate it at 0-25, otherwise we truncate it at 51-100. Then continue to truncate for positioning until the target operator is determined.

[0102] In addition, after obtaining the target operator, the target operator needs to be verified to determine whether the target operator located this time is correct, so as to ensure the accuracy of positioning. The step of searching for the target operator according to the positioning type for any of the sub-intermediate representation graphs also includes: verifying the operator matching the positioning type; in response to the verification being successful, determining the operator matching the positioning type as the target operator; in response to the verification being unsuccessful, repeatedly searching the sub-intermediate representation graph for an operator matching the positioning type as the target operator.

[0103] When verifying the target operator, the operator that matches the positioning type can be verified to obtain the verification result. When the verification passes, that is, the currently located operator is the operator that needs to be found, the operator that matches the positioning type can be determined as the target operator. When the verification fails, that is, the currently located operator is not the operator that needs to be found, the step of searching for the operator that matches the positioning type as the target operator in the sub-intermediate representation graph is repeated until the target operator that passes the verification is found.

[0104] Step S2044, determining the position corresponding to the target operator as the target operator position;

[0105] After the target operator is determined, the specific location of the target operator may be determined as the target operator location.

[0106] Specifically, when the positioning type is a fault type, the step of verifying the operator that matches the positioning type includes: setting the operator that matches the positioning type to zero; running the neural network model corresponding to the intermediate representation graph according to the set operator that matches the positioning type to generate operation output data; and performing verification based on the operation output data.

[0107] When locating a faulty operator, since the operator is faulty, setting it to 0 will not affect the current running state of the model. The operator matching the positioning type can be set to zero; based on the set operator matching the positioning type, the neural network model corresponding to the intermediate representation graph is run to generate running output data; verification is performed based on the changes in the running output data.

[0108] In addition, since some models may be tree structures, in order to improve the search efficiency of the intermediate representation graph of this type of deep learning framework, the step of searching for any of the sub-intermediate representation graphs according to the positioning type and locating the target operator includes: locating the target operator in the sub-intermediate representation graph based on a generalized first search or a depth-first search.

[0109] In the sub-intermediate representation graph, operators can be searched based on generalized first search or depth-first search to locate the target operator. Generalized first search is a priority-based search strategy that determines the order of expansion according to the priority of the nodes during the search process. Nodes with higher priorities are expanded first for search. Depth-first search is a search strategy used to traverse or search a tree or graph. The search process starts from a starting node and searches the branches of the graph as deeply as possible. When all the edges where the node is located have been explored, the search will backtrack to the starting node of the edge where the node was found. This process continues until all nodes reachable from the source node have been found. If there are still undiscovered nodes, one of them is selected as the source node and the above process is repeated. The whole process is repeated until all nodes have been visited.

[0110] Step 205, extracting the target operator at the target operator position to generate text information;

[0111] The target operator is extracted from the target operator position, and the target operator is recorded in the form of text to generate text information.

[0112] Step 206, reading the text information;

[0113] Step 207: Generate deep learning model compilation information based on the target operator in the text information.

[0114] Read the text information and generate deep learning model compilation information based on the target operator in the text information. That is, the next time you run the model, you only need to read the text information without importing the model, which saves the time spent on the operator conversion process and improves training efficiency.

[0115] In addition, the method may further include receiving a modification instruction; and modifying the property of the target operator according to the modification instruction to update the position of the target operator.

[0116] After locating the target operator, the target operator can be modified to achieve debugging or reasoning of the model. The content of the modification is set as needed to generate the corresponding modification instruction. The modification instruction can be received to modify the subsequent operators and the corresponding operator positions. The properties of the target operator are modified according to the information in the modification instruction so that the properties of the target operator meet the requirements in the modification instruction; to update the target operator position.

[0117] Specifically, the modification instruction includes a target precision attribute, and the step of modifying the attribute of the target operator according to the modification instruction includes: modifying the string precision attribute of the target operator to the target precision attribute.

[0118] In actual applications, the string precision attribute of the current operator may be modified to facilitate model training. The string precision attribute of the target operator can be modified to the target precision attribute, thereby realizing the modification of the string precision attribute of the target operator. For example, the string precision attribute of the target operator can be changed to half precision (that is, the target precision attribute).

[0119] Furthermore, the precision corresponding to the target precision attribute is smaller than the precision corresponding to the string precision attribute of the target operator, that is, the precision corresponding to the string precision attribute of the target operator is reduced, thereby compressing the model and further improving the efficiency of model training.

[0120] In addition, the deep learning framework data is updated based on the updated target operator. After obtaining the updated target operator, the deep learning framework data can also be updated based on the updated target operator, thereby updating the parameters of the model. The model can modify a certain parameter or a certain category of parameters separately without affecting other parameters for training reasoning, thereby improving the efficiency of training.

[0121] Furthermore, the target operator may be extracted at the target operator position to update the target depth model;

[0122] At the target operator position, the target operator is extracted. The target deep model is updated using the extracted target operator, so that the target deep model can be iterated to improve the recognition accuracy of the target deep model. The target deep model is a deep learning neural network model used for fault identification processing. The updated target deep model is used to identify fault information from the log data.

[0123] The embodiment of the present invention receives deep learning framework data and positioning requirement instructions; converts the deep learning framework data into intermediate representation information; indexes and reconstructs the intermediate representation information to generate an intermediate representation graph; searches the intermediate representation graph according to the positioning requirement instructions to locate the target operator; receives modification instructions; modifies the properties of the target operator according to the modification instructions to update the target operator; and updates the deep learning framework data based on the updated target operator. The differences between frameworks are eliminated by converting into intermediate representation information, and the dependence on the framework is eliminated; based on the reconstruction of the intermediate representation graph, the specific features of the operator are clearly displayed through the intermediate representation graph, so that the target operator can be quickly located from the intermediate representation graph, achieving fast and accurate operator positioning, so that the target operator can be modified, and model training can be performed based on the modified operator to improve training efficiency.

[0124] In order to make the implementation process of the embodiments of the present invention clear to those skilled in the art, the following examples are used for illustration:

[0125] You can refer to Figure 4 , showing a step flow chart of an example of a deep learning operator positioning method of the present invention.

[0126] 1. Convert the deep learning framework into the IR format of the deep learning compiler and save it in TXT format. 2. Regularize and extract the IR information saved in TXT format, automatically extract the model information, rebuild the index, and generate the IR graph. 3. Based on this IR graph, cut the model according to the index to facilitate the positioning of the operator.

[0127] You can also Figure 5 As shown in the figure, through the deep learning compiler, we convert the models of different frameworks into a unified intermediate representation format, and then extract the input and output of the IR through the regularization method, and number each operator in sequence, which can automatically generate an IR format that the deep learning compiler can recognize. The process will be saved in TXT format. The next time you run the model, you only need to read the TXT file without importing the model, which saves the time spent on the conversion process from operator to IR. Based on this, combined with Figure 6, edit the IR information directly. First, extract the input, output, and output dimensions through the regularization method, and reconstruct them into a new IR graph. The new IR graph can be defined according to the operator number corresponding to the operator rule of Yisuhe. After generating the IR information, locate the out-of-bounds operator, add a new attribute to the operator, and indicate that the operator is a property change. In this way, add a corresponding attribute to each operator, and control the construction of the entire model through the TXT file.

[0128] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0129] Reference Figure 7 , an embodiment of the present invention further provides an electronic device, including:

[0130] A processor 701 and a storage medium 702, wherein the storage medium 702 stores a computer program executable by the processor 701, and when the electronic device is running, the processor 701 executes the computer program to perform the deep learning operator positioning method as described in any one of the embodiments of the present invention.

[0131] Receive deep learning framework data and positioning requirement instructions;

[0132] Converting the deep learning framework data into intermediate representation information;

[0133] Reconstructing the index of the intermediate representation information to generate an intermediate representation graph;

[0134] The intermediate representation graph is queried according to the positioning requirement instruction to perform a search and determine the target operator position.

[0135] Optionally, the step of reconstructing the index of the intermediate representation information to generate an intermediate representation graph includes:

[0136] Regularizing and extracting the intermediate representation information to determine an operator index;

[0137] The intermediate representation graph is generated according to the operator index.

[0138] Optionally, the step of performing regularization extraction on the intermediate representation information and determining the operator index includes:

[0139] Regularizing and extracting the intermediate representation information to determine the operator structure and dependency relationship;

[0140] The operator structure and the dependency are combined to generate the operator index.

[0141] Optionally, the step of combining the operator structure with the dependency relationship to generate the operator index includes:

[0142] Encoding the operators in the operator structure according to a preset rule to obtain operator codes;

[0143] According to the dependency relationship, the operator codes are associated to generate the operator index.

[0144] Optionally, the step of searching the intermediate representation graph according to the positioning requirement instruction to determine the position of the target operator includes:

[0145] In the operator index, determining a target cutoff position;

[0146] Truncating the intermediate representation graph based on the target truncation position to generate the at least two sub-intermediate representation graphs;

[0147] Determine the positioning type according to the positioning requirement instruction;

[0148] For any of the sub-intermediate representation graphs, searching the sub-intermediate representation graph for an operator matching the positioning type;

[0149] Verifying the operator that matches the positioning type;

[0150] In response to the verification being passed, determining the operator matching the positioning type as the target operator;

[0151] In response to the verification failing, repeatedly performing the step of searching the sub-intermediate representation graph for an operator matching the positioning type as the target operator;

[0152] The position corresponding to the target operator is determined as the target operator position.

[0153] Optionally, when the positioning type is a fault type, the step of verifying the operator matching the positioning type includes:

[0154] Setting the operator matching the positioning type to zero;

[0155] According to the set operator matching the positioning type, the neural network model corresponding to the intermediate representation graph is run to generate running output data;

[0156] Verification is performed based on the operation output data.

[0157] Optionally, the method comprises:

[0158] Reading the text information;

[0159] Based on the target operator in the text information, deep learning model compilation information is generated.

[0160] Optionally, the method comprises:

[0161] Receive modification instructions;

[0162] The property of the target operator is modified according to the modification instruction to update the position of the target operator.

[0163] The embodiment of the present invention receives deep learning framework data and positioning requirement instructions; converts the deep learning framework data into intermediate representation information; indexes and reconstructs the intermediate representation information to generate an intermediate representation graph; searches the intermediate representation graph according to the positioning requirement instructions to determine the position of the target operator. The differences between frameworks are eliminated by converting to intermediate representation information, and the dependence on the framework is eliminated; based on the reconstruction of the intermediate representation graph, the specific features of the operator are clearly displayed through the intermediate representation graph, so that the position of the target operator can be quickly located from the intermediate representation graph, achieving fast and accurate operator positioning, thereby improving the efficiency of model training.

[0164] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor.

[0165] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0166] Reference Figure 8The embodiment of the present invention further provides a computer-readable storage medium 801, on which a computer program is stored. When the computer program is executed by a processor, the deep learning operator positioning method as described in any one of the embodiments of the present invention is executed.

[0167] The deep learning operator positioning method comprises:

[0168] Receive deep learning framework data and positioning requirement instructions;

[0169] Converting the deep learning framework data into intermediate representation information;

[0170] Reconstructing the index of the intermediate representation information to generate an intermediate representation graph;

[0171] The intermediate representation graph is queried according to the positioning requirement instruction to perform a search and determine the target operator position.

[0172] Optionally, the step of reconstructing the index of the intermediate representation information to generate an intermediate representation graph includes:

[0173] Regularizing and extracting the intermediate representation information to determine an operator index;

[0174] The intermediate representation graph is generated according to the operator index.

[0175] Optionally, the step of performing regularization extraction on the intermediate representation information and determining the operator index includes:

[0176] Regularizing and extracting the intermediate representation information to determine the operator structure and dependency relationship;

[0177] The operator structure and the dependency are combined to generate the operator index.

[0178] Optionally, the step of combining the operator structure with the dependency relationship to generate the operator index includes:

[0179] Encoding the operators in the operator structure according to a preset rule to obtain operator codes;

[0180] According to the dependency relationship, the operator codes are associated to generate the operator index.

[0181] Optionally, the step of searching the intermediate representation graph according to the positioning requirement instruction to determine the position of the target operator includes:

[0182] In the operator index, determining a target cutoff position;

[0183] Truncating the intermediate representation graph based on the target truncation position to generate the at least two sub-intermediate representation graphs;

[0184] Determine the positioning type according to the positioning requirement instruction;

[0185] For any of the sub-intermediate representation graphs, searching the sub-intermediate representation graph for an operator matching the positioning type;

[0186] Verifying the operator that matches the positioning type;

[0187] In response to the verification being passed, determining the operator matching the positioning type as the target operator;

[0188] In response to the verification failing, repeatedly performing the step of searching the sub-intermediate representation graph for an operator matching the positioning type as the target operator;

[0189] The position corresponding to the target operator is determined as the target operator position.

[0190] Optionally, when the positioning type is a fault type, the step of verifying the operator matching the positioning type includes:

[0191] Setting the operator matching the positioning type to zero;

[0192] According to the set operator matching the positioning type, the neural network model corresponding to the intermediate representation graph is run to generate running output data;

[0193] Verification is performed based on the operation output data.

[0194] Optionally, the method comprises:

[0195] Reading the text information;

[0196] Based on the target operator in the text information, deep learning model compilation information is generated.

[0197] Optionally, the method comprises:

[0198] Receive modification instructions;

[0199] The property of the target operator is modified according to the modification instruction to update the position of the target operator.

[0200] The embodiment of the present invention receives deep learning framework data and positioning requirement instructions; converts the deep learning framework data into intermediate representation information; indexes and reconstructs the intermediate representation information to generate an intermediate representation graph; searches the intermediate representation graph according to the positioning requirement instructions to determine the position of the target operator. The differences between frameworks are eliminated by converting to intermediate representation information, and the dependence on the framework is eliminated; based on the reconstruction of the intermediate representation graph, the specific features of the operator are clearly displayed through the intermediate representation graph, so that the position of the target operator can be quickly located from the intermediate representation graph, achieving fast and accurate operator positioning, thereby improving the efficiency of model training.

[0201] Reference Fig. 9 , shows a structural block diagram of a computer program product provided by an embodiment of the present invention. The computer program product includes computer instructions 901, and when the computer instructions 901 are executed by a processor, the steps of the deep learning operator positioning method described above are implemented.

[0202] The deep learning operator positioning method comprises:

[0203] Receive deep learning framework data and positioning requirement instructions;

[0204] Converting the deep learning framework data into intermediate representation information;

[0205] Reconstructing the index of the intermediate representation information to generate an intermediate representation graph;

[0206] The intermediate representation graph is queried according to the positioning requirement instruction to perform a search and determine the target operator position.

[0207] Optionally, the step of reconstructing the index of the intermediate representation information to generate an intermediate representation graph includes:

[0208] Regularizing and extracting the intermediate representation information to determine an operator index;

[0209] The intermediate representation graph is generated according to the operator index.

[0210] Optionally, the step of performing regularization extraction on the intermediate representation information and determining the operator index includes:

[0211] Regularizing and extracting the intermediate representation information to determine the operator structure and dependency relationship;

[0212] The operator structure and the dependency are combined to generate the operator index.

[0213] Optionally, the step of combining the operator structure with the dependency relationship to generate the operator index includes:

[0214] Encoding the operators in the operator structure according to a preset rule to obtain operator codes;

[0215] According to the dependency relationship, the operator codes are associated to generate the operator index.

[0216] Optionally, the step of searching the intermediate representation graph according to the positioning requirement instruction to determine the position of the target operator includes:

[0217] In the operator index, determining a target cutoff position;

[0218] Truncating the intermediate representation graph based on the target truncation position to generate the at least two sub-intermediate representation graphs;

[0219] Determine the positioning type according to the positioning requirement instruction;

[0220] For any of the sub-intermediate representation graphs, searching the sub-intermediate representation graph for an operator matching the positioning type;

[0221] Verifying the operator that matches the positioning type;

[0222] In response to the verification being passed, determining the operator matching the positioning type as the target operator;

[0223] In response to the verification failing, repeatedly performing the step of searching the sub-intermediate representation graph for an operator matching the positioning type as the target operator;

[0224] The position corresponding to the target operator is determined as the target operator position.

[0225] Optionally, when the positioning type is a fault type, the step of verifying the operator matching the positioning type includes:

[0226] Setting the operator matching the positioning type to zero;

[0227] According to the set operator matching the positioning type, the neural network model corresponding to the intermediate representation graph is run to generate running output data;

[0228] Verification is performed based on the operation output data.

[0229] Optionally, the method comprises:

[0230] Reading the text information;

[0231] Based on the target operator in the text information, deep learning model compilation information is generated.

[0232] Optionally, the method comprises:

[0233] Receive modification instructions;

[0234] The property of the target operator is modified according to the modification instruction to update the position of the target operator.

[0235] The embodiment of the present invention receives deep learning framework data and positioning requirement instructions; converts the deep learning framework data into intermediate representation information; indexes and reconstructs the intermediate representation information to generate an intermediate representation graph; searches the intermediate representation graph according to the positioning requirement instructions to determine the position of the target operator. The differences between frameworks are eliminated by converting to intermediate representation information, and the dependence on the framework is eliminated; based on the reconstruction of the intermediate representation graph, the specific features of the operator are clearly displayed through the intermediate representation graph, so that the position of the target operator can be quickly located from the intermediate representation graph, achieving fast and accurate operator positioning, thereby improving the efficiency of model training.

[0236] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0237] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0238] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0239] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0240] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0241] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0242] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0243] The above is a detailed introduction to a deep learning operator positioning method, an electronic device, a storage medium and a program product provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A deep learning operator positioning method, characterized in that: include: Receive deep learning framework data and positioning requirement instructions; Converting the deep learning framework data into intermediate representation information; Reconstructing the index of the intermediate representation information to generate an intermediate representation graph; According to the positioning requirement instruction, the intermediate representation graph is searched to determine the target operator position; The target operator is extracted at the target operator position to generate text information.

2. The deep learning operator positioning method according to claim 1, characterized in that: The step of reconstructing the index of the intermediate representation information to generate an intermediate representation graph comprises: Regularizing and extracting the intermediate representation information to determine an operator index; The intermediate representation graph is generated according to the operator index.

3. The deep learning operator positioning method according to claim 2, characterized in that: The step of regularizing and extracting the intermediate representation information and determining the operator index comprises: Regularizing and extracting the intermediate representation information to determine the operator structure and dependency relationship; The operator structure and the dependency are combined to generate the operator index.

4. The deep learning operator positioning method according to claim 3, characterized in that: The step of combining the operator structure and the dependency relationship to generate the operator index comprises: Encoding the operators in the operator structure according to a preset rule to obtain operator codes; According to the dependency relationship, the operator codes are associated to generate the operator index.

5. The deep learning operator positioning method according to claim 2, characterized in that: The step of searching the intermediate representation graph according to the positioning requirement instruction to determine the position of the target operator includes: In the operator index, determining a target cutoff position; Truncating the intermediate representation graph based on the target truncation position to generate the at least two sub-intermediate representation graphs; Determine the positioning type according to the positioning requirement instruction; For any of the sub-intermediate representation graphs, searching the sub-intermediate representation graph for an operator matching the positioning type; Verifying the operator that matches the positioning type; In response to the verification being passed, determining the operator matching the positioning type as the target operator; In response to the verification failing, repeatedly performing the step of searching the sub-intermediate representation graph for an operator matching the positioning type as the target operator; The position corresponding to the target operator is determined as the target operator position.

6. The deep learning operator positioning method according to claim 5, characterized in that: When the positioning type is a fault type, the step of verifying the operator matching the positioning type includes: Setting the operator matching the positioning type to zero; According to the set operator matching the positioning type, the neural network model corresponding to the intermediate representation graph is run to generate running output data; Verification is performed based on the operation output data.

7. The deep learning operator positioning method according to claim 1, characterized in that: The method comprises: Reading the text information; Based on the target operator in the text information, deep learning model compilation information is generated.

8. An electronic device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the deep learning operator positioning method as described in any one of claims 1 to 17 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the deep learning operator positioning method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by the processor, the steps of the deep learning operator positioning method as described in any one of claims 1 to 7 are implemented.