Artificial intelligence processor operator overflow optimization method and system

Abstract

The application relates to the technical field of operator overflow control, in particular to an AI processor operator overflow optimization method and system, state combination and sequencing are performed on a numerical value introduction finite automaton, a continuous state sequence covering an operator execution stack is generated, a state number carrying bit width section information and stack position information at the same time is obtained for an intermediate result, a section to which an operator intermediate value belongs is provided with discrete identification in a fixed point space, a graph neural network is introduced to perform a weighted truncation reduction transformation, a clipping weight is derived under a topological connection constraint between operators, each operator intermediate value is simultaneously provided with differentiated correction strength in a longitudinal accumulation direction and a horizontal link direction, the intermediate value is guided to generate an operator correction set through the clipping weight, the utilization rate of an effective dynamic range after right shift and truncation cooperation reduction is improved, and the stability, repeatability and tracking ability for abnormal calculation behavior of an inference process in a long chain accumulation scene are enhanced.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of operator overflow control, in particular to an AI processor operator overflow optimization method and system. BACKGROUND

[0002] The technical field of operator overflow control is a processor system for executing machine learning model inference operations, including hardware units for executing convolution operators, matrix multiplication operators, activation operators, and normalization operators. The technical field of operator overflow control involves limiting the numerical range, determining the overflow, and correcting the results of intermediate results of multiplication, addition, shifting, and accumulation operations in fixed-point and mixed-precision operations. The field focuses on establishing threshold intervals, detection logic, bit width constraint rules, and correction strategies for processors when performing fixed-bit width operations to ensure that operator outputs meet stable intervals and avoid calculation abnormalities.

[0003] The AI processor operator overflow optimization method is a processing scheme for detecting, limiting, and correcting intermediate numerical values generated during the execution of operators in a fixed-bit width operation environment. The purpose of the method is to establish a clear numerical interval during operator execution and adjust the numerical values when the intermediate results exceed the interval to keep the operator output within the effective range supported by the processor. The goal is to enable various operators to maintain stable calculations in a hardware-limited bit width environment, avoid sign flipping, saturation errors, and calculation interruptions, and maintain the continuity and repeatability of the inference process.

[0004] The prior art relies on fixed threshold intervals and simple comparison logic in operator overflow control. The intermediate numerical values generated by multiplication, addition, shifting, and accumulation are generally configured with uniform upper and lower limits. The differences in overflow sensitivity of various operators at different operation stages and different accumulation depths cannot be reflected in the numerical interval setting. In long-chain inference processes, some intermediate numerical values of operators are close to the bit width boundary for a long time, and some intermediate numerical values remain far from the boundary for a long time, resulting in overprotection and insufficient protection of the same threshold interval for different operators. Overflow determination is mostly based on single comparison and a small number of cascaded comparison levels, and lacks continuous depiction of the evolution trajectory of intermediate numerical values on the execution stack, making it difficult to provide accurate determination basis combined with operator position. Bit width constraint rules and correction strategies are mostly configured separately from operation units in the form of static rule tables. After the intermediate numerical values undergo right shift, truncation, and reduction, they usually lack traceable link-level records. Once the inference result deviates abnormally, it is difficult to quickly locate the abnormal operator and corresponding operation stage based on existing records. When executing machine learning model inference, it affects the stability of inference accuracy and the repeated performance under different input distributions, and increases the engineering burden of tuning and verifying for specific model scenarios. SUMMARY

[0005] The application aims to solve the problems in the prior art and provides an AI processor operator overflow optimization method and system.

[0006] To achieve the above-mentioned purpose, the application adopts the following technical scheme: an AI processor operator overflow optimization method, comprising the following steps:

[0007] S1: comparing the upper limit of the bit width and the lower limit of the bit width in the configuration table item by item through the operator hardware bit width configuration table and reading the operator numerical slot boundary, arranging the read content with the three overflow sections, and obtaining the section boundary graph;

[0008] S2: based on the section boundary graph, aligning the section numerical value with the intermediate result of the operator accumulation module, combining and sorting the alignment result and the stack state number of the operator using a finite automaton, rearranging the nodes after splitting and adjacency judgment of the sorting chain state nodes, and generating a state node book;

[0009] S3: based on the state node book, comparing the state number with the critical section in the overflow section projection graph, combining the critical section number and the right shift stack right shift value truncation value reduction ratio, arranging it into a control record according to the operator execution stack order, and establishing an overflow control book;

[0010] S4: based on the overflow control book, corresponding to the right shift truncation reduction control item and the intermediate numerical value of the operator accumulation link, transforming the corresponding numerical value graph neural network with weighted truncation reduction, rearranging the transformed intermediate numerical value into a correction structure according to the operator, and forming an operator correction set;

[0011] S5: based on the overflow control book and the operator correction set, comparing the corrected numerical value with the control item and writing it into the specified position information of the operator accumulation module according to the right shift truncation reduction order, combining the written link section into link data according to the operator execution stack order, and outputting a reasoning link file.

[0012] As a further scheme of the application, the section boundary graph includes the upper limit of the bit width corresponding section, the lower limit of the bit width corresponding section and the three overflow section corresponding boundary, the state node book includes the aligned section numerical value record, the stack state number record and the node record arranged according to the operator position, the overflow control book includes the critical section number record, the right shift amount record and the truncation value reduction ratio record, the operator correction set includes the right shift truncation reduction transformed value, the numerical value rearranged according to the operator stack order and the correction structure record, and the reasoning link file includes the right shift written section, the truncation written section and the reduction written section.

[0013] As a further scheme of the application, the specific steps to obtain the section boundary graph are as follows:

[0014] Based on the operator hardware bit width configuration table, the bit width upper limit item in the configuration table is compared with the bit width lower limit item one by one, and the upper limit value and the lower limit value compared are corresponded with the operator numerical slot boundary respectively, the corresponding records are arranged in order according to the table, and a bit width boundary cluster is generated;

[0015] Based on the bit width boundary cluster, the boundary records in the bit width boundary cluster are sequentially corresponded with the three overflow sections, and a section arrangement structure is established according to the corresponding order, the section arrangement structure is written into the section relationship record table, and a section limit graph is generated.

[0016] As a further scheme of the application, the specific steps for generating the state node book are:

[0017] Based on the section limit graph, the section numerical values in the section limit graph are compared one by one with the intermediate results of the operator accumulation module, the paired numerical values after comparison are arranged in order according to the record order, and the arrangement content is written into a temporary alignment table, and an alignment node group is generated.

[0018] Based on the alignment node group, the paired numerical values in the alignment node group are sequentially combined with the stack state number of the operator by using the finite automaton, the arranged nodes are arranged according to the combination generation order, and the arranged nodes are split into a segmented structure according to the continuous relationship, a sorting and splitting cluster is generated.

[0019] Based on the sorting and splitting cluster, the segmented nodes in the sorting and splitting cluster are compared one by one with the adjacent numbers, and the adjacency judgment is performed, the nodes satisfying the adjacency relationship are rearranged according to the operator position order, and the rearranged structure is written into the node sequence, and a state node book is generated.

[0020] As a further scheme of the application, the finite automaton takes the paired numerical values recorded in the alignment node group as input items, reads the section range corresponding to the paired numerical values from the section limit graph one by one, takes the current automaton state and the section range as a group of joint input into the state transition table, determines the target state number by matching the index position of the joint input in the state transition table, writes the target state number into the stack state number record corresponding to the paired numerical value, and sequentially performs the above reading, matching and number writing actions according to the order of the paired numerical values in the alignment node group, to form a continuous state sequence, the state sequence is then used as the state basis for node arrangement and segmented splitting of the sorting and splitting cluster, and the final state record is completed according to the operator position order when the state node book is constructed.

[0021] As a further scheme of the application, the specific steps for establishing the overflow control book are:

[0022] Based on the state node book, the state numbers are compared one by one with the critical section of the overflow section projection graph, and the reduction ratio of the falling number and the right shift amount truncation value is sequentially combined, the combined content is written into the temporary record according to the number order, and the record is arranged according to the line, to generate a critical joint cluster.

[0023] Based on the critical joint cluster, the joint records are taken out one by one in the order of the operator execution stack, and the taken-out contents are written into the control records in the order of the link, the records are sorted by segments, and the sorted contents are sorted into the index set, and the overflow control book is established.

[0024] As a further scheme of the present application, the specific steps for constituting the operator correction set are:

[0025] Based on the overflow control book, the right shift item truncation item reduction item in the overflow control book is extracted one by one to the intermediate value of the operator accumulation link, the corresponding values are classified and split into multiple item structures according to the right shift truncation reduction, and the split results are written into the temporary table to generate the control corresponding cluster;

[0026] Based on the control corresponding cluster, a graph structure is constructed with the operator in the operator accumulation link as the node and the right shift value, the truncation value and the reduction value in the control corresponding cluster as the edge attribute, a message passing is performed on the node and the edge attribute using a graph neural network to obtain a pruning weight, the pruning weight is written into the corresponding control item, the right shift value, the truncation value and the reduction value in the control corresponding cluster are taken out, the right shift, truncation and reduction are performed according to the control item type, and the transformation result is adjusted and transformed with the pruning weight, the transformed output is written into the index record according to the operator stack number, and a transformation index group is generated;

[0027] Based on the transformation index group, the values of the transformation index group records are traced back one by one according to the operator execution stack number, the traced back values are combined into a continuous correction link in the order of the level, and the correction values of each level in the link are integrated according to the corresponding pruning weight proportion, the correction link is written into the correction record table, and an operator correction set is generated.

[0028] As a further scheme of the present application, the graph neural network takes the operator in the operator accumulation link as the node identifier, writes the right shift value, the truncation value and the reduction value in the control corresponding cluster into the edge attribute table according to the connection order between the operators, takes the node features and the edge attributes as the initial input, reads the messages, combines the messages and updates the nodes in turn between the adjacent operator nodes, writes the updated node values into the node temporary record table, reads the updated values from the node temporary record table after the completion of the multiple rounds of message passing, and writes the updated values into the pruning weight record table according to the operator number, so as to provide the pruning weight data corresponding to the operator number for the subsequent steps.

[0029] As a further scheme of the present application, the specific steps for outputting the inference link file are:

[0030] Based on the overflow control book and the operator correction set, the correction values are compared one by one, the corresponding order is generated, and the segment data records are written into the link writing table to generate a writing link group.

[0031] Based on the write link group, read the write fragment data according to the operator sequence number recorded in the write link group, and lock the corresponding operator accumulation module position of each fragment, write the fragment content to the locked position, and then merge each write position according to the sequence to establish the inference link file.

[0032] The AI processor operator overflow optimization system is used for executing the above-mentioned AI processor operator overflow optimization method, and the system comprises:

[0033] The segment construction module reads and arranges the upper limit of bit width, the lower limit of bit width and the numerical slot boundary based on the operator hardware bit width configuration table, establishes the bit width limit corresponding structure, and obtains the segment limit graph after completing the corresponding arrangement.

[0034] The node arrangement module performs pairing, numbering and node splitting of segment values and intermediate results based on the segment limit graph, performs associated scanning and position arrangement of node structure, completes rearrangement according to operator position, and generates a state node book.

[0035] The control integration module compares the state number and the critical segment based on the state node book, combines the corresponding right shift amount, the truncation value and the reduction ratio, and completes sequence arrangement and record regularization according to the operator execution stack sequence, so as to establish an overflow control book.

[0036] The correction generation module corresponds the right shift item, the truncation item and the reduction item to the intermediate value based on the overflow control book, introduces a graph neural network to generate a weighted transformation result, and performs backtracking and reorganization according to the operator stack sequence, to form an operator correction set for compensating overflow.

[0037] The link output module compares the modified numerical value and the multiple control contents based on the overflow control book and the operator correction set to form a write fragment and write to the specified position information, completes fragment merging according to the operator execution stack sequence, and outputs the inference link file.

[0038] Compared with the prior art, the advantages and positive effects of the present application are that:

[0039] 1. In the present application, the state combination and ordering of the numerical value are performed by introducing a finite automaton, a continuous state sequence covering the operator execution stack is generated, the intermediate result obtains a state number carrying bit width segment information and stack position information at the same time, and the segment to which the intermediate numerical value of the operator belongs has discrete identification in the fixed point space.

[0040] 2. In the present application, the weighted truncation and reduction transformation is performed by introducing a graph neural network, the clipping weight is derived under the topological connection constraint between operators, and the intermediate numerical value of each operator obtains differentiated correction strength in the vertical accumulation direction and the horizontal link direction at the same time.

[0041] 3. In the present application, by guiding the intermediate value generation operator correction set with the clipping weight, the overflow risk identification accuracy is improved in the fixed bit width environment, the utilization rate of the effective dynamic range after the cooperation of the right shift and the truncation is reduced, and the stability, repeatability and tracking ability of the inference process in the long chain accumulation scene are enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The workflow diagram of the present application is shown in the figure.

[0043] Figure 2 The system flowchart of the present application is shown in the figure. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0045] Example 1

[0046] Please refer to Figure 1 The present application provides a technical scheme: an AI processor operator overflow optimization method, comprising the following steps:

[0047] S1: By comparing the bit width upper limit and the bit width lower limit in the operator hardware bit width configuration table and reading the operator value slot boundary, the read content is arranged corresponding to the three segment overflow section, and the section limit diagram is obtained;

[0048] S2: Based on the section limit diagram, align the section value with the intermediate result of the operator accumulation module, combine the alignment result with the stack state number of the operator using finite automata, and sort the state nodes after splitting and adjacency judgment, and rearrange the nodes according to the operator position to generate the state node book;

[0049] S3: Based on the state node book, compare the state number with the critical section in the overflow section projection diagram, combine the critical section number with the operator shift stack right shift amount truncation value reduction ratio, arrange the control record according to the operator execution stack order to establish the overflow control book;

[0050] S4: Based on the overflow control book, corresponding to the right shift truncation reduction control item and the intermediate value of the operator accumulation link, the corresponding value graph neural network is weighted and truncated to reduce the transformation, and the transformed intermediate value is arranged according to the operator to form a correction structure, which constitutes an operator correction set;

[0051] S5: Comparing the modified value with the control item and writing the information in the specified position of the operator accumulation module in the right shift truncation reduction order, merging the written link segments into link data in the order of the operator execution stack, and outputting the inference link file.

[0052] The segment boundary graph includes a bit width upper limit corresponding segment, a bit width lower limit corresponding segment, and a three-segment overflow segment corresponding boundary. The state node book includes an aligned segment value record, a stack state number record, and a node record arranged by operator position. The overflow control book includes a falling into critical segment number record, a right shift amount record, and a truncation value reduction ratio record. The operator correction set includes a right shift truncation reduction transformation value, a value arranged by operator stack sequence number, and a correction structure record. The inference link file includes a right shift writing segment, a truncation writing segment, and a reduction writing segment.

[0053] The specific steps for obtaining the segment boundary graph are as follows:

[0054] Based on the operator hardware bit width configuration table, the bit width upper limit item and the bit width lower limit item in the configuration table are compared one by one, and the upper limit value and the lower limit value obtained by comparison are respectively corresponding to the operator value slot boundary. The corresponding records are arranged in order according to the table, and a bit width boundary cluster is generated;

[0055] Based on the bit width boundary cluster, the boundary records in the bit width boundary cluster are sequentially corresponding to the three-segment overflow segment, and the segment arrangement structure is established in the corresponding order. The segment arrangement structure is written into the segment relationship record table to generate the segment boundary graph;

[0056] Based on the operator hardware bit width configuration table, the bit width upper limit item value 127 and the bit width lower limit item value-128 of each row record in the configuration table are read in sequence with the row number as the order index. The upper limit item and the lower limit item are used as a pair of input parameters to compare the size relationship with the operator value slot boundary range-128 to 127. According to the comparison result, the upper boundary field and the lower boundary field are marked in the row record. The upper boundary field is written into the upper boundary position of the operator value slot boundary, and the lower boundary field is written into the lower boundary position of the operator value slot boundary. In the same row record, three index information of row number field, upper limit value field and lower limit value field are appended. All records are written into the boundary record table in order from small to large according to the row number. After completing the traversal of all row numbers and record writing, the bit width boundary cluster is generated;

[0057] Based on the bit width boundary cluster, the row number recorded in each boundary of the bit width boundary cluster is taken as the traversal sequence, the numerical range of the three overflow sections -128 to -64, -63 to 63, 64 to 127 is taken as the division parameter, the upper boundary field and the lower boundary field stored in each boundary record are respectively subjected to interval attribution judgment, when the boundary value is located in the range of -128 to -64, the overflow front section mark is marked in the record, when the boundary value is located in the range of -63 to 63, the safe section mark is marked, when the boundary value is located in the range of 64 to 127, the overflow rear section mark is marked, and the section number field is additionally recorded in the same record, the records with section marks and section numbers are sequentially arranged according to the row number to form a section arrangement structure, and the section arrangement structure is written into the section relationship record table according to the record order, and the section limit graph is generated after the writing of all boundary records is completed.

[0058] The specific steps for generating the state node book are as follows:

[0059] Based on the section limit graph, the section values in the section limit graph are compared with the intermediate results in the operator accumulation module one by one, the paired values after comparison are sorted according to the record order, and the sorting content is written into the temporary alignment table to generate an alignment node group;

[0060] Based on the alignment node group, the finite automaton is used to perform state positioning on the paired values, the paired values in the alignment node group are sequentially combined with the operator to form a stack state number group, the arranged nodes are arranged according to the combination generation order, and the arranged nodes are split according to the continuous relationship to form a segmentation structure, and a sorting and splitting cluster is generated;

[0061] Based on the sorting and splitting cluster, the segmented nodes in the sorting and splitting cluster are compared with the adjacent numbers one by one and the adjacency is judged, the nodes satisfying the adjacency relationship are reordered according to the operator position order, and the reordered structure is written into the node sequence to generate a state node book;

[0062] Based on the section limit graph, the boundary values in the records of the section limit graph are read one by one in order, each boundary value is combined with the corresponding value in the intermediate results of the operator accumulation module to form a paired value during the reading process, each boundary value is taken as the basis according to the specific boundary point recorded in the section limit graph, if the boundary point is less than -64, the section number 1 is marked, if the boundary point is between -20 and 20, the section number 2 is recorded, and if the boundary point is greater than 64, the section number 3 is recorded, the combined paired values are written into the temporary alignment table according to the sorting order of the record numbers from 1 to N, and the alignment node group is generated after all paired combinations are written;

[0063] Based on the alignment node group, the state positioning is performed on the pair of numerical values by using the finite automaton state transition method. When the finite automaton is input, the section number in the pair of numerical values is taken as the state input item, the intermediate result in the pair of numerical values is divided into three categories according to the size, wherein the input smaller than -80 is taken as the first category input, the input between -10 and 10 is taken as the second category input, and the input greater than 90 is taken as the third category input. Three different migration paths are set in the automaton, so that the state number is sequentially generated as numbers 4, 5 and 6 according to the input category. The state chain is formed by arranging all the numbers in the generation order. The state chain is split according to the continuous jump relationship between the numbers 4, 5 and 6 in the state chain. Each group of continuous number segments is combined as a node segment combination. The sorting split cluster is formed by arranging each segment combination.

[0064] Based on the sorting split cluster, the state numbers of each segment node in the sorting split cluster are taken out one by one. In the taking-out process, the difference between the current number and the adjacent number is taken as the judgment basis. If the difference is 1, it is recorded as a continuous node. If the difference is 2, it is recorded as an interval node. In order to maintain the record order, the original position of the operator in the accumulation link is taken as the positioning basis. The accumulation link position is rearranged according to the order from the first operator to the twentieth operator as the rearrangement reference. The continuous nodes are chain rearranged. The rearranged nodes are sequentially written into the node sequence table. After all the nodes are written, the state node book is generated.

[0065] The finite automaton takes the pair of numerical values recorded in the alignment node group as the input item. The section range corresponding to the pair of numerical values is read from the section boundary graph one by one. The current automaton state and the section range are taken as a group of joint input into the state transition table. The target state number is determined by matching the index position of the joint input in the state transition table. The target state number is written into the stack state number record corresponding to the pair of numerical values. The above reading, matching and number writing actions are sequentially performed according to the order of the pair of numerical values in the alignment node group, so as to form a continuous state sequence. The state sequence is then used as the state basis for the node arrangement and segment split of the sorting split cluster. Finally, the state record is completed according to the order of the operator position when the state node book is constructed.

[0066] The finite automaton is according to the formula:

[0067]

[0068] Wherein: is the target state number, is the current state number, is the section input code, is the state transition rule, which represents the basis next state number mapping found in the state transition table based on the input, is the section weight coefficient, is the section weight coefficient, For location weighting coefficients, For the operator accumulation link in the th The middle value of each position. intermediate value The absolute value, The operator execution stack position number indicates the number of the... Each pair of values ​​corresponds to the operator's index in the execution stack, and the constant... As a reference for normalizing the range of fixed-point numerical values, a constant As a normalization benchmark for operator execution stack length, the exponent The exponentiation of the square operation of the position normalization term, subscript Indexed by serial number;

[0069] Execution process: After the alignment node group is constructed, each pair of values ​​is assigned an index according to the order of appearance. Based on the segment boundary map, read the segment codes that match the paired values ​​and write them into the code. Simultaneously, intermediate values ​​at the same position are read from the operator accumulation link and written to... And write the position of the corresponding operator in the execution stack. Set in the initial stage of state localization =0, for the th When updating the state of a record, first check the state transition table. To find the next state number in the index, denote the found number as... Then, based on the current segment code and the sign of the intermediate value, it is determined In segment coding =1 and When less than 0, give Assigning a value of -2, in the segment encoding 2 and When it is between -20 and 20, give Assigning a value of 0 to the segment code. 3 and When greater than 0, give Assign a value of 2, then number according to the stack position. Sure ,when Give when the range is 1 to 10 When the value is assigned to 0, When the value is in the range of 11 to 20, give Assign a value of 1 to complete. , , , Calculation after setting the four parameters As a normalization factor for intermediate values, then calculate As a position-normalized squared term, , and three execution additions result in and are written into the state node book in the stack state number position corresponding to the first pair of numerical values, while is taken as the participate in subsequent calculations, and from 1 to the above-mentioned reading, parameter setting and state updating process is repeated in order, so that all intermediate numerical values in the entire link of the operator accumulation chain obtain the state number sequence associated with the section information, intermediate numerical value amplitude and stack position in the finite automaton.

[0070] The specific steps for establishing the overflow control book are as follows:

[0071] Based on the state node book, compare the state numbers with the critical sections of the overflow section projection diagram one by one, and merge the numbers falling into the number and right shift amount truncation value reduction ratio in turn, write the merged content in the temporary record according to the number order, and arrange the record according to the line, generate the critical joint cluster;

[0072] Based on the critical joint cluster, take out the joint records one by one according to the operator execution stack order and write the taken out content into the control record according to the link order, arrange the records according to the section and sort the arranged content according to the index set of the order column, and establish the overflow control book;

[0073] Based on the state node book, read the state numbers recorded in the state node book one by one and compare them with the critical sections in the overflow section projection diagram. In the comparison process, use the state number as the input number source, use the three critical values -100, 0, 100 extracted from the section boundary in the overflow section projection diagram as the critical reference points, perform the greater than, less than and equal to item-by-item judgment on the input number and the three critical reference points respectively, and in the judgment process, the numbers falling into the number less than -100 are classified into critical number group 1, the numbers between -25 and 25 are classified into critical number group 2, and the numbers greater than 80 are classified into critical number group 3, while the corresponding right shift amount, truncation value and reduction ratio three control parameters are attached to each number, wherein the right shift amount is recorded as 1, 2, 3 in order, the truncation value is recorded as 4, 5, 6 in order, and the reduction ratio is recorded as 10, 20, 30 in order, and all numbers and three parameters are written into the temporary record table according to the number from small to large, and the record order is consistent with the number order after arranging the temporary record table according to the line number order, and the critical joint cluster is generated after the arrangement is completed;

[0074] Based on the critical joint cluster, the joint records stored in the critical joint cluster are taken out one by one in the order of the operator execution stack from the first operator to the twentieth operator, and in the process of taking out, the row number of the record in the critical joint cluster is taken as the primary index, the order of the operator execution stack is taken as the link index, each record is written into the link position in the control record table, and in the process of writing, the right shift amount, the truncation value and the reduction ratio are sequentially stored in the three fixed fields of the link position. Then, the control record table is sorted according to the link number from 1 to 20, the continuous segment obtained each time is written into the index sequence in the order of the number, the index sequence is sorted from small to large, and the record set after sorting is written into the control record output table. After the record is written, the overflow control book is established.

[0075] The specific steps for constructing the operator correction set are as follows:

[0076] Based on the overflow control book, the right shift item, the truncation item and the reduction item in the overflow control book are extracted one by one to the intermediate value of the operator accumulation link. The corresponding values are classified and split into multiple items according to the right shift, truncation and reduction, and the split results are written into the temporary table to generate the control corresponding cluster.

[0077] Based on the control corresponding cluster, a graph structure is constructed with the operator in the operator accumulation link as the node and the right shift value, the truncation value and the reduction value in the control corresponding cluster as the edge attribute. The graph neural network is used to perform message passing on the node and the edge attribute to obtain the pruning weight, and the pruning weight is written into the corresponding control item. The right shift value, the truncation value and the reduction value in the control corresponding cluster are taken out, the right shift, the truncation and the reduction are performed according to the control item type, and the transformation result is adjusted and transformed with the pruning weight. The transformation output is written into the index record according to the operator stack number to generate the transformation index group.

[0078] Based on the transformation index group, the values of the transformation index group records are traced back one by one according to the operator execution stack number. The values taken out by backtracking are combined into a continuous correction link in the order of the level, and the correction values in the link are integrated according to the corresponding pruning weight proportion. The correction link is written into the correction record table to generate the operator correction set.

[0079] Based on the overflow control book, the right shift item, the truncation item and the reduction item recorded in the overflow control book are extracted in turn to the intermediate value of the operator accumulation link according to the row number from 1 to 20. In the extraction process, the fixed values 1, 2 and 3 of the right shift item are taken as the right shift parameter group, the fixed values 4, 8 and 12 of the truncation item are taken as the truncation parameter group, and the fixed values 5, 10 and 15 of the reduction item are taken as the reduction parameter group. The three parameters corresponding to each row are matched with the intermediate values of the same sequence number in the operator accumulation link in parallel, and in the matching action, the input value is divided into three types of structures according to the right shift, truncation and reduction categories. The three types of structures obtained by the division are written into the temporary table according to the original row sequence number, and the row sequence number field, the parameter source field and the category flag field are added to each record during writing, so that the records form a continuously numbered storage collection, and the control corresponding cluster is generated after the completion of all the division actions;

[0080] Based on the control corresponding cluster, the graph neural network message passing method is adopted. In the construction of the graph structure, each operator in the operator accumulation link is taken as the node number 1 to 20, and the right shift value, the truncation value and the reduction value recorded in the control corresponding cluster are taken as the edge attribute. The edge attribute is constructed as a directed structure graph in the order from node 1 to node 20. In the message passing action of the graph neural network, the node number is taken as the input source, and the three types of parameters in the edge attribute are taken as the attributes to be processed. The parameter values are input into the message reading unit in order, and in the message reading unit, the parameter values are divided into reading batches according to the node number parity. The nodes with even numbers are processed according to the reading order of parameter groups 1, 4 and 5, and the nodes with odd numbers are processed according to the reading order of parameter groups 2, 8 and 10. In the processing batch, the clipping weights 1, 2 and 3 are generated according to the node number difference. The clipping weights are written into the corresponding control item and the right shift value, the truncation value and the reduction value are taken out from the control corresponding cluster. The three types of parameters are right shifted, truncated and reduced according to the clipping weights. The transformed output is written into the index record in order according to the operator stack sequence number from 1 to 20. The transformed index group is generated after all the records are written;

[0081] Based on the transformed index group, the values recorded in the transformed index group are read in order according to the stack sequence number of the operator from 20 back to 1, and in the reading process, the sequence number difference between the current level and the upper level is taken as the backtracking depth. The backtracking output values are combined into a continuous correction link in order of levels, and in the correction link, each level record is appended with the corresponding clipping weight, wherein the clipping weight is sequentially taken as 1, 2 and 3 according to the order of the records in the transformed index group. The weight and the record value are integrated in a one-to-one relationship, and the integrated correction record is written into the sequence number 1 to 20 position in the correction record table. The operator correction set is generated after the writing of the correction link of each level is completed.

[0082] The graph neural network uses operators in the operator accumulation link as node identifiers. It writes the right shift value, truncation value, and reduction value recorded in the corresponding cluster into the edge attribute table according to the connection order between operators. Then, using node features and edge attributes as initial inputs, it sequentially performs message reading, message combination, and node updating between adjacent operator nodes. The updated node value is written into the node temporary record table. After multiple rounds of message passing, the updated value is read from the node temporary record table and written into the pruning weight record table according to the operator sequence number, providing pruning weight data corresponding to the operator sequence number for subsequent steps.

[0083] Graph neural networks, according to the formula:

[0084]

[0085] in: This is the cutting factor. For node indexing, For the set of adjacent nodes, This represents the number of adjacent nodes. For adjacent node indexes, The edge connection weights, For node feature values, The weighting coefficient for the right shift term. The weighting coefficient for the truncation term. For the weighting coefficient of the reduction term, The positional weight coefficients are... This is the right shift value. This is the truncation value. For the reduced value, The position number is used, and the constant 20 is the normalization benchmark for the execution stack length;

[0086] Execution process: Each node in the operator accumulation chain is written to the index in the order of the execution stack. Reading nodes in a graph structure Adjacent sets And calculate the number of adjacencies. Then index the adjacent nodes. Read the node feature value obtained from the previous round of message passing Read the corresponding right shift value from the edge attributes. Cutoff value Reduction value Simultaneously read the edge connection weights Take out from the weight parameter table , , , The four weighting coefficients are for each Calculate linear combination terms And complete all the sum is divided by to get the normalized aggregate value, while the node position number is divided by 20 and multiplied by to get the position correction amount, and finally the position correction amount is added to the normalized aggregate value to generate the clipping coefficient , and is written into the clipping weight record table of the corresponding node, so that the operator node obtains an independent clipping coefficient for right shift, truncation and reduction correction.

[0087] The specific steps of outputting the inference link file are as follows:

[0088] Based on the overflow control book and the operator correction set, the correction value and the right shift item, the truncation item and the reduction item are compared in turn, and the write segment data is generated according to the corresponding order and recorded in the link write table, and the write link group is generated;

[0089] Based on the write link group, the write segment data is read and the operator accumulation module position corresponding to each segment is locked according to the operator sequence number recorded in the write link group, and after the segment content is written into the locked position, the write positions are merged in sequence to establish the inference link file;

[0090] Based on the overflow control book and the operator correction set, the correction value is compared with the right shift item, the truncation item and the reduction item recorded in the overflow control book, and in the comparison process, the fixed values 1, 2 and 3 of the right shift item are taken as the right shift input, the fixed values 4, 6 and 8 of the truncation item are taken as the truncation input, and the fixed values 10, 20 and 30 of the reduction item are taken as the reduction input. Each input is attached with a sequential identifier with the number 1 to 30 in the comparison action, and the five-tuple record obtained by comparison is assembled into write segment data according to the arrangement order of right shift, truncation and reduction. Each segment data contains a number field, a right shift field, a truncation field, a reduction field and a correction value field. The write segment data is written into the link write table according to the number from 1 to 30, so that the records in the table form a continuous structure according to the row number, and the write link group is generated after the write action is completed;

[0091] Based on the write link group, the write fragment data stored in the write link group is read item by item in the order of the operator sequence number from the first operator to the 20th operator, and in the reading process, the number field recorded in the write link group is used as the fragment index, each fragment is locked one by one with the position in the operator accumulation module, and in the locking action, the operator sequence number is used as the position reference, the fragment content is written into the corresponding storage interval in the operator accumulation module, and after each write, the row number of the write position is recorded, from row number 1 to row number 20, the write position performs merging according to the recording order, each fragment is arranged in a link sequence according to the recording order, and all fragments in the link sequence are continuously spliced according to the sequence number, after splicing, the structure is written into the reasoning link storage table, and after the splicing record is written, the reasoning link file is established.

[0092] Please refer to Figure 2 , an AI processor operator overflow optimization system, the system comprises:

[0093] The segment construction module: based on the operator hardware bit width configuration table, read and arrange the bit width upper limit item, the bit width lower limit item and the numerical slot boundary, establish the bit width limit corresponding structure, and after completing the corresponding arrangement, obtain the segment limit graph;

[0094] The node arrangement module: based on the segment limit graph, the segment value is executed with the intermediate result, the number is generated and the node is split, the node structure is associated with scanning and position arrangement, the rearrangement is completed according to the operator position, and the state node book is generated;

[0095] The control integration module: based on the state node book, the state number is compared with the critical segment, the corresponding right shift amount, the truncation value and the reduction ratio are combined, and the sequence arrangement and record regularization are completed according to the operator execution stack sequence, so as to establish the overflow control book;

[0096] The correction generation module: based on the overflow control book, the right shift item, the truncation item and the reduction item are corresponded to the intermediate value, the weighted transformation result is generated by introducing the graph neural network, and the backtracking reorganization is executed according to the operator stack sequence number, forming the operator correction set for compensating overflow;

[0097] The link output module: based on the overflow control book and the operator correction set, the modified numerical value is compared with the multiple control contents to form a write fragment and write the specified position information, the fragment merging is completed according to the operator execution stack sequence, and the reasoning link file is output.

[0098] The above is only the preferred embodiment of the present application, and does not limit the present application in other forms. Any skilled person in the art can modify or change the above disclosed technical content to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made on the basis of the technical essence of the present application to the above embodiments still belongs to the protection scope of the technical solution of the present application.

Claims

1. An AI processor operator overflow optimization method, characterized in that, Includes the following steps: S1: By comparing the upper and lower limits of the bit width in the operator hardware bit width configuration table item by item and reading the operator value slot boundary, the read content is arranged with the three overflow segments to obtain the segment boundary map. S2: Based on the segment boundary map, align the segment values ​​with the intermediate results of the operator accumulation module, combine and sort the alignment results with the stack state numbers of the operators using a finite automaton, split and adjacency judgment of the sorted chain state nodes, organize the nodes and rearrange them according to the operator positions to generate a state node book. S3: Based on the state node book, compare the state number with the critical segment in the overflow segment projection diagram, merge the number of falling into the critical segment with the reduction ratio of the right shift cutoff value of the operator shift stack, arrange them as control records according to the operator execution stack order, and establish an overflow control book. S4: Based on the overflow control book, the intermediate values ​​of the corresponding right-shift truncation and reduction control item and the operator accumulation link are transformed by the weighted truncation and reduction transformation of the corresponding numerical graph neural network. The transformed intermediate values ​​are stacked backtracked and organized into a correction structure according to the operator, forming the operator correction set. S5: Based on the overflow control book and the operator correction set, compare the correction value with the control item and write it into the specified position information of the operator accumulation module in the right shift truncation reduction order. Merge the written link segments into link data in the order of operator execution stack and output the inference link file.

2. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The segment boundary diagram includes segments corresponding to the upper limit of bit width, segments corresponding to the lower limit of bit width, and three overflow segments. The state node book includes aligned segment value records, stack state number records, and node records arranged by operator position. The overflow control book includes critical segment number records, right shift records, and truncation value reduction ratio records. The operator correction set includes right shift, truncation, and reduction transformation values, backtracking and organizing values ​​according to operator stack sequence numbers, and correction structure records. The inference link file includes right shift write segments, truncation write segments, and reduction write segments.

3. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The specific steps to obtain the segment boundary map are as follows: Based on the operator hardware bit width configuration table, the upper and lower bit width entries in the configuration table are compared one by one, and the upper and lower values ​​are matched with the operator numerical slot boundaries respectively. The corresponding records are organized in the order of the table to generate bit width boundary clusters. Based on the bit-width boundary cluster, the boundary records and three overflow segments in the bit-width boundary cluster are sequentially matched and a segment arrangement structure is established in the corresponding order. The segment arrangement structure is written into the segment relationship record table to generate a segment boundary map.

4. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The specific steps for generating the state node book are as follows: Based on the segment boundary map, the segment values ​​in the segment boundary map are compared with the intermediate results of the operator accumulation module one by one. The paired values ​​after comparison are sorted according to the record order and the sorted content is written into a temporary alignment table to generate an alignment node group. Based on the alignment node group, a finite automaton is used to perform state positioning on paired values. The paired values ​​and operators in the alignment node group are sequentially combined with stack state numbers. The nodes are arranged according to the combination generation order and the arranged nodes are split into segmented structures according to the continuous relationship to generate sorted split clusters. Based on the sorted split cluster, each segment node in the sorted split cluster is compared with its adjacent number and an adjacency judgment is performed. The nodes that satisfy the adjacency relationship are rearranged according to the operator position order and the rearranged structure is written into the node sequence to generate a state node book.

5. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The finite automaton uses the paired values ​​recorded in the alignment node group as input. It reads the segment range corresponding to the paired values ​​one by one from the segment boundary diagram. The current automaton state and the segment range are entered into the state transition table as a joint input. The target state number is determined by matching the index position of the joint input in the state transition table. The target state number is written into the stack state number record corresponding to the paired values. The above reading, matching and number writing operations are performed in sequence according to the order of the paired values ​​in the alignment node group to form a continuous state sequence. The state sequence is then used to sort the node arrangement of the split cluster and the state basis when splitting the segment. The final state record is completed in the order of the operator position when constructing the state node book.

6. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The specific steps for establishing the overflow control book are as follows: Based on the state node book, each state number is compared with the critical segment of the overflow segment projection diagram, and the falling number and the reduction ratio of the right shift cutoff value are merged in turn. The merged content is written into a temporary record in the order of the numbers and the records are organized by row to generate a critical joint cluster. Based on the critical joint cluster, the joint records are retrieved one by one in the order of operator execution stack and the retrieved content is written into the control record in the order of link. The records are then organized into segments and the organized content is arranged into a sequence set by index to establish an overflow control book.

7. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The specific steps for constructing the operator correction set are as follows: Based on the overflow control book, the right shift, truncation, and reduction items in the overflow control book are extracted one by one to the intermediate value of the operator accumulation link. The corresponding values ​​are split into multiple structures according to the right shift, truncation, and reduction categories, and the splitting results are written into a temporary table to generate the corresponding control cluster. Based on the control corresponding cluster, a graph structure is constructed with operators in the operator accumulation link as nodes and right shift truncation value reduction value in the control corresponding cluster as edge attributes. A graph neural network is used to perform message passing on the nodes and edge attributes to obtain pruning weights. After writing the pruning weights into the corresponding control items, the right shift truncation value reduction value in the control corresponding cluster is retrieved. Right shift truncation and reduction are performed according to the control item type, and the transformation result is adjusted with pruning weights. The transformation output is written into the index record according to the operator stack sequence number to generate a transformation index group. Based on the transformation index group, the values ​​recorded in the transformation index group are backtracked level by level according to the operator execution stack number. The backtracked values ​​are combined into a continuous correction link in hierarchical order. The correction values ​​of each level in the link are weighted and integrated according to the corresponding pruning weight ratio. The correction link is written into the correction record table to generate the operator correction set.

8. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The graph neural network uses operators in the operator accumulation link as node identifiers. It writes the right shift value, truncation value, and reduction value recorded in the corresponding cluster into the edge attribute table according to the connection order between operators. Then, using node features and edge attributes as initial inputs, it sequentially performs message reading, message combination, and node updating between adjacent operator nodes. The updated node value is written into the node temporary record table. After multiple rounds of message passing, the updated value is read from the node temporary record table and written into the pruning weight record table according to the operator sequence number, providing pruning weight data corresponding to the operator sequence number for subsequent steps.

9. The AI ​​processor operator overflow optimization method according to claim 1, characterized in that, The specific steps for outputting the inference link file are as follows: Based on the overflow control book and the operator correction set, the correction values ​​are compared with the right shift, truncation, and reduction terms one by one, and the write segment data is generated and recorded in the link write table in the corresponding order to generate the write link group. Based on the write link group, the write segment data is read and locked according to the operator sequence number recorded in the write link group, and the position of the operator accumulation module corresponding to each segment is locked one by one. After the segment content is written to the locked position, the write positions are merged in sequence to establish the inference link file.

10. An AI processor operator overflow optimization system, characterized in that, The AI ​​processor operator overflow optimization method according to any one of claims 1-9, wherein the system comprises: Segment construction module: Based on the operator hardware bit width configuration table, read and arrange the upper bit width limit item, lower bit width limit item and numerical slot boundary, establish the bit width limit corresponding structure, and obtain the segment limit map after completing the corresponding arrangement; Node arrangement module: Based on the segment boundary map, the segment values ​​and intermediate results are paired, numbered and split, and nodes are split. The node structure is scanned and its position is arranged. The nodes are rearranged according to the operator position to generate a state node book. Control integration module: Based on the state node book, it compares the state number with the critical section, combines the corresponding right shift amount, truncation value and reduction ratio, and completes the sequence sorting and record regularization according to the operator execution stack order, thereby establishing an overflow control book; Correction generation module: Based on the overflow control book, the right shift term, truncation term and reduction term are mapped to the intermediate value, a graph neural network is introduced to generate a weighted transformation result and backtracking is performed according to the operator stack number to form an operator correction set for compensating for overflow; Link output module: Based on the overflow control book and the operator correction set, the correction value is compared with multiple control contents to form a write segment and written to the specified position information. The segments are merged according to the operator execution stack order, and the inference link file is output.

Images