Optimization method for beam search for completely independent break points

Abstract

The invention provides an optimization method for beam search for completely independent break points, which comprises the following steps of: S1, judging and finding out the completely independent break points: solving the independent break points in a beam search process only by judging whether the maximum numbered names of each beam scheme are consistent or not, if so, judging whether unconsumed ancestor nodes exist or not, and if not, judging whether unconsumed ancestor nodes exist or not; if the ancestor nodes are all consumed, the ancestor nodes are completely independent segmentation points; s2, according to the completely independent segmentation points, the whole network diagram is segmented into independent sub-diagrams, the previous independent sub-diagram has N schemes, the current N schemes do not affect sequencing at the next moment, so that the N schemes can select an optimal scheme, and numbers from the current moment 1 to the moment t are recorded; independently analyzing the (t + 1) th moment and the subsequent serial numbers, also respectively processing each independent sub-graph, and respectively entering beam search for sorting each independent sub-graph; and S3, finally, all the independent sub-graph schemes are selected to be despliced with the minimum memory peak value, and sorting information of the whole network graph is formed.

Description

Technical Field

[0001] The invention belongs to the field of computer technology, and in particular relates to an optimization method for beam search of completely independent segmentation points. Background Art

[0002] With the development of the Internet, artificial intelligence (AI) has developed rapidly, and AI models have been applied to various products and services. Generally, AI models are constructed using deep neural networks or convolutional neural networks, and then the constructed models are trained using training samples to obtain AI models that can be used for reasoning in application scenarios.

[0003] The network graph will have multiple operators and multiple edges. Each edge will be associated with two operators. The edge is also used to represent the input / output relationship between the two associated operators. During the AI ​​reasoning process, the terminal side needs to calculate one by one according to the execution order of each operator in the network. Each operator has input data and output data. For example, the convolution operator performs convolution calculation on the input data and then outputs it. Therefore, at least one block of memory is required for each operator to store the corresponding output data, and this output data is used as the input data of the next operator associated with it. In actual applications, some special operators such as GRU and LSTM will have multiple output tensors. If the memory of these operators is not managed, the executing operator cannot know where to schedule the input data, nor can it determine the location where the output data should be stored in the memory, resulting in data confusion and inability to complete reasoning.

[0004] If a piece of memory is allocated to each operator in the computation graph, it will cause a waste of memory resources. Because the operators in the network are executed in sequence, when the subsequent operator is executed, the memory of the previous operator in the previous execution order may have been cleared. In this way, the subsequent operator can use all or part of the memory of the previous operator. When pre-allocating memory, multiple operators with non-overlapping life cycles can share memory. The industry has explored a variety of solutions to reduce the actual deployment memory peak of the computation graph.

[0005] However, in all current solutions, in practical applications, it is assumed that a certain sorting order has been determined, and how to pre-allocate memory. In theoretical research, since the solution space of NP-hard problems is extremely large, using learning and training AI algorithms and supervised training to study sorting will consume a lot of resources and may fall into local optimality.

[0006] Technical problems existing in existing technologies:

[0007] Neural networks have independent partitioning points, also called dominating points. The network can be divided into multiple independent subgraphs that do not affect each other by independent partitioning points. Therefore, the usual practice is to find independent partitioning points before entering the beam to divide the entire network graph into independent subgraphs, and each independent subgraph enters the beam search for sorting. Finally, the sorting of each subgraph is spliced ​​together.

[0008] However, solving the independent split point itself is a difficult algorithm. There are many ways to solve the independent split point of the directed acyclic graph in the industry, but the complexity of each method is also relatively large. In a naive way, in order to find the independent split point, it is necessary to traverse each node in the network, cut out the ancestor node set of the point, find out the descendant node set of the point, and then determine whether the network has other nodes, that is, its edge nodes. If there is a corresponding edge node, it means that the point is not an independent split point.

[0009] Solving independent split points is also time-consuming. To determine whether each node is an independent split point, you need to continue traversing the N nodes in the entire graph, or if part of the subgraph has been split, you need to traverse all the M nodes in the remaining subgraph.

[0010] Commonly used terms in the prior art include:

[0011] 1) Node, operator, operator: In the embodiments of the present application, the node in the neural network can also be called an operator, operator, etc., which is the smallest unit of the network structure. In the embodiments of the present application, the operator is used as an example for description.

[0012] 2)Tensor:

[0013] The output data results of the operator must be saved in a tensor. An operator is generally an output result, and only one tensor is needed to save the output result. In practical applications, an operator may output multiple tensors, such as LSTM operators and GRU operators. The life cycles of various tensors may be different, as stated in WO2022105187A1 "A method, device and system for memory management", "Each node requires at least one piece of memory to store the corresponding output data, and this output data serves as the input data of the next node associated with it." This application only discusses general cases.

[0014] 3) Computational graphs, AI models, neural networks, deep learning networks, directed acyclic graphs: all belong to the same concept, and are directed acyclic graph structures composed of multiple nodes.

[0015] 4) Operator memory:

[0016] When an operator has one output tensor, the memory occupied by the tensor is the memory of the operator.

[0017] When an operator has multiple output tensors, the sum of the memory occupied by the multiple tensors is the memory of the operator.

[0018] This application only discusses one operator with one output tensor.

[0019] 5) Network memory at time t:

[0020] The execution order of the network has calculated the memory required for the operator executed at time t:

[0021] First: input tensor memory,

[0022] Second: the memory of this operator,

[0023] Third: The memory of the tensors in the network that have not been consumed.

[0024] 6) The maximum memory peak of the network, the first memory peak of the network, the second memory peak of the network, the third memory peak of the network, and the tth memory peak:

[0025] The execution order of the network has been calculated at time t;

[0026] Calculate the network memory at the 1st moment, the 2nd moment, the 3rd moment, ..., the tth moment, and sort them in descending order, so as to know the first memory peak value of the network, the second memory peak value of the network, ..., the tth memory peak value of the network.

[0027] 7) Scheduling, sorting, and execution order:

[0028] Common network sorting schemes are depth-first sorting and breadth-first sorting, but the actual number of network sorting schemes is very large. When calculating offline on the PC side, it is even impossible to store all the scheduling schemes.

[0029] 8) Completely independent split points:

[0030] The entire network is cut into two network diagrams by this point, and the upper and lower diagrams are completely unrelated. In other words, if this point is removed, the entire network diagram is disconnected. Figure 1 , Figure 2 As shown, the Figure 1 The completely independent split points in are 1, 6, 7, and 10. Figure 2 The completely independent split points in are 1 and 10. Summary of the invention

[0031] In order to solve the above problems, the purpose of this application is to:

[0032] This method is a heuristic algorithm, which belongs to the traditional algorithm. Different from the AI ​​learning and training method, it does not require additional data for training, does not require the use of a large amount of training data, and uses the beam search global search solution to ensure the global optimal solution as much as possible and find the scheduling solution with the smallest memory peak. Among current researchers, it is the only and best method in the heuristic algorithm when the offline server space is limited.

[0033] The independent split point in this scheme can be solved in the beam search process. It only needs to determine whether the maximum numbered name of each beam scheme is consistent. If it is consistent, then determine whether there is an unconsumed ancestor node. If it is consumed, it means that the node with the largest code is a completely independent split point. Sometimes it may be possible to determine that it is not an independent split point by judging the node with the largest number of the first two schemes, and then exit the judgment immediately to save time. According to the independent split point, the previous independent subgraph has N schemes, and the current N schemes do not affect the sorting at the next moment, so the N schemes can select the best one and record the numbers from the current 1 to t. Next, independently analyze the numbers at the t+1 moment and thereafter. Each independent subgraph can also be processed separately.

[0034] Specifically, the present invention provides an optimization method for beam search of completely independent segmentation points, the method comprising:

[0035] S1, determine and find completely independent segmentation points:

[0036] The solution of the independent split point in the beam search process only needs to determine whether the maximum numbered name of each beam solution is consistent. If they are consistent, then determine whether there are unconsumed ancestor nodes. If all ancestor nodes are consumed, it means that the point is a completely independent split point.

[0037] S2, according to the completely independent split points, the entire network graph is divided into independent subgraphs. The previous independent subgraph has N solutions, and the current N solutions do not affect the sorting at the next moment, so the N solutions can select the best one and record the numbers from the current 1 to t moment; next, the numbers at the t+1th moment and later are analyzed independently, and each independent subgraph can also be processed separately, and each independent subgraph enters the beam search for sorting;

[0038] S3, finally, the sorting information of each subgraph is spliced ​​together to form the sorting information of the entire network graph.

[0039] The method further comprises step S0,

[0040] S0, before entering sorting, records the name of each node and the memory value of the node's output tensor into the name_to_memory dictionary.

[0041] In the step S1, when it is determined that there are two nodes, it can be determined that they are not independent segmentation points, and the determination is immediately exited.

[0042] The step S1 further comprises: assuming there is a node AJ,

[0043] S1.1, T = 1, assuming A is numbered 1,

[0044] The current beam scheme has only one numbered node, the maximum numbered name is the same, and there are no unconsumed descendant nodes, then A is a completely independent split point,

[0045] The ordering before and after the completely independent split point does not affect each other. The completely independent split point itself needs to participate in the calculation of the memory peak value at the next moment. There is no node before node A, so it does not need to be split;

[0046] S1.2, at time T = 2, the node A is numbered 1, then the node numbered 2 should be selected. Assume that the optional nodes numbered 2 are B and D, and the beam width is 2.

[0047] {A=1,B=2,{C,D,E,F,G,H,I,J}},

[0048] {A=1,D=2,{B,C,E,F,G,H,I,J}};

[0049] The maximum numbered names B and D of the two beam schemes are inconsistent, so there is no independent split point at t = 2;

[0050] S1.3, at T = 3, assume that the next number of the solution {A = 1, B = 2} can be selected as C and D; the next number of the solution {A = 1, D = 2} can be selected as B and E; therefore, at T = 3, the beam width is 4, and there are 4 solutions:

[0051] {A=1,B=2,C=3,{D,E,F,G,H,I,J}},

[0052] {A=1,B=2,D=3,{C,E,F,G,H,I,J}},

[0053] {A=1,D=2,B=3,{C,E,F,G,H,I,J}},

[0054] {A=1,D=2,E=3,{B,E,F,G,H,I,J}};

[0055] Among them, the names of the largest numbered nodes in the first and second schemes are C and D respectively, and the names are inconsistent, so there is no independent split point;

[0056] And so on.

[0057] At S1,4, T=4, the beam scheme has

[0058] {A=1, B=2, C=3, D=4, {E, F, G, H, I, J}},

[0059] {A=1,B=2,D=3,C=4,{E,F,G,H,I,J}},

[0060] {A=1,B=2,D=3,E=4,{C,F,G,H,I,J}},

[0061] {A=1,D=2,B=3,C=4,{E,F,G,H,I,J}},

[0062] {A=1,D=2,B=3,E=4,{C,F,G,H,I,J}},

[0063] {A=1,D=2,E=3,B=4,{C,F,G,H,I,J}};

[0064] Among them, the names of the largest numbered nodes in the first and second schemes are D and C respectively. The names are inconsistent, so there is no independent split point;

[0065] S1.5, T = 5, the beam solution has

[0066] {A=1,B=2,C=3,D=4,E=5,{F,G,H,I,J}},

[0067] {A=1,B=2,D=3,C=4,E=5,{F,G,H,I,J}},

[0068] {A=1,B=2,D=3,E=4,C=5,{F,G,H,I,J}},

[0069] {A=1,D=2,B=3,C=4,E=5,{F,G,H,I,J}},

[0070] {A=1,D=2,B=3,E=4,C=5,{F,G,H,I,J}},

[0071] {A=1,D=2,E=3,B=4,C=5,{F,G,H,I,J}};

[0072] Among them, the names of the largest numbered nodes in the first, second, and third schemes are E, E, and C respectively. The names are inconsistent, so there is no independent split point;

[0073] S1.6, T = 6, the beam solution has

[0074] {A=1,B=2,C=3,D=4,E=5,F=6,{G,H,I,J}},

[0075] {A=1,B=2,D=3,C=4,E=5,F=6,{G,H,I,J}},

[0076] {A=1,B=2,D=3,E=4,C=5,F=6,{G,H,I,J}},

[0077] {A=1,D=2,B=3,C=4,E=5,F=6,{G,H,I,J}},

[0078] {A=1,D=2,B=3,E=4,C=5,F=6,{G,H,I,J}},

[0079] {A=1,D=2,E=3,B=4,C=5,F=6,{G,H,I,J}};

[0080] The name of the maximum numbered node in all schemes is F, which is consistent, and there is no ancestor node that has not consumed all the memory, so there is an independent split point F.

[0081] The step S2 further comprises:

[0082] S2.1, so at the current time T = 6, there are 5 nodes in total {A, B, C, D, E, F}, judge the first memory peak to the sixth memory peak, and select the best solution. Assume that the best number selected is {A = 1, B = 2, D = 3, C = 4, E = 5, F = 6}, determine the number of ABDCEF, and then re-judge the order of the following nodes. At this time, the whole graph is divided into numbered {ABCDE} and the remaining unnumbered subgraphs {F = 6, {GHIJ}}. Cut out the sorted and numbered nodes, reduce the remaining network length, participate in the subsequent sorting calculation to save memory, and retain F = 6 to participate in the memory peak calculation at the next moment.

[0083] The step S1 further comprises:

[0084] S1.7, at time T = 7, the remaining subgraphs can be recalculated from a width of 0, rather than continuing from the 6 solutions at time T = 6;

[0085] At this time, the number of the numbered node F is 6, so the next step is to determine the node numbered 7; the node numbered 7 can currently be selected as G; at this time, the beam width is 1:

[0086] {F=6,G=7,{H,I,J}}

[0087] Similarly,

[0088] S1.8, beam solution at T=8:

[0089] {F=6,G=7,H=8,{I,J}},

[0090] {F=6,G=7,I=8,{H,J}};

[0091] S1.9, beam solution at T=9:

[0092] {F=6,G=7,H=8,I=9,{J}},

[0093] {F=6,G=7,I=8,H=9,{J}};

[0094] S1.10, beam solution at T=10:

[0095] {F=6,G=7,H=8,I=9,J=10},

[0096] {F=6,G=7,I=8,J=9,J=10};

[0097] The step S2 further comprises:

[0098] S2.2, finally, the solution with the smallest memory peak value is selected from the two solutions with T=10 as the final solution, assuming that {F=6, G=7, H=8, I=9, J=10}.

[0099] The step S3 further comprises:

[0100] {F=6, G=7, H=8, I=9, J=10} is then concatenated with the previously stored {A=1, B=2, D=3, C=4, E=5, F=6} to form the sorting information of the entire network graph {A=1, B=2, D=3, C=4, E=5, F=6, G=7, H=8, I=9, J=10}.

[0101] In the step S1, in actual implementation, it is necessary to record the current numbering information to facilitate obtaining the number at the next moment. The last numbered node can be retained to store the current numbering information, or other means can be used to record, including implementing the use of a global dictionary to store the memory information of all nodes.

[0102] Therefore, the advantage of the present application is that, since the width of the beam search is limited, each time an independent segmentation point is reached, one solution is selected from N solutions, which reduces the difficulty. This is the optimization method of the present application for beam search, and it is more time-efficient than splitting the independent subgraphs before entering the beam search. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0104] Figure 1 It is a schematic diagram of the existence of independent segmentation points in the figure.

[0105] Figure 2 It is a schematic diagram of the existence of non-independent segmentation points in the figure.

[0106] Figure 3 It is a diagram used in the method for determining whether it is an independent segmentation point in the embodiment of the method.

[0107] Figure 4 yes Figure 3 The numbering of node A does not affect the numbering at the next moment, thus forming the decomposed diagram.

[0108] Figure 5 In the embodiment of the method, the node F that has been numbered is obtained as 6, and then the diagram of the node numbered 7 needs to be determined next.

[0109] Figure 6 is a schematic diagram of the process of this method. DETAILED DESCRIPTION

[0110] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0111] like Figure 1 As shown, in general, the method of first determining the independent segmentation points is not recommended by this application method:

[0112] First: determine whether 1 is an independent segmentation point:

[0113] 1's ancestor node: empty

[0114] 1's descendant nodes: 2, 3, 4, 5, 6, 7, 8, 9, 10

[0115] Edge node of 1: The node set of the entire network graph {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} minus 1 itself, minus ancestor nodes, minus descendant nodes, the remainder is empty.

[0116] Therefore, 1 is a split point. Then determine whether it is an independent split point, and determine whether the ancestor node of 1 has output to the descendant node. 1 has no ancestor node, so there is no ancestor node that outputs to the descendant node, so 1 is an independent split point.

[0117] Second: Determine whether 2 is an independent segmentation point:

[0118] 2's ancestor node: 1

[0119] 2's descendant nodes: 4, 6, 7, 8, 9, 10

[0120] 2's edge nodes: 3, 5.

[0121] 2 has an edge node, so 2 is not an independent split point.

[0122] Third: Determine whether 6 is an independent segmentation point.

[0123] 6's ancestor nodes: 1, 2, 3, 4, 5.

[0124] 6's descendant nodes: 7, 8, 9, 10.

[0125] 6 edge source node: empty

[0126] 6 has no edge nodes, which means 6 is a split point. Then determine whether it is an independent split point and whether the ancestor node of 6 has outputs to descendant nodes. The output nodes of ancestor nodes 12, 3, 4, and 5 of 6 are inside the ancestor node set, so 6 is an independent split point.

[0127] Similarly, nodes 7 and 10 are also independent split points.

[0128] Note that Figure 2 As shown, 6, 7, and 10 are not independent split points. Figure 6 ,7,8,9,10, we still need the information of 4 nodes. Therefore, we cannot divide the entire network graph into two subgraphs: 1, 2, 3, 4, 5, 6 and 7, 8, 9, 10.

[0129] From this we can see that solving independent split points itself is also time-consuming. To determine whether each node is an independent split point, it is necessary to continue traversing the N nodes of the entire graph, or if part of the subgraph has been split, the M nodes in the remaining subgraph must be traversed.

[0130] The embodiment of the present application relates to an optimization method for beam search of completely independent split points. In this scheme, independent split points can be solved during the beam search process. It is only necessary to determine whether the maximum numbered names of each beam scheme are consistent. Sometimes, judging two nodes can determine that they are not independent split points, and then immediately exit the judgment, saving time.

[0131] According to the independent split point, the previous independent subgraph has N solutions, and the current N solutions do not affect the order of the next moment, so the N solutions can select the best one and record the numbers from the current 1 to t moment; next, independently analyze the numbers of the t+1 moment and later. Each independent subgraph can also be processed separately.

[0132] Since the width of beam search is limited, every time an independent split point is reached, one solution is selected from N solutions, which reduces the difficulty. This is the optimization method of the patented method for beam search, and it is more time-efficient than splitting independent subgraphs before entering beam search.

[0133] Specifically, in beam search, the method for determining whether it is an independent split point is to determine whether the maximum numbered name of each beam solution is consistent, and to determine whether there are any unconsumed ancestor nodes.

[0134] like Figure 6 As shown, the process steps of this method are as follows:

[0135] S1, determine and find completely independent segmentation points;

[0136] The solution of the independent split point in the beam search process only needs to determine whether the maximum numbered name of each beam solution is consistent. If they are consistent, then determine whether there is an ancestor node that has not consumed all the memory. If all are consumed, it means that the point is a completely independent split point.

[0137] S2, according to the completely independent split points, the entire network graph is divided into independent subgraphs. The previous independent subgraph has N solutions, and the current N solutions do not affect the sorting at the next moment, so the N solutions can select the best one and record the numbers from the current 1 to t moment; next, the numbers at the t+1th moment and later are analyzed independently, and each independent subgraph can also be processed separately, and each independent subgraph enters the beam search for sorting;

[0138] S3, finally, the rankings of each subgraph are stitched together to form the ranking information of the entire network graph. Figure 3 As shown, assuming there is an AJ node, the step S1 further includes:

[0139] S1.1, T = 1, assuming A is numbered 1,

[0140] The current beam solution has only one numbered node, the maximum numbered name is the same, and there is no descendant node that does not consume memory, then A is an independent split point. The order of completely independent split points before and after does not affect each other. The completely independent split point itself needs to participate in the calculation of the memory peak value at the next moment. There is no node before node A, so it does not need to be split; so the whole graph is divided into Figure 4 As shown, for the convenience of description, in actual implementation, it is also necessary to retain the last numbered node, such as A-1, to store the current numbering information. Of course, other means can be used to record the current numbering information to facilitate the acquisition of the number at the next moment. The entire graph is divided into {A=1} and {A=1, B, C, D, E, F, G, H, I, J};

[0141] S1.2, at time T = 2, the node A is numbered 1, then the node numbered 2 should be selected. Assume that the optional nodes numbered 2 are B and D, and the beam width is 2.

[0142] {A=1,B=2,{C,D,E,F,G,H,I,J}},

[0143] {A=1,D=2,{B,C,E,F,G,H,I,J}};

[0144] The maximum numbered names B and D of the two beam schemes are inconsistent, so there is no independent split point at t = 2;

[0145] S1.3, T = 3, assuming {A = 1, B = 2, {C, D, E, F, G, H, I, J}},

[0146] The next number of the scheme can be C and D; the next number of the scheme {A=1,D=2,{B,C,E,F,G,H,I,J}} can be B and E; so at T=3, the beam width is 4, and there are 4 schemes:

[0147] {A=1,B=2,C=3,{D,E,F,G,H,I,J}},

[0148] {A=1,B=2,D=3,{C,E,F,G,H,I,J}},

[0149] {A=1,D=2,B=3,{C,E,F,G,H,I,J}},

[0150] {A=1,D=2,E=3,{B,E,F,G,H,I,J}};

[0151] Among them, the names of the largest numbered nodes in the first and second schemes are C and D respectively, and the names are inconsistent, so there is no independent split point;

[0152] And so on.

[0153] At S1,4, T=4, the beam scheme has

[0154] {A=1, B=2, C=3, D=4, {E, F, G, H, I, J}},

[0155] {A=1,B=2,D=3,C=4,{E,F,G,H,I,J}},

[0156] {A=1,B=2,D=3,E=4,{C,F,G,H,I,J}},

[0157] {A=1,D=2,B=3,C=4,{E,F,G,H,I,J}},

[0158] {A=1,D=2,B=3,E=4,{C,F,G,H,I,J}},

[0159] {A=1,D=2,E=3,B=4,{C,F,G,H,I,J}};

[0160] Among them, the names of the largest numbered nodes in the first and second schemes are D and C respectively. The names are inconsistent, so there is no independent split point;

[0161] S1.5, T = 5, the beam solution has

[0162] {A=1,B=2,C=3,D=4,E=5,{F,G,H,I,J}},

[0163] {A=1,B=2,D=3,C=4,E=5,{F,G,H,I,J}},

[0164] {A=1,B=2,D=3,E=4,C=5,{F,G,H,I,J}},

[0165] {A=1,D=2,B=3,C=4,E=5,{F,G,H,I,J}},

[0166] {A=1,D=2,B=3,E=4,C=5,{F,G,H,I,J}},

[0167] {A=1,D=2,E=3,B=4,C=5,{F,G,H,I,J}};

[0168] Among them, the names of the largest numbered nodes in the first, second, and third schemes are E, E, and C respectively. The names are inconsistent, so there is no independent split point;

[0169] S1.6, T = 6, the beam solution has

[0170] {A=1,B=2,C=3,D=4,E=5,F=6,{G,H,I,J}},

[0171] {A=1,B=2,D=3,C=4,E=5,F=6,{G,H,I,J}},

[0172] {A=1,B=2,D=3,E=4,C=5,F=6,{G,H,I,J}},

[0173] {A=1,D=2,B=3,C=4,E=5,F=6,{G,H,I,J}},

[0174] {A=1,D=2,B=3,E=4,C=5,F=6,{G,H,I,J}},

[0175] {A=1, D=2, E=3, B=4, C=5, F=6, {G, H, I, J}};

[0176] The name of the maximum numbered node in all schemes is F, which is consistent, and there is no ancestor node that has not consumed all the memory, so there is an independent split point F.

[0177] Step S2.1, therefore, for all the solutions at the current moment T=6, determine the first memory peak to the t=6th memory peak, and select the best solution. Assume that the best number selected is {A=1, B=2, D=3, C=4, E=5, F=6}, determine the number of ABDCEF, and then re-determine the order of the subsequent nodes. At this time, the whole graph is divided into numbered {ABCDE} and the remaining unnumbered subgraphs {F=6, {GHIJ}}. The sorted and numbered nodes are cut out, the remaining network length is reduced, and the subsequent sorting calculation saves memory. Keep F=6 to participate in the memory peak calculation at the next moment.

[0178] For the convenience of description, in actual implementation, it is also necessary to retain the last numbered node to store the current numbering information. Of course, other means can be used to record the current numbering information to facilitate obtaining the number at the next moment.

[0179] like Figure 5 As shown, step S1 further includes:

[0180] S1.7, at time T=7, the order of the remaining subgraphs can be recalculated from a width of 0, rather than continuing the calculation from the 6 options at time T=6.

[0181] At this time, the node F that has been numbered is 6, so the next step is to determine the node numbered 7. The node numbered 7 can currently be G. At this time, the beam width is {F=6, G=7}; similarly,

[0182] S1.8, beam solution at T=8:

[0183] {F=6,G=7,H=8,{I,J}},

[0184] {F=6,G=7,I=8,{H,J}};

[0185] S1.9, beam solution at T=9:

[0186] {F=6,G=7,H=8,I=9,{J}},

[0187] {F=6,G=7,I=8,H=9,{J}};

[0188] S1.10, beam solution at T=10:

[0189] {F=6,G=7,H=8,I=9,J=10},

[0190] {F=6, G=7, I=8, J=9, J=10}.

[0191] Step S2.2, finally select the one with the smallest memory peak from the two solutions with T=10 as the final solution, assuming it is {F=6, G=7, H=8, I=9, J=10}.

[0192] In step S3, the final solution sorting of S2.2 is concatenated with the previously stored {A=1, B=2, D=3, C=4, E=5, F=6} to form the sorting information of the entire network graph {A=1, B=2, D=3, C=4, E=5, F=6, G=7, H=8, I=9, J=10}.

[0193] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An optimization method for beam search of completely independent segmentation points, characterized in that: The method comprises: S1, determine and find completely independent segmentation points: The solution of the independent split point in the beam search process only needs to determine whether the maximum numbered name of each beam solution is consistent. If they are consistent, then determine whether there are unconsumed ancestor nodes. If all ancestor nodes are consumed, it means that the point is a completely independent split point. S2, according to the completely independent split points, the entire network graph is divided into independent subgraphs. The previous independent subgraph has N solutions, and the current N solutions do not affect the sorting at the next moment, so the N solutions can select the best one and record the numbers from the current 1 to t moment; next, the numbers at the t+1th moment and later are analyzed independently, and each independent subgraph can also be processed separately, and each independent subgraph enters the beam search for sorting; S3, finally, the sorting information of each subgraph is spliced ​​together to form the sorting information of the entire network graph.

2. The optimization method for beam search for completely independent segmentation points according to claim 1, characterized in that: The method further comprises step S0, S0, before entering sorting, records the name of each node and the memory value of the node's output tensor into the name_to_memory dictionary.

3. The optimization method for beam search for completely independent segmentation points according to claim 2, characterized in that: In the step S1, when it is determined that there are two nodes, it can be determined that they are not independent segmentation points, and the determination is immediately exited.

4. The optimization method for beam search for completely independent segmentation points according to claim 3, characterized in that: The step S1 further includes: assuming that there is a node AJ, S1.1, at time T=1, assuming that A is numbered 1, The current beam scheme has only one numbered node, the maximum numbered name is the same, and there are no unconsumed descendant nodes, then A is a completely independent split point, The ordering before and after the completely independent split point does not affect each other. The completely independent split point itself needs to participate in the calculation of the memory peak value at the next moment. There is no node before node A, so it does not need to be split; S1.2, at time T = 2, the node A is numbered 1, then the node numbered 2 should be selected. Assume that the optional nodes numbered 2 are B and D, and the beam width is 2. {A=1,B=2,{C,D,E,F,G,H,I,J}}, {A=1,D=2,{B,C,E,F,G,H,I,J}}; The maximum numbered names B and D of the two beam schemes are inconsistent, so there is no independent split point at t = 2; S1.3, at T = 3, assume that the next number of the solution {A = 1, B = 2} can be selected as C and D; the next number of the solution {A = 1, D = 2} can be selected as B and E; therefore, at T = 3, the beam width is 4, and there are 4 solutions: {A=1,B=2,C=3,{D,E,F,G,H,I,J}}, {A=1,B=2,D=3,{C,E,F,G,H,I,J}}, {A=1,D=2,B=3,{C,E,F,G,H,I,J}}, {A=1,D=2,E=3,{B,E,F,G,H,I,J}}; Among them, the names of the largest numbered nodes in the first and second schemes are C and D respectively, and the names are inconsistent, so there is no independent split point; And so on. At S1,4, T=4, the beam scheme has {A=1, B=2, C=3, D=4, {E, F, G, H, I, J}}, {A=1,B=2,D=3,C=4,{E,F,G,H,I,J}}, {A=1,B=2,D=3,E=4,{C,F,G,H,I,J}}, {A=1,D=2,B=3,C=4,{E,F,G,H,I,J}}, {A=1,D=2,B=3,E=4,{C,F,G,H,I,J}}, {A=1,D=2,E=3,B=4,{C,F,G,H,I,J}}; Among them, the names of the largest numbered nodes in the first and second schemes are D and C respectively. The names are inconsistent, so there is no independent split point; S1.5, T = 5, the beam solution is {A=1,B=2,C=3,D=4,E=5,{F,G,H,I,J}}, {A=1,B=2,D=3,C=4,E=5,{F,G,H,I,J}}, {A=1,B=2,D=3,E=4,C=5,{F,G,H,I,J}}, {A=1,D=2,B=3,C=4,E=5,{F,G,H,I,J}}, {A=1,D=2,B=3,E=4,C=5,{F,G,H,I,J}}, {A=1,D=2,E=3,B=4,C=5,{F,G,H,I,J}}; Among them, the names of the largest numbered nodes in the first, second, and third schemes are E, E, and C respectively. The names are inconsistent, so there is no independent split point; S1.6, T = 6, the beam solution has {A=1,B=2,C=3,D=4,E=5,F=6,{G,H,I,J}}, {A=1,B=2,D=3,C=4,E=5,F=6,{G,H,I,J}}, {A=1,B=2,D=3,E=4,C=5,F=6,{G,H,I,J}}, {A=1,D=2,B=3,C=4,E=5,F=6,{G,H,I,J}}, {A=1,D=2,B=3,E=4,C=5,F=6,{G,H,I,J}}, {A=1,D=2,E=3,B=4,C=5,F=6,{G,H,I,J}}; The name of the maximum numbered node in all schemes is F, which is consistent, and there is no ancestor node that has not consumed all the memory, so there is an independent split point F.

5. The optimization method for beam search for completely independent segmentation points according to claim 4, characterized in that: The step S2 further comprises: S2.1, so at the current moment T=6, there are 5 nodes {A, B, C, D, E, F}. Determine the first memory peak to the sixth memory peak and select the best solution. Assume that the best number selected is {A=1, B=2, D=3, C=4, E=5, F=6}, determine the number of ABDCEF, and then re-determine the order of the following nodes. At this time, the whole graph is divided into numbered {ABCDE} and the remaining unnumbered subgraphs {F=6, {GHIJ}}. Cut out the sorted and numbered nodes, reduce the remaining network length, participate in the subsequent sorting calculation to save memory, and retain F=6 to participate in the memory peak calculation at the next moment.

6. The optimization method for beam search for completely independent segmentation points according to claim 5, characterized in that: The step S1 further comprises: S1.7, at time T = 7, the remaining subgraphs can be recalculated from a width of 0, rather than continuing from the 6 solutions at time T = 6; At this time, the number of the numbered node F is 6, so the next step is to determine the node numbered 7; the node numbered 7 can currently be selected as G; at this time, the beam width is 1: {F=6,G=7,{H,I,J}} Similarly, S1.8, beam solution at T=8: {F=6,G=7,H=8,{I,J}}, {F=6,G=7,I=8,{H,J}}; S1.9, beam solution at T=9: {F=6,G=7,H=8,I=9,{J}}, {F=6,G=7,I=8,H=9,{J}}; S1.10, beam solution at T=10: {F=6,G=7,H=8,I=9,J=10}, {F=6, G=7, I=8, J=9, J=10}.

7. The optimization method for beam search for completely independent segmentation points according to claim 6, characterized in that: The step S2 further comprises: S2.2, finally, the solution with the smallest memory peak value is selected from the two solutions with T=10 as the final solution, assuming that {F=6, G=7, H=8, I=9, J=10}.

8. The optimization method for beam search for completely independent segmentation points according to claim 6, characterized in that: The step S3 further comprises: {F=6, G=7, H=8, I=9, J=10} is then concatenated with the previously stored {A=1, B=2, D=3, C=4, E=5, F=6} to form the sorting information of the entire network graph {A=1, B=2, D=3, C=4, E=5, F=6, G=7, H=8, I=9, J=10}.

9. The optimization method for beam search for completely independent segmentation points according to claim 1, characterized in that: In the step S1, in actual implementation, it is necessary to record the current numbering information to facilitate obtaining the number at the next moment. The last numbered node can be retained to store the current numbering information, or other means can be used to record, including implementing the use of a global dictionary to store the memory information of all nodes.

Images