Target strategy model training method and device, instruction execution method and device and medium

Abstract

The invention discloses a target strategy model training method and device, an instruction execution method and device and a medium. The training method of the target strategy model comprises the following steps: determining a to-be-executed instruction sequence; determining memory occupancy information used for executing instruction sequence hardware; based on the instruction sequence, the memory usage information and the initial strategy model, determining an execution mode corresponding to the instruction sequence; evaluating the execution mode to obtain evaluation information; performing iterative training on the initial strategy model based on the instruction sequence, the memory usage information, the execution mode and the evaluation information; and in response to the condition that the iteratively trained initial strategy model meets a preset training ending condition, taking the iteratively trained initial strategy model as a target strategy model. According to the embodiment of the invention, the execution efficiency of the instruction sequence can be improved.

Description

Technical Field

[0001] The present disclosure relates to neural network model technology, and in particular to a training method, instruction execution method, device and medium for a target strategy model. Background Art

[0002] Neural network models are widely used in many fields such as computer vision, natural language processing, speech recognition, and autonomous driving. For example, using neural network models, target detection, target tracking, and instance segmentation can be performed on environmental images collected by on-board cameras.

[0003] How to ensure the operating efficiency of the neural network model is an issue worthy of attention for the technology in this field. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a training method, an instruction execution method, a device and a medium for a target strategy model.

[0005] According to one aspect of an embodiment of the present disclosure, a method for training a target policy model is provided, comprising:

[0006] determining a sequence of instructions to be executed;

[0007] Determining memory usage information of hardware used to execute the instruction sequence;

[0008] Determining an execution mode corresponding to the instruction sequence based on the instruction sequence, the memory occupancy information and the initial strategy model;

[0009] Evaluate the execution mode to obtain evaluation information;

[0010] Iteratively training the initial strategy model based on the instruction sequence, the memory occupancy information, the execution mode, and the evaluation information;

[0011] In response to the initial policy model after iterative training satisfying a preset training end condition, the initial policy model after iterative training is used as a target policy model.

[0012] According to another aspect of an embodiment of the present disclosure, there is provided an instruction execution method, including:

[0013] Determine the model to be compiled;

[0014] Compiling the model to be compiled to obtain a target instruction sequence corresponding to the model to be compiled;

[0015] Determining current memory usage information of hardware used to execute the target instruction sequence;

[0016] Based on the target instruction sequence, the current memory occupancy information and the target policy model, an execution mode corresponding to the target instruction sequence is determined.

[0017] According to another aspect of an embodiment of the present disclosure, a training device for a target policy model is provided, comprising:

[0018] A first determination module, used to determine an instruction sequence to be executed;

[0019] A second determining module, used to determine memory occupancy information of hardware used to execute the instruction sequence determined by the first determining module;

[0020] a third determining module, configured to determine an execution mode corresponding to the instruction sequence determined by the first determining module based on the instruction sequence determined by the first determining module, the memory occupancy information determined by the second determining module, and an initial policy model;

[0021] An evaluation module, used for evaluating the execution mode determined by the third determination module to obtain evaluation information;

[0022] a training module, configured to iteratively train the initial policy model based on the instruction sequence determined by the first determination module, the memory occupancy information determined by the second determination module, the execution mode determined by the third determination module, and the evaluation information obtained by the evaluation module;

[0023] The fourth determination module is used to use the initial policy model after iterative training by the training module as the target policy model in response to the initial policy model after iterative training by the training module meeting a preset training end condition.

[0024] According to another aspect of the embodiments of the present disclosure, there is provided an instruction execution device, including:

[0025] A fifth determination module, used to determine a model to be compiled;

[0026] A compiling module, used for compiling the model to be compiled determined by the fifth determining module to obtain a target instruction sequence corresponding to the model to be compiled determined by the fifth determining module;

[0027] a sixth determining module, used to determine current memory occupancy information of the target instruction sequence hardware obtained by executing the compiling module;

[0028] The seventh determination module is used to determine the execution mode corresponding to the target instruction sequence obtained by the compilation module based on the target instruction sequence obtained by the compilation module, the current memory occupancy information determined by the sixth determination module, and the target policy model.

[0029] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the above-mentioned training method of the target policy model, or to execute the above-mentioned instruction execution method.

[0030] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:

[0031] processor;

[0032] a memory for storing instructions executable by the processor;

[0033] The processor is used to read the executable instructions from the memory and execute the instructions to implement the training method of the target policy model or the instruction execution method.

[0034] According to another aspect of an embodiment of the present disclosure, a computer program product is provided. When instructions in the computer program product are executed by a processor, the training method of the target policy model mentioned above is executed, or the instruction execution method mentioned above is executed.

[0035] Based on the training method, instruction execution method, device, medium, electronic device and program product of the target policy model provided by the above-mentioned embodiments of the present disclosure, in the model training stage, the execution mode corresponding to the instruction sequence can be determined based on the instruction sequence to be executed, the memory occupancy information of the hardware used to execute the instruction sequence, and the initial policy model. The instruction sequence, memory occupancy information, execution mode, and the evaluation results obtained by evaluating the execution mode can all be used for iterative training of the initial policy model. During the iterative training process, the model parameters of the initial policy model can be optimized, and the initial policy model can learn how to assist in obtaining a more superior, more reasonable, and more conducive to ensuring the execution efficiency. In the model reasoning stage, based on the target policy model obtained through iterative training in the training stage, a suitable execution mode can be determined for a given instruction sequence, and the given instruction sequence is executed according to the determined execution mode, which is conducive to improving the execution efficiency of the instruction sequence during model reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic diagram of two stages involved in an embodiment of the present disclosure.

[0037] Figure 2 It is a flowchart of a method for training a target strategy model provided by some exemplary embodiments of the present disclosure.

[0038] Figure 3 It is a flowchart of a method for determining an execution mode corresponding to an instruction sequence provided by some exemplary embodiments of the present disclosure.

[0039] Figure 4-1 It is a flowchart of a method for obtaining policy information generated by an initial policy model provided by some exemplary embodiments of the present disclosure.

[0040] Figure 4-2 is a diagram of the working principle of the initial strategy model in some exemplary embodiments of the present disclosure.

[0041] Figure 5-1 It is a flowchart of a method for determining an execution mode corresponding to an instruction sequence provided by some exemplary embodiments of the present disclosure.

[0042] Figure 5-2 It is a flowchart of a method for determining an execution mode corresponding to an instruction sequence provided by some other exemplary embodiments of the present disclosure.

[0043] Figure 6 It is a flowchart of a method for obtaining evaluation information provided by some exemplary embodiments of the present disclosure.

[0044] Figure 7 It is a flowchart of an instruction execution method provided by some exemplary embodiments of the present disclosure.

[0045] Figure 8-1 It is a schematic diagram of a training method of a target policy model in some exemplary embodiments of the present disclosure.

[0046] Figure 8-2 It is a schematic diagram of a training method for a target strategy model in some other exemplary embodiments of the present disclosure.

[0047] Fig. 9 It is a structural diagram of a training device for a target strategy model provided by some exemplary embodiments of the present disclosure.

[0048] Fig.10 is a structural diagram of the third determination module in some exemplary embodiments of the present disclosure.

[0049] Fig.11 is a schematic structural diagram of a first determination submodule in some exemplary embodiments of the present disclosure.

[0050] Fig.12 It is a structural schematic diagram of the first determination submodule in some other exemplary embodiments of the present disclosure.

[0051] Fig.13 It is a schematic diagram of the structure of the acquisition submodule in some exemplary embodiments of the present disclosure.

[0052] Fig.14 is a schematic diagram of the structure of the evaluation module in some exemplary embodiments of the present disclosure.

[0053] Fig.15 It is a schematic diagram of the structure of an instruction execution device provided by some exemplary embodiments of the present disclosure.

[0054] Fig.16 It is a schematic diagram of the structure of an electronic device provided by some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0055] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0056] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0057] Application Overview

[0058] Neural network models are widely used in many fields such as computer vision, natural language processing, speech recognition, and autonomous driving. Compiling a neural network model with a compiler can generate a corresponding instruction sequence. Executing the instruction sequence corresponding to the neural network model on hardware is equivalent to running the neural network model.

[0059] In the related art, the execution efficiency of the instruction sequence is low, and accordingly, the operation efficiency of the neural network model is low, which is difficult to meet the actual needs. Therefore, it is necessary to take certain measures to improve the execution efficiency of the instruction sequence to ensure the operation efficiency of the neural network model.

[0060] Exemplary Systems

[0061] like Figure 1 As shown, the embodiments of the present disclosure may involve two stages, namely a model training stage and a model reasoning stage.

[0062] In the model training phase, the initial policy model can be iteratively trained to obtain the target policy model. The initial policy model can be a policy model to be trained, and the target policy model can be a trained policy model. Both the initial policy model and the target policy model can be used to assist in obtaining the execution mode of a given instruction sequence.

[0063] In the model reasoning stage, an execution mode can be determined for a given instruction sequence based on the target policy model. Executing the given instruction sequence according to the determined execution mode is conducive to improving the execution efficiency of the instruction sequence during model reasoning.

[0064] Exemplary Methods

[0065] Figure 2 It is a flowchart of a method for training a target strategy model provided by some exemplary embodiments of the present disclosure. Figure 2 The method shown may include step 210 , step 220 , step 230 , step 240 , step 250 , and step 260 .

[0066] Step 210, determining the instruction sequence to be executed.

[0067] Optionally, the neural network model may be compiled using a compiler to obtain an instruction sequence corresponding to the neural network model. The instruction sequence corresponding to the neural network model may be an instruction sequence executable by a computer. The instruction sequence corresponding to the neural network model may be used as an instruction sequence to be executed. The instruction sequence to be executed may be in a text representation.

[0068] Of course, the method for determining the instruction sequence to be executed is not limited to this. For example, the neural network model can be divided into multiple sub-models, and each of the multiple sub-models can include part of the network layer in the neural network model. By using a compiler to compile any of the multiple sub-models, the instruction sequence corresponding to the sub-model can be obtained, and the instruction sequence corresponding to the sub-model can be used as the instruction sequence to be executed.

[0069] It should be noted that, in the following text, all instruction sequences without modifying adjectives can be understood as the instruction sequences determined in step 210 .

[0070] Step 220, determining memory usage information of hardware used to execute the instruction sequence.

[0071] Optionally, the hardware used to execute the instruction sequence may be a neural network accelerator (Neural network Processing Unit, NPU).

[0072] Optionally, the neural network accelerator may include an on-chip memory. As an example, the on-chip memory may be a static random access memory (SRAM). The on-chip memory may include a plurality of bits for data storage, and the memory occupancy information of the neural network accelerator may be used to characterize which bits of the plurality of bits are occupied and which bits are not occupied. The memory occupancy information of the neural network accelerator may be in a text representation. The memory occupancy information of the neural network accelerator may be obtained through a random initialization operation.

[0073] It should be noted that the memory usage information without any modifying adjectives in the following text can be understood as the memory usage information determined in step 220 .

[0074] Step 230, determining the execution mode corresponding to the instruction sequence based on the instruction sequence, the memory usage information and the initial policy model.

[0075] Optionally, the instruction sequence and memory usage information can be used as input data of the initial policy model, and the initial policy model can perform operations based on the input data to obtain corresponding output data. Based on the output data, the execution mode corresponding to the instruction sequence can be determined.

[0076] Optionally, the execution mode corresponding to the instruction sequence can be used to characterize how the instruction sequence is executed. For example, the execution mode corresponding to the instruction sequence can be used to characterize the execution order of all instructions in the instruction sequence.

[0077] In one example, the instruction sequence may include four instructions, namely, instruction R1, instruction R2, instruction R3, and instruction R4; wherein instruction R1 is used to instruct the loading of data A1 and data A2; instruction R2 is used to instruct the loading of data B1 and data B2; instruction R3 is used to instruct the addition of data A1 and data B1 to obtain data C1, and the addition of data A2 and data B2 to obtain data C2; instruction R4 is used to instruct the multiplication of data C1 and data C2 to obtain data C5. The execution mode corresponding to the instruction sequence can be used to represent that instruction R1 is executed first, then instruction R2, then instruction R3, and finally instruction R4, that is, instruction R1, instruction R2, instruction R3, and instruction R4 are executed in sequence. Alternatively, the execution method corresponding to the instruction sequence can be used to represent that data A1 is first loaded according to instruction R1, and data B1 is loaded according to instruction R2, and data A1 and data B1 are added according to instruction R3 to obtain data C1. While adding data A1 and data B1, A2 is loaded according to instruction R1, and data B2 is loaded according to instruction R2. After the addition of data A1 and data B1 is completed, data A2 and data B2 are added according to instruction R3 to obtain data C2. Thereafter, data C1 and data C2 are multiplied according to instruction R4 to obtain data C5. That is, instructions R1, instruction R2, instruction R3, and instruction R4 are not executed sequentially, but the parts that can be executed synchronously are executed synchronously, which is equivalent to executing the instruction sequence in the form of a pipeline operation.

[0078] Of course, the meaning represented by the execution mode corresponding to the instruction sequence is not limited to this. For example, the execution mode corresponding to the instruction sequence can also be used to represent whether it is necessary to pre-allocate resources for certain instructions in the instruction sequence. As an example, the allocated resources here can be memory space in the on-chip memory.

[0079] It should be noted that the execution modes without modifying attributes in the following text can be understood as the execution modes determined in step 230 .

[0080] Step 240, evaluating the execution mode to obtain evaluation information.

[0081] Optionally, the execution mode includes the order of instruction execution, or parallel execution of instructions, serial execution mode, etc. The evaluation of the execution mode can be implemented by a pre-set evaluation algorithm or evaluation model. The evaluation model can be, for example, a convolutional neural network model. The evaluation information obtained by evaluating the execution mode can be used to represent the execution performance of the instruction sequence corresponding to the execution mode. The execution performance can reflect information such as the time and power consumption required for the instruction sequence to be executed on the neural network accelerator according to the execution mode.

[0082] Specifically, the instruction sequence and execution mode (e.g., parallel execution of some instructions) can be used as input data of the evaluation model, and the time and / or power consumption required for the instruction sequence to be executed on the neural network accelerator according to the execution mode can be used as output data of the evaluation model, and evaluation information can be obtained based on the output data of the evaluation model. For example, the output data of the evaluation model can be converted into an evaluation value, and the obtained evaluation value can be used as the evaluation information.

[0083] It should be noted that the evaluation information without modifying attributes in the following text can be understood as the evaluation information obtained in step 240 .

[0084] Step 250, iteratively train the initial policy model based on the instruction sequence, memory usage information, execution mode and evaluation information.

[0085] Optionally, the optimizer can be used to collect data such as instruction sequences, memory usage information, execution methods, and evaluation information. Based on the collected data, the optimizer can continuously optimize the model parameters of the initial policy model to achieve iterative training of the initial policy model. Specifically, the instruction sequence, memory usage information, execution method, and evaluation information can be used to iteratively train the initial policy model based on the reinforcement learning algorithm.

[0086] In some embodiments, the instruction sequence and the memory occupancy information may be preprocessed respectively, and based on the preprocessed instruction sequence, the preprocessed memory occupancy information and the initial strategy model, the execution mode corresponding to the preprocessed instruction sequence is determined, the determined execution mode is evaluated to obtain evaluation information, and based on the preprocessed instruction sequence, the preprocessed memory occupancy information, the determined execution mode, and the obtained evaluation information, the initial strategy model is iteratively trained based on the reinforcement learning algorithm. Optionally, the preprocessing may include but is not limited to abnormal data cleaning, redundant data removal, text dictionary unified processing, etc.

[0087] Step 260 , in response to the initial policy model after iterative training satisfying a preset training end condition, the initial policy model after iterative training is used as a target policy model.

[0088] Optionally, it can be determined whether the initial policy model after iterative training converges. If the initial policy model after iterative training converges, it can be determined that the initial policy model after iterative training meets the preset training end condition. If the initial policy model after iterative training does not converge, it can be determined that the initial policy model after iterative training does not meet the preset training end condition.

[0089] In some embodiments, the training duration of the initial policy model may also be counted, and if the training duration is greater than a preset duration, it may be determined that the initial policy model after iterative training meets a preset training end condition. The preset duration may be a preset upper limit of the training duration.

[0090] When the initial policy model after iterative training meets the preset training end condition, the initial policy model after iterative training can be used as the target policy model.

[0091] If the initial policy model after iterative training does not meet the preset training end condition, iterative training can be continued.

[0092] In the embodiments of the present disclosure, in the model training stage, the execution mode corresponding to the instruction sequence can be determined based on the instruction sequence to be executed, the memory usage information of the hardware used to execute the instruction sequence, and the initial policy model. The instruction sequence, the memory usage information, the execution mode, and the evaluation results obtained by evaluating the execution mode can all be used for iterative training of the initial policy model. During the iterative training process, the model parameters of the initial policy model can be optimized, and the initial policy model can learn how to assist in obtaining a more superior, more reasonable, and more conducive to ensuring the execution efficiency. In the model reasoning stage, based on the target policy model obtained through iterative training in the training stage, a suitable execution mode can be determined for a given instruction sequence, and the given instruction sequence is executed according to the determined execution mode, which is conducive to improving the execution efficiency of the instruction sequence during model reasoning.

[0093] Figure 3 It is a flowchart of a method for determining an execution mode corresponding to an instruction sequence provided by some exemplary embodiments of the present disclosure. Figure 3 The method shown may include step 310 and step 320. Optionally, the combination of step 310 and step 320 may be used as an optional implementation of step 230 of the present disclosure.

[0094] Step 310: Obtaining policy information generated by the initial policy model based on the instruction sequence and the memory occupancy information.

[0095] Step 320: Determine the execution mode corresponding to the instruction sequence based on the policy information.

[0096] Optionally, the initial policy model may be a policy model to be trained, and the initial policy model may include multiple sub-networks, such as the first sub-network, the second sub-network, the third sub-network, etc. The initial policy model may be used to generate corresponding policy information based on a given instruction sequence and given memory usage information. The corresponding policy information generated by the initial policy model may be used to represent the overall execution strategy of the given instruction sequence. On the basis of the overall execution strategy, the execution details may be planned to obtain the execution mode corresponding to the given instruction sequence.

[0097] Specifically, in step 310, the instruction sequence and memory occupancy information may be used as input data of the initial policy model, and the initial policy model may perform operations based on the instruction sequence and memory occupancy information to generate policy information, thereby determining the execution mode corresponding to the instruction sequence.

[0098] In some optional embodiments of the present disclosure, Figure 4-1 As shown, step 310 may include step 3101 , step 3103 , step 3105 and step 3107 .

[0099] Step 3101, pre-process the instruction sequence and memory occupancy information respectively.

[0100] Optionally, the preprocessing of the instruction sequence and the memory occupancy information may include but is not limited to abnormal data cleaning, redundant data removal, text dictionary unification, etc. The preprocessed instruction sequence and the preprocessed memory occupancy information may both be in text representation.

[0101] Step 3103, through the first sub-network in the initial strategy model, feature extraction is performed on the preprocessed instruction sequence to obtain a first tensor.

[0102] Optionally, the initial strategy model may include a first sub-network, a second sub-network and a third sub-network; wherein the first sub-network and the second sub-network may be used for feature extraction, and the third sub-network may be used for feature fusion and strategy information generation.

[0103] Optionally, the first sub-network in the initial policy model can be a multi-layer recurrent neural network or a Transformer; wherein the Transformer is a network based on an encoder-decoder architecture and an attention mechanism. The first sub-network in the initial policy model can also be called an instruction feature encoding network. Figure 4-2As shown, the preprocessed instruction sequence can be used as input data of the instruction feature encoding network, and the instruction feature encoding network can perform feature extraction on the input data to obtain a corresponding tensor, which can be used as the first tensor.

[0104] Step 3105: Perform feature extraction on the preprocessed memory occupancy information through the second sub-network in the initial strategy model to obtain a second tensor.

[0105] Optionally, the second sub-network in the initial strategy model can be a multi-layer recurrent neural network or a fully connected network. The second sub-network in the initial strategy model can also be called a memory feature encoding network. Figure 4-2 As shown, the preprocessed memory occupancy information can be used as input data of the memory feature encoding network, and the memory feature encoding network can perform feature extraction on the input data to obtain a corresponding tensor, which can be used as the second tensor.

[0106] Step 3107, performing feature fusion on the first tensor and the second tensor via the third sub-network in the initial strategy model to obtain a fused tensor, and generating strategy information based on the fused tensor.

[0107] Optionally, the third sub-network in the initial strategy model can be a multi-layer fully connected network. The third sub-network in the initial strategy model can also be called a feature fusion network. Figure 4-2 As shown, the first tensor and the second tensor can be used as input data of the feature fusion network. The feature fusion network can fuse the first tensor and the second tensor to obtain a fused tensor, and operate on the fused tensor to generate strategy information.

[0108] Figure 4-1 In the illustrated implementation manner, the accuracy of the instruction sequence and the memory occupancy information is improved by preprocessing the instruction sequence and the memory occupancy information respectively; further, based on the preprocessed instruction sequence and the preprocessed memory occupancy information, feature extraction, feature fusion and strategy information generation are performed, thereby improving the rationality and reliability of the strategy information.

[0109] In some optional embodiments of the present disclosure, Figure 5-1 As shown, step 320 may include step 3201 , step 3203 , step 3205 and step 3207 .

[0110] Step 3201, based on the policy information, determine the first grouping quantity of the instruction sequence.

[0111] Optionally, the policy information may include first information for indicating the number of groups, and determining the first number of groups of the instruction sequence based on the policy information may include:

[0112] Using the first information in the policy information as the first grouping number of the instruction sequence;

[0113] or,

[0114] Based on the first information in the policy information, a first grouping quantity of the instruction sequence is obtained through a first preset processing method.

[0115] Specifically, the first information can be represented by numerical information. If the first information is an integer, the first information can be used as the first grouping quantity of the instruction sequence; if the first information is a decimal, the first preset processing method includes rounding the first information, and the rounding result can be used as the first grouping quantity of the instruction sequence. The rounding method here can be, for example, rounding. In an example, the first information is 3.2, and the first grouping quantity is 3.

[0116] Step 3203, determining multiple grouping modes of the instruction sequence corresponding to the first number of groups.

[0117] Optionally, the first number of groups may be the number of portions into which the instruction sequence needs to be divided. For example, the first number of groups is 3, which indicates that the instruction sequence needs to be divided into three portions. Then, it can be determined how many feasible grouping methods there are for grouping the instruction sequence according to the first number of groups, thereby obtaining multiple grouping methods of the instruction sequence corresponding to the first number of groups.

[0118] In one example, the instruction sequence includes 8 instructions, namely instruction S1, instruction S2, instruction S3, instruction S4, instruction S5, instruction S6, instruction S7 and instruction S8, and the number of first groups is 3. Then, among the multiple grouping methods of the instruction sequence corresponding to the number of first groups, the first grouping method can be: instruction S1 and instruction S2 are in one group, instruction S3 and instruction S4 are in another group, and instruction S5, instruction S6, instruction S7 and instruction S8 are in another group. The second grouping method can be: instruction S1 and instruction S2 are in one group, instruction S3, instruction S4 and instruction S5 are in another group, and instruction S6, instruction S7 and instruction S8 are in another group. The third grouping method can be: instruction S1 and instruction S2 are in one group, instruction S3, instruction S4, instruction S5 and instruction S6 are in another group, and instruction S7 and instruction S8 are in another group. Other grouping methods can be deduced in the same way, ensuring that there are three groups in each grouping method, each of the three groups includes one or more instructions, and the total number of instructions included in each of the three groups is 8, which will not be listed one by one here.

[0119] Step 3205, filter the target grouping method from multiple grouping methods.

[0120] Optionally, a grouping method can be randomly selected from multiple grouping methods as a target grouping method. Of course, a grouping method can also be selected from multiple grouping methods according to a pre-set screening rule as a target grouping method. The screening rule here can be, for example, a grouping method that selects the number of instructions in different groups as evenly as possible.

[0121] Step 3207, based on the target grouping method, determine the execution method corresponding to the instruction sequence.

[0122] Optionally, the execution mode corresponding to the instruction sequence can be used to characterize the execution order of instructions in different groups under the target grouping mode. For example, under the target grouping mode, instructions S1 and instruction S2 are located in one group (hereinafter referred to as group U1 for the convenience of description), instructions S3, instructions S4 and instructions S5 are located in another group (hereinafter referred to as group U2 for the convenience of description), and instructions S6, instructions S7 and instructions S8 are located in another group (hereinafter referred to as group U3 for the convenience of description). Then, it can be determined which instructions in group U1 can be executed synchronously with which instructions in group U2, which instructions in group U1 can be executed synchronously with which instructions in group U3, which instructions in group U2 can be executed synchronously with which instructions in group U3, etc., thereby reasonably determining which instructions from instructions S1 to instruction S8 are executed synchronously, which instructions are executed successively, etc. On this basis, the execution mode corresponding to the instruction sequence can be obtained.

[0123] Figure 5-1 In the illustrated implementation, based on the policy information generated by the initial policy model, a target grouping method can be reasonably determined to determine the execution method corresponding to the instruction sequence. In this way, in the model reasoning stage, based on the target policy model, a suitable grouping method can be determined for a given instruction sequence, and on this basis, a suitable execution method can be determined for a given instruction sequence, which is beneficial to the execution efficiency of the instruction sequence during model reasoning.

[0124] In some other optional embodiments of the present disclosure, Figure 5-2 As shown, step 320 may include step 3211 , step 3213 and step 3215 .

[0125] Step 3211, based on the policy information, determine the target parameter information related to the predetermined operation option.

[0126] Step 3213: Determine the selection state corresponding to the predetermined operation option based on the target parameter information.

[0127] Optionally, the policy information may include second information corresponding to a predetermined operation option. The predetermined operation option may be, for example, an option of whether to release memory space in an on-chip memory in advance. Based on the policy information, determining target parameter information related to the predetermined operation option may include:

[0128] using the second information in the policy information as target parameter information related to the predetermined operation option;

[0129] or,

[0130] Based on the second information in the policy information, target parameter information related to the predetermined operation option is obtained through a second preset processing method.

[0131] Specifically, the second information can be represented by numerical information, and the second information can be used as target parameter information related to the predetermined operation option. Accordingly, in step 3213, the selection state corresponding to the predetermined operation option can be determined based on the target parameter information and the preset value. For example, if the target parameter information is less than the preset value, the selection state corresponding to the predetermined operation option can be determined to be an unselected state; if the target parameter information is greater than or equal to the preset value, the selection state corresponding to the predetermined operation option can be determined to be a selected state. As an example, the preset value can be 0.

[0132] Alternatively, the second information may be represented by numerical information, and the second preset processing method may include determining the magnitude relationship between the second information and the preset numerical value, and using the magnitude relationship as target parameter information related to the predetermined operation option. Accordingly, in step 3213, the selection state corresponding to the predetermined operation option may be determined based only on the target parameter information. For example, if the target parameter information is used to represent that the second information is less than the preset numerical value, the selection state corresponding to the predetermined operation option may be determined to be an unselected state. If the target parameter information is used to represent that the second information is greater than or equal to the preset numerical value, the selection state corresponding to the predetermined operation option may be determined to be a selected state.

[0133] Step 3215, based on the selection state, determine the execution mode corresponding to the instruction sequence.

[0134] Optionally, the execution mode corresponding to the instruction sequence can be used to characterize the selection state corresponding to the predetermined operation option.

[0135] Assuming that the execution mode corresponding to the instruction sequence is used to characterize that instruction V1 in the instruction sequence is executed before instruction V2, if the instruction sequence is actually executed according to the execution mode corresponding to the instruction sequence, instruction V1 will be executed before instruction V2, and the result data G1 obtained by executing instruction V1 can be stored in the on-chip memory. Assuming that the execution mode corresponding to the instruction sequence is used to characterize that the selection state corresponding to the predetermined operation option is the selected state, which means that the memory space in the on-chip memory needs to be released in advance, then the result data G1 can be transferred to the non-on-chip memory, and the result data G1 in the on-chip memory is deleted to release the memory space in the on-chip memory for storing the result data G1, so that in the next execution of instruction V2, the memory space originally used to store the result data G1 can be used. Assuming that the execution mode corresponding to the instruction sequence is used to characterize that the selection state corresponding to the predetermined operation option is the unselected state, which means that the memory space in the on-chip memory does not need to be released in advance, then the result data G1 can be retained in the on-chip memory, and in the next execution of instruction V2, the memory space storing the result data G1 cannot be used.

[0136] Figure 5-2 In the illustrated implementation, based on the policy information generated by the initial policy model, the selection state corresponding to the predetermined operation option can be determined efficiently and quickly, so as to determine the execution mode corresponding to the instruction sequence. In this way, in the model reasoning stage, based on the target policy model, the appropriate selection state can be determined for a given instruction sequence, and on this basis, the appropriate execution mode can be determined for the given instruction sequence, which is conducive to improving the execution efficiency of the instruction sequence during model reasoning.

[0137] Optionally, Figure 5-1 The embodiment shown and Figure 5-2 The illustrated implementations may be used in combination. That is, based on the policy information, the first grouping quantity and the target parameter information may be determined, and the first grouping quantity and the target parameter information may both be used to determine the execution mode corresponding to the instruction sequence.

[0138] In the embodiments of the present disclosure, based on the instruction sequence and memory usage information, the initial policy model can generate policy information to provide an effective reference for determining the execution mode corresponding to the instruction sequence. The instruction sequence, memory usage information, execution mode, and the evaluation results obtained by evaluating the execution mode can all be used for iterative training of the initial policy model. In this way, in the model reasoning stage, based on the given instruction sequence and the real memory usage information of the neural network accelerator, the target policy model can generate reasonable and reliable policy information. Based on the generated policy information, a suitable execution mode can be determined for a given instruction sequence, which is conducive to improving the execution efficiency of the instruction sequence during model reasoning.

[0139] In some optional examples, step 250 may include:

[0140] The initial policy model is iteratively trained based on the reinforcement learning algorithm, using the instruction sequence and memory usage information as state information, the execution mode as action information, and the evaluation information as reward information.

[0141] Optionally, the instruction sequence and memory usage information can be input as input data into the initial policy model, and the initial policy model can be operated accordingly to obtain policy information. Based on the policy information, the execution mode can be determined. In addition, the execution mode can be evaluated to obtain evaluation information. Here, the instruction sequence and memory usage information can be used as the state information of the first step of the reinforcement learning algorithm, the execution mode can be used as the action information of the first step of the reinforcement learning algorithm, and the evaluation information can be used as the reward information of the first step of the reinforcement learning algorithm. Next, the instruction sequence and memory usage information can be updated, and the updated instruction sequence and the updated memory usage information can be input as input data into the initial policy model. The initial policy model can be operated accordingly to obtain new policy information. Based on the new policy information, a new execution mode can be determined. In addition, the new execution mode can be evaluated to obtain new evaluation information. Here, the updated instruction sequence and the updated memory usage information can be used as the state information of the second step of the reinforcement learning algorithm, the new execution mode can be used as the action information of the second step of the reinforcement learning algorithm, and the new evaluation information can be used as the reward information of the third step of the reinforcement learning algorithm. Subsequently, we can follow this path to obtain the state information corresponding to multiple steps of the reinforcement learning algorithm, the action information corresponding to multiple steps, and the reward information corresponding to multiple steps.

[0142] Optionally, the initial policy model can be iteratively trained according to the following formula:

[0143]

[0144] Wherein, w represents the model parameters of the initial strategy model before parameter optimization, w' represents the model parameters of the initial strategy model after parameter optimization, β represents the preset learning rate, γ represents the preset discount coefficient, t represents the iteration step number, and n represents the total number of iteration steps. represents the gradient, a t represents the action information corresponding to the tth step, s t Indicates the state information corresponding to step t, u t Represents the reward information corresponding to the tth step.

[0145] In this way, we only need to bring the state information corresponding to multiple steps, the action information corresponding to multiple steps, and the reward information corresponding to multiple steps into the above formula for calculation, and we can optimize the parameters of the initial strategy model efficiently and reliably based on the reinforcement learning algorithm, and the model parameters can be optimized to a better state.

[0146] In some embodiments, the instruction sequence and the memory occupancy information may be preprocessed respectively, and based on the preprocessed instruction sequence, the preprocessed memory occupancy information and the initial policy model, the execution mode corresponding to the preprocessed instruction sequence is determined, and the determined execution mode is evaluated to obtain evaluation information. In this way, the preprocessed instruction sequence and the preprocessed memory occupancy information may be used as state information, the determined execution mode may be used as action information, and the obtained evaluation information may be used as reward information, and the initial policy model may be iteratively trained based on the reinforcement learning algorithm.

[0147] Figure 6 It is a flowchart of a method for obtaining evaluation information provided by some exemplary embodiments of the present disclosure. Figure 6 The method shown may include step 610 and step 620. Optionally, the combination of step 610 and step 620 may be used as an optional implementation of step 240 of the present disclosure.

[0148] Step 610, determine the estimated execution time of the execution mode.

[0149] Optionally, the time required to execute the instruction sequence on the neural network accelerator according to the execution mode can be estimated according to a pre-set evaluation algorithm, and the determined time can be used as the estimated execution time of the execution mode. For example, the time required to execute each instruction in the instruction sequence can be estimated, and then combined with the execution mode to characterize which instructions are executed concurrently and which instructions are executed sequentially, to determine the estimated execution time of the execution mode.

[0150] Step 620: Determine evaluation information corresponding to the execution mode based on the estimated execution time.

[0151] Optionally, the objective function may be pre-set in which the independent variable is the duration, the dependent variable is the evaluation value, and the independent variable and the dependent variable are negatively correlated. As an example, the objective function may be an inverse proportional function or an exponential function with a base less than 1.

[0152] Optionally, the estimated execution time may be substituted into the objective function as the value of the independent variable to perform calculations to obtain a corresponding evaluation value, and the obtained evaluation value may be used as evaluation information corresponding to the execution mode.

[0153] Generally speaking, the length of the estimated execution time is closely related to the quality of the execution method. For example, the shorter the estimated execution time, the better the execution method. Therefore, referring to the estimated execution time, the quality of the execution method can be effectively evaluated to obtain corresponding evaluation information. The obtained evaluation information is used for iterative training of the initial policy model, which enables the initial policy model to learn how to assist in obtaining a better, more reasonable, and more efficient execution method. In the model reasoning stage, based on the target policy model, a suitable execution method can be determined for a given instruction sequence, which is conducive to improving the execution efficiency of the instruction sequence during model reasoning.

[0154] Figure 7 It is a flowchart of an instruction execution method provided by some exemplary embodiments of the present disclosure. Figure 7 The method shown may include step 710 , step 720 , step 730 , and step 740 .

[0155] Step 710: determine the model to be compiled.

[0156] Optionally, the model to be compiled may be a complete neural network model, or the model to be compiled may be any sub-model among a plurality of sub-models obtained by splitting the complete neural network model.

[0157] Step 720, compile the model to be compiled to obtain a target instruction sequence corresponding to the model to be compiled.

[0158] Optionally, a compiler may be used to compile the model to be compiled to obtain a target instruction sequence corresponding to the model to be compiled. For example, the compiler may first generate an intermediate representation corresponding to the model to be compiled, and then generate a target instruction sequence corresponding to the model to be compiled based on the intermediate representation. The target instruction sequence may be an instruction sequence executable by a computer.

[0159] Step 730, determine current memory occupancy information of hardware used to execute the target instruction sequence.

[0160] Optionally, the hardware for executing the target instruction sequence may be a neural network accelerator. The neural network accelerator may include an on-chip memory. Here, all bits included in the on-chip memory may be traversed to determine which bits are occupied and which bits are not occupied, thereby obtaining the current memory occupancy information of the neural network accelerator.

[0161] Step 740, determining the execution mode corresponding to the target instruction sequence based on the target instruction sequence, the current memory usage information and the target policy model.

[0162] Optionally, the target instruction sequence and the current memory usage information can be used as input data of the target policy model, and the target policy model can be operated according to the input data to obtain corresponding policy information. Based on the determined policy information, the execution mode corresponding to the target instruction sequence can be determined, wherein the target policy model is obtained by the training method of the above-mentioned target policy model, and the similarities are not repeated here.

[0163] In an embodiment of the present disclosure, in the model reasoning stage, based on the target policy model obtained through iterative training in the training stage, a suitable execution mode can be determined for a given instruction sequence (for example, a target instruction sequence), and the given instruction sequence is executed according to the determined execution mode, which is beneficial to improving the execution efficiency of the instruction sequence during model reasoning, which is equivalent to improving the operating efficiency of the model to be compiled.

[0164] In some optional examples, such as Figure 8-1 , Figure 8-2 As shown, in the model training stage, the compiler can obtain the instruction sequence to be executed by model compilation, and in addition, the memory occupancy information of the neural network accelerator can be determined. Next, the instruction sequence and the memory occupancy information can be preprocessed respectively, and the preprocessed instruction sequence and the preprocessed memory occupancy information can be input into the initial policy model. The initial policy model can output policy information to the compiler. The compiler can determine the execution mode corresponding to the instruction sequence based on the policy information. For example, the compiler can determine the first grouping number of the instruction sequence and the target parameter information related to the predetermined operation option based on the policy information, and determine the execution mode corresponding to the instruction sequence based on the first grouping number and the target parameter information. By evaluating the execution mode corresponding to the instruction sequence, evaluation information can be obtained. The instruction sequence, memory occupancy information, execution mode and evaluation information can all be used for iterative training of the initial policy model, thereby obtaining the target policy model. Optionally, the optimizer can be used to calculate the loss value and perform gradient back propagation to optimize the model parameters of the initial policy model, thereby obtaining the target policy model.

[0165] In the model reasoning stage, there is no need to evaluate the execution mode or use an optimizer. In the model reasoning stage, the target instruction sequence can be obtained through model compilation, and the current memory usage information of the neural network accelerator can be determined. The target instruction sequence and the current memory usage information are input into the target policy model, and the target policy model can output policy information. Based on the policy information, the execution mode can be determined for the target instruction sequence. Executing the target instruction sequence according to the determined execution mode is conducive to improving the execution efficiency of the target instruction sequence during model reasoning.

[0166] In summary, in the embodiments of the present disclosure, the execution mode corresponding to the target instruction sequence can be efficiently generated based on the target strategy model without the need to use artificial rules or heuristic algorithms, which is conducive to ensuring the execution efficiency of the target instruction sequence and the operating efficiency of the model to be compiled.

[0167] Exemplary Devices

[0168] Fig. 9 It is a structural diagram of a training device for a target strategy model provided by some exemplary embodiments of the present disclosure. Fig. 9 The apparatus shown may include:

[0169] A first determination module 910, configured to determine an instruction sequence to be executed;

[0170] A second determination module 920, used to determine memory usage information of hardware used to execute the instruction sequence determined by the first determination module 910;

[0171] The third determination module 930 is used to determine the execution mode corresponding to the instruction sequence determined by the first determination module 910 based on the instruction sequence determined by the first determination module 910, the memory occupancy information determined by the second determination module 920, and the initial policy model;

[0172] An evaluation module 940, configured to evaluate the execution mode determined by the third determination module 930 to obtain evaluation information;

[0173] A training module 950, configured to iteratively train the initial policy model based on the instruction sequence determined by the first determination module 910, the memory occupancy information determined by the second determination module 920, the execution mode determined by the third determination module 930, and the evaluation information obtained by the evaluation module 940;

[0174] The fourth determination module 960 is used to use the initial policy model after iterative training by the training module 950 as the target policy model in response to the initial policy model after iterative training by the training module 950 meeting a preset training end condition.

[0175] In some optional examples, the training module 950 is used to iteratively train the initial policy model based on a reinforcement learning algorithm using instruction sequences and memory usage information as state information, execution methods as action information, and evaluation information as reward information.

[0176] In some optional examples, such as Fig.10 As shown, the third determination module 930 includes:

[0177] An acquisition submodule 1010 is used to obtain policy information generated by an initial policy model based on the instruction sequence determined by the first determination module 910 and the memory occupancy information determined by the second determination module 920;

[0178] The first determination submodule 1020 is used to determine the execution mode corresponding to the instruction sequence determined by the first determination module 910 based on the policy information obtained by the acquisition submodule 1010.

[0179] In some optional examples, such as Fig.11 As shown, the first determination submodule 1020 includes:

[0180] A first determining unit 1110, configured to determine the first grouping quantity of the instruction sequence determined by the first determining module 910 based on the policy information obtained by the obtaining submodule 1010;

[0181] A second determining unit 1120, configured to determine that the instruction sequence determined by the first determining module 910 corresponds to multiple grouping modes of the first number of groups determined by the first determining unit 1110;

[0182] A screening unit 1130, configured to screen a target grouping method from the multiple grouping methods determined by the second determining unit 1120;

[0183] The third determining unit 1140 is used to determine the execution mode corresponding to the instruction sequence determined by the first determining module 910 based on the target grouping mode screened by the screening unit 1130.

[0184] In some optional examples, such as Fig.12 As shown, the first determination submodule 1020 includes:

[0185] A fourth determining unit 1210, configured to determine target parameter information related to a predetermined operation option based on the policy information obtained by the obtaining submodule 1010;

[0186] A fifth determining unit 1220, configured to determine a selection state corresponding to a predetermined operation option based on the target parameter information determined by the fourth determining unit 1210;

[0187] The sixth determining unit 1230 is used to determine the execution mode corresponding to the instruction sequence determined by the first determining module 910 based on the selection state determined by the fifth determining unit 1220.

[0188] In some optional examples, such as Fig.13 As shown, the acquisition submodule 1010 includes:

[0189] The processing unit 1310 is used to pre-process the instruction sequence determined by the first determination module 910 and the memory occupancy information determined by the second determination module 920 respectively;

[0190] A first feature extraction unit 1320 is used to extract features from the instruction sequence preprocessed by the processing unit 1310 via a first sub-network in the initial strategy model to obtain a first tensor;

[0191] A second feature extraction unit 1330 is used to extract features from the memory occupancy information preprocessed by the processing unit 1310 via a second sub-network in the initial strategy model to obtain a second tensor;

[0192] The generating unit 1340 is used to perform feature fusion on the first tensor obtained by the first feature extraction unit 1320 and the second tensor obtained by the second feature extraction unit 1330 via the third sub-network in the initial strategy model to obtain a fused tensor, and generate strategy information based on the fused tensor.

[0193] In some optional examples, such as Fig.14 As shown, the evaluation module 940 includes:

[0194] The second determination submodule 1410 is used to determine the estimated execution time of the execution mode determined by the third determination module 930;

[0195] The third determining submodule 1420 is used to determine evaluation information corresponding to the execution mode based on the estimated execution time determined by the second determining submodule 1410.

[0196] Fig.15 It is a schematic diagram of the structure of an instruction execution device provided by some exemplary embodiments of the present disclosure. Fig.15 The apparatus shown may include:

[0197] A fifth determining module 1510 is used to determine a model to be compiled;

[0198] A compiling module 1520, configured to compile the model to be compiled determined by the fifth determining module 1510, and obtain a target instruction sequence corresponding to the model to be compiled determined by the fifth determining module 1510;

[0199] A sixth determination module 1530, used to determine current memory occupancy information of the target instruction sequence hardware obtained by the compilation module 1520 for executing the target instruction sequence;

[0200] The seventh determination module 1540 is used to determine the execution mode corresponding to the target instruction sequence obtained by the compilation module 1520 based on the target instruction sequence obtained by the compilation module 1520, the current memory occupancy information determined by the sixth determination module 1530, and the target policy model.

[0201] In the device of the present disclosure, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve corresponding functions and effects, and the present disclosure does not list them one by one.

[0202] Exemplary Electronic Devices

[0203] Fig.16 The block diagram of an electronic device according to an embodiment of the present disclosure is illustrated. The electronic device 1600 includes one or more processors 1610 and a memory 1620 .

[0204] The processor 1610 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1600 to perform desired functions.

[0205] The memory 1620 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1610 may execute one or more computer program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0206] In one example, the electronic device 1600 may further include: an input device 1630 and an output device 1640 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0207] The input device 1630 may also include, for example, a keyboard, a mouse, etc.

[0208] The output device 1640 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0209] Of course, to simplify, Fig.16 Only some of the components related to the present disclosure in the electronic device 1600 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 1600 may also include any other appropriate components.

[0210] Exemplary computer program products and computer-readable storage media

[0211] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0212] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0213] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0214] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0215] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present disclosure. The specific details disclosed above are only for the purpose of illustration and ease of understanding, not limitation. The above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0216] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for training a target strategy model, comprising: determining a sequence of instructions to be executed; Determining memory usage information of hardware used to execute the instruction sequence; Determining an execution mode corresponding to the instruction sequence based on the instruction sequence, the memory occupancy information and the initial strategy model; Evaluate the execution mode to obtain evaluation information; Iteratively training the initial strategy model based on the instruction sequence, the memory occupancy information, the execution mode, and the evaluation information; In response to the initial policy model after iterative training satisfying a preset training end condition, the initial policy model after iterative training is used as a target policy model.

2. The method according to claim 1, wherein: The iterative training of the initial strategy model based on the instruction sequence, the memory occupancy information, the execution mode and the evaluation information includes: The initial strategy model is iteratively trained based on a reinforcement learning algorithm, using the instruction sequence and the memory occupancy information as state information, the execution mode as action information, and the evaluation information as reward information.

3. The method according to claim 1, wherein: The determining, based on the instruction sequence, the memory occupancy information and the initial policy model, an execution mode corresponding to the instruction sequence includes: Based on the instruction sequence and the memory occupancy information, obtaining the policy information generated by the initial policy model; Based on the policy information, an execution mode corresponding to the instruction sequence is determined.

4. The method according to claim 3, wherein: The determining, based on the policy information, an execution mode corresponding to the instruction sequence includes: Based on the policy information, determining a first number of groups of the instruction sequence; Determine a plurality of grouping modes in which the instruction sequence corresponds to the first number of groups; Selecting a target grouping method from the plurality of grouping methods; Based on the target grouping method, an execution method corresponding to the instruction sequence is determined.

5. The method according to claim 3, wherein: The determining, based on the policy information, an execution mode corresponding to the instruction sequence includes: Based on the policy information, determining target parameter information related to a predetermined operation option; Based on the target parameter information, determining a selection state corresponding to the predetermined operation option; Based on the selection state, an execution mode corresponding to the instruction sequence is determined.

6. The method according to claim 3, wherein: The obtaining, based on the instruction sequence and the memory occupancy information, the policy information generated by the initial policy model includes: Preprocessing the instruction sequence and the memory occupancy information respectively; Performing feature extraction on the preprocessed instruction sequence via a first sub-network in the initial strategy model to obtain a first tensor; Performing feature extraction on the preprocessed memory occupancy information via a second sub-network in the initial strategy model to obtain a second tensor; The first tensor and the second tensor are feature fused via a third subnetwork in the initial strategy model to obtain a fused tensor, and the strategy information is generated based on the fused tensor.

7. The method according to any one of claims 1 to 6, wherein: The evaluating the execution mode to obtain evaluation information includes: Determining an estimated execution time of the execution method; Based on the estimated execution time, evaluation information corresponding to the execution mode is determined.

8. A method for executing an instruction, comprising: Determine the model to be compiled; Compiling the model to be compiled to obtain a target instruction sequence corresponding to the model to be compiled; Determining current memory usage information of hardware used to execute the target instruction sequence; Based on the target instruction sequence, the current memory occupancy information and the target policy model, an execution mode corresponding to the target instruction sequence is determined.

9. A training device for a target strategy model, comprising: A first determination module, used to determine an instruction sequence to be executed; A second determining module, used to determine memory occupancy information of hardware used to execute the instruction sequence determined by the first determining module; a third determining module, configured to determine an execution mode corresponding to the instruction sequence determined by the first determining module based on the instruction sequence determined by the first determining module, the memory occupancy information determined by the second determining module, and an initial policy model; An evaluation module, used for evaluating the execution mode determined by the third determination module to obtain evaluation information; a training module, configured to iteratively train the initial policy model based on the instruction sequence determined by the first determination module, the memory occupancy information determined by the second determination module, the execution mode determined by the third determination module, and the evaluation information obtained by the evaluation module; The fourth determination module is used to use the initial policy model after iterative training by the training module as the target policy model in response to the initial policy model after iterative training by the training module meeting a preset training end condition.

10. An instruction execution device, comprising: A fifth determination module, used to determine a model to be compiled; A compiling module, used for compiling the model to be compiled determined by the fifth determining module to obtain a target instruction sequence corresponding to the model to be compiled determined by the fifth determining module; a sixth determining module, used to determine current memory occupancy information of the target instruction sequence hardware obtained by executing the compiling module; The seventh determination module is used to determine the execution mode corresponding to the target instruction sequence obtained by the compilation module based on the target instruction sequence obtained by the compilation module, the current memory occupancy information determined by the sixth determination module, and the target policy model.

11. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method according to any one of claims 1 to 8.

12. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-8.

Images