An acceleration method, terminal device and storage medium in intelligent decision-making

Abstract

The present invention discloses an acceleration method, terminal device and storage medium in intelligent decision-making, and the method includes: step 1, optimizing multiple parameters of a target object, and obtaining the historical iteration value of each parameter within the current optimization number interval; step 2, inputting the historical iteration value of each parameter into the trained policy network model in turn, and obtaining the posterior probability that each parameter converges to each discrete candidate state; step 3, judging whether the posterior probability corresponding to each parameter is greater than a preset early threshold value, if so, fixing the state of the parameter, obtaining the historical iteration values ​​of the remaining parameters except the parameters with fixed state within the next optimization number interval, repeating steps 2 to 3, until the preset termination condition is reached, if not, obtaining the historical iteration value of each parameter within the next optimization number interval, repeating steps 2 to 3, until the preset termination condition is reached. The method of the present invention ensures the accuracy of decision-making while improving the speed.

Description

Technical Field

[0001] The invention discloses an acceleration method in intelligent decision-making, terminal equipment and computer-readable storage medium, belonging to the field of intelligent decision-making. Background Art

[0002] Intelligent decision-making refers to the process of using a variety of intelligent technologies and tools to model, analyze and make decisions on relevant data based on established goals. It is used to solve increasingly complex production and life problems in the new growth era. The decision-making problem can be standardized as an integer programming problem.

[0003] Integer programming is a general modeling tool for discrete or combinatorial optimization problems with wide applications. Generally speaking, integer programming solution algorithms can be divided into two categories: exact methods and approximate methods. Although exact methods can obtain optimal solutions, they repeatedly solve relaxed linear problems, which is time-consuming.

[0004] Among the approximate methods, the Lp-Box Alternating Direction Method of Multipliers (ADMM) has attracted more and more attention. Lp-Box ADMM replaces the binary constraint with the intersection of the box constraint and the Lp norm ball constraint equivalently. However, it is found in actual research that the Lp-Box ADMM method converges at a sublinear rate, that is, most of the variables fluctuate around their final convergence state in a very long iteration. To further accelerate this approximate algorithm, a possible solution in the prior art is "early stopping" or "early exiting".

[0005] However, this "early stopping" or "early exiting" has a trade-off between efficiency and accuracy: if you choose to stop earlier, the running time can be saved more, but the accuracy will be reduced more. Summary of the invention

[0006] The purpose of this application is to provide an acceleration method, terminal device and computer-readable storage medium in intelligent decision-making to solve the technical problem of reduced accuracy when using early stopping or early exiting acceleration approximation methods in the decision-making process.

[0007] A first aspect of the present invention provides an acceleration method in intelligent decision making, comprising:

[0008] Step 1: Optimize multiple parameters of the target object and obtain the historical iteration value of each parameter within the current optimization number interval;

[0009] Step 2: input the historical iteration value of each parameter into the trained policy network model in turn to obtain the posterior probability that each parameter converges to each discrete candidate state;

[0010] Step 3, determine whether the posterior probability corresponding to each of the parameters is greater than a preset early threshold value. If so, fix the state of the parameter, obtain the historical iteration values ​​of the remaining parameters except the parameters with fixed states within the next optimization number interval, repeat steps 2 to 3 until the preset termination condition is reached. If not, obtain the historical iteration values ​​of each of the parameters within the next optimization number interval, repeat steps 2 to 3 until the preset termination condition is reached.

[0011] Preferably, the state of fixing the parameter specifically includes:

[0012] The state of the parameter is fixed to a discrete candidate state corresponding to the posterior probability greater than a preset early threshold.

[0013] Preferably, the historical iteration values ​​of each parameter are sequentially input into the trained policy network model to obtain the posterior probability that each parameter converges to each discrete candidate state, specifically including:

[0014] Inputting the historical iteration value of each parameter into the trained strategy network model in turn;

[0015] Reconstructing nodes based on the historical iteration values;

[0016] Based on the reconstructed data of the nodes, an attention mechanism is used to determine the posterior probability that each of the parameters converges to each discrete candidate state.

[0017] Preferably, based on the reconstructed data of the node, the attention mechanism is used to determine the posterior probability that each parameter converges to each discrete candidate state, specifically including:

[0018] Based on the reconstructed data of the nodes, the attention mechanism is used to determine the output data of each node;

[0019] The output data of each node is reduced in dimension to obtain the posterior probability that each parameter converges to each discrete candidate state.

[0020] Preferably, the strategy network model includes an attention layer and an output layer;

[0021] The training process of the strategy network model includes:

[0022] Acquire a training data set, wherein the training data set includes a real action of a parameter and a real state corresponding to the real action; the real action refers to whether to determine the state of the parameter in advance;

[0023] Inputting the real action into the attention layer to obtain the predicted state of the parameter;

[0024] According to the predicted state and the corresponding actual state, combined with the loss function, the policy network model is trained to obtain a trained policy network model.

[0025] Preferably, the loss function is a weighted binary cross entropy loss function.

[0026] Preferably, the loss function is determined according to a first formula, which is:

[0027]

[0028] In the formula, J θ is the loss function, θ is the parameter of the policy network, ω e,r,i is the weight of the i-th parameter of the r-th training block of the e-th training instance in the training dataset, q e,r,i is the binary cross entropy of the i-th parameter of the r-th training block of the e-th training instance in the training data set, N is the total number of the training instances, γ is the total number of the training blocks in each of the training instances, and n is the total number of the parameters in each of the training blocks.

[0029] Preferably, the binary cross entropy is determined according to a second formula, which is:

[0030] q e,r,i =a * logπ θ (a|s)+(1-a * )log(1-π θ (a|s))

[0031] In the formula, π θ (a|s) is the policy network π with parameter θ θ The probability that the current action a is set to state s earlier, a * is the real action corresponding to action a.

[0032] A second aspect of the present invention provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0033] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0034] Compared with the prior art, the acceleration method, terminal device and storage medium in intelligent decision-making of the present invention have the following beneficial effects:

[0035] The present invention proposes an accelerated method for solving integer programming based on early fixing in intelligent decision-making, wherein the parameters are independent of each other, and for a single parameter, the present invention predicts and determines whether it is early fixed based on its values ​​in a series of past iterations. Compared with the "early exit" or "early stop" in the prior art, the present invention proposes the concept of "early fixing", which not only pays attention to the depth of iteration, but also pays attention to the dimension of the parameters in the iteration process, so as to achieve the effect of improving the optimization speed of the approximate method while ensuring the accuracy. The method of the present invention is of great significance for improving the efficiency of solving problems with integer programming in intelligent decision-making, especially in the fields of computer vision or machine learning. Specifically, if the method is applied to the field of image segmentation, the speed of image segmentation can be improved and the accuracy of image segmentation can be guaranteed.

[0036] The present invention formulates the process of solving integer programming problems as a Markov decision process, specifically, the parameters are independent of each other, and for a single parameter, it is predicted and judged whether it is determined in advance based on its past series of iterative values. The present invention adopts the learning technology of the attention mechanism in the policy network to determine whether to determine the parameter in advance. Since the attention mechanism does not depend on the order between words, but mines information by calculating the similarity between words, there will be no information loss. At the same time, it can be calculated in parallel, which greatly improves the calculation efficiency, thereby achieving the optimization speed of the approximate method while ensuring accuracy. At the same time, when training the policy network model, the present invention uses behavior cloning as an imitation learning method for training, which can improve the training speed. The present invention uses weighted binary cross-entropy loss in the training process to further ensure the accuracy of the output results of the trained policy network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the acceleration method in intelligent decision making in an embodiment of the present invention;

[0038] Figure 2 A framework diagram of the strategy network model provided by the present invention;

[0039] Figure 3 This is a diagram showing the advantages of the early determination of the present invention compared to the early stopping. DETAILED DESCRIPTION

[0040] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0041] At present, China's industry is at an important stage of development from digitalization to intelligence. Every scenario of intelligent development faces a large number of decision-making needs, which are often in the millions or tens of millions, and require real-time decision-making. The original human decision-making method cannot meet this requirement at all. Therefore, intelligent decision-making is becoming more and more important in the current society. For example, in the automotive industry, through AI decision-making model analysis, sales conversion rate is predicted, follow-up priority is defined, and input-output ratio is improved; in the tourism industry, based on the analysis of airline member data, intelligent algorithms extract the characteristics of the customer group with the intention to purchase air tickets, and derive personalized recommendation models and demand forecasts for air tickets, thereby improving marketing and operation efficiency; in the financial industry, banks use AI decision-making technology and intelligent algorithms to achieve more accurate and comprehensive customer service, marketing management, and risk management; in the e-commerce industry, decision-making intelligence is integrated with machine learning and deep learning algorithms to deeply analyze users, items, and scenarios, and make personalized product recommendations to improve user purchasing efficiency and customer experience.

[0042] The present invention proposes an acceleration method in intelligent decision making, such as Figure 1 As shown, including:

[0043] Step 1: Optimize multiple parameters of the target object and obtain the historical iteration value of each parameter within the current optimization number range.

[0044] In the embodiment of the present invention, the parameters of the target object are optimized using the approximate method in the existing integer programming. The approximate method may be the alternating direction method (ADMM) based on multipliers and its related improved methods, such as BetheADMM, BregmanADMM, Lp-BoxADMM, etc., which are not limited in the present invention.

[0045] In the embodiment of the present invention, the optimization number interval is a plurality of continuous intervals, each of which can be of equal length or unequal length and can be set as required, and the present invention does not limit this. For example, the current optimization number interval is an interval consisting of 1-100 optimization processes, and the next optimization number interval is an interval consisting of 101-300 optimization processes.

[0046] The historical iteration value in the embodiment of the present invention includes each iteration value of each parameter in the iteration process within the optimization number interval.

[0047] Step 2: Input the historical iteration value of each parameter into the trained policy network model in turn to obtain the posterior probability that each parameter converges to each discrete candidate state.

[0048] The framework of the strategy network model in the present invention is as follows Figure 2 As shown, it at least includes an input layer, a node reconstruction layer, an attention layer, an MLP layer and an output layer. The model is used to determine whether a single parameter is fixed early during the iteration process. Unlike the early stopping and other works in the prior art that regard the parameter set as a whole, the early determination framework of the present invention independently judges and analyzes each parameter. It takes the iterative value of the parameter as input and gives the probability as output. Specifically: the iterative value of the parameter of the past multiple iterations is input, and the policy network will evaluate the posterior probability of each parameter converging to all discrete candidate states (that is, eventually converging to 0 or 1).

[0049] The process of using the policy network model to obtain the posterior probability of each parameter converging to each discrete candidate state is:

[0050] The historical iteration value of each parameter is input into the trained strategy network model through the input layer in turn;

[0051] The node reconstruction layer reconstructs nodes based on historical iteration values;

[0052] Based on the reconstructed data of the nodes, the attention layer uses the attention mechanism to determine the posterior probability of each parameter converging to each discrete candidate state, including:

[0053] Based on the reconstructed data of the nodes, the attention layer uses the attention mechanism to determine the output data of each node;

[0054] The output data of each node is reduced in dimension using the MLP layer to obtain the posterior probability that each parameter converges to each discrete candidate state.

[0055] Step 3: Determine whether the posterior probability corresponding to each parameter is greater than the preset early threshold. If so, fix the state of the parameter, obtain the historical iteration values ​​of the remaining parameters except the parameters with fixed state in the next optimization number interval, repeat steps 2 to 3 until the preset termination condition is reached. If not, obtain the historical iteration value of each parameter in the next optimization number interval, repeat steps 2 to 3 until the preset termination condition is reached.

[0056] The state of the parameter is fixed, specifically, by fixing the state of the parameter to a discrete candidate state corresponding to a posterior probability greater than a preset early threshold.

[0057] The preset termination condition is that the stopping criterion of the approximate method reaches convergence or all parameters are fixed.

[0058] For example, when the preset early threshold is 0.8, the posterior probability of the parameter convergence to 0 obtained by the policy network model is 0.9, and the posterior probability of convergence to 1 is 0.1, then the state of the parameter is fixed to 0.

[0059] The pseudo code of the working process of the policy network model in the present invention is shown in Table 1, which takes the historical iteration value of the parameter within the current optimization number interval as input and gives the posterior probability as output. The iterative value of the variable in the past multi-step iteration is input, and the policy network model will evaluate the posterior probability of each parameter converging to all discrete candidate states (that is, converging to 0 or 1 in the end). If the posterior probability of a state exceeds a threshold, that is, the early threshold, the action of fixing this parameter to that discrete state is executed, and this parameter will no longer participate in the iteration in the future; otherwise, no fixing operation will be performed, the iteration will continue, and the parameter will be further updated.

[0060] Table 1 Pseudo code of the strategy network working process

[0061]

[0062] According to the policy network model, the state of the parameters can be updated, thereby converting some parameters from free parameters to fixed parameters. The iteration will terminate when the stopping criterion reaches convergence or all variables are fixed. As a Markov decision process, this early determination process is divided into stages, and a part of the variables are determined early in each stage, so that the problem size after each round of early determination becomes smaller, thereby accelerating the optimization process.

[0063] The advantages of the early setting of the present invention compared to the early stopping are as follows: Figure 3 shown.

[0064] Furthermore, the training process of the policy network model in step 2 of the present invention includes:

[0065] Obtain a training data set, the training data set includes multiple training instances, each training instance includes multiple training blocks, each training block includes multiple parameters, each parameter includes a real action and a real state corresponding to the real action; the real action refers to whether the state of the parameter needs to be determined in advance;

[0066] Input the real action into the attention layer of the initially constructed policy network model to obtain the predicted state of the training instance;

[0067] According to the predicted state and the corresponding actual state, combined with the loss function, the policy network is trained to obtain a trained policy network.

[0068] The loss function is the weighted binary cross entropy loss function, specifically:

[0069]

[0070] In the formula, J θ is the loss function, θ is the parameter of the policy network, ω e,r,i is the weight of the i-th parameter of the r-th training block of the e-th training instance in the training dataset, q e,r,i is the binary cross entropy of the i-th parameter of the r-th training block of the e-th training instance in the training dataset, N is the total number of training instances, γ is the total number of training blocks in each training instance, and n is the total number of parameters in each training block.

[0071] where q e,r,i As shown in formula (2):

[0072] q e,r,i =a * logπ θ (a|s)+(1-a * )log(1-π θ (a|s)) (2)

[0073] In the formula, π θ (a|s) is the policy network π with parameter θ θ The probability that the current action a is set to state s earlier, a * is the real action corresponding to action a.

[0074] The present invention adopts an imitation learning scheme to train the early-determined strategy network model. Instead of training the network from scratch, the early-determined features are learned and directly mapped to convergence, thereby achieving the purpose of acceleration.

[0075] Specifically, the policy network of the present invention is trained by behavior cloning as an expert-driven imitation learning method. The present invention uses the accelerated approximation method itself as the expert rule, first runs the expert strategy on a set of several training instances, and selects a data set of expert <state-action> pairs. The strategy is learned by minimizing the weighted binary cross entropy loss.

[0076] A second aspect of the present invention provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0077] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0078] The method of the present invention will be described in detail below with more specific embodiments.

[0079] Step 1: Use the ADMM method to optimize the image segmentation problem and obtain multiple parameters in the optimization process. In this example, the image segmentation problem contains 10,000 variable parameters. Preset the optimization intervals, each interval is equal in length, and the interval is 10 iterations. Obtain the historical iteration value of each parameter in the current optimization interval.

[0080] Step 2: Input the historical iteration value of each parameter into the trained policy network model in turn to obtain the early determination probability of the parameter. Taking a single parameter as an example, after the first round of early determination judgment, the probability of the parameter converging to 1 is 0.95, and the probability of converging to 0 is 0.05. Assuming the judgment threshold is 0.9, then 0.95>0.9, that is, the parameter is early determined and converges to 1; taking another parameter as an example, after the first round of early determination judgment, the probability of the parameter converging to 1 is 0.8, and the probability of converging to 0 is 0.2. Assuming the judgment threshold is 0.9, then 0.8<0.9, that is, the early determination operation is not performed on the parameter.

[0081] Step 3: Repeat step 2 until all parameters are fixed or the optimization algorithm converges.

[0082] The above are only a few embodiments of the present application and do not constitute any form of limitation to the present application. Although the present application is disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the profession, without departing from the scope of the technical solution of the present application, using the technical content disclosed above to make slight changes or modifications are equivalent to equivalent implementation cases and fall within the scope of the technical solution.

Claims

1. An acceleration method in intelligent decision making, characterized in that: include: Step 1: Optimize multiple parameters of the target object and obtain the historical iteration value of each parameter within the current optimization number interval; Optimize multiple parameters of the target object, specifically: optimize image segmentation using the ADMM method; Step 2: input the historical iteration value of each parameter into the trained policy network model in turn to obtain the posterior probability that each parameter converges to each discrete candidate state; Step 3: determine whether the posterior probability corresponding to each of the parameters is greater than a preset early threshold value. If so, fix the state of the parameter, obtain the historical iteration values ​​of the remaining parameters except the parameters with fixed states within the next optimization number interval, and repeat steps 2 to 3 until the preset termination condition is reached. If not, obtain the historical iteration value of each of the parameters within the next optimization number interval, and repeat steps 2 to 3 until the preset termination condition is reached. The state of fixing the parameter specifically includes: The state of the parameter is fixed to a discrete candidate state corresponding to the posterior probability greater than a preset early threshold.

2. The acceleration method in intelligent decision-making according to claim 1, characterized in that: The historical iteration values ​​of each parameter are sequentially input into the trained policy network model to obtain the posterior probability that each parameter converges to each discrete candidate state, specifically including: Inputting the historical iteration value of each parameter into the trained strategy network model in turn; Reconstructing nodes based on the historical iteration values; Based on the reconstructed data of the nodes, an attention mechanism is used to determine the posterior probability that each of the parameters converges to each discrete candidate state.

3. The acceleration method in intelligent decision-making according to claim 2, characterized in that: Based on the reconstructed data of the node, the attention mechanism is used to determine the posterior probability that each parameter converges to each discrete candidate state, specifically including: Based on the reconstructed data of the nodes, the attention mechanism is used to determine the output data of each node; The output data of each node is reduced in dimension to obtain the posterior probability that each parameter converges to each discrete candidate state.

4. The acceleration method in intelligent decision-making according to claim 1, characterized in that: The strategy network model includes an attention layer and an output layer; The training process of the strategy network model includes: Acquire a training data set, wherein the training data set includes a real action of a parameter and a real state corresponding to the real action; the real action refers to whether to determine the state of the parameter in advance; Inputting the real action into the attention layer to obtain the predicted state of the parameter; According to the predicted state and the corresponding actual state, combined with the loss function, the policy network model is trained to obtain a trained policy network model.

5. The acceleration method in intelligent decision-making according to claim 4, characterized in that: The loss function is a weighted binary cross entropy loss function.

6. The acceleration method in intelligent decision-making according to claim 5, characterized in that: The loss function is determined according to a first formula, which is: ， In the formula, is the loss function, are the parameters of the policy network, The training data set The first training instance The training block The weight of the parameters, The training data set The first training instance The training block The binary cross entropy of the parameters, is the total number of training instances, is the total number of the training blocks in each of the training instances, is the total number of parameters in each of the training blocks.

7. The acceleration method in intelligent decision-making according to claim 6, characterized in that: The binary cross entropy is determined according to a second formula, which is: ， In the formula, The parameters are Strategy Network Current Action Early set status The probability of For and Action Corresponding real actions.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images