A probabilistic decision-making method for biological-like machines

Abstract

The present invention discloses a biological machine probability decision algorithm, which is carried out in the order of behavior decision, parameter adjustment, and action execution, and is repeated. Each complete action sequence is called an execution cycle; in parameter adjustment, for each behavior, each time it is executed, its probability of occurrence decreases by a%; if the behavior is selected for execution in the current cycle, a will increase; if the behavior is not selected for execution in the current cycle, a will decay. The beneficial effects of the present invention are as follows: the constraints can be defined and limited by a number of parameters, and the parameters have a certain meaning, which is convenient for humans to quickly grasp the principle of parameter adjustment; the algorithm should have a certain self-learning ability, and can dynamically optimize the behavior decision based on the artificially defined constraints according to the environment and application scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine behavior decision algorithms, and specifically is a biological-like machine probability decision algorithm. Background Art

[0002] When robots and intelligent agents interact with humans, they need to exhibit biological-like behaviors and decisions in order to gain a sense of intimacy with humans. There are two main types of commonly used machine behavior decision algorithms. One type is a decision mechanism with certain rules, which forms a machine behavior sequence through pre-programming, or pre-compiles a table of external stimulus-machine response correspondence to achieve machine behavior decision-making.

[0003] The other type is random decision-making, which is to predefine the set of machine behaviors corresponding to each external stimulus and the probability of occurrence of each machine behavior in the set, and randomly decide which behavior will occur.

[0004] The first type of behavior decision algorithm with fixed rules is mainly used in situations where machines are strictly required to determine their behavior, such as industrial scenarios. If it is used in scenarios such as robot companions and entertainment robots, it is not easy for humans to accept it as a "companion" because its behavior can be accurately predicted, and it will only be regarded as a machine.

[0005] For the second random decision algorithm, if the probability of each behavior is not strictly limited, it is easy to form an overly random behavior sequence, making it completely unpredictable and causing panic to humans. However, strictly limiting the probability of behavior is a complex and time-consuming task.

[0006] Therefore, a behavior decision algorithm is needed that can form a "basic" predictable behavior through simple parameter adjustment. Moreover, if different parameter values ​​can be set to form machine behavior decisions with different "characters" according to the environment in which each robot is located, it will greatly reduce the debugging workload and enrich the product variety. Summary of the invention

[0007] The purpose of the present invention is to provide a biological machine probability decision algorithm, which solves the problem that the random decision algorithm with strict probability limitation is complicated and time-consuming.

[0008] The object of the present invention is achieved through the following technical solutions:

[0009] A biological-like machine probability decision algorithm is implemented, which is repeated in the order of behavior decision, parameter adjustment, and action execution. Each complete action sequence is called an execution cycle. In parameter adjustment, for each behavior, its probability of occurrence decreases by a% each time it is executed. If the behavior is selected for execution in the current cycle, a will increase. If the behavior is not selected for execution in the current cycle, a will decay.

[0010] Furthermore, the initial value of a is 2 to 5. If the behavior is selected for execution in the current cycle, a will increase by e, where e is a random number between 2 and 5. If the behavior is not selected for execution in the current cycle, a will decay by b%, where b is a random number between 10 and 30.

[0011] Furthermore, in behavioral decision-making, there are active behaviors and passive behaviors. Active behaviors have a probability of occurrence. Executing each behavior consumes energy. The current total remaining energy of the machine is E. The energy levels are divided according to the amount of E. In each energy level, each active behavior has an initial probability of occurrence.

[0012] Furthermore, the sum of the occurrence probabilities of all active behaviors is 1. After the occurrence probabilities of active behaviors change, the occurrence probabilities of all active behaviors are normalized.

[0013] Furthermore, active behaviors include dormant behaviors, the probability of which increases as E decreases, and performing dormant behaviors increases energy.

[0014] Furthermore, energy increases with the increase of sleep time.

[0015] Furthermore, action execution includes active behavior execution and passive behavior execution. After the passive behavior is executed, penalty adjustment is required. When the absolute value of E is less than th, the probability of dormant behavior in all energy levels is increased.

[0016] Furthermore, for energy level m, the probability of dormancy behavior increases to the mth power of c%, where c ranges from 2 to 5.

[0017] Furthermore, the action execution includes active behavior execution and passive behavior execution. After the active behavior is executed, a sleep determination is required. If it is determined to be yes, random fine-tuning is performed, and the probability of the behavior in the energy level is randomly increased or decreased. If it is determined to be no, the next cycle is performed.

[0018] Furthermore, the probability of the behavior in the energy level increases or decreases by d%, where d is a random number between -20 and 20.

[0019] Beneficial effects of the present invention:

[0020] 1. Constraints can be defined and limited by a number of parameters. The parameters have certain meanings, which makes it easier for humans to quickly grasp the principles of parameter adjustment.

[0021] 2. The algorithm should have a certain degree of self-learning ability and be able to dynamically optimize behavioral decisions based on the environment and application scenarios and on the basis of artificially defined constraints.

[0022] The aforementioned main scheme of the present invention and its further options can be freely combined to form multiple schemes, all of which are schemes that can be adopted and claimed for protection by the present invention; and in the present invention, (non-conflicting options) options and other options can also be freely combined. After understanding the scheme of the present invention, those skilled in the art can understand that there are many combinations based on the prior art and common knowledge, all of which are technical schemes to be protected by the present invention, and they are not exhaustively listed here. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0024] The following non-limiting examples serve to illustrate the invention.

[0025] Embodiment 1:

[0026] refer to Figure 1 As shown, a biological-like machine probabilistic decision-making algorithm is performed in the order of behavioral decision-making, parameter adjustment, and action execution, and is repeated. Each complete action sequence is called an execution cycle.

[0027] Action execution includes active behavior execution and passive behavior execution. After active behavior execution, a dormancy determination is required. If it is determined to be yes, random fine-tuning is performed. If it is determined to be no, the next cycle is performed. After passive behavior execution, penalty adjustment is required.

[0028] In behavioral decision making, specifically based on energy E and the probability of active behavior, there are active behaviors and passive behaviors. Active behaviors are actions that the machine does not have to perform, and passive behaviors are actions that the machine must respond to external stimuli. A machine must have a certain number of active and passive behaviors. Active behaviors are recorded as P_ACT_n, and passive behaviors are recorded as N_ACT_n, where n = 0, 1, 2…

[0029] Active behavior has a probability of occurrence, and the sum of the probabilities of all active behaviors is 1. Passive behavior has no probability of occurrence, or in other words, after being stimulated by the outside world, 100% of the actions must be taken.

[0030] Executing each behavior (except sleeping behavior) consumes energy. This "energy" has no direct relationship with the energy consumed by the real machine. It is just a parameter used to make behavioral decisions. The energy consumption of each behavior is set manually. The energy consumed by each active behavior is set to P_ACT_E_n, and the energy consumed by the passive behavior is set to N_ACT_E_n, where n=0, 1, 2...

[0031] The total remaining energy of the machine is E. According to the amount of E, it is divided into several energy levels (0, 1, 2...). It is recommended to divide the energy levels into 0-3 or 0-5, with 0 energy level as the lowest energy level. In each energy level, the probability of each active behavior is recorded as p_n_m, that is, the probability of active behavior n when E is at energy level m. The initial value of this probability is manually specified, and different parameter values ​​correspond to different machine personalities.

[0032] In particular, active behavior must include a behavior, namely, dormant behavior, which does not define energy consumption, but defines the rate of energy growth during dormancy, that is, after the machine performs this operation, the energy will increase as the dormancy time increases. Its probability of occurrence increases as E decreases, and executing dormant behavior will increase energy.

[0033] In parameter adjustment, for each active behavior, each time it is executed, its probability of occurrence decreases by a%, which is called the active behavior occurrence probability correction. The recommended initial value of a is 2 to 5. If the behavior is selected for execution in the current cycle, a will increase by e, where e is a random number between 2 and 5. If the behavior is not selected for execution in the current cycle, a will decay by b%, and the recommended value of b is 10 to 30.

[0034] That is, before the probability adjustment is performed, the probability of a certain active behavior is p, and the calculation process of the probability p' after adjustment is: p'=p*(1-a%).

[0035] Then to update the value of a, do the following:

[0036] If this behavior currently occurs: a'=a+e, if a'>100, then set a'=100.

[0037] If the behavior is not currently occurring: a'=a*(1-b%), if a'<1, set a'=0.

[0038] The other behaviors at this energy level evenly distribute the excess or reduced probabilities so that the total probability of all behaviors is still 1. The probability correction of active behaviors is to avoid the continuous occurrence of the same behavior and increase the randomness of the machine decision-making system.

[0039] After the active behavior is executed, a dormancy determination is required. If it is determined to be yes, random fine-tuning is performed. Random fine-tuning is an optional operation used to further increase the randomness of the decision-making behavior. The probability of this behavior in this energy level increases or decreases by d%.

[0040] That is, for a certain behavior within a certain energy level, a random number d with a value between -20 and 20 is generated, and the probability adjustment formula for the behavior within the energy level is: p'=p*(1+d%), and then the occurrence probability of all behaviors within the energy level is normalized, and finally the next cycle is carried out.

[0041] If the determination is otherwise, directly proceed to the next cycle.

[0042] After the passive behavior is executed, penalty adjustment is required. When the absolute value of E is less than th, that is, abs(E) < th, where th is a threshold set manually, and it is recommended that th <= 10% of the maximum value of E, then increase the probability of the sleep behavior among all energy levels, and the probabilities of other behaviors are reduced on average, while the total behavior occurrence probability still remains 1.

[0043] For the m-th energy level, the formula for adjusting the sleep probability is: p’ = p * (1 + c% ** m), where c is recommended to take values from 2 to 5, p’ is the adjusted sleep behavior probability, and ** is the power operator, so c% ** m means the m-th power of c%.

[0044] This mechanism ensures that the system tends to use energy conservatively to cope with possible emergency events at any time. The specific energy utilization strategy is dynamically adjusted according to the probability of emergency events occurring in the actual environment.

[0045] The basic example of the present invention and its various further selected examples can be freely combined to form multiple embodiments, all of which are embodiments that can be adopted and claimed by the present invention. In the solution of the present invention, each selected example can be arbitrarily combined with any other basic example and selected example.

[0046] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A biological-like machine probability decision-making method, characterized by: Follow the order of behavior decision, parameter adjustment, and action execution, and repeat the operation. Each complete action sequence is called an execution cycle. In parameter adjustment, for each behavior, each time it is executed, its probability of occurrence decreases by a%; if the behavior is selected for execution in the current cycle, a will increase; if the behavior is not selected for execution in the current cycle, a will decay; In behavioral decision-making, there are active behaviors and passive behaviors. Active behaviors have a probability of occurrence. Executing each behavior consumes energy. The current total remaining energy of the machine is E. The energy levels are divided according to the amount of E. In each energy level, each active behavior has an initial probability of occurrence.

2. The biological-like machine probability decision-making method according to claim 1, characterized in that: The initial value of a is 2~5. If the behavior is selected for execution in the current cycle, a will increase by e, where e is a random number between 2 and 5. If the behavior is not selected for execution in the current cycle, a will decay by b%, where b is a random number between 10 and 30.

3. The biological-like machine probability decision-making method according to claim 1, characterized in that: The sum of the occurrence probabilities of all active behaviors is 1. After the occurrence probabilities of active behaviors change, the occurrence probabilities of all active behaviors are normalized.

4. The biological-like machine probability decision-making method according to claim 1, characterized in that: Active behaviors include dormant behaviors, the probability of which increases as E decreases, and performing dormant behaviors increases energy.

5. The biological-like machine probability decision-making method according to claim 4, characterized in that: Energy will increase as the sleep time increases.

6. The biological-like machine probability decision-making method according to claim 1, characterized in that: Action execution includes active behavior execution and passive behavior execution. After passive behavior execution, penalty adjustment is required. When the absolute value of E is less than th, the probability of dormant behavior in all energy levels is increased.

7. The biological-like machine probability decision-making method according to claim 6, characterized in that: For energy level m, the probability of dormancy behavior increases by c% to the mth power, where c is 2~5.

8. The biological-like machine probability decision-making method according to claim 1, characterized in that: Action execution includes active behavior execution and passive behavior execution. After active behavior execution, a sleep determination is required. If it is determined to be yes, random fine-tuning is performed, and the probability of this behavior in this energy level is randomly increased or decreased. If it is determined to be no, the next cycle is carried out.

9. The biological-like machine probability decision-making method according to claim 8, characterized in that: The probability of this behavior at this energy level increases or decreases by d%, where d is a random number between -20 and 20.

Images