Method for controlling an actuator by means of a control system and system for controlling an actuator

Abstract

The present invention relates to a method for controlling an actuator, the method comprising: - a phase of generating a control law for controlling the actuator for a performance objective, the generation phase comprising the steps of: - collecting signals from the measurement sensors; - creating fuzzy sets corresponding to the collected signals; - generating the associated rule or creating new fuzzy sets; - implementing reinforcement learning on the fuzzy sets comprising an operation of modifying the position of the centre and the width of the fuzzy sets; - converting the result of the reinforcement learning into a set of control rules forming the control law; and - a control phase in which the control system applies the control law.

Description

[0001]Method for controlling an actuator by a control system and system for controlling an actuator The present invention relates to a method for controlling an actuator by a control system. It also relates to a system for controlling an actuator. In the field of control systems, it is common to seek to control a process, often based on physical quantities, with the objective of regulating the behavior of a system. The development of transparent and interpretable artificial intelligence models has become essential in recent years since AI is becoming an essential tool in more and more fields. These models make it possible to achieve higher performance than models requiring complete knowledge of a system. Trusted artificial intelligence models are increasingly critical in critical or sensitive fields. To inspire confidence in the operator or user of the system,Two properties are required: the user's ability to anticipate the system's decisions and the ease of understanding a result or decision made by the system that does not conform to what was expected. This implies that the artificial intelligence model created is interpretable. For this, it is known to carry out machine learning of an interpretable model for control systems. There are several machine learning algorithms applicable to these systems, including evolutionary algorithms and reinforcement learning algorithms. In particular, hybrid algorithms between rule-based models and evolutionary or reinforcement learning algorithms are used. However, in practice, it is observed that similar rules obtained by these algorithms are not consistent with each other. Thus,for the following two rules: - rule 1: IF low temperature AND high humidity THEN slow element speed, - rule 2: IF high temperature AND high humidity THEN fast element speed, it is observed that the definition of high humidity is not the same between the two rules. Furthermore, due to the overlap of the rules, it is not possible to easily determine the causes of the malfunction of the control system. There is therefore a need for a method of controlling an actuator by a control system having better reliability. To this end, the description describes a method of controlling an actuator by a control system, the control method comprising: - a phase of generating a control law for the actuator, the control law controlling the actuator with a view to a performance objective from values ​​taken by measurement sensors,the generation phase comprising the steps of: - collecting signals from the measurement sensors, - creating fuzzy sets corresponding to the collected signals, - when a truth value associated with a fuzzy set exceeds a threshold value, generating the associated rule and otherwise, creating new fuzzy sets, - implementing reinforcement learning to the fuzzy sets comprising several iterations of an application of an action and updates of the action-state values, to determine the actions to be carried out to obtain the performance objective as a function of the fuzzy sets, the implementation comprising at each iteration an operation of modifying at least one of the position of the center and the width of the fuzzy sets to obtain modified fuzzy sets on which the reinforcement learning is applied at the following iteration,- converting the result of reinforcement learning into a set of control rules forming the control law, and - a control phase during which the control system applies the control law to the actuator. A truth value associated with a fuzzy set represents the probability that the fuzzy set is true. In binary logic, the truth value would be 0 or 1 whereas in fuzzy logic, as in the case of the present invention, the truth value is a real value between 0 and 1. According to particular embodiments, the control method has one or more of the following characteristics, taken in isolation or in all technically possible combinations: - during the modification operation, both the position of the center and the width of each fuzzy set is modified. - during the modification operation,the time difference errors of each of the different state-action associations are calculated and the modified position of the center depends on the calculated time difference errors. - the modified position of the center is the sum of the current position of the center plus the arithmetic mean of the time difference errors. - during the modification operation, the expected time differences of each of the different state-action associations are calculated and the modified width is a function of the expected time differences. - the modified width is the sum of the current width plus the arithmetic mean of the expected time differences. - the implementation further includes an operation for testing the usefulness of deleting each fuzzy set,the fuzzy set being deleted if the test is validated. The description also relates to a control system adapted to implement: - a phase of generating a control law for the actuator, the control law controlling the actuator with a view to a performance objective from values ​​taken by measurement sensors, the generation phase comprising the steps of: - collecting signals from the measurement sensors, - creating fuzzy sets corresponding to the collected signals, - when a truth value associated with a fuzzy set exceeds a threshold value, generating the associated rule and otherwise, creating new fuzzy sets, - implementing reinforcement learning on the fuzzy sets comprising several iterations of an application of an action and updates of the action-state values, to determine the actions to be carried out to obtain the performance objective according to the fuzzy sets,the implementation comprising at each iteration an operation of modifying at least one of the position of the center and the width of the fuzzy sets to obtain modified fuzzy sets on which the reinforcement learning is applied at the following iteration, - conversion of the result of the reinforcement learning into a set of control rules forming the control law, and - a control phase during which the control system (12) applies the control law to the actuator. In the present description, the expression "specific to" means indifferently "adapted for", "adapted to" or "configured for". Characteristics and advantages of the invention will appear on reading the description which follows, given solely by way of non-limiting example, and made with reference to the appended drawings, in which: - figure 1 is a schematic representation of an automated system,- Figure 2 is a schematic representation of the different positions of a pendulum, - Figure 3 graphically illustrates the creation of a new fuzzy set in the context of the pendulum of Figure 2, - Figure 4 is a schematic representation of an operation for modifying the position of the center and the width of fuzzy sets in the implementation of a DFQL algorithm, - Figure 5 is a schematic representation of an operation for modifying the position of the center and the width of fuzzy sets in the implementation of a part of a method for controlling a part of the system of Figure 1, and - Figure 6 is a schematic representation of a system and a computer program product. A control method is now described. The control method is a method for controlling an actuator 10 by a control system 12. The actuator 10 is, in the sense of automated systems, an element capable of performing an action. For example,an actuator 10 is a cylinder, a rotor or an effector. The control system 12 is a system suitable for controlling the actuator 10, in particular a controller. The control is obtained by sending a control signal. The actuator 10 and the control system 12 are part of an automated system 14 as shown schematically in FIG. 1. The automated system 14 also comprises measurement sensors 16 (a variant with a single sensor is possible) and a comparison unit 18. The measurement sensors 16 are suitable for measuring elements relating to the environment of the actuator 10 or to its operation. The output of the measurement sensors 16 is compared to a setpoint C which is sent to the comparison unit 18. In the example proposed,the comparison unit 18 is a subtractor suitable for obtaining the difference between the setpoint C and the outputs of the measurement sensors 16. It is assumed here that the setpoint C and the outputs of the measurement sensors 16 have been made comparable by processing ensuring that these elements are expressed in the same space. For example, if the setpoint C is a voltage setpoint and the measurement sensor 16 is a temperature, either the setpoint C is converted into a temperature or the temperature signal is converted into a voltage. Some specific examples of control of an automated system 14 are given in the following: - in the case of anti-lock braking, the control of the brakes in dangerous situations as a function of the speed and acceleration of the car and the speed and acceleration of the wheels, - in the case of automatic transmission, the control of fuel injection and ignition as a function of the throttle setting,of the cooling water temperature or rpm, - in the case of automatic vehicles, speed control depending on engine load, driving style and road conditions, - in the case of a photocopier, drum tension control depending on image density, humidity and temperature, - in the case of cruise control, accelerator control to regulate the speed and acceleration of the car, or - in the case of a dishwasher, control of the cleaning cycle, rinsing and washing strategies depending on the number of dishes and the amount of food on the dishes. Examples could also be derived from other very different contexts such as microwave ovens, computers, engraving, trains or autonomous robots to name but a few. Before describing the control process further,It is appropriate to introduce notions on which the present method is based. It is possible to describe an automated system in terms of its state on the one hand and the actions that the system can perform on the other. In such a formalism, each action-state pair is associated with a real number which is called the q-value (or sometimes the q-value). The q-value measures the quality of an action performed in a given state of the system. It can be noted that for cases with a limited number of pairs, it is possible to establish tables of q-values. This is no longer possible when the number of states becomes too large. This is particularly the case when the actions and / or states are continuous and not discrete. Therefore,an approximation function is used to limit the occupied memory space and the processing time. This approximation function then gives a value q even for action-state pairs that are not observed in practice. A fuzzy inference system makes it possible to operate in the case of continuous spaces for actions and / or states. A fuzzy inference system is often referred to by the abbreviation FIS in reference to the corresponding English term "Fuzzy Inference System". Also, in the following, such a system will be referred to as a FIS system. More precisely, fuzzy logic allows the transition from numerical data to semantic data. This means that fuzzy logic makes it possible to convert a numerical value into a degree of truth of belonging to a semantic value. Thus, a numerical data ^^, ^^ is translated into truth value ^^ ^^, ^^ ( ^^ ^^ ) of the semantic variable ^^ ^^, ^^ . ^^ ^^, ^^denotes the j-th semantic variable of the i-th rule. The indices i and j are here two non-zero natural integers. More precisely, as far as the index i is concerned, this index varies from 1 to n, n being the number of rules. A FIS system is a set of fuzzy rules, the fuzzy rules being conjunctions of semantic variables. Generically, a fuzzy rule indexed i is written in the following form: IF premise THEN conclusion Where IF / THEN denotes the IF / THEN operation in English terminology. The premises are the conjunction of fuzzy sets characterizing a property of each coordinate of the input vector. The premise of a rule i is generally written as: s1est ^^ ^^,1 AND… AND s n is ^^ ^^, ^^ In this expression "ET" denotes the AND operation in English terminology. Furthermore, s corresponds to the state vector of the environment. Each coordinate of the vector s is noted s i, so that s1 is the first coordinate of the state vector s and s n the n-th coordinate of the state vector s. The state vector s is received by collecting data from the measuring sensors 16. The strength of a premise ^^ ^^ is defined as the degree of membership of s in the fuzzy set that results from the conjunction of the fuzzy sets of the premise. This is written mathematically as The conclusions vary depending on the actions chosen for each of these premises. The conclusion is generally written as: a est ^^ ^^ ^^ ^^ here denotes the value of the action. For example, if the action is to increase the temperature, ^^ ^^takes a value corresponding to the increase in temperature to be applied to the environment. In summary, the notation used here reflects the fact that a is the value corresponding to the conclusion of rule i. It is conceivable that the FIS system uses a reinforcement algorithm, and more precisely a Q-learning algorithm better known by the corresponding English term "Q-Learning". Q-Learning makes it possible to learn a strategy, which indicates which action to perform in each state of the system. Q-Learning works by learning an action-state value function denoted Q which makes it possible to determine the potential gain, that is to say the long-term reward provided by the choice of a certain action in a certain state by following an optimal policy.When this action-state value function is known or learned by the agent, the optimal strategy can be constructed by selecting the action with the maximum value for each state, i.e., by selecting the action a that maximizes the value Q(s,a) when the agent is in state s. Because this algorithm is used for fuzzy sets, it is customary to indicate that the FIS system uses an FQL algorithm. The abbreviation FQL refers to the English term "Fuzzy Q-Learning" which could literally be translated as Q-learning for fuzzy sets. The FQL algorithm is thus an approach for learning a set of fuzzy rules by reinforcement. In this approach, agents represent different rules that are evaluated by different q values. Additionally, FQL uses Takagi-Sugeno FIS of order 0. The order refers to the degree of the conclusions which are polynomials in the coefficients of the input vector.Thus, since the conclusions are constants, we associate them directly with a sample of actions A = {A , · · · ,A} taken from the set of possible actions in the environment. This sample is independent of the rule so that the q-values ​​of the lookup table are associated with the premises on the rows and with each action of the sample on the columns. Finally, the activation function is of the form: ^^ ^^ is the strength of the premise mentioned above. It is expressed as a function of the operator dedicated to the conjunction, for example ×. In this case, ^^ ^^ = ∏ ^^^^=1 ^^ ^^, ^^ ( ^^ ^^) where the ^^ ^^ are the coordinates of the input vector. These mathematical elements correspond to the fact that the actions (the conclusions of the rules) are constants. The DFQL algorithm is an improvement of this algorithm using a self-adjusting FIS system based on reinforcement signals.The abbreviation DFQL refers to the English term "Dynamic Fuzzy Q-Learning" which could literally be translated as dynamic Q-learning for fuzzy sets. The DFQL algorithm is thus an extension of the FQL algorithm allowing the creation of the set of rules of the FIS system during training. In the DFQL algorithm, the actions will be associated with premises with respect to the q-values ​​(or q). These values ​​are calculated throughout the training phase in a look-up table (more often referred to by the corresponding English term "look-up table") which has the actions in columns and the premises in rows. This table constitutes Q. The activation function calculates the degree of truth of the premise. Thus, unlike the FQL algorithm, the DFQL algorithm does not involve the intervention of a business expert either at the start or during training and only creates the rules that seem useful.To do this, instead of relying on a set S fixed at the start, it is a set St that is variable and continuously adapted during additional steps compared to Q-Learning. This DFQL algorithm has six successive steps. These six successive steps are as follows: - first step: observation of the data, - second step: generation of rules if necessary, - third step: selection and application of an action, - fourth step: updating of the q values, - fifth step: refinement of the fuzzy sets used by the premises, and - sixth step: deletion of the fuzzy sets deemed useless if there are any. To fully understand this succession of six steps, it will now be detailed through a particular example. This example is that of the control of a pendulum.The goal of this example is to bring the pendulum to equilibrium (dotted line in Figure 2) from a random position and to maintain it in equilibrium by applying a force to the pendulum. Such an example corresponds to an environment in which both inputs and actions are continuous. The first step is to observe data to know the state of the environment. In this example, there are three inputs. The first input is the cosine of the angle between the pendulum and the equilibrium position. This first input is written as ^^ ^^ ^^( ^^). The second input is the sine of the angle between the pendulum and the equilibrium position. This second input is written as ^^ ^^ ^^( ^^). The third input is the angular velocity of the pendulum. This third input is written as ^^̇. These are transmitted to the learning agent. The learning agent represents the FIS.The learning agent takes the inputs and decides on an action and depending on the reward it updates the Q function. The second step corresponds to a generation of rules if necessary. This second step is effectively implemented only if the input data does not activate rules. More precisely, the condition for implementing the second step is that the values ​​of the input data do not allow exceeding a certain threshold of truth value for a dimension of the state. In such a hypothesis, fuzzy sets corresponding to the input data are created. The new fuzzy set created is centered on the input value corresponding to the dimension. An example is shown in Figure 3 where the two fuzzy sets present for the dimension ^^ ^^ ^^. ( θ )do not allow to reach the minimum threshold of truth value (left graph in Figure 3). This leads to the creation of a new fuzzy set centered on the value ^^ ^^ ^^ ( ^^ ) received as input (right graph in Figure 3). In the third step, reinforcement learning via the q value allows us to obtain which action is to be implemented while the fuzzy rules indicate the associated weighting. In this case, the Γ value of -2 is weighted by 0.5, the Γ value of 0 is weighted by 0.1 and the Γ value of 2 is weighted by 0.5. With the previous notations, the values ​​of -2, 0 or 2 correspond to a ^^ ^^ while values ​​of 0.1 or 0.5 correspond to a ^^ ^^. The fourth step consists of an update. More precisely, the q values ​​are updated to adapt the choice of action made in the third step according to the reward received. The reward is, for example, defined by the domain experts according to the desired objective. The fifth refinement step consists of modifying the centers and widths of the fuzzy sets to better correspond to the environment to be controlled. With reference to Figure 4 (top part), the centers of the fuzzy set updates are averaged for each rule in which the fuzzy set intervenes. We can note ^^ ^^, ^^ the position of the center corresponding to the set ^^ ^^,1 at time t and ^^ ^^, ^^ the position of the center corresponding to the set ^^ ^^,1 at the same time t. In this case, for illustrative purposes, it is assumed that ^^ ^^, ^^ = ^^ ^^, ^^. The fifth step allows to obtain respectively the position of the center ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1 at time t+1 and that of the center ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1 at this same instant t+1. As visible in figure 4 (upper part), a correction specific to each center is applied, which is written mathematically for the position of the center corresponding to the set ^^ ^^,1: In this equation, ( Δ ^^ ) ^^, ^^ corresponds to the value of the update of the position of the center of the Gaussian corresponding to the set ^^ ^^,1 at time t. With a similar notation, it also comes for the position of the center corresponding to the set ^^ ^^,1: Because the corrections ( Δ ^^ ) ^^, ^^ And ( Δ ^^ ) ^^, ^^ are not the same, at time t+1, the position of the center ^^ ^^, ^^+1corresponding to the whole ^^ ^^,1 is different from the center position ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1 . Referring to Figure 4 (lower part), in the same way as the centers, the widths are updated but with an additional constraint on the update to control the distinguishability. We can note ^^ ^^, ^^ the width corresponding to the whole ^^ ^^,1 at time t and ^^ ^^, ^^ the width corresponding to the whole ^^ ^^,1 at the same time t. In this case, for illustrative purposes, it is assumed that ^^ ^^, ^^ = ^^ ^^, ^^ . The fifth step allows you to obtain the width ^^ respectively ^^, ^^+1 corresponding to the whole ^^ ^^,1 at time t+1 and the width ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1at this same time t+1. As visible in figure 4 (lower part), a correction specific to each width is applied, which is written mathematically for the width corresponding to the set ^^ ^^,1: In this equation, ( Δσ ) ^^, ^^ corresponds to the value of the update of the width of the Gaussian corresponding to the set ^^ ^^,1 at time t. With a similar notation, it also comes for the width corresponding to the set ^^ ^^,1:^^ ^^, ^^+1 = ^^ ^^, ^^ + (Δ ^^) ^^, ^^Due to the fact that the corrections (Δσ) ^^, ^^ and (Δ ^^) ^^, ^^ are not the same, at time t+1, the width ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1 is different from the width ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1The sixth step is to delete the fuzzy sets deemed useless after the fifth refinement step. The deletion uses, for example, a criterion based on half size ^^ ^^ of the set of possible values ​​of the variable j when it is an interval [ ^^ ^^ ; ^^ ^^ ]. This half size ^^ ^^ is written:^^ ^^ = ( ^^ ^^ − ^^ ^^) / 2The deletion criterion is that > ^^ ^^ Or ^^ ^^is a suppression threshold. This corresponds to the fact that a fuzzy set is modeled by a Gaussian. If this Gaussian is too wide, that is to say it is greater than a threshold, the fuzzy set is suppressed because this set will disturb the system. This criterion amounts to increasing the width of the fuzzy set. If the fuzzy set is too wide, it disturbs the neighboring fuzzy sets and must therefore be removed. The method which will now be described uses the previous steps by making refinements to them. The method comprises a phase of generating a control law for the actuator 10 and a control phase. The generation phase aims to obtain a control law controlling the actuator 10 with a view to a performance objective from values ​​taken by measurement sensors 16. The generation phase comprises a collection step, a creation step, a generation step, an implementation step and a conversion step.During the collection step, signals from the sensors are collected. During the first creation step, fuzzy sets corresponding to the collected signals are created. Thus, during the first creation step, the center is the received value of the collected signal (that from the measurement sensors 16) and the width is set to a predetermined value. This predetermined value is, for example, chosen according to the number of linguistic variables desired for each rule. Similar to what was previously described for the DFQL algorithm, when a truth value associated with a fuzzy set exceeds a threshold value, a step is implemented to generate the associated rule and otherwise, create new fuzzy sets.Still referring to the DFQL algorithm, during the implementation step, reinforcement learning is then implemented on fuzzy sets comprising several iterations of an application of an action and updates of the state values ​​to determine the actions to be performed to obtain the objective based on the fuzzy sets. The reader is invited to refer to the initial notions to find more information on the operations of this step. However, in the example described, the previous fifth step, i.e. the operation of modifying at least one of the center position and the width of the fuzzy sets is implemented differently. For illustration purposes, it is assumed in a non-limiting manner in the following, that both the center position and the width are modified.Unlike the case of the DFQL algorithm where the modification operation is different depending on the rules, the modification operation is common to all rules using the fuzzy set. More precisely, the centers ρ and widths σ of the fuzzy sets are modified depending on the reward received by applying the action a in the state ^^. ^^−1 The first is to harmonize the updates of all the modifications made on the ^^ ^^, ^^ and ^^ ^^, ^^by averaging them, as shown in Figure 5. More precisely, for the center case (top part of Figure 5), the time difference errors of each of the different action-state associations are calculated and the modified position of the center depends on the calculated time difference errors. The modified position of the center is then the sum of the current position of the center plus the arithmetic mean of the time difference errors. Using the notations corresponding to those used for the description of Figure 4, the position of the center corresponding to the set ^^ ^^,1 at time t+1 is written:(Δρ) ^ + (Δρ)^^ = ^^ ^, ^^ ^^, ^^^^, ^^+1 ^^, ^^ +2In this equation, (Δρ) ^^, ^^ corresponds to the value of the update of the position of the center of the Gaussian corresponding to the set ^^ ^^,1 at time t while (Δρ) ^^, ^^corresponds to the value of the update of the position of the center of the Gaussian corresponding to the set ^^ ^^,1 at time t. With a similar notation, it also comes for the position of the center corresponding to the set ^^ ^^,1 :( Δρ) ( ^^ 1 = ^^ ^^, ^^ + Δρ) ^^, ^^^^, ^^+ ^^, ^^ +2Even if the corrections ( Δ ^^ ) ^^, ^^ And ( Δ ^^ ) ^^, ^^are not the same, at time t+1, the position of the center ^^ ^^, ^^+1 corresponding to the set ^^ ^^,1 is identical to the position of the center^^ ^^, ^^+1 corresponding to the set ^^ ^^,1.Similarly, for the case of the width (bottom part of Figure 5), the width is also modified as a function of the expected temporal differences. More precisely, here, the modified width is the sum of the current width added to the arithmetic mean of the expected temporal differences. Using the notations corresponding to those used for the description of Figure 4, the width corresponding to the set ^^ ^^,1 at time t+1 is written: In this equation, ( Δ ^^ ) ^^, ^^ corresponds to the value of the update of the width of the Gaussian corresponding to the set ^^ ^^,1 at time t while (Δ corresponds to the value of the update of the width of the Gaussian corresponding to the set ^^ ^^,1 at time t. With a similar notation, it also comes for the width corresponding to the set ^^ ^^,1: Even if the corrections are not the same, at time t+1, the width ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1 is the same as the width ^^ ^^, ^^+1 corresponding to the whole ^^ ^^,1 . According to an alternative embodiment, it is also proposed to take into account the distinguishability during the update by modulating the effects on the centers and the widths by a coefficient. For example, the following formula can be used: HAS vec, e t ^^∗ = ^^ ^^ ^^ ^^, ^^ ^^∗^^, ^^ ( ^^ ^^, ^^)In this case, it is imposed that the distinguishability of the fuzzy sets is at least ^^ ^^. This gives us the set of actions to be performed to obtain the objective based on the fuzzy sets. During the conversion step, this result of reinforcement learning is converted into a set of control rules forming the control law. This is a simple rewriting. The generation phase that has just been described is relatively simple to implement since it does not require modeling of the physical system. It also does not involve using a priori knowledge, and a fortiori not the knowledge of an expert. This generation phase makes it possible to obtain a control law with better reliability. To understand this, it is interesting to return to the case of the DFQL algorithm. In this case, the updates are different depending on the rules and participate in the differentiation of the definitions of fuzzy sets having the same semantic meaning.The refinement of the DFQL algorithm thus does not take into account the sharing of fuzzy sets into several rules. From the first update, an initially shared fuzzy set gives two very close ones, which significantly increases the number of rules, slows down the calculations and, above all, impoverishes the performance of the algorithm, which no longer associates a coordinate of a state with a given characteristic. In other words, this means that If ^^. ^^, ^^ and ^^ ^^, ^^ are shared between premises i and j (i.e. their membership function is equal) and their errors are different THEN ^^ ^^, ^^ and ^^ ^^, ^^are no longer shared in the next iteration. In the method described, the average of the updates, according to the rules, allows to have only one update common to all the rules using the fuzzy set. The method makes it possible to guarantee that for a given semantic variable, only one fuzzy set is used to define it. This allows the rules to be verifiable and this contributes to increasing the robustness of the control law. Controlling the evolution of the width of the widths ^^ ^^, ^^ also allows to ensure the distinguishability of fuzzy sets and therefore of the associated semantic variables. We define the distinguishability within the local partition from the normalized membership function of the fuzzy set ^^ ^^, ^^ is the conclusion associated with this fuzzy set, ultimately with the premise, is taken into account with respect to the others. If is not distinguishable then we cannot find the exact cause of an error, so our contribution must control the distinguishability. Here again, this allows the rules to be verifiable and this contributes to increasing the robustness of the control law. In addition, there is a gain in memory because the redundant rules are eliminated. During the control phase, the control system 12 applies the control law thus obtained to benefit from its better robustness in operation of the automated system. To improve this effect, the implementation further comprises an operation of testing the usefulness of the deletion of each fuzzy set, the fuzzy set being deleted if the test is validated. The method which has just been described can be implemented by a device 100 as shown in FIG. 6.The interaction between the device 110 and the computer program product 112 enables the implementation of the generation phase of the control method, which is thus a computer-implemented phase. The device 110 is a desktop computer. Alternatively, the device 110 is a rack-mounted computer, a laptop, a tablet, a personal digital assistant (PDA), or a smartphone. In specific embodiments, the computer is adapted to operate in real time and / or is in an embedded system, in particular in a vehicle such as an airplane. In the case of FIG. 6, the device 110 comprises a computing unit 114, a user interface 116, and a communication device 118.The computation unit 114 is an electronic circuit configured to manipulate and / or transform data represented by electronic or physical quantities in registers of the device 110 and / or memories into other similar data corresponding to physical data in the register memories or other types of display devices, transmission devices, or storage devices. As specific examples, the computation unit 114 includes a single-core or multi-core processor (such as a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, and a digital signal processor (DSP)), a programmable logic circuit (such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic device (PLD), and programmable logic arrays (PLAs)), a state machine, a logic gate, and discrete hardware components.The computing unit 114 comprises a data processing unit 120 adapted to process data, in particular by performing calculations, memories 122 adapted to store data and a reader 124 adapted to read a computer-readable medium. The user interface 116 comprises an input device 126 and an output device 128. The input device 126 is a device allowing the user of the system to enter information or commands on the device 110. In FIG. 6, the input device 126 is a keyboard. Alternatively, the input device 126 is a pointing device (such as a mouse, a touchpad and a graphics tablet), a voice recognition device, an eye tracker or a haptic device (motion analysis). The output device 128 is a graphical user interface, i.e., a display unit designed to provide information to the user of the device 110.In Figure 1, the output device 128 is a display screen enabling visual presentation of the output. In other embodiments, the output device is a printer, an augmented and / or virtual display unit, a speaker or other sound generating device for presenting the output as sound, a unit producing vibrations and / or odors, or a unit adapted to produce an electrical signal. In a specific embodiment, the input device 126 and the output device 128 are the same component forming human-machine interfaces, such as an interactive display. The communication device 118 enables one-way or two-way communication between the components of the device 110. For example, the communication device 118 is a bus communication system or an input / output interface.The presence of the communication device 118 allows that, in some embodiments, the components of the device 110 are spaced apart from each other. The computer program product 112 includes a computer-readable medium 132. The computer-readable medium 132 is a tangible device readable by the reader 124 of the computing unit 114. Notably, the computer-readable medium 132 is not a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, such as light pulses or electronic signals. Such a computer-readable storage medium 132 is, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.As a non-exhaustive list of more specific examples, the computer-readable storage medium 132 is a mechanically encoded device, such as punched cards or raised structures in a groove, a floppy disk, a hard disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EROM), an electrically erasable readable memory (EEPROM), a magneto-optical disk, a static random access memory (SRAM), a compact disk (CD-ROM), a digital versatile disk (DVD), a USB flash drive, a floppy disk, a flash memory, a solid-state disk (SSD), or a PC card such as a PCMCIA memory card. A computer program is stored on the computer-readable storage medium 132. The computer program comprises one or more sequences of stored program instructions.Such program instructions, when executed by the data processing unit 120, cause steps of the method to be executed. For example, the form of the program instructions is a source code form, a computer-executable form, or any intermediate form between a source code and a computer-executable form, such as the form resulting from the conversion of the source code via an interpreter, an assembler, a compiler, a linker, or a locator. Alternatively, the program instructions are microcode, firmware instructions, state definition data, integrated circuit configuration data (e.g., VHDL), or object code. The program instructions are written in any combination of one or more languages, for example, an object-oriented programming language (FORTRAN, C++, JAVA, HTML), a procedural programming language (e.g., C language).Alternatively, the program instructions are downloaded from an external source via a network, as is the case in particular for applications. In this case, the computer program product comprises a computer-readable data carrier on which the program instructions are stored or a data carrier signal on which the program instructions are encoded. In each case, the computer program product 112 comprises instructions that can be loaded into the data processing unit 120 and adapted to cause the execution of the method when executed by the data processing unit 120. Depending on the embodiments, the execution is fully or partially carried out either on the device 110, i.e., a single computer, or in a system distributed among several computers (in particular via the use of cloud computing).

Claims

CLAIMS 1. Method for controlling an actuator (10) by a control system (12), the control method comprising: - a phase of generating a control law for the actuator (10), the control law controlling the actuator (10) with a view to a performance objective from values ​​taken by measurement sensors (16), the generation phase comprising the steps of: - collecting signals from the measurement sensors (16), - creating fuzzy sets corresponding to the collected signals, - when a truth value associated with a fuzzy set exceeds a threshold value, generating the associated rule and otherwise, creating new fuzzy sets, - implementing reinforcement learning on the fuzzy sets comprising several iterations of an application of an action and updates of the action-state values, to determine the actions to be carried out to obtain the performance objective as a function of the fuzzy sets,the implementation comprising at each iteration an operation of modifying at least one of the position of the center and the width of the fuzzy sets to obtain modified fuzzy sets on which the reinforcement learning is applied at the following iteration, - converting the result of the reinforcement learning into a set of control rules forming the control law, and - a control phase during which the control system (12) applies the control law to the actuator (10).

2. Control method according to claim 1, wherein, during the modification operation, both the position of the center and the width of each fuzzy set is modified.

3. Control method according to claim 1 or 2, wherein, during the modification operation,the time difference errors of each of the different state-action associations are calculated and the modified position of the center depends on the calculated time difference errors.

4. A control method according to claim 3, wherein the modified position of the center is the sum of the current position of the center plus the arithmetic mean of the time difference errors.

5. A control method according to any one of claims 1 to 4, wherein, during the modification operation, the expectation of the time differences of each of the different state-action associations is calculated and the modified width is a function of the time difference expectations.

6. A control method according to claim 5, wherein the modified width is the sum of the current width plus the arithmetic mean of the time difference expectations.

7. A control method according to any one of claims 1 to 6, wherein the implementation further comprises an operation for testing the usefulness of deleting each fuzzy set,the fuzzy set being deleted if the test is validated.

8. Control system (12) adapted to implement: - a phase of generating a control law for the actuator (10), the control law controlling the actuator (10) with a view to a performance objective from values ​​taken by measurement sensors (16), the generation phase comprising the steps of: - collecting signals from the measurement sensors (16), - creating fuzzy sets corresponding to the collected signals, - when a truth value associated with a fuzzy set exceeds a threshold value, generating the associated rule and otherwise, creating new fuzzy sets, - implementing reinforcement learning on the fuzzy sets comprising several iterations of an application of an action and updates of the action-state values, to determine the actions to be carried out to obtain the performance objective according to the fuzzy sets,the implementation comprising at each iteration an operation of modifying at least one of the position of the center and the width of the fuzzy sets to obtain modified fuzzy sets on which the reinforcement learning is applied at the following iteration, - conversion of the result of the reinforcement learning into a set of control rules forming the control law, and, - a control phase during which the control system (12) applies the control law to the actuator (10).

Images