Sensorless motor control

Abstract

A motor controller 18 is configured to operate a motor 16 of a printing apparatus 14. The motor controller stores a first set of parameters defining a first control function mapping control inputs and

Description

FIELD We provide a method and apparatus for sensorless control of electric motors. In particular, but not exclusively, the present invention relates to a method and apparatus for optimising control of electric motors in printing apparatus, using a model trained using reinforcement machine-learning techniques. BACKGROUND Electric motors have many applications in industrial devices, and motors such as DC stepper motors and solenoid-actuated motors are commonly used in printing apparatus (such as thermal transfer printers and thermal-direct printers). Such motors are used to drive ribbon spools in thermal transfer printing apparatus, and to drive printheads in most types of printing technology. Motor controllers operate by controlling motor speed using the voltage and / or current supplied to the motor. Commonly, Field-Oriented Control (FOC) strategies are used, which require sensors on or adjacent the motor to detect the rotational position and speed of the motor. Such a control strategy works well at all motor speeds, so that the voltage and / or current supplied to the motor can be controlled, accordingly. However, it is advantageous to provide a motor controller that operates without sensors, to reduce the parts involved in manufacturing motors. Doing so may also reduce calibration steps required to set up the motor for operation, to account for minor deviations in the positioning of sensors and related components. BRIEF DESCRIPTION OF THE INVENTION Sensorless control of a motor can be performed using a control model (or “control function”) trained through reinforcement learning techniques. In other words, the control function is trained to make control decisions (i.e., output of voltage / current to the motor) in order to maximise a ‘reward’ function. For example, it is known to control a permanent magnet synchronous motor (PMSM) using a control function trained in this way. While relatively high computational power may be required to train a model to perform in the desired way, once the model is trained, relatively lower computing power is needed to run the model during operation of the motor since at that stage the model acts as a function mapping combinations of inputs to control outputs. Sensorless control may be employed across a wide range of motor speeds by using variable schemes. For example, at low speeds (or zero speed) of the motor, a high frequency signal injection (HFSI) algorithm may be used, whereas a machine-learning trained function may be employed for medium and high speeds of the motor. In this way a motor controller may employ different control functions across different ranges of motor speed to achieve optimised control. However, there are two main problems with implementing a model that is trained and optimised, before use, by machine learning techniques. One is that each motor installation in a printing apparatus (for example) is different, and usually it is necessary to fine-tune control of a motor once installed. This is difficult or impossible to achieve where the model is trained before use, as the computation power (and training data) required to update the trained model makes doing so impractical in situ. In addition, overtime, components of the motor, or equipment being driven by the motor, may change through wear, and so the performance of the motor alters. In hand with this, control of the motor must be updated to maintain optimal performance. A drawback of reinforcement learning techniques is that they are costly in terms of computational power required to carry out the training process. Therefore, retraining or optimising a model in this way for a ‘live’ motor during run-time is inefficient. This means that where a model that has been trained using reinforcement learning is employed, and where the model hasn’t been trained ‘in situ’ (i.e. where different respective systems are responsible for the training and inference phases, which can often, but not always, be the case), then it is hard to calibrate the model, leading to suboptimal control at run-time” And further to that, as the motor components and the printer equipment driven by the motor experience wear, the model becomes suboptimal. The complexity of retraining the model makes customisation and optimisation impractical. The present invention aims to overcome or reduce one or more problems associated with the state of the art. According to a first aspect of the invention, we provide a motor controller configured to operate a motor of a printing apparatus, the motor controller configured to store a first set of parameters defining a first control function mapping control inputs and motor state data to control outputs for operating the motor; the motor controller being operable to receive a second set of parameters defining a second control function, and to store the second set of parameters, such that subsequent control of the motor is performed using the second set of parameters defining the second control function; wherein the second set of parameters is determined using a reinforcement learning model. In some embodiments of the technology the motor controller is configured to control a permanent magnet synchronous motor. In some embodiments of the technology the motor controller operates a motor driving a ribbon spool of a printing apparatus. The controller may be configured to operate a second motor of the printing apparatus, and the first motor may operate a supply spool and the second motor may operate a takeup spool. In some embodiments of the technology the control inputs include at least one of: a desired speed of rotation, a desired tension on a ribbon supported on the spool, ora desired speed of the ribbon. In some embodiments of the technology the motor controller operates a motor driving a print head of a printing apparatus. In some embodiments of the technology the motor state data includes at least one of: motor position or motor torque. In some embodiments of the technology the control outputs define at least a voltage and / or current to be applied to the motor. In some embodiments of the technology the first set of parameters are determined using a reinforcement learning model. In some embodiments of the technology the motor is a sensorless motor. According to a second aspect of the invention, we provide a system for controlling a motor of a printing apparatus, the system including: the motor controller of the first aspect, and a processing device configured to run a reinforcement learning model, wherein the processing device is operable to receive environmental data relating to the environment in which the printing apparatus is installed, and I or relating to operating conditions under which the printing apparatus is operating, and to update the reinforcement learning model using the environmental data, and following the update to the reinforcement learning model, subsequently outputting from the reinforcement learning model the second set of parameters, and the processing device being operable to communicate the second set of parameters to the motor controller to update control of the motor. In some embodiments of the technology the environmental data includes one or more of: a load on the motor, speed of the motor, speed of a ribbon driven by the motor, a temperature of the motor, vibration, shock forces, a position of the motor, the nature of a load such as driven mass or spring constant, a most frequently used motor speed, estimated degradation of one or more components of the printing apparatus. The load represents the combined mechanical effect of all items actuated by the motor, which might include springs or similar resilient members as well as inert masses such as a printhead. In some embodiments of the technology the environmental data includes feedback from a digital twin simulation of at least a portion of the printing apparatus, including the motor controller and motor. In some embodiments of the technology the environmental data includes feedback from a simulated version of the motor controller, simulated within multiple different motors with different applied loads. In some embodiments of the technology the system further includes a communication link between the motor controller and the processing device, the communication link being one of: a physical network connection, a Wi-Fi connection, a Bluetooth connection, connection via an internet. In some embodiments of the technology the reinforcement learning model employs a Deep Deterministic Policy Gradient algorithm. In some embodiments of the technology the system further includes an observation module configured to gather environmental data during operation of the printing apparatus, and to send the environmental data to the processing device. In some embodiments of the technology the processing device is configured to receive environmental data relating to the installation environments, or operating conditions, of multiple printing apparatuses. In some embodiments of the technology the processing device is located remotely from the motor controller and printing apparatus. According to a third aspect of the invention we provide a motor controller configured to operate a motor of a printing apparatus, the motor controller configured to store a first set of parameters defining a first control function mapping control inputs and motor state data to control outputs for operating the motor; the motor controller providing a processing device configured to: run a reinforcement learning model, and receive environmental data relating to the environment in which the printing apparatus is installed, and / or relating to operating conditions under which the printing apparatus is operating, update the reinforcement learning model using the environmental data, and following the update to the reinforcement learning model, subsequently outputting from the reinforcement learning a second set of parameters defining a second control function, and store the second set of parameters, such that subsequent control of the motor is performed using the second set of parameters defining the second control function. According to a fourth aspect of the invention we provide a method of optimising configuration of a motor controller for operating a motor of a printing apparatus, within a system providing a processing device configured to run a reinforcement learning model, the method including: installing the motor controller relative to a printing apparatus, the motor controller being connected to a motor of the printing apparatus, and configured with a first set of parameters defining a first control function mapping control inputs and motor state data to control outputs for operating the motor; providing to the processing device environmental data relating to the environment in which the printing apparatus is installed, and / or relating to operating conditions under which the printing apparatus is operating, updating the reinforcement learning model at the processing device using the environmental data, outputting from the reinforcement learning model the second set of parameters, communicating the second set of parameters to the motor controller, and storing the second set of parameters at the motor controller, such that subsequent control of the motor is performed using the second set of parameters. BRIEF DESCRIPTION OF THE FIGURES In order that the present disclosure maybe more readily understood, preferable embodiments thereof will now be described, by way of example only, with reference to the accompanying drawings, in which: FIGURES 1 and 2 are diagrammatic views of a system embodying the present disclosure; FIGURE 3 is a diagram illustrating flow of data within an exemplary system embodying the present disclosure; and FIGURE 4 is a flow diagram illustrating steps of a method embodied by the present disclosure. DETAILED DESCRIPTION OF THE DISCLOSURE With reference to Figures 1 and 2 of the drawings, the present invention relates to a system 10 for controlling a motor 16 of a printing apparatus 14. The system 10 broadly provides a motor controller 18 for operating one or more motors 16 of the printing apparatus 14, and a processing device 100 for computing optimised parameters to be used by the motor controller 18. In embodiments of the technology, the processing device 100 is remote from the printing apparatus 14 and the motor controller 18. For example, the processing device 100 may be cloud-based, or may be provided on a local computer or processing cluster, but need not be directly connected or located close to the printing apparatus 14. As illustrated in Figure 1 using broken lines, the motor controller 18 may form part of a control unit including a controller communication module 20 operable to receive data relating to parameters for operating the motor 16, as discussed in more detail below. In some embodiments of the technology, the controller communication module 20 is configured to send / receive data via a wired or wireless network connection, or via Bluetooth for example, to one or more remote systems monitoring the operation of the printing apparatus 14 and in particular the motor controller 18 and / or motor 16. The sent data may include operation data relating to control of the motor 16, for example (position / speed data), and / or data relating to the load being driven by the motor 16. The control unit further includes controller storage / memory 22 for storing operating parameters of the motor controller 18, and optionally storing such operation data. We refer to the motor controller 18 and its associated control unit components collectively herein as the motor controller 18, for simplicity. In some embodiments of the technology, and with reference to Figure 2, the motor(s) 16 are typically motors 16a, 16b fordriving ribbon-supporting spools of the printing apparatus 14, which typically include a supply spool driven by a first motor 16a, and a take-up spool driven by a second motor 16b. In some embodiments of the technology, the system 10 may be applied to controlling a motor 16c fordriving a print head of the printing apparatus 14. The motor 16 may be a permanent magnet synchronous motor (PMSM). In other embodiments, the motor 16 may be another type of motor suitable fordriving a spool of a printer, ora print head, for example, such as a DC stepper motor or a solenoid-actuated motor. Looking now at the motor controller 18 in more detail, the motor controller 18 is configured to operate the motor(s) 16 of the printing apparatus 14. The motor controller 18 is configured to store a first set of parameters defining a first control function in controller storage / memory 22. The first control function maps control inputs (i.e. instructions to carry out certain operations) and motor state data to control outputs for operating the motor 16. The control outputs define at least a voltage and / or current to be applied to the motor 16 so as to operate the motor 16. Control inputs to the motor controller 18 may be provided as inputs from a motherboard of the controller, any relevant sensors and / or external sources as required to provide output to the motor(s) 16. During operation of the printing apparatus 14, the motor controller 18 sends commands to the motor 16 based on the output of the current control function as defined by the parameters to meet the desired speed / position of the motor and load from upstream inputs (such as a printer UI). The control inputs may include, for example, a desired speed of rotation, a desired tension on a ribbon supported on the spool, or a desired speed of the ribbon. The motor state data may represent the position of the motor, the torque in the motor, and / or its current speed, for example. The motor controller 18 is operable to receive (via the controller communication module 20) a second set of parameters defining a second control function. The second control function provides one or more updates to the way the motor 16 is to be controlled, so that for a subset of the motor state data, and control inputs, the motor controller 18 provides different control outputs to the motor 16 under the second control function than it would have done under the first control function. The motor controller 18 stores the second set of parameters, to replace the first set, such that subsequent control of the motor 16 is then performed using the second set of parameters defining the second control function. In this way, the performance of the motor controller 18 can be updated overtime to reflect changes to the operating conditions or the performance of the components of the printing apparatus 14. While reference is made herein to a first set of parameters, and an updated second set of parameters, it should be understood that the process of refining and updating the reinforcement learning model may be ongoing during operation of the printing apparatus 14. Therefore, further sets of parameters may be determined by the processing device 100 and supplied to the motor controller 18, each time replacing / updating the previous parameters to update the control of the motor 14. The updated control function and associated parameters are determined by the processing device 100. Looking now at the processing device 100 in more detail, it broadly comprises a processing device communication module 102, one or more processors 104, one or more memory devices 106 and one or more storage devices 108. While the term processing device 100 is used, it should be understood that the processors and other components may be distributed, and may not necessarily be in a single location. For example, the processing device 100 may be cloud-based and comprise multiple servers. The processing device 100 is configured to run a reinforcement learning model. The reinforcement learning model may involve one or more artificial neural networks, and may employ a Deep Deterministic Policy Gradient algorithm, for example, or any other suitable learning framework as would be understood by the skilled person in this field. The reinforcement learning model undergoes initial training to perform a motor control function, generating a model containing a base control function common to all models / instances of a specific type of motor or type of motor installation. Typically, the reinforcement learning model is trained at the outset using inputs from a training motor controller, representing a desired speed of a motor to be used. A representative motor system with relevant sensors is then used to provide real time feedback on the state of the motor (such as its speed and position). Other sensors on the representative system may also give parameters such as temperatures, vibrations, shock forces, etc., to optimise the control function and related control parameters for a particular motor / load combination. In embodiments, a fixed set of rules may be loaded onto the motor controller 18 to provide, for example, fixed limits of current or voltage values which the outputs from the motor controller are not allowed to exceed, to avoid damage to the motor 161 controller 18. As noted elsewhere, these initialising steps may be achieved using a digital twin of the motor 16 and its installation. The training may also be conducted for different motors and different loads / installation details, so as to provide a library of motor settings to be applied as required during operation. In broad terms, the model learns, via the training steps and sensor feedback, a set of parameters defining a control function for mapping a desired control input (e.g. a desired speed) to a control output (e.g. defining voltage and current outputs) which when applied to the motor 16 via the motor controller 18 will cause a measured property (e.g. measured speed) of the test motor to match the desired control input (e.g. desired speed). Where sensorless control is desired in use, the processing device 100 may, for example, further generate a second sub-model capable of predicting a property, e.g. motor speed and / or position by only observing the outputs of the reinforcement learning model. In this way, the motor controller may be programmed to determine a property, e.g. the position and / or speed of the motor 16 without requiring sensor feedback during use. The processing device 100 preferably constrains the structure of its reinforcement learning model such that later updates can be most easily provided. The processing device 100 is operable to receive environmental data via a device communication module 102. The device communication module 102 can be used in combination with the controller communication module 20 so that a communication link, between the motor controller 18 and the processing device 100 is established. The communication modules 102, 20 may communicate via one of: a physical network connection, a Wi-Fi connection, a Bluetooth connection, connection via an internet, for example. In embodiments of the technology, the environmental data relates to the printing environment 12 in which the printing apparatus 14 is installed and I or relates to operating conditions under which the printing apparatus 14 is operating. This may include the operation data relating to control of the motor 16, for example position I speed data, and / or data relating to the load being driven by the motor 16. The environmental data may also include observed data recorded by an observation module 24 within the printing environment 12. The observation module 24 may provide (or receive data from) one or more sensors for obtaining feedback on the position of the motor 16, the speed of the motor 16, or on the quality of one or more aspects of a printing operation. The sensors may include cameras, load meters to capture the component wear, installation and / or operation details. For example, the tension in a ribbon, or the quality of print resulting from a printing operation, may be determined using one or more of the sensors. In some embodiments, predictions or estimations may be determined about the expected wear of components of the printing apparatus 14, including the motor 16, in order to simulate and take account of wear overtime and its impact on performance. In this way, updates to parameters may be based in part on expected changes in performance over time so as to pre-empt the negative impact on the motor control and performance and update the parameters accordingly in advance. To assist in predicting the degradation of components, for example, humidity sensors and / or thermometers may provide environmental data to the processing device 100 via the observation module 24. In some embodiments of the technology, the observation module 24 may receive feedback input manually by a human operator, for example. This may be provided via a digital form submission. The observation module 24 may provide such feedback directly to the processing device 100, or to the motor controller 18, for example. If provided to the motor controller 18, this data may subsequently be relayed to the processing device 100 via the controller communication module 20. In this way the processing device 100 receives feedback on the motor 16 performance under control of the motor controller 18 using a known set of parameters, and / or feedback on the operation of the printing apparatus in terms of observable operation parameters or its printed output. In embodiments of the technology, the environmental data may include data measured at the point of installation of the printing apparatus 14. In other words, as the motor 16 and its motor controller 18 are installed on site, and details of the load being driven and the likely implementation of the printing apparatus in situ are understood, steps may be taken to measure I record those parameters, during calibration steps, for example, and to provide that data to the processing device 100. Relevant parameters for ‘day zero’ installation may include specific installed loads, or an observed mechanical environment (e.g., a high vibration environment) which might require specific control settings to avoid damage to the load. In broad terms, the environmental data may include one or more of: a load on the motor, speed of the motor, speed of a ribbon driven by the motor, a temperature of the motor, vibration, shock forces, a position of the motor, the nature of a load such as driven mass or spring constant, a most frequently used motor speed, estimated degradation of one or more components of the printing apparatus. In either of the above cases, the processing device 100 is provided with data about the performance or intended future use of the printing apparatus 14 and motor 16, and so steps can be taken to optimise the motor controller 18 prior to beginning operation of the printing apparatus 14 and / or as an update during operation. To achieve this, the reinforcement learning model is updated using the environmental data to optimise the performance of the model based on that data. From the model, a second control function and associated set of optimised second parameters are produced. The processing device 100 is operable to output these second parameters, sending them to the motor controller 18, so that the second parameters can be stored in place of the first parameters (i.e., the current parameters), to update control of the motor 16. In some embodiments of the technology, it may be advantageous to use the reinforcement learning model to determine initial parameters for the motor controller 18, for use with a specific type of motor 16 for example, for a specific purpose and / or in a specific installation environment. With known operating criteria and known system mechanics, the model can be trained to provide an optimised default set of parameters to be shipped with the motor controller 18 and / or motor 16. In this way, in embodiments, the first parameters may be determined using a reinforcement learning model. In some embodiments of the technology, if the control function provided by the reinforcement learning model is found not to work well at certain motor speed ranges, a variable scheme may be adopted in which the parameters output by the model are used only for control over certain speed ranges. For example, at low speeds (or zero speed) of the motor 16, a high frequency signal injection (HFSI) algorithm may be used. For higher speeds of the motor 16, the parameters determined by the machine-learning trained function may be used (for medium and high speeds of the motor, for example). An exemplary system is illustrated in Figure 3, in which the processing device 100 is illustrated as comprising a first function for generating an initial model for installing the motor controller 18 and motor 16 with an initial set of parameters, and a second function training model for providing updated parameters over time. The motor controller 18 is illustrated as providing an RL model / Control function which reflects the currently loaded parameters to be used to operate the motor 16. It can be seen that voltage and current are provided as the output from the motor controller 18 to the motor 16, and that optionally a load measurement is provided to the motor controller 18 as feedback. Installation information at the outset, and environmental data captured and logged within the printing environment 12 on an ongoing basis, are subsequently communicated to the processing device 100. The reinforcement learning model is then used to provide an optimised control function and related parameters for communication to the motor controller 18, so as to update its control functionality. In some embodiments of the technology, the environmental data includes feedback from a digital twin simulation of at least a portion of the printing apparatus 14, including the motor controller 18 and motor 16. The digital twin may be a forward-looking twin predicting future behaviour of the printing apparatus 14. In some embodiments, the environmental data includes feedback produced from a simulated version of the motor controller 18. In this way the performance of the controller with a variety of different motors 16 and different applied loads I operating conditions I installations may be modelled. This approach allows multiple default sets of parameters to be established, so that corresponding parameters can be loaded into the motor controller 18 when using the motor controller 18 in different installations or with different motors 16. In some embodiments, the processing device 100 is configured to receive environmental data relating to the installation environments, and / or operating conditions, of multiple printing apparatuses. In this way, the reinforcement learning model may receive relatively larger amounts of data than produced from a single motor installation, and so through the increased data, provide better optimised control functions and parameters. In some embodiments of the technology, some or all of the components of the processing device 100 may instead be located at the printing apparatus 14, and may form part of the control unit and motor controller 18. For example, the motor controller 18 may be configured to run the reinforcement learning model, and to receive the environmental data, updating the reinforcement learning model using the environmental data. In this way, the second set of parameters defining the second control function may be obtained from the model at the printing apparatus 14, so that the control of the motor is then updated to use the updated second set of parameters. In broad terms, the technology outlined above enables a way to optimise the configuration of the motor controller 18, as shown in relation to Figure 4. This involves the general steps of first installing the motor controller 18 relative to a printing apparatus 14, the motor controller 18 being connected to the motor(s) 16 of the printing apparatus 14 and configured with the first (i.e. default) set of parameters defining the first control function. At this initial step 202, in embodiments of the technology, the reinforcement learning model is used to train the motor controller 18 based on the initial calibration information captured or observed at the installation of the printing apparatus 14, about the printing environment 12. This is subsequently used to determine an initial first control function for controlling the motor 16 at step 204, which is sent to the motor controller 18. During operation, environmental data relating to the environment 12 in which the printing apparatus 14 is installed, and / or relating to operating conditions under which the printing apparatus 14 is operating, is provided to the processing device 100. This allows the reinforcement learning model (or a further model) to be trained so as to take account of performance issues, as shown in step 206. As a result, the reinforcement learning model at the processing device 100 is updated (or a further model trained) based on this environmental data, and at step 208, the second set of parameters is output from the reinforcement learning model. At step 210, it is shown that the first / second sets of parameters are provided to the motor controller and stored at the motor controller 18. Subsequently control inputs are sent to the motor 16 in step 212 based on the updated parameters such that subsequent control of the motor 16 is performed using the updated set of parameters. In this way, a motor-driven system is enabled to maintain optimum performance across the system lifetime by implementation of a control function optimised via reinforcement learning techniques, in which the control function is tuned offline. In this way, pure motor control parameters alongside the estimated remaining performance of the motor, spring(s) and other system components, may be taken into account. Relevant triggers for such ‘ongoing’ updates might include frequent observed or predicted use of specific settings (like high speeds or long strokes), which may cause specific degradation modes, and which can be at least partially compensated for by (for example) disallowing certain parts of the motor control space. In addition, or alternatively, observed or predicted degradation to some part of the load (e.g., loss of stiffness of a spring) which would cause an alternative set of parameters to control the motor 16 more efficiently if installed. The control strategy may be fine-tuned on-the-fly, e.g., by updating the control function to account for an observed installation environment on day zero, or providing updates, for example, to deal with measured or predicted wear. In addition, using reinforcement learning techniques allows for optimised control of the motor 16 to be achieved for controlling a sensorless motor. While mention is made above of the possibility of sensors being used to capture data about the printing apparatus 14 and the printing environment 12, the motor 16 itself may not be controlled using feedback from sensors. The printing apparatus 14 is controlled using control parameters established via the reinforcement learning model, in which the control strategy takes account both of the parameters necessary to control the motor 16 (i.e. based on motor state data and desired functionality), and of environmental data relating to the estimated usage, life or status of the motor and / or associated structures such as springs, as well as the observed installation environment. Due to the split structure of the system model (online inference carried out at the controller, but offline training carried out remotely at the processing device), the parameters can be updated over time without interruption to the normal function of the motor-driven system. This is achieved through updating the parameters to take account of expected or measured degradation, known changes in the usage context (e.g. speeds, stroke) and other measured data. The updates may be supplied as part of a network connected industrial equipment setup - and may optionally be derived from a digital twin of the equipment model as mentioned above. Notably, use of the method and apparatus of the described embodiments provides optimised motor performance over the full lifetime of the motor and printing apparatus, by enabling fine-tuning of the reinforcement learning based motor control. In addition, the motor control is optimised for each installation scenario, as the offline training of the model based on environmental and usage effects for the motor being modelled are taken into account when the control function is updated. Further, motor performance is optimised following the initial motor set-up so that changes in the motor-driven system’s characteristics over time are taken into account following the initial training phase. This allows the system to account for wear of components, for example. As an example of its implementation, a forward-looking digital twin model may suggest that in a specific installed environment e with user settings of s, after x hours of use, the stiffness of a spring attached to the motor degrades by k N / m. This allows an updated or new reinforcement learning agent to be trained by the processing device, optimising the performance of the motor 16 to compensate for or prevent the degradation of the spring. For example, given an expected degradation and change in performance, the simulation of the motor may be programmed so as to exhibit the expected degradation, resulting in the performance of the trained control function becoming sub-optimal over time as the exhibited behaviour changes, resulting in the reward function of the reinforcement learning algorithm assigning negative reward (also referred to as punishment) which over time will increase if the trained model fails to adapt. In this way, the reinforcement learning algorithm will adapt with the aim of maximising reward so as to thereby optimise performance to match the updated behaviour being modelled. More generally, an action-reward feedback loop may be used in which actions that cause desirable behaviour(s) can be positively rewarded by the algorithm, whereas actions that cause undesirable behaviour(s) can be negatively rewarded (penalised) by the algorithm, in which a positive reward increases a likelihood for that action and a negative reward decreases a likelihood for that action. Accordingly, through reinforcement learning, a control function can be learned to optimise performance to match (or at least better match) the updated behaviour being modelled. Moreover, an optimised control function can be learned, and following changes in the behaviours) being modelled, an updated or new control function can be learned which is optimised for changes in one or more behaviours. Various reinforcement learning algorithms may be used forthe purposes described herein. One or more deep reinforcement learning algorithms may be used. In some embodiments, one or more policy-based reinforcement learning algorithms may be used. In some embodiments, a Deep Deterministic Policy Gradient (DDPG) algorithm may be used. Running the model in this way may also help to predict the change in the environment and performance based on the feedback received, input / output relationship and / or changes in relevant sensor readings that would indicate this degradation was starting. In this way, the system allows for further fine tuning in response to the changing mechanics of the motor system itself. For example, if owner (a) uses a low pack rate, a short stroke, a different platen material or has a different printer mount, then their printhead motor and spring will be subject to different mechanical forces. Therefore, different degradation modes will be exhibited compared to a second owner (b) with a different installation. Although both may use the same model of printing apparatus, and thus have the same initial parameters installed on their motor controllers 18, or same initial reinforcement learning models loaded onto their processing devices 100, and even assuming their installation environment is the same (temperature, mechanical attachments, properties of printer mount, mass of print head etc.), the fine tuning of the updated control functions will differ and thus the motor heads will be subject to different control inputs in order to optimise function overtime. When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components. The invention may also broadly consist in the parts, elements, steps, examples and / or features referred to or indicated in the specification individually or collectively in any and all combinations of two or more said parts, elements, steps, examples and / or features. In particular, one or more features in any of the embodiments described herein may be combined with one or more features 5 from any other embodiment(s) described herein. Protection may be sought for any features disclosed in any one or more published documents referenced herein in combination with the present disclosure. 10 Although certain example embodiments of the invention have been described, the scope of the appended claims is not intended to be limited solely to these embodiments. The claims are to be construed literally, purposively, and / or to encompass equivalents.

Claims

1. A motor controller configured to operate a motor of a printing apparatus, the motor controller configured to store a first set of parameters defining a first control function mapping control inputs and motor state data to control outputs for operating the motor;the motor controller being operable to receive a second set of parameters defining a second control function, and to store the second set of parameters, such that subsequent control of the motor is performed using the second set of parameters defining the second control function;wherein the second set of parameters is determined using a reinforcement learning model.

2. The motor controller of claim 1, configured to control a permanent magnet synchronous motor.

3. The motor controller of any preceding claim, wherein the motor controller operates a motor driving a ribbon spool of a printing apparatus.

4. The motor controller of claim 3, wherein the controller is configured to operate a second motor of the printing apparatus, and wherein the first motor operates a supply spool and the second motor operates a take-up spool.

5. The motor controller of any preceding claim, in which the control inputs include at least one of: a desired speed of rotation, a desired tension on a ribbon supported on the spool, or a desired speed of the ribbon.

6. The motor controller of claim 1 or claim 2, wherein the motor controller operates a motor driving a print head of a printing apparatus.

7. The motor controller of any preceding claim, in which the motor state data includes at least one of: motor position or motor torque.

8. The motor controller of any preceding claim, in which the control outputs define at least a voltage and / or current to be applied to the motor.

9. The motor controller of any preceding claim, wherein the first set of parameters are determined using a reinforcement learning model.

10. The motor controller of any preceding claim, wherein the motor is a sensorless motor.

11. A system for controlling a motor of a printing apparatus, the system including:the motor controller of any preceding claim, anda processing device configured to run a reinforcement learning model,wherein the processing device is operable to receive environmental data relating to the environment in which the printing apparatus is installed, and / or relating to operating conditions under which the printing apparatus is operating, and to update the reinforcement learning model using the environmental data, and following the update to the reinforcement learning model, subsequently outputting from the reinforcement learning model the second set of parameters, andthe processing device being operable to communicate the second set of parameters to the motor controller to update control of the motor.

12. The system of claim 11, wherein the environmental data includes one or more of: a load on the motor, speed of the motor, speed of a ribbon driven by the motor, a temperature of the motor, vibration, shock forces, a position of the motor, the nature of a load such as driven mass or spring constant, a most frequently used motor speed, estimated degradation of one or more components of the printing apparatus.

13. The system of claim 11 or claim 12, wherein the environmental data includes feedback from a digital twin simulation of at least a portion of the printing apparatus, including the motor controller and motor.

14. The system of one of claims 11 to 13, wherein the environmental data includes feedback from a simulated version of the motor controller, simulated within multiple different motors with different applied loads.

15. The system of any one of claims 11 to 14, further including a communication link between the motor controller and the processing device, the communication link being one of: a physical network connection, a Wi-Fi connection, a Bluetooth connection, connection via an internet.

16. The system of any one of claims 11 to 15, wherein the reinforcement learning model employs a Deep Deterministic Policy Gradient algorithm.

17. The system of any one of claims 11 to 16, the system further including an observation module configured to gather environmental data during operation of the printing apparatus, and to send the environmental data to the processing device.

18. The system of claim 17, wherein the processing device is configured to receive environmental data relating to the installation environments, or operating conditions, of multiple printing apparatuses.

19. The system of any one of claims 11 to 18, wherein the processing device is located remotely from the motor controller and printing apparatus.

20. A motor controller configured to operate a motor of a printing apparatus, the motor controller configured to store a first set of parameters defining a first control function mapping control inputs and motor state data to control outputs for operating the motor;the motor controller providing a processing device configured to:run a reinforcement learning model, andreceive environmental data relating to the environment in which the printing apparatus is installed, and / or relating to operating conditions under which the printing apparatus is operating,update the reinforcement learning model using the environmental data, andfollowing the update to the reinforcement learning model, subsequently outputting from the reinforcement learning a second set of parameters defining a second control function, andstore the second set of parameters, such that subsequent control of the motor is performed using the second set of parameters defining the second control function.

21. A method of optimising configuration of a motor controller for operating a motor of a printing apparatus, within a system providing a processing device configured to run a reinforcement learning model, the method including:installing the motor controller relative to a printing apparatus, the motor controller being connected to a motor of the printing apparatus, and configured with a first set of parameters defining a first control function mapping control inputs and motor state data to control outputs for operating the motor;providing to the processing device environmental data relating to the environment in which the printing apparatus is installed, and / or relating to operating conditions under which the printing apparatus is operating,updating the reinforcement learning model at the processing device using the environmental data,outputting from the reinforcement learning model the second set of parameters, communicating the second set of parameters to the motor controller, andstoring the second set of parameters at the motor controller, such that subsequent control of the motor is performed using the second set of parameters.

Images