Method for updating a current operating model of a device in a communication network

Abstract

The invention relates to a method for updating a current operating model (MOC) of a first device (E1) following a modification to at least one software function (FL) of a second device (E2) with which the first device is interfaced. Such a method comprises, in the first device: - receiving (11) at least one item of data for accessing a digital replica (RNE2) of the second device, wherein the digital replica implements the modified software function; - training (12) an experimental model associated with the first device according to items of data exchanged between the first device and the modified software function implemented by the digital replica delivering an updated operating model (MOA); - deploying (13) the updated operating model as a new current operating model of the first device.

Description

Description METHOD FOR UPDATING A CURRENT OPERATIONAL MODEL OF A COMMUNICATION NETWORK EQUIPMENT Technical field

[0001] The invention relates to the field of communication networks. More particularly, the invention relates to the problems of maintaining the continuity and quality of the service provided via such networks, when the software functionalities of a particular piece of network equipment are required to evolve. Prior art

[0002] A communications network is generally based on an infrastructure comprising numerous devices of various origins and types, which communicate with each other to ensure the operation of the network and the provision of a service. To this end, each device is composed of a hardware part and a software part.

[0003] For many reasons, the software part of a piece of equipment often needs to evolve. For example, as part of corrective maintenance operations (solving operating problems identified within the equipment, by deploying patches) or evolutionary maintenance (adding new functionalities to the equipment), existing software functions are generally corrected or modified, and / or new software functions are regularly added, thus modifying the behavior of the equipment in question.

[0004] Although they generally aim to improve the performance of the communications network (and, ultimately, of the service provided) as a whole, such updates to equipment are not without risks, in particular because they are likely to cause side effects on other equipment in the communications network, and primarily on those interfacing with the updated equipment.

[0005] Since the interconnected equipment of a communications network is generally supplied by different equipment manufacturers, it is not always easy or possible for the network operator to identify and manage in advance the potential negative and undesirable impacts of updating the software part of a particular equipment on the other equipment in the network, especially since the operator does not always have control over these update operations, and the equipment manufacturers themselves are sometimes reluctant to provide technical details of the content of the proposed updates.

[0006] It follows that software update operations on communications network equipment sometimes have a counterproductive effect initially, in- causing performance drops or even service interruptions that directly impact users of the communications network. The time taken to detect and report these impacts to the operator, added to the time needed to adapt the software of other equipment interfaced with the updated equipment accordingly, means that these negative effects are sometimes long-lasting, further increasing the risk of generating dissatisfaction on the part of users with this degraded service.

[0007] There is therefore a need for a solution to better manage the impacts caused by software modifications to a piece of equipment in a communications network on other equipment in the network. Summary of the invention

[0008] The present technique makes it possible to propose a solution aimed at remedying certain drawbacks of the prior art. According to one aspect, the present technique relates to a method for updating a current operational model of a first device of a communication network, following a modification of at least one software function of a second device of said communication network, with which the first device is interfaced. Such a method comprises, at the level of said first device, the following steps:

[0009] - receiving at least one access data to a digital replica of said second equipment, said digital replica implementing said modified software function;

[0010] - training an experimental model associated with said first equipment, based on data exchanged between said first equipment and said modified software function implemented by said digital replica, delivering at the end of said training an updated operational model;

[0011] - the deployment of said updated operational model as the new current operational model of said first equipment.

[0012] In this way, by using a digital replica of the second equipment, the generation of an updated operational model of the first equipment can be implemented in parallel with the production operation governed by the current operational model of the first equipment. Thus, the production operation according to the current operational model remains maintained as long as the generation of a new model is in progress, and then updated as soon as the generation of this new model is completed.

[0013] In a particular embodiment, said at least one item of access data to the digital replica of the second equipment takes the form of a container comprising said digital replica. According to a particular characteristic, said method comprises a step of instantiating, locally, from said container, said digital replica.

[0014] In this way, the training of the experimental model is carried out locally, within the first equipment or at least within equipment close to the first equipment, thus making it possible to increase the speed of implementation of this training, to secure its execution, and to avoid overloading the communication network thanks to the confinement thus obtained of the data exchanges between the first equipment and the digital replica of the second equipment.

[0015] In a particular embodiment, said method comprises, prior to said reception step, a step of transmitting a request for access to the digital replica of the second device.

[0016] According to a particular characteristic, said access request is issued:

[0017] - periodically; or

[0018] - upon receipt of an order to recover said digital replica; or

[0019] - upon receipt of information representing the availability of an updated digital replica of said second equipment; or

[0020] - detection, via monitoring of at least one dedicated performance indicator, of a drop in performance associated with exchanges between the first equipment and the second equipment.

[0021] In this way, the training of an experimental model with a view to obtaining a new current operational model of the first equipment can in particular be triggered automatically, on the basis of one or more criteria, which makes it possible to obtain an updated operational model more quickly for the first equipment. The duration during which a current operational model of the first equipment remains in force - even when this model has become obsolete or unsuitable following the modification or deployment of at least one software function on a second equipment with which the first equipment is interfaced - is thus considerably reduced.

[0022] In a particular embodiment, said digital replica is implemented in the form of a reinforcement learning environment.

[0023] In this way, the tedious and time-consuming task of manually specifying and writing a test set each time a software function is introduced or updated within a communication network device is no longer required, since trial-and-error learning can be implemented thanks to the provision of a reinforcement learning environment. The manufacturer or administrator of the device whose software function is modified is thus also freed from the constraint of having to document the deployed updates in detail, and can notably provide its customers with a device and / or a digital replica in a “black box” mode.

[0024] According to a particular characteristic, in this embodiment, said training step comprises:

[0025] - a so-called exploratory step, corresponding to a first training phase of the reinforcement learning type, in which at least one exchange between said first equipment and said modified software function implemented by said digital replica is used to train a first experimental model, said exploratory step delivering a data set in which at least part of said exchanges is traced;

[0026] - a so-called consolidation step, corresponding to a second training phase of the supervised learning type, in which said data set is used to train a second experimental model, said consolidation step delivering, at the end of said supervised learning, said updated operational model.

[0027] In a particular embodiment, during said exploratory step, each exchange of said at least one exchange comprises:

[0028] - the issue of an action to said digital replica;

[0029] - receiving, in response to said action, status data and reward data from said digital replica;

[0030] the successive actions issued to the digital replica within the framework of said at least one exchange being determined so as to tend to maximize over time the reward data received in response to these actions, said exchanges being traced within said data set only when a predetermined reward level is reached.

[0031] In this way, the proposed technique allows the implementation of hybrid learning combining the advantages of reinforcement learning and supervised learning: the exploratory step makes it possible to identify without operational impact the best communication sequences between devices, and the consolidation step to deduce the best forced operation (i.e. the message sequences closest to the most relevant message sequences tested during the exploratory step) to be implemented in practice in production conditions.

[0032] According to a particular characteristic, said training step comprises a curative step of manual and / or automatic processing of said data set, prior to said consolidation step.

[0033] In this way, it is notably possible to retain human expertise in the training process of the experimental model, for example to clean, filter, and / or enrich the dataset obtained at the end of the exploratory stage before it is used as a learning base for the implementation of supervised training during the consolidation stage.

[0034] According to another aspect, the present technique also relates to a device for updating a current operational model of a first piece of equipment of a communication network, following a modification of at least one software function of a second piece of equipment of said communication network, with which the first piece of equipment is interfaced. Such a device comprises, within said first piece of equipment:

[0035] - means for receiving at least one item of access data to a digital replica of said second equipment, said digital replica implementing said modified software function;

[0036] - means for training an experimental model associated with said first equipment, based on data exchanged between said first equipment and said modified software function implemented by said digital replica, delivering at the end of said training an updated operational model;

[0037] - means of deploying said updated operational model as a new current operational model of said first equipment.

[0038] According to another aspect, the proposed technique also relates to a computer program product downloadable from a communication network and / or stored on a computer-readable medium and / or executable by a microprocessor, comprising program code instructions for executing a method of updating a current operational model of a first equipment of a communication network as described previously in any of its embodiments, when executed on a computer.

[0039] The proposed technique also relates to a computer-readable recording medium on which is recorded a computer program comprising program code instructions for executing the steps of the method as described above, in any of its embodiments.

[0040] Such a recording medium may be any entity or device capable of storing the program. For example, the medium may include a storage medium, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or a magnetic recording medium, for example a USB flash drive or a hard disk.

[0041] On the other hand, such a recording medium may be a transmissible medium such as an electrical or optical signal, which may be conveyed via an electrical or optical cable, by radio or by other means, so that the computer program contained therein is remotely executable. The program according to the invention may in particular be downloaded over a network, for example the Internet.

[0042] The various embodiments mentioned above can be combined with each other to implement the invention. Figures

[0043] Other characteristics and advantages of the invention will appear more clearly on reading the following description of a preferred embodiment, given as a simple illustrative and non-limiting example, and the appended drawings, among which:

[0044] [Fig-1] illustrates the general principle of a method for updating a current operational model of equipment in a communication network, in a particular embodiment of the proposed technique;

[0045] [Fig.2] presents a sequence diagram of the method of updating a current operational model of equipment in a communication network, in a particular embodiment of the proposed technique;

[0046] [Fig.3] describes a simplified architecture of a device for implementing the proposed technique, in a particular embodiment. Detailed description of the invention

[0047] This application makes it possible to remedy some of the aforementioned drawbacks.

[0048] In all figures of this document, elements and steps of the same nature are designated by the same numerical reference. Furthermore, in the following description, the terms "first" and "second" are intended only, unless explicitly stated otherwise, to allow a distinction to be made between two elements, and do not imply a priori the existence of any order relationship between these elements.

[0049] According to a first aspect, the present technique relates to a method for updating a current operational model of a device in a communication network. By current operational model, we mean here the set of software rules - implemented in the form of program code instructions executed by at least one processor - which govern the current operation of the device in question, including the management of its interactions with other devices in the communication network with which it is interfaced. In this context, a "device" according to the present technique is understood in the broad sense: it can correspond in a conventional manner to a physical resource (e.g. all or part of a machine), but also to a service implemented within a machine such as for example a virtualization service used for the implementation of virtualized architectures.Software rules can then, for example, correspond to rules associated with instances of virtualized functions of such a virtualized architecture.

[0050] The general principle of the proposed method is illustrated in relation to [Fig. 1]. In the particular embodiment presented in this figure, this method relates more particularly to the updating of the current operational model MOC of a first equipment El of a communication network, following a modification of at least one FL software function of a second E2 equipment of the communication network, with which the first equipment is interfaced. Such a software modification is part, for example, of corrective and / or evolutionary maintenance operations implemented on the second E2 equipment, and may in particular result from an update of one or more already existing software functions, and / or a deployment of new software functions within the second E2 equipment.

[0051] The method according to the present technique is implemented within the first equipment El, and it comprises the steps detailed below, in a particular embodiment.

[0052] In a step 11, the first equipment E1 receives, from a third-party equipment of the communication network (typically from the second equipment E2, but this example is not limiting), at least one access data to a digital replica RNE2 of said second equipment E2. By digital replica, is meant here a digital copy of the physical object that is the second equipment E2, i.e. a virtual representation designed to faithfully simulate, in software, the states, characteristics and operating dynamics of the second equipment E2. According to a particular characteristic, this digital replica corresponds for example to a digital twin of the second equipment E2. Such a digital replica - the creation of which is not the subject of the present technique - implements in particular the modified software function FL of the second equipment E2.

[0053] The access data to the RNE2 digital replica can take different forms, depending on the implementation mode of this replica.

[0054] According to a particular characteristic, the access data received in step 11 take for example the form of coordinates (e.g. IP address, URL, port number, etc.) allowing the first equipment El to connect to a remote server (which may or may not be integrated into the equipment E2) within which the digital replica RNE2 is deployed: the first equipment El then remotely accesses the digital replica of the second equipment E2, this digital replica being installed on a machine different from the first equipment El.

[0055] Alternatively, according to another particular characteristic, the access data received in step 11 may also take the form of a container (or “package” in English) comprising the digital replica, in other words all the data necessary for the instantiation of such a digital replica. This container is for example received from a remote server, which may or may not be integrated into the equipment E2. In this case, the method according to the present technique then comprises a step of instantiating, locally, from the received container, the replica digital replica of the second E2 equipment: the first El equipment then locally accesses the digital replica of the second E2 equipment, this RNE2 digital replica being installed within the first El equipment itself (for example within a memory of the first El equipment) or at least within a nearby equipment forming part of the same local or private network (eg the same intranet) as the first El equipment, which has the advantage of not unnecessarily overloading the communication network with multiple exchanges of messages between remote network equipment during the subsequent use of the digital replica, in particular when the El and E2 equipment are interfaced via a wide area communication network.

[0056] In any event, at the end of step 11, the first device E1 is able to communicate with the digital replica of the device E2, either because it has the information allowing it to connect to a device within which an instance of such a digital replica is deployed, or because it has itself been able to download and deploy within itself an instance of such a digital replica. Optionally, a verification operation is carried out by the first device before launching such a download, in order to ensure that the digital replica about to be downloaded corresponds to a recent version (i.e. a new, up-to-date version) of the digital replica, and not to an old version which would have already been previously downloaded and installed within the first device (in which case the download is not carried out).Such verification can for example be implemented by comparing distinct control hash data identifying different versions of a digital replica of the same equipment.

[0057] In a step 12, the digital replica is used to train an experimental model associated with the first equipment El. More particularly, by “experimental model”, it is meant that such training is carried out in an experimental mode (as opposed to an operational mode), that is to say without impact on the current operational operation of the first equipment El within the communication network. In other words, during this training step 12 of the experimental model, the current operational model MOC which governs the operation of the first equipment El remains in force. The training is therefore carried out in parallel with the production operation of the first equipment El, without impact on this operation (there is therefore no interruption of service at the level of the equipment El which would be due to this training).According to the proposed technique, access to the digital replica of the second equipment E2 is therefore used by the first equipment El to determine a new operating model of the first equipment El adapted to the developments of the second equipment E2, i.e. to the modifications previously made to one or more of the software functions of this. equipment E2 with which the first equipment El is interfaced. To this end, the experimental model is trained on the basis of data exchanged between the first equipment El and the digital replica of the second equipment E2. More particularly, as detailed later in this document, machine learning techniques are implemented, in which on the one hand data associated with requests issued by the first equipment El to the digital replica and on the other hand data obtained from the digital replica (and in particular from the modified software function implemented by the digital replica) in response to these requests are analyzed and used to train the experimental model.

[0058] In a step 13, once the experimental model is considered sufficiently trained, it can be deployed as a new current operational model of the first equipment El, thus governing the new operation of this equipment in production conditions. In other words, this new current operational model replaces the operational model previously implemented within the first equipment El.

[0059] In a particular embodiment, step 11 of receiving access data to the digital replica RNE2 of the second equipment E2 follows the transmission, by the equipment E1, of a request for access to such a digital replica. Such a request is for example transmitted to the second equipment E2, or to a third-party equipment (for example a server dedicated to hosting or providing digital replicas of equipment of the communication network).

[0060] The transmission by the equipment El of such an access request can be triggered under various conditions, in complementary or alternative embodiments of the proposed technique described below for illustrative and non-limiting purposes.

[0061] Thus, the access request is for example issued repeatedly, possibly periodically. For example, such a request is issued daily by the first device, preferably at a time when traffic on the communication network is reduced (e.g., at night). In this way, in particular when the access data to the digital replica takes the form of a potentially relatively large container to be downloaded from a remote server, it is possible to limit the impact of this download operation on the performance of the communication network, by triggering it preferentially during a period of low load on this network.

[0062] The access request can also be issued upon receipt, by the first equipment E1, of a command to recover the digital replica, or upon receipt of information representing the availability of a new digital replica (i.e. a recently updated digital replica) from the second equipment E2. such command and / or such information are for example received from the second device E2, or from a third-party device different from the second device. According to a particular characteristic, the command to recover a digital replica can in particular be forced manually, for example by interaction of a user on a dedicated element of a graphical interface made available to him. Furthermore, the reception of information representative of the availability of a new digital replica can be part of a prior subscription, by the first device, with another device (typically the second device), to a notification service for events of this type.

[0063] According to yet another possibility, the access request is issued upon detection, by the first equipment E1, of a drop in performance associated with the exchanges between the first equipment E1 and the second equipment E2. Such a drop in performance is evaluated via the monitoring of at least one dedicated performance indicator (or KPI, from the English “Key Performance Indicator”), such as for example an abnormally high rate of error messages received from the second equipment in response to requests issued to its destination by the first equipment, or any other relevant indicator.

[0064] Various embodiments of the training step 12 are now described in more detail. More particularly, in a particular embodiment, the digital replica RNE2 of the second equipment is implemented in the form of a reinforcement learning environment, thus allowing the implementation of reinforcement learning techniques in this step 12 of training an experimental model. As described below, the originality of the proposed technique lies however in that the reinforcement learning implemented here does not have as its primary objective to train a model (as is generally the case in conventional approaches), but to construct a JD data set intended to serve as a learning base for the implementation of another, subsequent learning phase, this time based no longer on reinforcement learning but on supervised learning.

[0065] Thus, in a particular embodiment of the proposed technique, training step 12 comprises:

[0066] - a so-called exploratory step 121, corresponding to a first training phase of the reinforcement learning type, in which a succession of exchanges (at least one) between the first equipment E1 and the modified software function implemented by the digital replica RNE2 of the second equipment E2 is used to train a first experimental model, this exploratory step delivering a data set JD in which at least part of said exchanges is traced;

[0067] - a so-called consolidation step 122, corresponding to a second phase supervised learning type training, in which the JD data set obtained at the end of the first phase is used (possibly after control and adaptation by a human operator) to train a second experimental model, this consolidation step delivering, once the supervised learning is completed, the updated operational model MOA intended to be deployed as the new current operational model MOC of the first equipment (in accordance with step 13 of [Fig.l]).

[0068] More particularly, during the first training phase by reinforcement learning, according to a trial-and-error type approach, multiple data exchanges (generally very numerous, e.g. several thousands or even millions, without this being limiting, a limited number of exchanges, e.g. only a few tens or hundreds, which may prove sufficient in certain cases) are carried out between the first device El and the digital replica RNE2 of the second device, with a view to broadly exploring the results obtained in response to different parameter values ​​transmitted within a message and / or to different sequences of messages.

[0069] Each of these exchanges includes, according to the usual vocabulary in reinforcement learning:

[0070] - the transmission by the first device of action data to the digital replica of the second device;

[0071] - receiving, in response to said action, status data and reward data from said digital replica.

[0072] State data are representative of a new state of the reinforcement learning environment, i.e. of the digital replica, following the action carried out on this environment.

[0073] The reward data is representative of one or more metrics of effectiveness of the action performed on the environment, or at least data allowing the first device to calculate such metrics.

[0074] The successive actions issued to the digital replica are determined by a reinforcement algorithm, so as to tend to maximize over time the reward data received in response to these actions. According to the proposed technique, once a predetermined reward level is reached, all subsequent data exchanges carried out during this exploratory step are traced (for example in a trace file, in a database, or any other suitable data structure), thus participating in the construction of a JD data set representative of sequences of actions which can be considered relevant or effective (because associated with high obtained rewards) with respect to the digital replica implementing the modified software function.

[0075] More particularly, in a particular embodiment of the proposed technique, each sample of such a data set JD comprises salient information relating to a data exchange between the first equipment E1 and the digital replica RNE2 (and in particular the modified software function) of the second equipment E2. Such salient information comprises for example:

[0076] - data relating to the action sent by the first device to the digital replica of the second device as part of the data exchange in question, comprising for example a transmitted command and associated parameters, an identifier of the first device (taking for example the form of a decentralized identifier or DID, for “Decentralized identifier”), and possibly a timestamp representing the time of emission of this action;

[0077] - data relating to the result returned by the digital replica of the second device in response to this action, including for example state data representative of a consequence of the action on the environment, associated reward data, additional data (for example data representative of one or more efficiency metrics with regard to the exchange considered), an identifier of the digital replica of the second device (for example also taking the form of a decentralized identifier), and possibly a timestamp representative of the time of receipt of this response.

[0078] As indicated previously, the JD dataset is then used for the implementation of the consolidation step 122, in which it serves as a learning base for the supervised training of a second experimental model (such training may, depending on the case, be carried out on the basis of a blank model, i.e. from scratch, or from a previous version of the model if its architecture allows it - e.g. “Transformer” type model - and if there is no major change in the actions and states associated with the digital replica implementing the modified software function).

[0079] Optionally, in a particular embodiment, the JD data set obtained at the end of the exploratory step 121 is made available to a user (for example via a dedicated graphical interface provided by the first device) before the consolidation step 122 is carried out, for consultation and possible modifications and / or adaptations before the implementation of the supervised training. Such a curative step of processing the data set prior to its use as a learning base makes it possible to retain human expertise in the process of training the experimental model.

[0080] For clarity, we now present in relation to [Fig.2] a sequence diagram illustrating in time an example of possible interactions between different entities of the first equipment El, including the implementation of the method of updating the current operational model of this equipment, according to some of the particular embodiments described above.

[0081] In production conditions, in an operational mode OP, the current operational model MOC of the first equipment E1 governs the operation of this equipment, and is notably requested, in a step 201, to determine an action to be implemented with respect to a second equipment E2, within the framework of a data exchange 202 between these two equipments. Such a data exchange is established between a communication interface II of the first equipment E1, and a communication interface 12 of the second equipment E2 (the communication interface 12 being the only entity of the second equipment E2 represented in [Fig. 2], this equipment being illustrated here as a “black box”). At the end of each exchange 202, in a step 203, at least one performance indicator associated with the exchange carried out is sent to a control module CTRL of the first equipment E1.Such steps 201, 202 and 203 are repeated for all exchanges between the first equipment E1 and the second equipment E2 in operational mode.

[0082] When the control module CTRL detects, on the basis of one of the performance indicators obtained in step 203, a drop in performance associated with the exchanges between the first equipment E1 and the second equipment E2, it requests from the second equipment E2, via its communication interface II, in a step 204, access to a digital replica RNE2 of this equipment E2. Such a drop in performance may in particular result from the addition and / or modification, within the second equipment E2, of at least one software function FL. In the particular embodiment illustrated in [Fig.2], following this request, the control module CTRL is authorized to download, in a step 205, a container comprising all the data necessary for an installation of the digital replica of the second equipment.By means of this container, the control module CTRL then instantiates within a memory of the first equipment El, in a step 206, an instance of the digital replica RNE2 of the second equipment, this digital replica taking the form of a reinforcement learning environment implementing the modified software function FL. The control module CTRL also instantiates, in a step 207, a first experimental model MEXP1.

[0083] A first PHI phase of reinforcement learning type is then implemented, in which the experimental model MEXP1 is trained on the basis of multiple data exchanges 208 (generally a very large number, e.g. thousands or even millions of exchanges) with the digital replica RNE2, by trial and error, as already described in relation to [Fig.l]. When a predetermined reward level is reached, the subsequent data exchanges are traced, in a step 209, within a data set.

[0084] At the end of the first phase of PHI reinforcement learning, the control module CTRL requests and retrieves, in a step 210, the data set thus constituted.

[0085] This data set is transmitted to a second experimental model MEXP2 instantiated by the control module CTRL in a step 211.

[0086] A second phase PH2 of the supervised learning type is then implemented, in which the experimental model MEXP2 is trained on the basis of the samples present in the data set, during numerous iterations of a step 212.

[0087] A Tissue of the second supervised learning phase PH2, the experimental model MEXP2 is considered sufficiently trained to be used as an updated operational model of the first equipment EL II is therefore recovered by the control module CTRL in a step 213, then deployed in production in a step 214, as a new current operational model MOC of the first equipment El, relevant for the implementation of efficient data exchanges with the second equipment E2 implementing the modified FL software function.

[0088] The embodiments presented previously in relation to figures 1 and 2, which are based on the implementation of two successive learning phases but of distinct types, offer numerous advantages presented in part below.

[0089] Firstly, the implementation of the first phase of reinforcement learning avoids the tedious and time-consuming task of manually specifying and writing a test set each time a software function is introduced or updated within a device in the communication network. Furthermore, when the digital replica of the second device is instantiated locally, for example within a memory of the first device or within a nearby device that is part of the same local or private network (e.g. the same intranet), the acquisition of experience intrinsic to reinforcement learning is carried out locally, possibly directly on the learning machine (i.e.the first equipment), which makes it possible on the one hand to increase the speed of implementation of the tests and to guarantee a certain security in their execution (via the local confinement of these tests), and on the other hand to avoid overloading the communication network (the thousands or millions of message exchanges carried out within the framework of these tests do not transit on the communication network). The update of the operational model of a piece of equipment, and in particular of its communication model with the other equipment of the network, can thus be prepared autonomously within this equipment. In other words, the generation of an updated operational model is implemented in parallel with the production operation governed by the current operational model of this equipment, this operation remaining maintained and not impacted as long as this generation is in progress.

[0090] Secondly, combining two learning methods is of definite interest, by offering a mechanism allowing, throughout the life cycle of the equipment, by the use of efficiency metrics, to combat the inevitable obsolescence in the long term of an operational model which would result solely from supervised learning (i.e. without taking risks), by allowing an update of the operational model on the basis of new data resulting from reinforcement learning (i.e. non-fixed data, resulting from the exploration of new options and possibilities of dialogue with the equipment whose software function has been updated). In other words, the proposed technique allows, thanks to the exploratory step, to identify without operational impact the best sequences of communications between equipment, then to deduce from them, thanks to the consolidation step, the best forced march (i.e.the message sequences closest possible to the most relevant message sequences previously tested during the exploratory stage) to be implemented in practice under production conditions.

[0091] Thirdly, the proposed technique is also of interest to the manufacturer or administrator of the equipment whose at least one software function is modified, since he is freed from the constraint of having to systematically document in detail the proposed updates or share their source code. The present technique can indeed be implemented even when the second equipment or its digital replica are implemented in a "black box" mode, with obfuscated or undocumented code for example.

[0092] Fourthly, it is interesting to note that the proposed technique can also be applied to the update of a current operational model of a device of a communication network when not only one but several other devices with which it is interfaced have been the subject of recent software function updates. In this case, a reinforcement learning phase is conducted for each of these other devices, in relation to a digital replica of each device, and the different data sets obtained following these reinforcement learning phases are concatenated to form a single set used in the second supervised learning phase to train an experimental model and deliver an updated operational model that is efficient and relevant for multi-device communications.

[0093] Fifthly, the proposed technique can also be used to select a preferred software function to be deployed within a piece of equipment from a plurality of available software functions (e.g. developed for testing purposes), by testing (e.g. using different versions of a digital replica of a piece of equipment) these different possible options in relation to the equipment already deployed. In this way, it is possible to anticipate and / or compare the potential impacts of each software function with regard to different criteria (performance, protocol compatibility, etc.), and choose the most suitable software function on this basis.

[0094] According to another aspect, the proposed technique also relates to a device for updating a current operational model of a first device of a communication network, following a modification of at least one software function of a second device of said communication network, with which the first device is interfaced. Such a device, integrated within the first device, is capable of carrying out the method previously described in any one of its embodiments. More particularly, such a device according to the present technique comprises:

[0095] - means for receiving at least one item of access data to a digital replica of said second equipment, said digital replica implementing said modified software function;

[0096] - means for training an experimental model associated with said first equipment, based on data exchanged between said first equipment and said modified software function implemented by said digital replica, delivering at the end of said training an updated operational model;

[0097] - means of deploying said updated operational model as a new current operational model of said first equipment.

[0098] [Fig. 3] represents, in a schematic and simplified manner, the structure of such a device, in a particular embodiment. The device according to the proposed technique, implemented within a first equipment of a communication network, comprises for example a memory 31 consisting of a buffer memory M, a processing unit 32, equipped for example with a microprocessor iiP, and controlled by the computer program Pg 33, implementing steps of the method for updating a current operational model of the first equipment according to at least one embodiment of the invention. To this end, the device also comprises at least one communication interface (for example an Ethernet communication interface), allowing it to receive and transmit messages from and to other equipment present in the communication network.

[0099] Upon initialization, the code instructions of the computer program 33 are loaded into the buffer memory before being executed by the processor of the processing unit 32. The processing unit 32 receives as input E, for example, a message signaling to it the availability of a new digital replica of a second device with which the first device is interfaced.

[0100] The microprocessor of the processing unit 32 then carries out the steps of the method for updating the current operational model of the first equipment, according to the ins- instructions of the computer program 33. More particularly, the first equipment receives access data to the digital replica of the second equipment, then uses this digital replica to train an experimental model based on data exchanged in particular with a modified software function implemented by said digital replica. This training delivers at its end an updated operational model, which is then transmitted by the processing unit 32 at output S, for deployment in production within the first equipment, in place of the current operational model which has become obsolete.

Claims

Claims

1. Method for updating a current operational model (COM) of a first device (El) of a communication network, following a modification of at least one software function (FL) of a second device (E2) of said communication network, with which the first device is interfaced, said method being characterized in that it comprises, at the level of said first device, the following steps: - reception (11) of at least one access data to a digital replica (RNE2) of said second equipment, said digital replica implementing said modified software function; - training (12) of an experimental model associated with said first equipment, as a function of data exchanged between said first equipment and said modified software function implemented by said digital replica, delivering at the end of said training an updated operational model (MO A); - deployment (13) of said updated operational model as a new current operational model of said first equipment.

2. Method according to claim 1, characterized in that said at least one item of access data to the digital replica of the second equipment takes the form of a container comprising said digital replica.

3. Method according to claim 2, characterized in that it comprises a step of instantiating, locally, from said container, said digital replica.

4. Method according to claim 1, characterized in that it comprises, prior to said reception step (11), a step of transmitting a request for access to the digital replica of the second equipment.

5. Method according to claim 4, characterized in that said access request is issued: - periodically; or - upon receipt of an order to recover said digital replica; or - upon receipt of information representing the availability of an updated digital replica of said second equipment; or - detection, via monitoring of at least one dedicated performance indicator, of a drop in performance associated with exchanges between the first equipment and the second equipment.

6. Method according to claim 1, characterized in that said digital replica is implemented in the form of a reinforcement learning environment.

7. Method according to claim 6, characterized in that said training step (12) comprises: - a so-called exploratory step (121), corresponding to a first training phase (PHI) of the reinforcement learning type, in which at least one exchange between said first equipment and said modified software function implemented by said digital replica is used to train a first experimental model, said exploratory step delivering a data set (JD) in which at least part of said exchanges is traced; - a so-called consolidation step (122), corresponding to a second training phase (PH2) of the supervised learning type, in which said data set (JD) is used to train a second experimental model, said consolidation step delivering, at the end of said supervised learning, said updated operational model (MOA).

8. Method according to claim 7, characterized in that during said exploratory step, each exchange of said at least one exchange comprises: - the issue of an action to said digital replica; - receiving, in response to said action, state data and reward data from said digital replica; the successive actions sent to the digital replica within the framework of said at least one exchange being determined so as to tend to maximize over time the reward data received in response to these actions, said exchanges being traced within said data set only when a predetermined reward level is reached.

9. Method according to claim 7, characterized in that said training step (12) comprises a curative step of manual or automatic processing of said data set (JD), prior to said consolidation step (122).

10. Device for updating a current operational model of a first piece of equipment of a communication network, following a modification of at least one software function of a second piece of equipment of said communication network, with which the first piece of equipment is interfaced, said device being characterized in that it comprises, within said first equipment: - means for receiving at least one item of access data to a digital replica of said second equipment, said digital replica implementing said modified software function; - means for training an experimental model associated with said first equipment, based on data exchanged between said first equipment and said modified software function implemented by said digital replica, delivering at the end of said training an updated operational model; - means of deploying said updated operational model as a new current operational model of said first equipment.

11. Computer program product downloadable from a communications network and / or stored on a computer-readable medium and / or executable by a microprocessor, characterized in that it comprises program code instructions for executing a method according to any one of claims 1 to 9, when executed by a computer.

Images