Method and apparatus for training a charging station information coordination model, and product

Abstract

To provide a method for training an information adjustment model of a charging station that can improve the real-time property of a charging station to determine target operation information, rationality, and the cooperativeness between the charging stations.SOLUTION: An above-mentioned method includes: acquiring a charging request and determining environment state information corresponding to charging stations in a charging station group; determining target operation information on a charging station to the charging request based on the environment state information corresponding to the charging stations via an initial policy network; determining an expected value of cumulative rewards corresponding to the charging request based on the environment state information corresponding to the charging stations and the target operation information via an initial value network; and training the initial policy network and the initial value network by using a deep deterministic policy gradient method, and determining the trained policy network as an information adjustment model corresponding to the charging station.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to the field of artificial intelligence technologies, in particular to the field of multi-agent reinforcement learning technologies, and in particular to a method and apparatus for training an information coordination model of a charging station, as well as a method, apparatus, electronic device, storage medium and computer program for selecting a charging station that can be used in a charging scenario of the charging station. [Background technology]

[0002] As social interest in clean energy and environmental protection issues increases, more and more people are choosing rechargeable vehicles. In urban areas, many public charging stations have been built to meet the ever-increasing demand for charging. However, these charging stations generally have a uniform dynamic price adjustment at fixed time intervals (e.g., every hour), so they universally face the problems of imbalance in charging demand and low utilization rate, resulting in poor charging experience for drivers of rechargeable vehicles. The low utilization rate of charging stations discourages operators from building more charging stations, which hinders the further spread of rechargeable vehicles. Summary of the Invention [Means for solving the problem]

[0003] The present disclosure provides a method and apparatus for training an information coordination model of a charging station, as well as a method, apparatus, electronic device, storage medium and computer program for selecting a charging station.

[0004] According to a first aspect, there is provided a method for training an information coordination model of a charging station, the method comprising: receiving a charging request and determining environmental state information corresponding to each charging station in a collection of charging stations; determining, via an initial policy network, target operation information of each charging station in the set of charging stations for a charging request based on environmental state information corresponding to each charging station in the set of charging stations; determining an expected value of a cumulative reward corresponding to the charging request based on environmental state information and target operation information corresponding to each charging station in the set of charging stations via an initial value network; A step of training an initial policy network and an initial value network using a deep decision policy gradient method to obtain a trained policy network and a value network, in which during the training process, the initial value network is updated using a time difference method, and the initial policy network is updated with the goal of maximizing the expected value of the cumulative reward corresponding to the charging request; determining the trained policy network as an information coordination model corresponding to each charging station in the charging station collection.

[0005] According to a second aspect, there is provided a method for selecting a charging station, the method comprising: obtaining a charging request; determining environmental state information corresponding to each charging station in the collection of charging stations; Determine target operation information of the charging station for the charging request according to the environmental state information corresponding to the charging station through the trained information coordination model corresponding to the charging station, where the information coordination model is obtained by performing multi-agent reinforcement learning based on a deep decision policy gradient method after sensing each other's environmental state information between the charging stations in the charging station set; displaying target operation information of each charging station in the set of charging stations in response to the charging request; receiving a selection command and determining a target charging station from the set of charging stations based on the selection command.

[0006] According to a third aspect, there is provided an apparatus for training an information coordination model of a charging station, the apparatus comprising: a first determining unit configured to obtain a charging request and determine environmental state information corresponding to each charging station in the collection of charging stations; a second determination unit configured to determine, via an initial policy network, target operation information of each charging station in the set of charging stations for a charging request based on environmental state information corresponding to each charging station in the set of charging stations; a third determination unit configured to determine, via the initial value network, an expected value of a cumulative reward corresponding to the charging request based on the environmental state information and the target operation information corresponding to each charging station in the charging station collection; A training unit, the training unit being configured to use a deep decision policy gradient method to train an initial policy network and an initial value network to obtain a trained policy network and a value network, where in the training process, the training unit updates the initial value network using a time difference method, and updates the initial policy network with the goal of maximizing the expected value of a cumulative reward corresponding to a charging request; and a fourth determination unit configured to determine the trained policy network as an information coordination model corresponding to each charging station in the charging station collection.

[0007] According to a fourth aspect, there is provided an apparatus for selecting a charging station, the apparatus comprising: an acquiring unit configured to acquire a charging request; a fifth determining unit configured to determine environmental state information corresponding to each charging station in the collection of charging stations; a sixth determination unit, configured for each charging station in the charging station set to determine target operation information of the charging station for a charging request based on environmental state information corresponding to the charging station through a trained information adjustment model corresponding to the charging station, where the information adjustment model is obtained by sensing each other's environmental state information between the charging stations in the charging station set and then performing multi-agent reinforcement learning based on a deep decision policy gradient method; a display unit configured to display target operation information of each charging station in the collection of charging stations in response to a charging request; and a receiving unit configured to receive a selection command and determine a target charging station from the collection of charging stations based on the selection command.

[0008] According to a fifth aspect, there is provided an electronic device, the electronic device comprising: At least one processor; a memory communicatively connected to the at least one processor; Here, the memory stores instructions executable by at least one processor, and the instructions are executed by the at least one processor such that the at least one processor can perform the method described in either the first or second implementation method.

[0009] According to a sixth aspect, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to carry out a method as described in either of the implementations of the first or second aspect.

[0010] According to a seventh aspect, there is provided a computer program comprising a computer program which, when executed by a processor, realises the method described in either of the realisation methods of the first or second aspect.

[0011] According to the technology disclosed herein, a method for training an information coordination model of a charging station is provided, in which the charging station is used as an agent, and multi-agent reinforcement learning is performed based on a deep decision policy gradient method. Through training, a policy network capable of determining target operation information in real time can be obtained as an information coordination model, thereby improving the real-time nature and rationality of the charging station in determining the target operation information. In the model training stage, the goal is to maximize the cumulative reward of the entire set of charging stations, and the agents can sense information about the entire environment from each other, thereby improving the coordination between charging stations in the set of charging stations. This helps solve the problem of poor charging coordination between charging stations, and improves the utilization rate of charging stations.

[0012] It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, and are not intended to limit the scope of the present disclosure. Other features of the present disclosure will be easily understood through the following description. [Brief description of the drawings]

[0013] The drawings are used for better understanding of the present scheme and are not intended to be limiting of the present disclosure.

[0014] [Figure 1] FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure may be applied. [Diagram 2] 1 is a flow chart of an embodiment of a method for training an information coordination model of a charging station according to the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of an application scenario of the method for training an information coordination model of a charging station according to the present embodiment; [Figure 4] FIG. 13 is a schematic diagram of a process flow of an agent pooling module according to the embodiment. [Diagram 5]FIG. 2 is a schematic diagram illustrating selection of a subset from a set of charging stations according to an embodiment of the present invention. [Figure 6] 11 is a flowchart of yet another embodiment of a method for training an information coordination model of a charging station in accordance with the present disclosure. [Figure 7] 1 is a flow chart of an embodiment of a method for selecting a charging station according to the present disclosure. [Figure 8] FIG. 2 is a structural diagram of an embodiment of an apparatus for training an information coordination model of a charging station according to the present disclosure; [Figure 9] FIG. 2 is a structural diagram of an embodiment of a device for selecting a charging station according to the present disclosure. [Figure 10] FIG. 1 is a schematic structural diagram of a computer system suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. In order to facilitate understanding, various details of the embodiments of the present disclosure are included, and they should be understood as merely illustrative. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, the following description omits the description of known functions and structures.

[0016] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and other processing of relevant user personal information shall all comply with the provisions of relevant laws and regulations and not violate public order and morals.

[0017] FIG. 1 illustrates an example architecture 100 of a method and apparatus for training an information coordination model of a charging station and a method and apparatus for selecting a charging station to which the present disclosure can be applied.

[0018] 1, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The terminal devices 101, 102, 103 are communicatively connected to form a topology network, and the network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0019] The terminal devices 101, 102, and 103 may be hardware devices or software that support network connection and perform data exchange and data processing. If the terminal devices 101, 102, and 103 are hardware, they may be various electronic devices that support functions such as network connection, information acquisition, exchange, display, and processing, including, but not limited to, smartphones, tablets, in-vehicle computers, e-book readers, laptop portable computers, and desktop computers. If the terminal devices 101, 102, and 103 are software, they may be installed in the electronic devices listed above. They may be implemented as multiple software or software modules used to provide distributed services, or as a single software or software module. There is no particular limitation here.

[0020] The server 105 may be a server that provides various services, such as obtaining a background processing server of a trained information coordination model corresponding to each charging station by performing multi-agent reinforcement learning using a deep decision policy gradient method based on a training sample provided by the terminal devices 101, 102, and 103. As another example, based on a charging request provided by the terminal devices 101, 102, and 103, for a target charging station, target operation information for the charging request is determined by an information coordination model corresponding to the target charging station, so that a user can select a background processing server of the target charging station from a set of charging stations based on the target operation information. As an example, the server 105 may be a cloud server.

[0021] The server may be hardware or software. If the server is hardware, it may be realized as a distributed server cluster consisting of multiple servers, or as a single server. If the server is software, it may be realized as multiple pieces of software or software modules (e.g., software or software modules for providing distributed services), or as a single piece of software or software module. There are no particular limitations here.

[0022] In addition, the method for training an information coordination model of a charging station and the method for selecting a charging station provided by the embodiment of the present disclosure may be executed by a server, may be executed by a terminal device, or may be executed by the server and the terminal device in cooperation with each other. Correspondingly, each part (e.g., each unit) included in the device for training an information coordination model of a charging station and the device for selecting a charging station may be all provided in the server, all provided in the terminal device, or may be provided in the server and the terminal device, respectively.

[0023] It should be understood that the number of terminal devices, networks, and servers in Fig. 1 is merely exemplary. Any number of terminal devices, networks, and servers may be included according to the needs of implementation. If the electronic device on which the method for training the information coordination model of the charging station and the method for selecting the charging station are executed does not need to transmit data with other electronic devices, the system architecture may include only the electronic device (e.g., a server or a terminal device) on which the method for training the information coordination model of the charging station and the method for selecting the charging station are executed.

[0024] Referring to FIG. 2, FIG. 2 is a flowchart of a method for training an information coordination model of a charging station provided by an embodiment of the present disclosure, where the flow 200 includes the following steps.

[0025] Step 201: Obtain a charging request, and determine environmental state information corresponding to each charging station in a collection of charging stations.

[0026] In this embodiment, an entity executing the method for training an information coordination model of a charging station (e.g., a terminal device or a server in FIG. 1 ) can obtain a charging request remotely or locally based on a wired network connection manner or a wireless network connection manner, and determine environmental state information corresponding to each charging station in a charging station collection.

[0027] The charging request may be a request issued by a user in the charging vehicle through a terminal device such as a smartphone or an on-board computer, characterizing that the charging vehicle needs to be charged. When the charging vehicle needs to be charged, a user such as a corresponding driver or passenger initiates a charging request at the charging platform;

number

[0028] For a received charging request, the target operation information of each charging station in the charging station set needs to be fed back to the user so that the user can select a charging station from the charging station set based on the fed back target operation information.

[0029] Step 202: Determine, via an initial policy network, target operation information of each charging station in the charging station collection for a charging request based on environmental state information corresponding to each charging station in the charging station collection.

[0030] In this embodiment, the execution entity can determine, via the initial policy network, target operation information of each charging station in the charging station set in response to a charging request based on environmental state information corresponding to each charging station in the charging station set.

[0031] The target operational information may be any operational information that can be adjusted during operation of the charging station, and as an example, the target operational information may be operational information such as price information, charging time information, charging speed information, etc. of the charging station.

[0032] Taking the price information as an example, the price information may specifically be a charging price per kWh of the charging station, which includes an electricity price per kWh of the charging station and a service fee price per kWh of the charging station.

[0033]

number

[0034] Furthermore, the total profit of the charging stations in the charging station collection is the sum of the profits corresponding to all charging requests.

[0035] The charging station set includes a plurality of charging stations. For example, the charging station set is all charging stations in a predefined division area (e.g., obtained by dividing by administrative district or by a specified area). Each charging station can be an agent,

number

[0036]

number

[0037] In this embodiment, an actor (policy network)-critic (value network) architecture is used, where the policy network is used to determine operation information, i.e., target operation information of each charging station in the charging station set for a charging request, based on environmental state information corresponding to each charging station in the charging station set.

[0038]

number

[0039]

number

[0040] Here, the policy network may use a network model such as a deep neural network, for example, a deep convolutional network or a residual network.

[0041] Step 203: Determine an expected value of a cumulative reward corresponding to the charging request based on the environmental state information and the target operation information corresponding to each charging station in the charging station set through the initial value network.

[0042] In this embodiment, the execution entity can determine an expected value of a cumulative reward corresponding to a charging request based on the environmental state information and the target operation information corresponding to each charging station in the charging station set through the initial value network. The expected value of the cumulative reward can be a score of the target operation information of the charging station determined by the policy network, and the higher the value, the better the target operation information characterizing the policy network.

[0043]

number

[0044]

number

[0045] In this embodiment, the value network may use a network model such as a deep neural network, for example, a deep convolutional network, a residual network, etc. Here, the value network may use the same network structure as the policy network, or may use a network structure different from the policy network.

[0046] Step 204: Using a deep decision policy gradient method, train the initial policy network and the initial value network to obtain a trained policy network and a value network.

[0047] In this embodiment, the execution entity uses a deep decision policy gradient method to train the initial policy network and the initial value network, thereby obtaining a trained policy network and a value network. Here, in the training process, the initial value network is updated by a time difference method, and the initial policy network is updated with the goal of maximizing the expected value of the accumulated reward corresponding to the charging request.

[0048] The Deep Deterministic Policy Gradient (DDPG) algorithm is a policy learning method that incorporates a deep learning neural network into the DPG (Deterministic Policy Gradient). Compared with the DPG algorithm, the DDPG algorithm uses a deep neural network as a simulation of the policy function and value function, and then uses deep learning methods to train the policy function and value function.

[0049] As an example,

number

[0050] Step 205: Determine the trained policy network as an information coordination model corresponding to each charging station in the charging station collection.

[0051] In this embodiment, the executive may determine the trained policy network as an information coordination model corresponding to each charging station in the charging station collection.

[0052] As an example, an information coordination model can be deployed to determine target operating information in real-time for each charging station in a collection of charging stations based on a received charging request.

[0053] Still referring to FIG. 3, FIG. 3 is a schematic diagram 300 of an application scene of the method for training an information coordination model of a charging station according to the embodiment. In the application scene of Figure 3, first, the server 301 obtains a charging request from the terminal device 302, and determines environmental state information corresponding to each charging station in the charging station set 303. Then, through the initial policy network 304, based on the environmental state information corresponding to each charging station in the charging station set 303, determines target operation information of each charging station in the charging station set for the charging request. Through the initial value network 305, based on the environmental state information and target operation information corresponding to each charging station in the charging station set 303, determines an expected value of a cumulative reward corresponding to the charging request. By training the initial policy network and the initial value network using a deep decision strategy gradient method, a trained policy network 306 and a value network 307 can be obtained. Here, during the training process, the initial value network is updated by a time difference method, and the initial policy network is updated with the goal of maximizing the expected value of the cumulative reward corresponding to the charging request. The trained policy network 306 is determined as an information adjustment model corresponding to each charging station in the charging station set.

[0054] In this embodiment, a method for training an information coordination model of a charging station is provided, in which the charging station is an agent, and multi-agent reinforcement learning is performed based on the deep decision policy gradient method. Through training, a policy network that can determine target operation information in real time can be obtained as the information coordination model, thereby improving the real-timeness and rationality of the charging station in determining the target operation information. In the model training stage, the goal is to maximize the cumulative reward of the entire set of charging stations, and the agents can sense information about the entire environment from each other, thereby improving the coordination between the charging stations in the charging station set. This helps solve the problem of uncoordinated charging between charging stations, and improves the utilization rate of the charging stations.

[0055] In some optional implementations of this embodiment, the execution entity can execute step 203 by the following steps:

[0056] First step: via an agent pooling module, determine integrated display information characterizing all charging stations in the charging station collection based on the environmental state information and the target operation information corresponding to each charging station in the charging station collection.

[0057] Second step: Determine the expected value of the cumulative reward corresponding to the charging request based on the integrated display information via the initial value network.

[0058]

number

[0059] In this implementation method, the environmental state information and target operation information corresponding to all charging stations in the charging station set are reduced in dimension through the agent pooling module, thereby solving the problem of vector dimensional explosion caused by too many charging stations in the centralized training process, thereby improving the applicability of the training process and the training efficiency.

[0060] In some optional implementations of this embodiment, The above executing body is: First, by using a mapping vector, the environmental state information and the target operation information corresponding to each charging station in the charging station set are mapped as score features characterizing the importance of the charging station; Next, determining a predetermined number of charging stations from the set of charging stations based on the score feature, and determining environmental state information, target operation information, and score features corresponding to the predetermined number of charging stations; Next, normalizing the score features corresponding to the preset number of charging stations to obtain a gating control vector; Next, determining a gating characteristic based on the environmental condition information, the target operation information, and the gating vector corresponding to the predetermined number of charging stations; and Finally, the first step can be performed by determining consolidated display information for all charging stations in the collection of charging stations based on the gate control characteristic.

[0061] As shown in FIG. 4, FIG. 4 shows a schematic diagram of the process flow of the agent pooling module. Specifically,

number

[0062]

number

[0063]

number

[0064] Then, the retention of knowledge is controlled using a gate control mechanism, as follows:

number

[0065]

number

[0066] In this implementation method, a specific operation flow of the agent pooling module is provided, and based on charging station filtering, important charging stations for charging requests are determined, and based on a gate control mechanism, knowledge retention is controlled, so that the accuracy of the integrated display information determined by the agent pooling module and the training efficiency of the training process can be further improved.

[0067] In some optional implementations of this embodiment, the execution entity can execute step 204 by the following steps.

[0068] First step: The first loss corresponding to the initial value network is determined using the time difference method.

[0069] The time difference methods include the on-policy Sarsa method and the off-policy Q-Learning method, and in this realization method, the first loss corresponding to the initial value network can be determined by either of the time difference methods.

[0070] Second step: The second loss corresponding to the agent pooling module is determined by self-supervised comparative learning method.

[0071]

number

[0072] In this implementation method, the integrated display information corresponding to the set of charging stations represented by the agent pooling module is used as a query example, and positive examples and negative examples corresponding to the integrated display information are determined, so that a self-supervised comparative learning loss InfoNCE between the positive instances and negative instances corresponding to the integrated display information can be determined as the second loss.

[0073] Third step: Update the first initial value network and the agent pooling module based on the first loss and the second loss.

[0074] In this implementation method, the executing entity can determine a total loss based on the first loss and the second loss based on methods such as summation, weighted summation, etc., and further determine a gradient based on the total loss, thereby updating the first initial value network and the agent pooling module based on the gradient descent method.

[0075] Fourth step: The initial policy network is updated with the goal of maximizing the expected value of the cumulative reward corresponding to the charging requests.

[0076]

number

[0077] In the present realization method, a specific training process of the policy network, the pooling module and the value network is provided, which helps to improve the accuracy of the policy network, the pooling module and the value network obtained by the training.

[0078] In some optional implementations of this embodiment, the execution entity can execute the second step as follows:

[0079] First, for a first subset of the joint features, first integrated display information is determined through an agent pooling module, where the joint features include environmental state information and target operation information corresponding to each charging station in the charging station set; then, for a second subset of the joint features, second integrated display information is determined through the agent pooling module; then, for a third subset of the joint features corresponding to another charging request different from the charging request, third integrated display information is determined through the agent pooling module; and finally, a self-supervised comparative learning loss determined based on the first integrated display information, the second integrated display information and the third integrated display information is used as a second loss.

[0080] In this realization method, one of the problems to be solved is how to train the agent pooling module to obtain an effective large-scale agent latent representation through the trained agent pooling module. A simple way is to update the agent pooling module with the optimization goal of reinforcement learning. However, the reinforcement learning algorithm optimizes the agent's policy through the feedback reward of the environment, and this kind of optimization method is much more difficult to control than supervised learning. It can be imagined that it is very difficult for reinforcement learning to learn an effective latent representation from a high-dimensional input. Here, this realization method proposes a comparison learning goal as an auxiliary task to promote the representation learning of large-scale agents by the agent pooling module.

[0081]

number

[0082]

number

[0083]

number

[0084] Then, we optimize the comparison learning objective using the InfoNCE loss.

[0085]

number

[0086] In this implementation method, the loss of the agent pooling module is determined using a self-supervised comparative learning method, which makes it easier to train and improves training efficiency compared to a policy that uses a reinforcement learning algorithm to optimize agents based on environmental feedback rewards.

[0087] In some optional implementations of this embodiment, the execution entity may obtain the first loss by performing the first step as follows.

[0088] First, a reward information is determined based on a charging operation of a charging target corresponding to a charging request according to a preset reward function, where each charging station in a charging station set shares reward information, and the preset reward function provides different rewards for different charging operations. Next, a first loss corresponding to an initial value network is determined based on an expected value of a cumulative reward corresponding to the charging request, a reward corresponding to the charging request, and an expected value of a cumulative reward corresponding to a charging request following the charging request according to a time difference algorithm.

[0089]

number

[0090]

number

[0091] In addition, in this implementation, all charging stations in the charging station collection share the same reward, so these charging stations can be encouraged to cooperate with each other to maximize the total profit.

[0092] Furthermore, the first loss is determined by the following formula.

[0093]

number

[0094]

number

[0095] In this implementation method, rewards corresponding to different charging requests are determined by a designed preset reward function, and the first loss is determined by a time difference method, so that the accuracy of the first loss can be improved. The reward obtained by the preset reward function is shared among the charging stations in the charging station set, which helps to encourage mutual cooperation among these charging stations and improves the coordination among the charging stations.

[0096] In this implementation method, the executing entity can determine the weighted sum of the first loss and the second loss using the following formula.

[0097]

number

[0098] Continuing to refer to FIG. 6, FIG. 6 illustrates an exemplary flow 600 of yet another embodiment of a method for training an information coordination model of a charging station according to the present disclosure, including the following steps.

[0099] Step 601: Obtain a charging request, and determine environmental state information corresponding to each charging station in a collection of charging stations.

[0100] Step 602: Determine, via an initial policy network, target operation information of each charging station in the charging station collection for a charging request based on environmental state information corresponding to each charging station in the charging station collection.

[0101] Step 603: Using a mapping vector, the environmental state information and the target operation information corresponding to each charging station in the charging station collection are mapped as a score feature characterizing the importance of the charging station.

[0102] Step 604: Determine a preset number of charging stations from the set of charging stations based on the score feature, and determine environmental state information, target operation information and score features corresponding to the preset number of charging stations.

[0103] Step 605: Normalize the score features corresponding to the preset number of charging stations to obtain a gating control vector.

[0104] Step 606: Determine a gate control characteristic based on the environmental state information, the target operation information, and the gate control vector corresponding to a preset number of charging stations.

[0105] Step 607: Determine consolidated display information of all charging stations in the charging station collection based on the gate control characteristic.

[0106] Step 608: Determine an expected value of a cumulative reward corresponding to the charging request based on the integrated display information through the initial value network.

[0107] Step 609: A first loss corresponding to the initial value network is determined using a time differencing method.

[0108] Step 610: A second loss corresponding to the agent pooling module is determined by a self-supervised comparison learning method.

[0109] Step 611: Update the first initial value network and the agent pooling module based on the first loss and the second loss, and update the initial policy network with the goal of maximizing the expected value of the cumulative reward corresponding to the charging request, thereby obtaining a trained policy network and a value network.

[0110] Step 612: Determine the trained policy network as an information coordination model corresponding to each charging station in the charging station collection.

[0111] As can be seen from this embodiment, compared with the embodiment corresponding to Figure 2, the method flow 600 for training the information coordination model of the charging station in this embodiment specifically describes the processing process by the agent pooling module and the multi-agent reinforcement learning process, which can further improve the coordination between the charging stations in the charging station collection, and improve the real-timeness and rationality of the charging station in determining the target operation information.

[0112] Continuing to refer to FIG. 7, FIG. 7 illustrates an example flow 700 of one embodiment of a method for selecting a charging station according to the present disclosure, including the following steps.

[0113] Step 701: Obtain a charging request.

[0114] In this embodiment, the entity executing the method for selecting a charging station (e.g., the terminal device or the server in FIG. 1 ) can obtain a charging request remotely or locally based on a wired network connection method or a wireless network connection method.

[0115] The charging request may be a request issued by a user in the charging vehicle through a terminal device such as a smartphone, an on-board computer, etc., to characterize that the charging vehicle needs to be charged. When the charging vehicle needs to be charged, a user such as a corresponding driver, a passenger, etc., initiates a charging request at the charging platform;

number

[0116] Step 702: Determine environmental condition information corresponding to each charging station in the collection of charging stations.

[0117] In this embodiment, the execution entity can determine environmental condition information corresponding to the target charging station.

[0118] The charging station set includes a plurality of charging stations, and as an example, the charging station set is all charging stations in a predefined divided area (e.g., divided by administrative district or by a specified area). Each charging station can be an agent,

number

[0119]

number

[0120] Step 703: For each charging station in the charging station collection, determine target operation information of the charging station for a charging request based on the environmental state information corresponding to the charging station by the trained information adjustment model corresponding to the charging station.

[0121] In this embodiment, for each charging station in the charging station set, the execution subject can determine target operation information of the charging station in response to a charging request based on the environmental state information corresponding to the charging station through the trained information adjustment model corresponding to the charging station. Here, the information adjustment model is obtained by sensing the environmental state information between the charging stations in the charging station set and then performing multi-agent reinforcement learning based on the deep decision policy gradient method. Specifically, the information adjustment model is obtained by training according to the above embodiments 200 and 600.

[0122]

number

[0123] Step 704: Display target operation information of each charging station in the charging station collection for the charging request.

[0124] In this embodiment, the execution subject can display target operation information of each charging station in the charging station set in response to the charging request.

[0125] As an example, a user submitting a charging request may be shown target operational information of each charging station in the charging station collection for the charging request via the charging platform.

[0126] Step 705: Receive a selection command, and determine a target charging station from the set of charging stations based on the selection command.

[0127] As an example, a user sending a charging request can select an appropriate charging station as a target charging station based on the target operation information of each displayed charging station, and send a selection command by an operation command such as touch or click, a voice command, etc., and further, the above-mentioned executing entity can determine the target charging station based on the selection command.

[0128] After determining the target charging station, the execution entity may also perform a navigation operation from the current location to the target charging station based on a navigation application.

[0129] In this embodiment, the real-time target operation information of the charging station is determined by the trained information adjustment model, which improves the real-timeness and rationality of the target operation information while improving the coordination between charging stations in the charging station group, thereby helping to solve the problem of poor charging coordination between charging stations and improving the utilization rate of the charging stations.

[0130] Continuing to refer to FIG. 8, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for training an information coordination model of a charging station, which embodiment of the apparatus corresponds to the embodiment of the method shown in FIG. 2, and which can be specifically applied to various electronic devices.

[0131] As shown in FIG. 8 , an apparatus 800 for training an information coordination model of a charging station includes: A first determining unit 801 configured to obtain a charging request and determine environmental state information corresponding to each charging station in the charging station collection; a second determining unit 802 configured to determine, via an initial policy network, target operation information of each charging station in the charging station collection for the charging request based on environmental state information corresponding to each charging station in the charging station collection; a third determination unit 803 configured to determine an expected value of a cumulative reward corresponding to the charging request based on the environmental state information and the target operation information corresponding to each charging station in the charging station collection through the initial value network; A training unit 804, the training unit 804 being configured to use a deep decision strategy gradient method to train an initial policy network and an initial value network to obtain a trained policy network and a value network, where in the training process, the training unit 804 updates the initial value network by a time difference method, and updates the initial policy network with the goal of maximizing the expected value of a cumulative reward corresponding to a charging request; and a fourth determining unit 805 configured to determine the trained policy network as an information coordination model corresponding to each charging station in the charging station collection.

[0132] In some optional implementations of this embodiment, The third determination unit 803 is Determine, via an agent pooling module, integrated display information characterizing all charging stations in the charging station collection based on the environmental state information and the target operation information corresponding to each charging station in the charging station collection; It is further configured to determine, via the initial value network, an expected value of a cumulative reward corresponding to the charging request based on the integrated display information.

[0133] In some optional implementations of this embodiment, The third determination unit 803 is Using a mapping vector, the environmental state information and the target operation information corresponding to each charging station in the charging station set are mapped as a score feature characterizing the importance of the charging station; Determine a predetermined number of charging stations from the set of charging stations based on the score feature, and determine environmental state information, target operation information, and score features corresponding to the predetermined number of charging stations; Normalizing the score features corresponding to the predetermined number of charging stations to obtain a gating vector; determining a gating characteristic based on the environmental condition information, the target operation information, and the gating vector corresponding to the predetermined number of charging stations; The charging station collection is further configured to determine consolidated display information for all charging stations in the charging station collection based on the gate control characteristic.

[0134] In some optional implementations of this embodiment, The training unit 804 is Determine the first loss corresponding to the initial value network using the time difference method; A second loss corresponding to the agent pooling module is determined by a self-supervised comparative learning method; Update the first initial value network and the agent pooling module based on the first and second losses; The system is further configured to update the initial policy network with a goal of maximizing an expected value of a cumulative reward corresponding to the charging requests.

[0135] In some optional implementations of this embodiment, The training unit 804 is Determine, via an agent pooling module, first integrated representation information for a first subset of common features including environmental state information and target operation information corresponding to each charging station in the set of charging stations; determining, via the agent pooling module, second integrated representation information for a second subset of the joint features; For a third subset of the common features corresponding to another charging request different from the charging request, determine, through the agent pooling module, a third integrated display information; The method is further configured to use the determined self-supervised comparison learning loss based on the first integrated representation information, the second integrated representation information, and the third integrated representation information as a second loss.

[0136] In some optional implementations of this embodiment, The training unit 804 is Determine reward information according to a preset reward function based on a charging operation of a charging object corresponding to the charging request, where each charging station in the charging station set shares the reward information, and the preset reward function provides different rewards for different charging operations; The time difference algorithm is further configured to determine a first loss corresponding to the initial value network based on an expected value of the accumulated reward corresponding to the charging request, the reward corresponding to the charging request, and an expected value of the accumulated reward corresponding to a charging request following the charging request.

[0137] In this embodiment, a method for training an information coordination model of a charging station is provided, in which the charging station is used as an agent, and multi-agent reinforcement learning is performed based on the deep decision policy gradient method. Through training, a policy network that can determine target operation information in real time can be obtained as the information coordination model, thereby improving the real-timeness and rationality of the charging station in determining the target operation information. In the model training stage, the goal is to maximize the cumulative reward of the entire set of charging stations, and the agents can sense information about the entire environment from each other, thereby improving the coordination between charging stations in the charging station set, which helps to solve the problem of poor charging coordination between charging stations, and thereby improves the utilization rate of the charging stations.

[0138] Continuing to refer to FIG. 9, as an implementation of the method shown in each of the above figures, the present disclosure provides an embodiment of an apparatus for selecting a charging station, which embodiment of the apparatus corresponds to the embodiment of the method shown in FIG. 7, and which can be specifically applied to various electronic devices.

[0139] As shown in FIG. 9 , an apparatus 900 for selecting a charging station includes: An acquiring unit 901 configured to acquire a charging request; a fifth determining unit 902 configured to determine environmental condition information corresponding to each charging station in the collection of charging stations; A sixth determination unit 903, which is configured to, for each charging station in the charging station set, determine target operation information of the charging station for a charging request based on environmental state information corresponding to the charging station through a trained information adjustment model corresponding to the charging station, where the information adjustment model is obtained by sensing each other's environmental state information between the charging stations in the charging station set and then performing multi-agent reinforcement learning based on a deep decision policy gradient method; a display unit 904 configured to display target operation information of each charging station in the collection of charging stations in response to a charging request; A receiving unit 905 configured to receive a selection command and determine a target charging station from the collection of charging stations based on the selection command.

[0140] In this embodiment, the target operation information of the charging station is determined in real time by the trained information adjustment model, which improves the real-timeness and rationality of the target operation information while improving the coordination between charging stations in a charging station group. This helps to solve the problem of poor charging coordination between charging stations and improves the utilization rate of the charging stations.

[0141] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, The electronic device includes at least one processor and a memory communicatively coupled to the at least one processor, where: The memory stores instructions executable by at least one processor, which, when executed by the at least one processor, are executed by the at least one processor to realize the method for training an information coordination model of a charging station and the method for selecting a charging station described in any of the above embodiments.

[0142] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium, which stores computer instructions that, when executed by a computer, can realize the method for training the charging station information adjustment model and the method for selecting a charging station described in any of the above embodiments.

[0143] An embodiment of the present disclosure provides a computer program, which, when executed by a processor, can realize the method for training an information coordination model of a charging station and the method for selecting a charging station described in any of the above embodiments.

[0144] 10 shows a schematic block diagram of an exemplary electronic device 1000 that may be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the practice of the present disclosure as described and / or claimed herein.

[0145] 10, the device 1000 includes a computing unit 1001, which can execute various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 1002 or loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 can also store various programs and data required for the operation of the device 1000. The computing unit 1001, the ROM 1002 and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0146] Multiple components in the device 1000 are connected to an I / O interface 1005, including input units 1006 such as a keyboard, mouse, etc., output units 1007 such as various types of displays, speakers, etc., storage units 1008 such as magnetic disks, optical disks, etc., and communication units 1009 such as a network card, modem, wireless communication transceiver, etc. The communication units 1009 enable the device 1000 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0147] The computing unit 1001 may be various general-purpose and / or dedicated processing components having processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various machine learning model algorithm computing units, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 executes various methods and processes described above, such as a method for training an information coordination model of a charging station. For example, in some embodiments, the method for training an information coordination model of a charging station may be implemented as a computer software program tangibly included in a machine-readable medium, such as the storage unit 1008. In some embodiments, some or all of the computer program may be loaded and / or installed in the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the method for training an information coordination model of a charging station described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the method for training the information coordination model of the charging station by any other suitable means (eg, by firmware).

[0148] Various embodiments of the above-described systems and techniques may be implemented in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard components (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be embodied in one or more computer programs that may be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be a special purpose or general purpose programmable processor, and may include a processor that is capable of receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

[0149] The program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, so that when the program codes are executed by the processor or controller, the functions / operations defined in the flowcharts and / or block diagrams are implemented. The program codes may be fully executed on the machine, partially executed on the machine, partially executed on the machine as an independent package and partially executed on the machine by remote control, or fully executed on the machine by remote control or on a server.

[0150] In the context of the present disclosure, a machine-readable medium may be a tangible medium, which may contain or store a program used by or in combination with an instruction execution system, apparatus or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0151] To provide for user interaction, the systems and techniques described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) for the user to provide input to the computer. Other types of devices may provide for user interaction, for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback) and may receive input from the user in any form (including voice input, audio input, or tactile input).

[0152] The systems and techniques described herein may be implemented in a computing system that includes a back-end component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a front-end component (e.g., a user computer having a graphical user interface or network browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected via any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0153] The computer system may include a client and a server. The client and the server are generally remote from each other and usually communicate with each other through a communication network. The relationship between the client and the server is generated through a computer program executed by a corresponding computer and having a client-server relationship with each other. The server may be a cloud server, also called a cloud computing server or a cloud host, which is a host in a cloud computing service system, thereby solving the defects of large management difficulty and weak service scalability that exist in traditional physical hosts and virtual private server (VPS) services. The server may be a server of a distributed system, or a server combined with a blockchain.

[0154] According to the technical solution of an embodiment of the present disclosure, a method for training an information coordination model of a charging station is provided, in which the charging station is an agent, and multi-agent reinforcement learning is performed based on a deep decision policy gradient method. Through training, a policy network that can determine target operation information in real time can be obtained as the information coordination model, thereby improving the real-timeness and rationality of the charging station in determining the target operation information. In the model training stage, the goal is to maximize the cumulative reward of the entire set of charging stations, and the agents can sense information about the entire environment from each other, thereby improving the coordination between charging stations in the charging station set. This helps solve the problem of poor charging coordination between charging stations, and improves the utilization rate of charging stations.

[0155] It should be understood that the various forms of processes shown above can be used, and steps can be rearranged, added, or removed. For example, each step described in the present disclosure can be performed in parallel, sequentially, or in a different order, and is not limited thereto, as long as the desired results of the technical solutions provided by the present disclosure can be achieved.

[0156] The above specific embodiments do not limit the scope of protection of the present disclosure. It should be understood that those skilled in the art can make various modifications, combinations, subcombinations and substitutions according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A computer-implemented method for training an information coordination model of a charging station, comprising: receiving a charging request and determining environmental state information corresponding to each charging station in a collection of charging stations; determining, via an initial policy network, target operation information for each charging station in the set of charging stations for the charging request based on environmental state information corresponding to each charging station in the set of charging stations; determining an expected value of a cumulative reward corresponding to the charging request based on environmental state information and target operation information corresponding to each charging station in the set of charging stations via an initial value network; Using a deep decision policy gradient method to train the initial policy network and the initial value network to obtain a trained policy network and a value network, and during the training process, updating the initial value network by a time difference method, and updating the initial policy network with the goal of maximizing the expected value of the accumulated reward corresponding to the charging request; determining the trained policy network as an information coordination model corresponding to each charging station in the set of charging stations; training the initial policy network and the initial value network using the deep decision policy gradient method, determining a first loss corresponding to the initial value network using the time differencing method; determining a second loss corresponding to an agent pooling module by a self-supervised comparative learning method, the agent pooling module being configured to determine integrated representation information characterizing all of the charging stations in the charging station collection based on environmental state information and target operation information corresponding to each charging station in the charging station collection; updating the initial value network and the agent pooling module based on the first loss and the second loss; and updating the initial policy network with the goal of maximizing an expected value of a cumulative reward corresponding to a charging request.

2. determining an expected value of a cumulative reward corresponding to the charging request based on environmental state information and target operation information corresponding to each charging station in the set of charging stations via the initial value network, determining, via an agent pooling module, integrated display information characterizing all of the charging stations in the collection of charging stations based on environmental state information and target operation information corresponding to each charging station in the collection of charging stations; and determining an expected value of a cumulative reward corresponding to the charging request based on the integrated display information via the initial value network.

3. determining, via the agent pooling module, integrated display information characterizing all of the charging stations in the charging station collection based on environmental state information and target operation information corresponding to each charging station in the charging station collection, Mapping, by a mapping vector, the environmental state information and the target operation information corresponding to each charging station in the set of charging stations as a score feature characterizing an importance of the charging station; determining a predetermined number of charging stations from the set of charging stations based on the score feature, and determining environmental state information, target operation information, and score features corresponding to the predetermined number of charging stations; normalizing the score features corresponding to the predetermined number of charging stations to obtain a gating vector; determining a gating characteristic based on the environmental condition information, the target operation information, and a gating vector corresponding to the predetermined number of charging stations; and determining integrated display information of all charging stations in the charging station collection based on the gate control feature.

4. The step of determining a second loss corresponding to the agent pooling module by a self-supervised comparison learning method includes: determining, via the agent pooling module, first integrated display information for a first subset of common features including environmental state information and target operation information corresponding to each charging station in the set of charging stations; determining, via the agent pooling module, second integrated display information for a second subset of the joint features; determining, via the agent pooling module, third integrated display information for a third subset of common features corresponding to other charging requests different from the charging request; and using a self-supervised comparative learning loss determined based on the first integrated display information, the second integrated display information, and the third integrated display information as the second loss.

5. The step of determining a first loss corresponding to the initial value network by a time differencing method includes: determining reward information based on a charging operation of a charging target corresponding to the charging request by a preset reward function, wherein each charging station in the charging station set shares the reward information, and the preset reward function provides different rewards for different charging operations; 2. The method for training an information coordination model of a charging station as described in claim 1, further comprising: determining, by the time difference method, a first loss corresponding to the initial value network based on an expected value of a cumulative reward corresponding to the charging request, a reward corresponding to the charging request, and an expected value of a cumulative reward corresponding to a charging request following the charging request.

6. obtaining a charging request; determining environmental state information corresponding to each charging station in the collection of charging stations; For each charging station in the set of charging stations, determining target operation information of the charging station for the charging request based on environmental state information corresponding to the charging station by a trained information coordination model corresponding to the charging station, the information coordination model being trained by the method for training an information coordination model of a charging station according to claim 1; displaying target operation information of each charging station in the set of charging stations in response to the charging request; receiving a selection command; and determining a target charging station from the collection of charging stations based on the selection command.

7. a first determination unit configured to obtain a charging request and determine environmental state information corresponding to each charging station in the collection of charging stations; a second determination unit configured to determine, via an initial policy network, target operation information of each charging station in the set of charging stations for the charging request based on environmental state information corresponding to each charging station in the set of charging stations; a third determination unit configured to determine, via an initial value network, an expected value of a cumulative reward corresponding to the charging request based on environmental state information and target operation information corresponding to each charging station in the set of charging stations; A training unit configured to use a deep decision strategy gradient method to train the initial policy network and the initial value network to obtain a trained policy network and a value network, wherein in the training process, the training unit updates the initial value network using a time difference method, and updates the initial policy network with the goal of maximizing an expected value of a cumulative reward corresponding to a charging request; a fourth determination unit configured to determine the trained policy network as an information coordination model corresponding to each charging station in the set of charging stations; The training unit includes: determining a first loss corresponding to the initial value network using the time differencing method; A second loss corresponding to the agent pooling module is determined by a self-supervised comparative learning method; updating the initial value network and the agent pooling module based on the first loss and the second loss; The method is further configured to update the initial policy network with a goal of maximizing an expected value of a cumulative reward corresponding to a charging request; The agent pooling module is an apparatus for training an information coordination model of a charging station configured to determine integrated display information characterizing all of the charging stations in the charging station collection based on environmental state information and target operation information corresponding to each charging station in the charging station collection.

8. The third determination unit: Determine, via an agent pooling module, integrated display information characterizing all of the charging stations in the charging station collection based on environmental state information and target operation information corresponding to each charging station in the charging station collection; The apparatus for training an information coordination model of a charging station as described in claim 7, further configured to determine an expected value of a cumulative reward corresponding to the charging request based on the integrated display information via the initial value network.

9. The third determination unit: Using a mapping vector, the environmental state information and the target operation information corresponding to each charging station in the set of charging stations are mapped as a score feature that characterizes the importance of the charging station; Determine a predetermined number of charging stations from the set of charging stations based on the score feature, and determine environmental state information, target operation information, and score features corresponding to the predetermined number of charging stations; normalizing the score features corresponding to the predetermined number of charging stations to obtain a gating control vector; determining a gating characteristic based on the environmental condition information, the target operation information, and the gating vector corresponding to the predetermined number of charging stations; The apparatus for training an information coordination model of a charging station of claim 8 , further configured to determine integrated display information of all charging stations in the charging station collection based on the gate control feature.

10. The training unit includes: Determine, via the agent pooling module, first integrated display information for a first subset of common features, the common features including environmental state information and target operation information corresponding to each charging station in the set of charging stations; determining, via the agent pooling module, second integrated representation information for a second subset of the joint features; Determine, via the agent pooling module, third integrated display information for a third subset of common features corresponding to another charging request different from the charging request; 8. The apparatus for training an information adjustment model of a charging station as described in claim 7, further configured to set a self-supervised comparative learning loss determined based on the first integrated display information, the second integrated display information, and the third integrated display information as the second loss.

11. The training unit includes: Determine reward information based on a charging operation of a charging target corresponding to the charging request according to a preset reward function, and each charging station in the charging station set shares the reward information, and the preset reward function provides different rewards for different charging operations; 8. The apparatus for training an information adjustment model of a charging station as described in claim 7, further configured to determine a first loss corresponding to the initial value network based on an expected value of a cumulative reward corresponding to the charging request, a reward corresponding to the charging request, and an expected value of a cumulative reward corresponding to a charging request following the charging request, by the time difference method.

12. an acquiring unit configured to acquire a charging request; a fifth determination unit configured to determine environmental state information corresponding to each charging station in the collection of charging stations; a sixth determination unit configured to determine, for each charging station in the set of charging stations, target operation information of the charging station for the charging request based on environmental state information corresponding to the charging station by a trained information coordination model corresponding to the charging station, wherein the information coordination model can be trained by an apparatus for training an information coordination model of a charging station according to any one of claims 7 to 11; a display unit configured to display target operation information of each charging station in the set of charging stations in response to the charging request; a receiving unit configured to receive a selection command and determine a target charging station from the collection of charging stations based on the selection command.

13. At least one processor; and a memory communicatively connected to the at least one processor, wherein: An electronic device, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor such that the at least one processor can perform the method of any one of claims 1 to 6.

14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to carry out the method according to any one of claims 1 to 6.

15. A computer program comprising: A computer program implementing the method according to any one of claims 1 to 6, when the computer program is executed by a processor.

Images