Training method, training device, equipment, system and storage medium

Abstract

The present disclosure provides a training method, training device, equipment, system and storage medium, which relate to the field of data processing, especially to the field of big data processing. The specific implementation scheme is as follows: the first training node in the distributed training system obtains the target data to be trained when it is determined that the cache startup conditions are met, wherein the target data to be trained is part of the total data to be trained, and the total data to be trained is the total data for asynchronous training of the model to be trained required by the distributed training system; the first training node performs asynchronous training on the pre-stored model to be trained based on the target data to be trained when it is determined that the training conditions are met, and obtains the current training result; the first training node sends the current training result to the second training node in the distributed training system. In this way, the training efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to the field of big data processing. Background Art

[0002] Distributed training technology uses multiple devices connected in a specific way to communicate and transmit data, jointly completing model training. How to use distributed training systems to efficiently complete model training has become a current research topic. Summary of the Invention

[0003] The present disclosure provides a training method, training device, equipment, system and storage medium.

[0004] According to one aspect of the present disclosure, a training method is provided, comprising:

[0005] The first training node in the distributed training system obtains target data to be trained when determining that a cache startup condition is met, wherein the target data to be trained is part of the total data to be trained, and the total data to be trained is the total data for asynchronous training of the to-be-trained model required to be trained by the distributed training system;

[0006] When determining that the training condition is met, the first training node performs asynchronous training on the pre-stored model to be trained based on the target data to be trained to obtain a current training result;

[0007] The first training node sends the current training result to the second training node in the distributed training system.

[0008] According to another aspect of the present disclosure, there is provided a training device comprising:

[0009] A data acquisition unit is configured to acquire target data to be trained when it is determined that the cache startup condition is met, wherein the target data to be trained is part of the total data to be trained, and the total data to be trained is the total data for asynchronous training of the to-be-trained model required to be trained by the distributed training system;

[0010] A model training unit is configured to asynchronously train the pre-stored model to be trained based on the target data to be trained, when it is determined that the training conditions are met, to obtain a current training result;

[0011] A sending unit is used to send the current training result to the second training node in the distributed training system.

[0012] According to another aspect of the present disclosure, a training system is provided, comprising at least: a first training node and a second training node; wherein,

[0013] The first training node is configured to, upon determining that a cache startup condition is satisfied, obtain target data to be trained, wherein the target data to be trained is a portion of the total data to be trained, and the total data to be trained is the total data for asynchronously training the to-be-trained model required to be trained by the distributed training system; upon determining that the training condition is satisfied, asynchronously train the pre-stored to-be-trained model based on the target data to be trained to obtain a current training result; and send the current training result to the second training node;

[0014] The second training node is configured to receive the current training result sent by the first training node.

[0015] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned method.

[0019] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above method.

[0020] According to yet another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0021] This provides a guarantee for effectively improving training efficiency.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0024] Figure 1 is a schematic diagram of an implementation flow of a training method according to an embodiment of the present disclosure;

[0025] Figure 2 is a schematic diagram of a training system according to an embodiment of the present disclosure;

[0026] Figure 3 is a schematic diagram of a scenario in a specific example of the training method according to an embodiment of the present disclosure;

[0027] Figure 4 is an interaction diagram between training nodes in another specific example of the training method according to an embodiment of the present disclosure;

[0028] Figure 5 is a schematic diagram of a scenario of two training nodes in a specific example of a training method according to an embodiment of the present disclosure;

[0029] Figure 6 This is a schematic diagram of the implementation process of the training method based on two training nodes in a specific example according to the embodiment of the present disclosure. Figure 1 ;

[0030] Figure 7 is an interaction diagram between training nodes in another specific example of the training method according to an embodiment of the present disclosure;

[0031] Figure 8 is a schematic diagram of functional modules in a training node in a specific example of a training method according to an embodiment of the present disclosure;

[0032] Figure 9 This is a schematic diagram of the implementation process of the training method based on two training nodes in a specific example according to the embodiment of the present disclosure. Figure 2 ;

[0033] Figure 10 is a schematic structural diagram of a first training node according to an embodiment of the present disclosure;

[0034] Figure 11 is a block diagram of an electronic device for implementing the training method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0036] In today's internet landscape, user behavior data and information data (for example) are exploding. Using a single machine (i.e., a single device) for model training is limited by its computing and memory resources, resulting in low efficiency and inability to meet timeliness requirements. In scenarios where deep learning is used for model training, to meet the memory and training time requirements of large data volumes and large models, and to address the drawbacks of single-machine training, additional training devices are added to expand computing and memory resources. However, this also leads to resource waste. Therefore, balancing the pros and cons of model training on a single device versus across multiple devices has become a pressing issue.

[0037] Based on this, the disclosed solution provides a training solution that uses multiple devices to cache data for training acceleration. On the one hand, it will not occupy additional memory resources, and on the other hand, it can recycle resources of used devices without wasting resources, effectively balancing the needs of resource utilization and efficiency.

[0038] Specifically, the present disclosure provides a training method, specifically, as Figure 1 and Figure 2 Shown, including:

[0039] Step S101: Upon determining that cache activation conditions are met, the first training node in the distributed training system obtains target training data, where the target training data is a portion of the total training data, which is the total data used to asynchronously train the model to be trained by the distributed training system. It should be noted that to ensure the normal progress of the training task, the training nodes participating in the current model training in the distributed training system may pre-download the model to be trained.

[0040] Step S102: When determining that the training conditions are met, the first training node performs asynchronous training on the pre-stored model to be trained based on the target data to be trained to obtain a current training result.

[0041] Step S103: The first training node sends the current training result to the second training node in the distributed training system.

[0042] In this way, the disclosed solution adopts an asynchronous training method. The training nodes perform asynchronous training when the data is cached, that is, when the data download is completed. Therefore, the training nodes do not need to consider the overhead of network resources and can use computing resources to the greatest extent for model training, providing a guarantee for effectively improving training efficiency.

[0043] Here, it can be understood that the asynchronous training refers to the distributed training system training nodes each performing their own training tasks and passing the training results to the next training node, so that the next training node updates the parameter information based on the training results of the previous node, and then relays the model training. Moreover, when each training node is training, the data to be trained used is part of the total data to be trained. At the same time, for the purpose of training effect, the data to be trained used by each training node is different. In this way, the purpose of asynchronously training the training model based on different training nodes and the total data to be trained is achieved. Moreover, the termination condition of the training task can be that the total data to be trained has been used up, or the model to be trained has converged, or the number of training rounds has reached a specified number, etc., and the disclosed solution does not impose any restrictions on this.

[0044] In a specific example of the disclosed solution, the first training node may obtain the target data to be trained at the following timing. Specifically, in the distributed training system described above, when determining that the cache startup condition is met, the first training node obtains the target data to be trained, which may specifically include:

[0045] The first training node in the distributed training system obtains the target data to be trained when it receives the first training start message from the third training node in the distributed training system to start asynchronous training; wherein, the first training start message is generated by the third training node after asynchronous training of the pre-stored model to be trained based on the first data to be trained; the first data to be trained is at least part of the total data to be trained except for the target data to be trained. It should be noted that the data to be trained used by different training nodes in the distributed training system described in the present disclosure are different when performing training tasks, but they are all part of the data in the total data to be trained. In this way, the validity of the training results is ensured, and the problem of insufficient training caused by repeated use of the same data to be trained is avoided.

[0046] That is to say, after starting the training task, the third training node will send a message to the first training node, that is, the first training start message; accordingly, the first training node can trigger the download task after receiving the message, that is, the first training start message, that is, obtain the target data to be trained. In this way, the training data required for the training task is pre-downloaded, which lays the foundation for effectively reducing the overall training time and improving training efficiency.

[0047] In the disclosed solution, in order to hide the data downloading time (i.e., data caching) of the first training node to the greatest extent, a first training start message can be sent to the first training node when the third training node starts training. At this time, while the third training node is performing asynchronous training, the first training node is enabled to pre-download the data for asynchronous training, i.e., the target data to be trained. In this way, it is possible to maximize the avoidance of the first training node being unable to normally relay training due to the incomplete downloading of the target data to be trained.

[0048] In a specific example of the disclosed solution, when the third training node completes training and obtains a first training result, the first training result may also be sent to the first training node, specifically including:

[0049] The first training node receives the first training result sent by the third training node;

[0050] Based on this, when the first training node determines that the training conditions are met, the first training node performs asynchronous training on the pre-stored model to be trained based on the target data to be trained to obtain the current training result, which specifically includes:

[0051] When the first training node completes caching the target data to be trained, it updates the pre-stored parameter information of the model to be trained based on the first training result, and after the update is completed, it asynchronously trains the pre-stored model to be trained based on the target data to be trained to obtain the current training result.

[0052] That is to say, after the first training node receives the first training result sent by the third training node, the training task can be started; it should be noted that at this time, if the first training node has cached the target data to be trained, the training task is performed directly, that is, the parameter information of the pre-stored model to be trained is updated based on the first training result, and after the update is completed, the pre-stored model to be trained is asynchronously trained based on the target data to be trained; otherwise, the first training node waits for the target data to be trained to be cached, that is, the download is completed, and performs the training task after the download is completed.

[0053] For example, if Figure 3As shown, the third training node can be understood as the previous node of the first training node, and the second training node is the next node of the first training node; at this time, the third training node, the first training node, and the second training node perform asynchronous training in sequence, and send their own training results to the next training node, so that the next node can perform training based on the training result of the previous node. For example, the third training node sends the first training result to the first training node, so that the first training node can perform training again based on the first training result, and sends the current training result obtained after the training to the second training node, so that the second training node can perform training again based on the training result of the first training node (i.e., the current training result), thereby completing the asynchronous training.

[0054] Detailed steps as follows Figure 3 and Figure 4 As shown, after the third training node receives the message from the previous training node corresponding to itself, it downloads the first data to be trained (i.e. Figure 3 After the download is completed, the training task is started, and at the same time, a first training start message is generated and sent to the first training node; accordingly, after the first training node receives the first training start message, it starts the download task, that is, obtains the target training data required for its own training task (that is, Figure 3 Further, after the third training node completes the training and obtains the first training result, the first training result is sent to the first training node. The first training node receives the first training result and, when it is determined that the cache has completed the target data to be trained, starts the training task, that is, updates the pre-stored parameter information of the model to be trained based on the first training result, and after the update is completed, asynchronously trains the pre-stored model to be trained based on the target data to be trained to obtain the current training result, and then sends the current training result to the second training node, so that the second training node can perform the training based on the current training result and the cached data to be trained (that is, Figure 3 Here, n is a positive integer greater than or equal to 1.

[0055] For example, if the current training node (such as the first training node) takes 10 minutes to download data, while the training task of the previous training node (such as the third training node) only takes 5 minutes, then, using the disclosed solution, since the current training node can download data synchronously when the training task of the previous training node is started, it can effectively shorten the proportion of data download time in the total time required for the entire system (i.e., the distributed training system) to complete model training. At the same time, it can also effectively reduce the total time, maximize the time spent on hiding data downloads, and let the time taken by the entire system to complete model training be fully reflected in the computational overhead that cannot be concealed, thereby improving the training efficiency as a whole.

[0056] In this way, since the disclosed solution can realize training cache data and start training tasks sequentially, it avoids the problem of fluctuating training effects due to data disorder in the existing distributed training process; at the same time, it ensures the orderliness of data reading and training.

[0057] In a specific example of the disclosed solution, the first training node may also obtain the target data to be trained at the following times:

[0058] The first training node obtains the historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves;

[0059] The first training node determines a cache start time for the first training node based on the acquired historical cache time and historical training time of the other training nodes;

[0060] Based on this, the first training node in the above-mentioned distributed training system obtains the target data to be trained when determining that the cache startup condition is met, including:

[0061] When the first training node in the distributed training system determines that the cache startup time has arrived, it obtains target data to be trained.

[0062] That is to say, the first training node can determine the timing of downloading the target data to be trained based on the historical time consumption of other training nodes, such as the historical cache time consumption (or historical download time, i.e., historical cache time) and the historical training time consumption (i.e., historical training time), that is, determine the cache startup time, and then download the target data to be trained when the download timing, i.e., the cache startup time, is reached. In this way, the proportion of data download time in the total time consumption of the entire system (i.e., distributed training system) to complete model training can be effectively shortened. At the same time, the total time consumption can also be effectively reduced, and the time consumption of data download can be maximized, so that the time consumption of the entire system to complete model training is fully reflected in the computational overhead that cannot be concealed, thereby improving the training efficiency as a whole.

[0063] It should be noted that in actual applications, there may be a situation where the cache start time has not been reached, but the first training start message of the third training node has been received. In this case, the priority of the download timing can be pre-set. For example, the priority of the cache start time is higher than the priority of receiving the first training start message of the third training node to trigger the download task. Therefore, at this time, even if the first training start message of the third training node is received, the download may not be triggered, and the download task will only be initiated after the cache start time is reached. Alternatively, there is a situation where the cache start time has been reached, but the first training start message of the third training node has not been received. At this time, since the priority of the cache start time is higher than the priority of receiving the first training start message of the third training node to trigger the download task, the download task can be triggered even if the first training start message of the third training node is not received. It can be understood that the above is only an example of priority, and the confirmation of priority can be set based on the actual needs of the actual scenario, and the present disclosure does not limit this.

[0064] At this time, the first training node in the distributed training system described above, upon determining that the cache startup condition is met, obtains the target data to be trained, which may further specifically include:

[0065] The first training node obtains the first training start message from the third training node but the cache start time has not arrived, and when it is determined that the priority of the first training start message is higher than the priority of the cache start time, obtains the target data to be trained; or

[0066] The first training node obtains the first training start message from the third training node but the cache start time has not arrived, and when it is determined that the priority of the first training start message is lower than the priority of the cache start time, obtains the target data to be trained when it is determined that the cache start time has arrived; or

[0067] The first training node obtains target data to be trained if it does not obtain the first training start message from the third training node but reaches the cache start time and it is determined that the priority of the first training start message is lower than the priority of the cache start time.

[0068] It should be noted that, in this example, the other training nodes are all the training nodes of the distributed training system that participate in this model training, for example, including Figure 3 and Figure 4 The third training node and the second training node are shown.

[0069] In a specific example of the disclosed solution, when determining the cache startup time, the data volume of the target data to be trained to be downloaded can also be referred to. In this way, the accuracy of the cache startup time obtained is improved, which lays the foundation for maximizing the time consumption of hidden data download, reducing the overall training time consumption, and improving the overall training efficiency. Specifically, the first training node described above determines the cache startup time for the first training node based on the historical cache time and historical training time obtained from the other training nodes, specifically including: the first training node determines the cache startup time for the first training node based on the historical cache time, historical training time, and data volume characteristics required for asynchronous training obtained from the other training nodes. It can be understood that the data volume characteristics required for asynchronous training described here can be understood as the data volume required for the current first training node to perform asynchronous training, in other words, the data volume of the target data to be trained, that is, the target data to be trained needs to meet the data volume characteristics.

[0070] In this way, by improving the accuracy of the cache startup time, the proportion of data download time in the total time required for the entire system (i.e., the distributed training system) to complete model training can be further shortened. At the same time, the total time can be further effectively reduced, and the time spent on hiding data downloads can be maximized, so that the time spent on the entire system to complete model training is fully reflected in the computational overhead that cannot be concealed, thereby improving the overall training efficiency.

[0071] In a specific example of the disclosed solution, the first training node can also determine the amount of data to be downloaded based on its own situation, such as node characteristics, including but not limited to: remaining storage resources, communication resources, etc., further improving the intelligence of the training node, and at the same time, laying the foundation for the efficient completion of the model training task. Specifically, the first training node determines the data volume characteristics required for asynchronous training; based on this, the above-mentioned acquisition of target data to be trained is specifically: obtaining target data to be trained that meets the data volume characteristics. In this way, the data to be trained can be cached in a targeted manner, which further provides support for maximizing the time spent on downloading hidden data.

[0072] In a specific example of the disclosed solution, after the first training node determines the amount of data to be downloaded, the first training node may also obtain target data to be trained at the following times, specifically including:

[0073] The first training node obtains the historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves;

[0074] The first training node adjusts the data volume feature based on the acquired historical cache time and historical training time of the other training nodes;

[0075] Wherein, the first training node in the distributed training system obtains target data to be trained when determining that the cache startup condition is met, including:

[0076] The first training node in the distributed training system obtains target to-be-trained data that meets the adjusted data volume feature when determining that the cache startup condition is met.

[0077] It should be noted that, in this example, the other training nodes are all the training nodes of the distributed training system that participate in this model training, for example, including Figure 3 and Figure 4 The third training node and the second training node are shown.

[0078] That is to say, the first training node can determine the amount of data to be downloaded based on the historical time consumption of other training nodes, such as the historical cache time consumption (or historical download time, i.e., historical cache time) and the historical training time consumption (i.e., historical training time). For example, based on the historical time consumption of other training nodes, it is known that the average historical download time is longer than the average historical training time. At this time, the size of the data can be reduced to shorten the download time. In this way, the proportion of data download time in the total time required for the entire system (i.e., distributed training system) to complete model training can be effectively shortened. At the same time, the total time can be effectively reduced to maximize the time consumption of hiding data downloads, so that the time consumption of the entire system to complete model training is fully reflected in the computational overhead that cannot be concealed, thereby improving the training efficiency as a whole.

[0079] In a specific example, after adjusting the data volume characteristics, the cache start time can also be determined. For example, based on the historical cache time and historical training time of the other training nodes obtained by the first training node, and the adjusted data volume characteristics, the cache start time is determined, and then when the download timing is reached, that is, when the cache start time is reached, the target data to be trained that meets the adjusted data volume characteristics is downloaded.

[0080] It is understandable that after determining the cache startup time based on the adjusted data volume characteristics, the above-mentioned download timing can be used to download the target training data that meets the adjusted data volume characteristics, which will not be repeated here.

[0081] In a specific example of the disclosed solution, the second training node and the third training node are the same node. That is, in this example, there are two training nodes in the distributed training system used for this model training, and thus asynchronous training is implemented based on the two training nodes.

[0082] For example, if Figure 5 and Figure 6 As shown in the figure, taking two training nodes as an example, the main process of asynchronous training based on the total training data includes:

[0083] During the first training, the first training node and the second training node download data simultaneously to avoid waiting for data download during training and reducing training efficiency.

[0084] The first training node asynchronously trains the pre-stored model to be trained based on the downloaded data block 1, obtains a training result 1, and sends the training result 1 to the second training node;

[0085] The second training node updates the pre-stored parameter information of the to-be-trained model based on the training result 1, and after the update is completed, performs asynchronous training based on the pre-downloaded data block 2. At the same time, for example, when the parameters are updated, the second training node sends a training start message 1 to the first training node, so that the first training node downloads the data block 3 required for re-training; further, the second training node obtains the training result 2 and sends the training result 2 to the first training node;

[0086] The first training node updates the parameter information of its own model to be trained based on the training result 2, and after the update is completed, performs asynchronous training based on the pre-downloaded data block 3. At the same time, for example, when the parameters are updated, the training start message 2 is sent to the second training node to enable the second training node to download the data block 4 required for re-training;

[0087] This cycle continues until the model to be trained converges, or the total training data to be trained has been trained on the model to be trained, or the number of training rounds, that is, the total number of training times for all training nodes combined, reaches a preset number, and the model training task is completed.

[0088] In a specific example of the disclosed solution, in order to ensure that the second training node can pre-present the required training data, such as Figure 7 As shown, the first training node can also send a second training start message to the second training node; wherein, the second training start message is generated by the first training node after asynchronously training the pre-stored model to be trained based on the target data to be trained, so that the second training node obtains (i.e., downloads) the data to be trained required for its own training based on the second training start message.

[0089] In a specific example of the disclosed solution, the first training node may also obtain the data to be trained required for further model training after sending the training result. Specifically, the first training node obtains new target data to be trained after determining to send the current training result to the second training node.

[0090] It should be noted that there are three possible times for obtaining download data in this disclosure:

[0091] The first method is to receive the training start message sent by the previous training node, and then obtain the training data required for its own training;

[0092] The second method is to determine the cache startup time based on historical time consumption, and obtain the training data required for self-training when the cache startup time is reached;

[0093] The third type: After obtaining the training results, the training results are sent to the next training node, which is the training data required for its own training.

[0094] In actual applications, the above three methods can be used one by one, or any two compatible ones can be used. It is worth noting that if any two compatible methods are used, such as the first and second methods, the priority of the two methods can be used to determine the timing of obtaining the training data required for self-training. Alternatively, in actual scenarios, as long as the current training node meets any of the above three conditions, it will obtain the training data required for its own training.

[0095] In this way, we can maximize the reduction of data download time as a percentage of the total time required for the entire system (i.e., the distributed training system) to complete model training. At the same time, we can also minimize the total time required, maximize the concealment of the time required for data download, and make the time required for the entire system to complete model training fully reflected in the computational overhead that cannot be concealed, thereby improving the overall training efficiency.

[0096] In a specific example of the disclosed solution, in order to ensure the normal progress of asynchronous training, after successfully starting the download task, the training node in the distributed training system will also feedback a download start message (also known as a cache start message) to the previous training node (such as the first training node) to inform the previous training node that the download task has been successfully started. On the contrary, if the previous training node does not receive the download start message, it means that the download has failed. At this time, the previous training node needs to reconfirm the new training node; specifically, if the first training node does not receive the cache start message fed back by the second training node, it will send the current training result to the next training node of the second training node in the distributed training system to trigger the next training node of the second training node to cache data. In other words, when the second training node cannot download normally, the first training node will skip the second training node and go directly to the next training node of the second training node, so as to ensure the normal progress of the entire asynchronous training.

[0097] In this way, on the one hand, it will not affect the stability of the current training task. On the other hand, the only link that needs to be quickly restored is the data cache, and the impact on the overall task is very limited. Therefore, the stability and robustness of the training are greatly improved.

[0098] It should be noted that the training nodes in the distributed training system described in the present disclosure are peer nodes. Based on this, the first, second, and third mentioned above are only for the convenience of description and are not used to limit the present disclosure.

[0099] In this way, the disclosed solution adopts an asynchronous training method. The training nodes perform asynchronous training when the data is cached, that is, when the data download is completed. Therefore, the training nodes do not need to consider the overhead of network resources and can use computing resources to the greatest extent for model training, providing a guarantee for effectively improving training efficiency.

[0100] The following is a further detailed description of the disclosed solution with reference to specific examples.

[0101] This example discloses a training method, specifically a distributed cluster training acceleration method with pre-caching of multiple devices (i.e., multiple training nodes). It mainly uses multiple devices to cache data streams in sequence to save the time caused by data downloading and copying during training. In this way, on the one hand, it can meet the demand for training efficiency, and on the other hand, it can effectively avoid the waste of resources caused by occupying too many devices.

[0102] It is understandable that the training node described in the disclosed solution can be specifically any electronic device with computing resources and storage resources, such as a personal computer, server, etc., and the disclosed solution does not impose any restrictions on this.

[0103] It should be noted that the disclosed solution is applicable to the situation where the training model (that is, the model to be trained required by the distributed training system described above) can be stored in the memory of a single machine (that is, a single device). In actual applications, if the scale of the training model exceeds the upper limit of the single machine memory, the disclosed solution cannot be applied.

[0104] Furthermore, the distributed cluster training acceleration method of the disclosed solution with multi-device pre-caching supports the design and implementation based on multi-node concurrent data caching and parameter exchange, specifically, Figure 8 As shown, it mainly includes three parts:

[0105] The first part is the scheduling end (also known as the scheduling function). Its main functions include: 1. Responsible for scheduling the size of the data block cached by the current training node, controlling and adjusting the data pre-download time or the size of the data block by balancing computing resources and data download time, or adjusting the data pre-download time and the size of the data block at the same time. In this way, the proportion of data download time in the entire model training process can be hidden to the greatest extent, and at the same time, the problem of long training time and low efficiency caused by long download time can be avoided to the greatest extent; 2. Responsible for starting the training task on the current training node.

[0106] The second part, the training end (also known as the training function), is mainly responsible for obtaining the training results of the previous training node before training, and updating the parameter information of the model to be trained of the current training node, and then starting local training.

[0107] The third part: the data cache end (also known as the data cache function) is mainly used to obtain the metadata of the data cache determined by the scheduling end (such as the number of data copies to be cached and the size of each data copy), cache the data based on the metadata, and notify the training end after the caching is completed. For example, after the caching is completed, it waits for the command to start training initiated by the scheduling end, and then notifies the training end to start training.

[0108] It should be understood that the scheduling end, training end and data cache end mentioned above are all logical functions. In actual applications, the training node described in the disclosed solution has the functions of the scheduling end, training end and data cache end mentioned above.

[0109] Based on the above functions, the disclosed solution effectively solves the following key problems

[0110] First, the problem of data sequential caching:

[0111] This disclosed solution uses a multi-node sequential caching approach to address the problem of fluctuating training results caused by data disorder during distributed training. Each training node sequentially caches the training data, thus ensuring the sequential nature of data reading. Furthermore, training tasks are initiated sequentially based on the order in which the data is cached. Even if different training nodes cache the training data, from the perspective of the distributed training system, this is equivalent to caching multiple copies of the data simultaneously. At this point, model training can still be performed sequentially.

[0112] Second, fault tolerance issues:

[0113] Due to the unreliability of network communication, in tasks that use large-scale distributed training, network anomalies will cause overall task anomalies, and retraining is a very resource-wasting operation. In the disclosed solution, since each training node is trained locally and independently, no network risk is introduced during the training process, and the stability and performance of training are better than distributed training. Moreover, if an abnormal training node appears during the data caching process, the disclosed solution can promptly apply for a new training node in the distributed training system to cache data. In this way, on the one hand, it will not affect the stability of the current training task, and on the other hand, it is only the data caching link that needs to be quickly restored, and the impact on the overall task is very limited, thereby greatly improving the stability and robustness of the training.

[0114] Third, the balance between data download and training time:

[0115] Each training node will feed back the time consumption and performance-related data of the data download and model training processes to other training nodes, so that the scheduling end of other training nodes can dynamically balance the download timing (that is, cache startup time) and / or the data volume characteristics of the training data to be downloaded according to the data download time consumption and model training time consumption in the training task. In this way, the data download time consumption and training time consumption can be balanced as much as possible to maximize the hiding of data download time consumption, so that the time consumption of the entire system to complete model training is fully reflected in the computational overhead that cannot be concealed, thereby improving the training efficiency as a whole.

[0116] Fourth, optimization of training performance:

[0117] Since there is no need to consider the overhead of data caching and network resources, the training node can use all computing resources for training tasks. In addition, it adopts a fully asynchronous training mode and performs asynchronous training at full power consumption, which maximizes device performance and reduces computing time as much as possible, thereby improving overall training performance.

[0118] Specifically, combined Figure 9 The steps of the disclosed solution are described in detail. Here, a distributed training system including two training nodes, namely a first training node (i.e., device 1) and a second training node (i.e., device 2) is used as an example for description, specifically including:

[0119] Step 1: Device 1 caches the first data. After data block 1 is cached, asynchronous training of device 1 is started.

[0120] Here, the total training data can be divided into blocks to obtain multiple data blocks. Each training session uses one or more data blocks. The one or more data blocks used by a device during a training session can be called a share. That is, the first share of data includes one or more data blocks. It should be noted that the data blocks used by different devices are different. In other words, data blocks cannot be reused to avoid ineffective training.

[0121] Step 2: While asynchronous training (i.e., multi-threaded asynchronous training) is started on device 1, device 2 starts caching the second copy of data based on the download task;

[0122] Step 3: After training is completed on device 1, the training results (model parameters) are directly synchronized to device 2.

[0123] Step 4: After device 2 completes caching of the second data, it performs asynchronous training again (i.e., multi-threaded asynchronous training) based on the updated model parameters based on the training results of training 1.

[0124] Step 5: While asynchronous training of device 2 is started, device 1 starts caching the third copy of data according to the download task.

[0125] Repeat this cycle until the training is completed.

[0126] This example can effectively reduce the high overhead, high distributed network risks and training data disorder problems brought about by traditional distributed training. Moreover, the training method of the disclosed solution can improve the training efficiency by 100% when the distributed training system includes two devices. In the event of an abnormality in the distributed network system, the training method of the disclosed solution can also quickly resume training, increasing the training speed by more than 500%.

[0127] The disclosed solution also provides a training device, for example, corresponding to the first training node, such as Figure 10 Shown, including:

[0128] The data acquisition unit 1001 is configured to acquire target data to be trained when it is determined that the cache startup condition is met, wherein the target data to be trained is part of the total data to be trained, and the total data to be trained is the total data for asynchronous training of the training model to be trained by the distributed training system;

[0129] The model training unit 1002 is configured to asynchronously train the pre-stored model to be trained based on the target data to be trained, when it is determined that the training conditions are met, to obtain a current training result;

[0130] The sending unit 1003 is configured to send the current training result to the second training node in the distributed training system.

[0131] In a specific example of the disclosed solution, the data acquisition unit is specifically configured to:

[0132] When a first training start message from a third training node in the distributed training system is obtained, obtaining target data to be trained;

[0133] The first training start message is generated by the third training node after asynchronously training the pre-stored model to be trained based on the first data to be trained; the first data to be trained is at least part of the total data to be trained except the target data to be trained.

[0134] In a specific example of the disclosed solution, it further includes: a receiving unit; wherein,

[0135] The receiving unit is configured to receive the first training result sent by the third training node;

[0136] The model training unit is specifically used to update the pre-stored parameter information of the model to be trained based on the first training result when the target data to be trained is cached, and after the update is completed, asynchronously train the pre-stored model to be trained based on the target data to be trained to obtain the current training result.

[0137] In a specific example of the disclosed solution, the second training node and the third training node are the same node.

[0138] In a specific example of the disclosed solution, the data acquisition unit is further configured to:

[0139] Obtaining historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves;

[0140] Determining a cache startup time for the first training node based on the acquired historical cache time and historical training time of the other training nodes;

[0141] When it is determined that the cache startup time has arrived, target data to be trained is obtained.

[0142] In a specific example of the disclosed solution, the data acquisition unit is specifically configured to:

[0143] Based on the acquired historical cache time, historical training time, and data volume characteristics required for asynchronous training of the other training nodes, a cache startup time for the first training node is determined.

[0144] In a specific example of the disclosed solution, the data acquisition unit is further used to determine the data volume characteristics required for asynchronous training; and acquire target training data that meets the data volume characteristics.

[0145] In a specific example of the disclosed solution, the data acquisition unit is further configured to:

[0146] Obtaining historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves;

[0147] The first training node adjusts the data volume feature based on the acquired historical cache time and historical training time of the other training nodes;

[0148] When it is determined that the cache startup condition is met, target to-be-trained data that meets the adjusted data volume feature is obtained.

[0149] In a specific example of the present disclosure, the sending unit is further used to send a second training start message to the second training node; wherein, the second training start message is generated by the first training node after asynchronously training the pre-stored model to be trained based on the target data to be trained.

[0150] In a specific example of the disclosed solution, the data acquisition unit is further configured to acquire new target data to be trained after determining to send the current training result to the second training node.

[0151] In a specific example of the disclosed solution, the sending unit is further used to send the current training result to the next training node of the second training node in the distributed training system when no cache startup message is received from the second training node, so as to trigger the next training node of the second training node to cache data.

[0152] The specific functions of each unit in the above-mentioned first training node can be described with reference to the above-mentioned method, which will not be repeated here.

[0153] The present disclosure also provides a training system, such as a distributed training system. Figure 2 As shown, it includes at least a first training node and a second training node; wherein,

[0154] The first training node is configured to, upon determining that a cache startup condition is satisfied, obtain target data to be trained, wherein the target data to be trained is a portion of the total data to be trained, and the total data to be trained is the total data for asynchronously training the to-be-trained model required to be trained by the distributed training system; upon determining that the training condition is satisfied, asynchronously train the pre-stored to-be-trained model based on the target data to be trained to obtain a current training result; and send the current training result to the second training node;

[0155] The second training node is configured to receive the current training result sent by the first training node.

[0156] In a specific example of the disclosed solution, the training system may further include a third training node; wherein,

[0157] The third training node is used to asynchronously train the pre-stored model to be trained based on the first data to be trained, generate a first training start message, and send it to the first training node; wherein the first data to be trained is at least part of the data in the total data to be trained except the target data to be trained.

[0158] Here, the relevant functions of the third training node can refer to the description of the above method and will not be repeated here.

[0159] It is understandable that the training system can refer to the relevant description of the above-mentioned distributed training system. Similarly, the specific functions of the first training node and the second training node can also refer to the relevant description of the above method, which will not be repeated here.

[0160] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0161] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0162] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0163] like Figure 11 As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. Various programs and data required for the operation of the device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0164] Various components in device 1100 are connected to I / O interface 1105, including an input unit 1106, such as a keyboard and mouse; an output unit 1107, such as various types of displays and speakers; a storage unit 1108, such as a magnetic disk and optical disk; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0165] The computing unit 1101 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the training method. For example, in some embodiments, the training method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the training method described above can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to perform the training method in any other appropriate manner (e.g., by means of firmware).

[0166] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0170] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0171] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0172] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0173] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A training method comprising: The first training node in the distributed training system obtains target data to be trained when determining that a cache startup condition is met, wherein the target data to be trained is part of the total data to be trained, and the total data to be trained is the total data for asynchronous training of the to-be-trained model required to be trained by the distributed training system; the cache startup condition indicates that a training startup message is detected; The first training node performs asynchronous training on the pre-stored model to be trained based on the target data to be trained to obtain a current training result when determining that a training condition is met; the training condition indicates that data caching is complete; The first training node sends the current training result to the second training node in the distributed training system.

2. The method according to claim 1, wherein The first training node in the distributed training system obtains target data to be trained when determining that a cache startup condition is met, including: The first training node in the distributed training system obtains target data to be trained when receiving a first training start message from a third training node in the distributed training system; The first training start message is generated by the third training node after asynchronously training the pre-stored model to be trained based on the first data to be trained; the first data to be trained is at least part of the total data to be trained except the target data to be trained.

3. The method according to claim 2, further comprising: The first training node receives the first training result sent by the third training node; Wherein, when determining that the training conditions are met, the first training node asynchronously trains the pre-stored to-be-trained model based on the target to-be-trained data to obtain a current training result, including: When the first training node completes caching the target data to be trained, it updates the pre-stored parameter information of the model to be trained based on the first training result, and after the update is completed, it asynchronously trains the pre-stored model to be trained based on the target data to be trained to obtain the current training result.

4. The method according to claim 2 or 3, wherein: The second training node and the third training node are the same node.

5. The method according to claim 1, further comprising: The first training node obtains the historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves; The first training node determines a cache start time for the first training node based on the acquired historical cache time and historical training time of the other training nodes; Wherein, the first training node in the distributed training system obtains target data to be trained when determining that the cache startup condition is met, including: When the first training node in the distributed training system determines that the cache startup time has arrived, it obtains target data to be trained.

6. The method according to claim 5, wherein: The first training node determines a cache startup time for the first training node based on the acquired historical cache time and historical training time of the other training nodes, including: The first training node determines a cache startup time for the first training node based on the acquired historical cache time, historical training time, and data volume characteristics required for asynchronous training of the other training nodes.

7. The method according to any one of claims 1 to 3, further comprising: The first training node determines a data volume characteristic required for asynchronous training; The step of obtaining target data to be trained includes: Obtain target training data that meets the data volume characteristics.

8. The method according to claim 7, further comprising: The first training node obtains the historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves; The first training node adjusts the data volume feature based on the acquired historical cache time and historical training time of the other training nodes; Wherein, the first training node in the distributed training system obtains target data to be trained when determining that the cache startup condition is met, including: The first training node in the distributed training system obtains target to-be-trained data that meets the adjusted data volume feature when determining that the cache startup condition is met.

9. The method according to any one of claims 1 to 3, 5, and 6, further comprising: The first training node sends a second training start message to the second training node; wherein the second training start message is generated by the first training node after asynchronously training the pre-stored model to be trained based on the target data to be trained.

10. The method according to any one of claims 1 to 3, 5, and 6, further comprising: After determining to send the current training result to the second training node, the first training node obtains new target data to be trained.

11. The method according to any one of claims 1 to 3, 5, and 6, further comprising: When the first training node does not receive the cache startup message fed back by the second training node, the first training node sends the current training result to the next training node of the second training node in the distributed training system to trigger the next training node of the second training node to cache data.

12. A training device, deployed in a first training node; comprising: a data acquisition unit, configured to acquire target data to be trained if it is determined that a cache startup condition is satisfied, wherein the target data to be trained is a portion of the total data to be trained, and the total data to be trained is the total data for asynchronous training of the to-be-trained model required to be trained by the distributed training system; the cache startup condition indicates detection of a training startup message; A model training unit is configured to asynchronously train the pre-stored model to be trained based on the target data to be trained to obtain a current training result when it is determined that a training condition is met; the training condition indicates that data caching is complete; A sending unit is used to send the current training result to the second training node in the distributed training system.

13. The training device according to claim 12, wherein: The data acquisition unit is specifically used to: When a first training start message from a third training node in the distributed training system is obtained, obtaining target data to be trained; The first training start message is generated by the third training node after asynchronously training the pre-stored model to be trained based on the first data to be trained; the first data to be trained is at least part of the total data to be trained except the target data to be trained.

14. The training device according to claim 13, further comprising: Receiving unit; wherein, The receiving unit is configured to receive the first training result sent by the third training node; The model training unit is specifically used to update the pre-stored parameter information of the model to be trained based on the first training result when the target data to be trained is cached, and after the update is completed, asynchronously train the pre-stored model to be trained based on the target data to be trained to obtain the current training result.

15. The training device according to claim 13 or 14, wherein: The second training node and the third training node are the same node.

16. The training device according to claim 12, wherein: The data acquisition unit is further configured to: Obtaining historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves; Determining a cache startup time based on the acquired historical cache time and historical training time of the other training nodes; When it is determined that the cache startup time has arrived, target data to be trained is obtained.

17. The training device according to claim 16, wherein: The data acquisition unit is specifically used to: The cache startup time is determined based on the acquired historical cache time, historical training time, and data volume characteristics required for asynchronous training of the other training nodes.

18. The training device according to any one of claims 12 to 14, wherein: The data acquisition unit is further configured to determine the data volume characteristics required for asynchronous training; and acquire target training data that meets the data volume characteristics.

19. The training device according to claim 18, wherein The data acquisition unit is further configured to: Obtaining historical cache time and historical training time of other training nodes in the distributed training system; wherein the historical cache time represents the cache time of the other training nodes for caching part of the total data to be trained required for their own asynchronous training; and the historical training time represents the training time required for the other training nodes to perform asynchronous training based on the part of the total data to be trained cached by themselves; Adjusting the data volume feature based on the acquired historical cache time and historical training time of the other training nodes; When it is determined that the cache startup condition is met, target to-be-trained data that meets the adjusted data volume feature is obtained.

20. The training device according to any one of claims 12 to 14, 16 and 17, wherein: The sending unit is further configured to send a second training start message to the second training node; wherein the second training start message is generated after asynchronous training of the pre-stored model to be trained based on the target data to be trained.

21. The training device according to any one of claims 12 to 14, 16 and 17, wherein: The data acquisition unit is further configured to acquire new target data to be trained after determining to send the current training result to the second training node.

22. The training device according to any one of claims 12 to 14, 16 and 17, wherein: The sending unit is further configured to send the current training result to the next training node of the second training node in the distributed training system when no cache startup message is received from the second training node, so as to trigger the next training node of the second training node to cache data.

23. A training system comprising at least: A first training node and a second training node; wherein, The first training node is configured to obtain target data to be trained if it is determined that a cache startup condition is satisfied, wherein the target data to be trained is part of the total data to be trained, and the total data to be trained is the total data for asynchronously training the model to be trained required by the training system; the cache startup condition indicates that a training startup message is detected; if it is determined that the training condition is satisfied, asynchronously train the pre-stored model to be trained based on the target data to be trained to obtain a current training result; wherein the training condition indicates that data caching is complete; and send the current training result to the second training node; The second training node is configured to receive the current training result sent by the first training node.

24. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

25. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

26. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images