Decision model training reasoning method, product, electronic equipment and medium

Abstract

The invention discloses a training reasoning method of a decision model, a product, electronic equipment and a medium, and relates to the technical field of artificial intelligence. In the method, when the training reasoning process of the decision model is divided into a plurality of calculation tasks to be distributed to different cluster nodes, the load pressure on the nodes is reduced; and the plurality of data subsets obtained by splitting the input data corresponding to the calculation task are allocated to different calculation nodes, so that the load pressure on the calculation nodes is reduced, the utilization rate of node resources is improved, parallel processing of the data subsets corresponding to the input data is realized, and the efficiency of training reasoning is improved. When the training reasoning process of the decision model is divided into one calculation task, the plurality of data subsets obtained by splitting the input data corresponding to the calculation task are distributed to different calculation nodes, so that the load balance on the calculation nodes is ensured, the load is reasonably distributed to the nodes in the cluster, and the calculation efficiency is improved. And the utilization rate of node resources is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a training and reasoning method, product, electronic device and medium for a decision model. Background Art

[0002] Intelligent autonomous decision-making technology is an important branch of artificial intelligence, which aims to enable the system to automatically make reasonable decisions in complex environments. It covers a variety of methods and technologies, including multiple decision-making models, such as rule-based systems, machine learning algorithms, and reinforcement learning-based methods. In order to achieve the training and reasoning of decision-making models, related technologies perform model training and reasoning on a platform based on a central processing unit (CPU) + a graphics processing unit (GPU). However, the core number and structure of traditional CPUs and GPUs are fixed, the hardware is not reconfigurable, and the structure cannot be dynamically adjusted according to application requirements.

[0003] Considering the unique reconfigurable characteristics of Field-Programmable Gate Array (FPGA), and the difficulty of a single FPGA board to support the operation of multiple components in training and reasoning, when training and reasoning the decision model, parallel distributed computing is achieved by building an FPGA cluster to accelerate the model training and reasoning process, shorten the training time, and improve the model performance. In the decision model training and reasoning system, if the load allocated to the nodes in the FPGA cluster is unreasonable, it may cause data transmission and synchronization delays, resulting in some tasks not being completed in time, or poor model training results.

[0004] It can be seen that how to reasonably distribute loads to nodes in an FPGA cluster is a technical problem that people in this field need to solve urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a training and reasoning method, product, electronic device and medium for a decision model to solve the technical problem of unreasonable node load distribution in an FPGA cluster.

[0006] In order to solve the above technical problems, the present invention provides a training and reasoning method for a decision model, which is applied to a processor of a host computer in a training and reasoning acceleration system; the accelerator cluster of the training and reasoning acceleration system is a cluster structure; the method comprises: Obtaining a decision model and obtaining network characteristics of the decision model; Dividing the training and reasoning process of the decision model into one or more computing tasks according to the network characteristics of the decision model, and assigning different computing tasks to different cluster nodes; Distribute the multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes to different computing nodes, and transfer the computing tasks on the cluster nodes to the computing nodes to which the input data are distributed; The training and reasoning results of the decision model are determined according to the output characteristics of the network model on the computing node to which the input data is distributed.

[0007] On the one hand, the training and reasoning process of the decision model is divided into multiple computing tasks according to the network characteristics of the decision model, and different computing tasks are assigned to different cluster nodes, including: Dividing the training and reasoning process of the decision model into one or more tasks according to the training and reasoning stages included in the decision model, and allocating the tasks to different cluster nodes of the first type; Dividing the task into one or more subtasks according to the network structure used when processing the task, and allocating different subtasks to different cluster nodes of the second type; The subtask is treated as one computing task or divided into a plurality of computing tasks according to the complexity of processing the subtask, and different computing tasks are allocated to different cluster nodes of the third type.

[0008] On the other hand, according to the complexity of processing the subtask, the subtask is treated as one computing task or divided into multiple computing tasks, and different computing tasks are allocated to different cluster nodes of the third type, including: When it is detected that the amount of computation when processing the subtask is less than the preset amount of computation, the subtask is regarded as a computation task, and the computation task is allocated to a cluster node of the third type; When it is detected that the amount of computation when processing the subtask is greater than or equal to the preset amount of computation, the subtask is divided into multiple computing tasks; the network structure of the network model used when processing the computing task is classified according to the complexity of the network structure of the network model used when processing the computing task; and the computing tasks corresponding to different categories of network structures are allocated to different cluster nodes of the third type.

[0009] On the other hand, obtaining the input data corresponding to the computing task on the cluster node includes: If the computing task is an initial computing task among all computing tasks, determining that the input data corresponding to the initial computing task is data sent by the processor of the host computer and received by the cluster node of the first type; If the computing task is not the initial computing task, it is determined that the input data corresponding to the computing task is the output feature obtained by computing the input data corresponding to the previous computing task of the computing task through the network model on the computing node.

[0010] On the other hand, the input data corresponding to the computing tasks on the cluster nodes are split into multiple data subsets including: When it is detected that the input data corresponding to the computing task on the cluster node is image data, if the data batch size is greater than or equal to the number of input channels, the input data corresponding to the computing task on the cluster node is split along the data batch dimension to obtain multiple data subsets; if the data batch size is less than the number of input channels, the input data corresponding to the computing task on the cluster node is split along the input channel dimension to obtain multiple data subsets; When it is detected that the input data corresponding to the computing task on the cluster node is not image data, the dimension with the largest number of dimensions is obtained from the data batch dimension and multiple feature dimensions; the input data corresponding to the computing task on the cluster node is split along the dimension with the largest number of dimensions to obtain multiple data subsets.

[0011] On the other hand, when it is detected that the dimension with the largest number of dimensions is a feature dimension, the step of allocating the multiple data subsets obtained by splitting the input data corresponding to the computing task on the cluster node to different computing nodes includes: Obtaining the number of computing nodes to which the preset input data is to be distributed; Obtain the result of performing a modulo operation on the dimension with the largest number of dimensions and the number of computing nodes to be allocated to determine a first number of computing nodes; Determining a second number of computing nodes according to a difference between the number of computing nodes to be allocated and the first number; Splitting the input data corresponding to the computing task on the cluster node according to the dimension with the largest number of dimensions determines a data subset of a first data size and a data subset of a second data size; wherein the first data size is determined by multiplying the result obtained by rounding down the dimension with the largest number of dimensions and the number of computing nodes to be allocated plus 1 and the data size on the remaining dimensions; the second data size is determined by multiplying the result obtained after the rounding down operation and the data size on the remaining dimensions; the remaining dimensions are dimensions other than the dimension with the largest number of dimensions among all dimensions; The data subsets of the first data size are respectively distributed to a first number of computing nodes, and the data subsets of the second data size are respectively distributed to a second number of computing nodes.

[0012] On the other hand, determining the training inference result of the decision model according to the output characteristics of the network model on the computing node to which the input data is distributed includes: Obtaining the current output features of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node; wherein the current subtask starts from the initial subtask of the training and reasoning process of the decision model; Obtain the next subtask of the current subtask according to the first order of the subtasks in the training and reasoning process of the decision model, and use the next subtask of the current subtask as the new current subtask; Transmitting the current output feature to the second type of cluster node where the current subtask is located via the third type of cluster node where the computing task corresponding to the current subtask is located; Obtain the second type of cluster node where the new current subtask is located, transmit the current output feature to the second type of cluster node where the new current subtask is located through the second type of cluster node where the current subtask is located, so as to obtain the input data corresponding to the new current subtask, and return the current output feature after the input data of the computing task corresponding to the current subtask passes through the network model on the computing node, until the current subtask is the last subtask in the training and reasoning process of the decision model, stop returning, and determine the training and reasoning result of the decision model according to the current output feature.

[0013] On the other hand, transmitting the current output feature to the new second type cluster node where the current subtask is located via the second type cluster node where the current subtask is located comprises: Controlling the communication between the second type of cluster node where the current subtask is located and the second type of cluster node where the new current subtask is located through the first signal transmission mode and obtaining communication information; wherein the communication information at least includes status monitoring information and status feedback information; When it is detected that the communication information is normal, the current output feature is transmitted to the second type of cluster node where the new current subtask is located through the second signal transmission mode via the second type of cluster node where the current subtask is located.

[0014] On the other hand, the current output features of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node include: In the case where it is detected that the current computing task is not obtained by dividing the current subtask, the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node is obtained, and the sum of the current output features is used as the current output feature of the computing task corresponding to the current subtask after the input data passes through the network model on the computing node; When it is detected that the current subtask is divided into multiple computing tasks, the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node is obtained; the next computing task of the current subcomputing task is obtained according to the second priority order of the computing tasks in the current subtask, and the next computing task of the current computing task is used as the new current computing task; the sum of the current output features is transmitted to the third type of cluster node where the new current computing task is located via the third type of cluster node where the current computing task is located, so as to obtain the input data corresponding to the new current computing task, and return to the step of obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node, until the current computing task is the last computing task in the current subtask, stop returning, output the sum of the current output features, and use the sum of the current output features as the current output feature of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node.

[0015] On the other hand, obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node includes: Obtaining a computing node of a first type and a computing node of a second type used when processing a data subset corresponding to a current computing task; wherein a network model is set in the computing node of the first type, and a network model is not set in the computing node of the second type; and there is one computing node of the second type; Obtain output features of each data subset corresponding to the current computing task on the corresponding computing node of the first type; Sending output features on all computing nodes of the first type to computing nodes of the second type; An aggregation operation is performed on all output features through the computing nodes of the second type to obtain the sum of current output features of the data subset corresponding to the current computing task on the corresponding computing nodes.

[0016] On the other hand, obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node includes: Obtaining the communication order between all computing nodes processing the current computing task; wherein the last computing node among all computing nodes arranged in the communication order is connected to the first computing node in communication; Obtain data segments of each data subset corresponding to the current computing task after being passed through the network model on the corresponding computing node and store them in a sending buffer of the computing node for sending data; Determine whether the current computing node processing the current computing task has data fragments of all other nodes; wherein the other nodes are computing nodes other than the current computing node among all computing nodes processing the current computing task; If not, then the data fragment on the current computing node is sent from the sending buffer to the receiving buffer for receiving data of the next computing node of the current computing node in accordance with the communication order; the next computing node of the current computing node is used as the new current computing node; the data of the receiving buffer of the new current computing node is aggregated with the data of the sending buffer of the new current computing node and used as the data fragment of the sending buffer of the new current computing node, and the step of determining whether there are data fragments of all other nodes in the current computing node processing the current computing task is returned; If so, the data fragments on the current computing node will be sent from the sending buffer to the receiving buffer for receiving data of the next computing node of the current computing node in accordance with the communication order; the next computing node of the current computing node will be used as the new current computing node; the data in the receiving buffer of the new current computing node will be overwritten with the data in the sending buffer of the new current computing node and used as the data fragments of the sending buffer of the new current computing node, and the step of determining whether there are data fragments of all other nodes in the current computing node processing the current computing task will be returned until it is detected that there are data fragments of all other nodes in all computing nodes processing the current computing task and the process stops returning.

[0017] On the other hand, transmitting the sum of the current output features to the cluster node of the third type where the new current computing task is located via the cluster node of the third type where the current computing task is located comprises: Controlling the communication between the third type of cluster node where the current computing task is located and the third type of cluster node where the new current computing task is located through the first signal transmission mode and obtaining communication information; wherein the communication information at least includes status monitoring information and status feedback information; When it is detected that the communication information is normal, the sum of the current output features is transmitted via the third type of cluster node where the current computing task is located to the third type of cluster node where a new current computing task is located through the second signal transmission mode.

[0018] On the other hand, before obtaining the output features of each data subset corresponding to the current computing task on the corresponding computing node of the first type, the method further includes: If it is detected that the load on the first type of computing nodes and the second type of computing nodes used to process the data subset corresponding to the current computing task is less than the preset load, the step of obtaining the output characteristics of each data subset corresponding to the current computing task on the corresponding first type of computing node is entered.

[0019] On the other hand, before obtaining the communication order between all computing nodes processing the current computing task, it also includes: Obtaining load conditions on the first type of computing nodes and the second type of computing nodes used when processing the data subset corresponding to the current computing task; If it is detected that the load on the first type of computing nodes and the second type of computing nodes used to process the data subset corresponding to the current computing task is greater than or equal to the preset load, the step of obtaining the communication order between all computing nodes processing the current computing task is entered.

[0020] On the other hand, the computing nodes include multiple types, and different types of computing nodes are used to process different types of network layer data; The step of allocating multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes to different computing nodes includes: Obtaining network layer characteristics of the computing task; Determine the target computing node based on the network layer characteristics; The multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes are distributed to different target computing nodes.

[0021] On the other hand, the first type of cluster nodes are located on the host computer; the second type of cluster nodes assigned to the subtasks corresponding to the same task and the third type of cluster nodes assigned to the same computing task are located on the same host; the third type of cluster nodes assigned to different computing tasks are located on different hosts.

[0022] On the other hand, allocating the tasks to different cluster nodes of the first type includes: Allocating the task to different cluster nodes of the first type by a third signal transmission method; The allocating different subtasks to different cluster nodes of the second type comprises: Using the first type of cluster nodes and allocating different subtasks to different second type of cluster nodes through the third signal transmission mode; The allocating different computing tasks to different third-type cluster nodes includes: The second type of cluster nodes are utilized and different computing tasks are allocated to different third type of cluster nodes through the third signal transmission manner.

[0023] On the other hand, the training reasoning of the decision model includes an interaction phase and a model update phase; before sending data to the first type of cluster nodes, it also includes: Obtaining data sent to a cluster node of the first type from a process of the graphics processing unit for training inference; The training inference result of determining the decision model according to the current output feature includes: receiving, in an interactive phase, a first current output feature outputted sequentially by the cluster nodes of the third type, the cluster nodes of the second type, and the cluster nodes of the first type; Storing the first current output feature in a data storage module located in a host computer; In the case where it is detected that the amount of data in the data storage module is greater than a preset amount of data, receiving, in the model updating stage, a second current output feature outputted sequentially by the cluster nodes of the third type, the cluster nodes of the second type, and the cluster nodes of the first type; Calculating a loss function based on the second current output feature to determine parameters of a decision model, so as to obtain a training inference result of the decision model; After obtaining the training and reasoning results of the decision model, the following steps are also included: The parameters of the decision model are transmitted to the cluster nodes of the first type to update the parameters of the decision model in the cluster nodes of the first type.

[0024] On the other hand, establishing a decision model located on the first type of cluster node includes: Capturing the complexity of the decision-making task; Decision models with different structures are established on the first type of cluster nodes according to the complexity of the decision task; wherein the complexity of the decision task is positively correlated with the complexity of the decision model structure.

[0025] On the other hand, determining the structure of the decision model includes: Acquire the operating status of the hardware platform; wherein the operating status of the hardware platform at least includes hardware resource utilization, latency and power consumption; Obtaining the current expected cumulative reward value under the current state and the current decision model structure; wherein the current state at least includes the current input data characteristics and the current operating state of the hardware platform; Determining an expected value according to a current expected cumulative reward and a current reward function; wherein the current reward function is determined by a current operating state of the hardware platform; Obtain the current decision model structure whose expected value meets the requirements; The current decision model structure whose expected value meets the requirements is used as the structure of the decision model.

[0026] On the other hand, determining the structure of the decision model includes: The structure of multiple pre-defined decision models; Obtaining hardware performance indicators under decision models of different structures, and establishing hardware constraints based on the hardware performance indicators; wherein the hardware performance indicators at least include latency, power consumption, and resource occupancy; Obtain the fitness of decision models with different structures; Under the condition of satisfying hardware constraints, obtain the decision model structure with the greatest fitness among all decision models of the structure; The decision model structure with the greatest fitness is taken as the structure of the decision model.

[0027] On the other hand, determining the structure of the decision model includes: Pre-set structures of various decision models and features of extracted input data; Predicting the probability of selecting decision models of different structures according to the characteristics of the input data; Obtain the structure of the decision model with the highest probability among all decision models of the structure; The structure of the decision model with the highest probability is taken as the structure of the decision model.

[0028] In order to solve the above technical problem, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the training and reasoning method of the above decision model when executed by a processor.

[0029] In order to solve the above technical problem, the present invention further provides an electronic device, comprising: Memory for storing computer programs; A processor is used to implement the steps of the training and reasoning method of the above-mentioned decision model when executing the computer program.

[0030] In order to solve the above technical problem, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the training and reasoning method of the above-mentioned decision model are implemented.

[0031] The beneficial effect of the present invention is that, in the training and reasoning process of the decision model, the training and reasoning process of the decision model is divided into one or more computing tasks according to the network characteristics of the decision model. Among them, in the case where the training and reasoning process of the decision model is divided into multiple computing tasks according to the network characteristics of the decision model, different computing tasks are assigned to different cluster nodes in the present invention. Compared with the method of directly assigning the entire process of the training and reasoning of the decision model to the node without division, the method provided by the present invention reduces the pressure of the load on the node; and the multiple data subsets obtained after the input data corresponding to the computing task is split are assigned to different computing nodes, which reduces the load pressure on the computing node and improves the utilization rate of the node resources compared to the method of not splitting the input data; when assigning computing nodes to computing tasks, the computing tasks are assigned to multiple computing nodes to which the input data is assigned, so as to realize the parallel processing of the data subsets corresponding to the input data and improve the efficiency of training and reasoning. In the case where the training and inference process of the decision model is divided into a computing task according to the network characteristics of the decision model, multiple data subsets obtained by splitting the input data corresponding to the computing task are distributed to different computing nodes, thereby ensuring load balancing on the computing nodes, achieving reasonable load distribution for nodes in the cluster, and improving the utilization of node resources.

[0032] In addition, when the training and reasoning process of the decision model is divided into multiple computing tasks according to the network characteristics of the decision model, and different computing tasks are assigned to different cluster nodes, the training and reasoning process of the decision model is divided into one or more tasks, and then the tasks are divided into one or more subtasks according to the network structure used when processing the tasks; and further, according to the complexity of processing the subtasks, the subtasks are used as a computing task or divided into multiple computing tasks, that is, the training and reasoning process of the decision model is divided into smaller granularities and handed over to the cluster nodes for processing. And the tasks, subtasks and computing tasks obtained by the training and reasoning of the decision model are assigned to different types of cluster nodes, so that the data traffic of each cluster node is similar in scale, ensuring the load balancing of each node, and making full use of node resources, improving the utilization rate of node resources.

[0033] When it is detected that the amount of computation when processing a subtask is greater than or equal to a preset amount of computation, the subtask is divided into multiple computing tasks; the network structure of the network model used when processing the computing task is classified according to the complexity of the network structure of the network model used when processing the computing task; and the computing tasks corresponding to different categories of network structures are allocated to different third-type cluster nodes, thereby ensuring the balance of the amount of network parameters allocated on the third-type cluster nodes as much as possible.

[0034] In the process of splitting the input data corresponding to the computing task on the cluster node to obtain multiple data subsets, different data splitting methods are adopted according to the type of the input data corresponding to the computing task (such as image data or non-image data), thereby realizing reasonable splitting of the input data; and when it is detected that the input data corresponding to the computing task on the cluster node is not image data, the input data corresponding to the computing task on the cluster node is split along the dimension with the largest number of dimensions to obtain multiple data subsets, thereby improving the parallelism of data processing and improving resource utilization.

[0035] Obtain the result of the modulo operation between the dimension with the largest number of dimensions and the number of computing nodes to be allocated to determine the first number of computing nodes; determine the second number of computing nodes according to the difference between the number of computing nodes to be allocated and the first number; split the input data corresponding to the computing task on the cluster node according to the dimension with the largest number of dimensions to determine the data subset of the first data size and the data subset of the second data size; respectively allocate the data subset of the first data size to the first number of computing nodes, and respectively allocate the data subset of the second data size to the second number of computing nodes. This method provides a way to allocate data subsets to computing nodes in the scenario where the feature dimension cannot divide the number of computing nodes set, and ensures that the input data is reasonably allocated to the computing nodes as much as possible.

[0036] In the process of determining the training and reasoning results of the decision model according to the output features of the network model on the computing node to which the input data is allocated, the output features of the previous subtask are sequentially transmitted to the next subtask according to the first order of the subtasks in the training and reasoning process of the decision model, thereby realizing the training and reasoning of the decision model. In addition, in the process of transmitting the current output features to the second type of cluster node where the current subtask is located to the second type of cluster node where the new current subtask is located via the second type of cluster node where the current subtask is located, the communication information between the second type of cluster node where the current subtask is located and the second type of cluster node where the new current subtask is located is detected via the first signal transmission method. When it is detected that the communication information is normal, the current output features are transmitted to the second type of cluster node where the new current subtask is located via the second type of cluster node where the current subtask is located via the second signal transmission method, thereby ensuring the stable operation of the entire system and the accurate transmission of data as much as possible.

[0037] In the process of obtaining the current output characteristics of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node, in the case where the current computing task is obtained without division, the sum of the current output characteristics of the data subset corresponding to the current computing task on the corresponding computing node is directly used as the current output characteristics of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node. In the case where the current subtask is divided to obtain multiple computing tasks, the sum of the current output characteristics is transmitted to the third type of cluster node where the new current computing task is located through the third type of cluster node where the current computing task is located, so as to obtain the input data corresponding to the new current computing task, and further obtain the sum of the current output characteristics of the data subset corresponding to the current computing task on the corresponding computing node, so as to realize the aggregation of the results of all computing tasks of the same subtask on the computing node, that is, to obtain the current output characteristics of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node.

[0038] In the process of obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node, different types of computing nodes are set, and the output features of all the first type of computing nodes are sent to the second type of computing nodes; the second type of computing nodes aggregate all the output features to obtain the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node. That is, centralized data parallelization is realized based on the structure of central node-multiple sub-nodes.

[0039] In the process of obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node, the communication order between all computing nodes processing the current computing task is obtained, and the last computing node among all computing nodes arranged in the communication order is connected to the first computing node for communication. Each computing node sends its own data to the next computing node and receives data from the previous computing node. As the communication proceeds, each computing node gradually integrates data fragments from other nodes, and finally all nodes have complete synchronized data, that is, the data integration of each computing node is completed based on the ring communication method of decentralized nodes.

[0040] In the process of transmitting the sum of current output features via the third type of cluster node where the current computing task is located to the third type of cluster node where the new current computing task is located, the communication status between the third type of cluster node where the current computing task is located and the third type of cluster node where the new current computing task is located is first monitored. Data transmission is performed only when the communication information is normal, thereby ensuring the stable operation of the entire system and the accurate transmission of data as much as possible.

[0041] Set up different computing nodes for processing different types of network layer data. Obtain the network layer characteristics of the computing task; determine the target computing node based on the network layer characteristics; and distribute multiple data subsets obtained by splitting the input data corresponding to the computing task on the cluster node to different target computing nodes, thereby improving the efficiency of training and reasoning.

[0042] The first type of cluster nodes are located on the host computer; the second type of cluster nodes assigned to the subtasks corresponding to the same task and the third type of cluster nodes assigned to the same computing task are located on the same host; the third type of cluster nodes assigned to different computing tasks are located on different hosts. Through different hosts, parallel processing can be achieved, thereby improving the efficiency of decision model training and reasoning.

[0043] Different decision models are used for decision-making tasks of different complexity, so that the selected decision model is more appropriate; when determining the structure of the decision model, the structure of the decision model can be selected in different ways, so as to reasonably set the structure of the decision model and enable the structure of the decision model to be adaptively selected, thereby improving the flexibility of the structure of the decision model.

[0044] In addition, the present invention also provides a computer product, an electronic device and a non-volatile storage medium, which have the same or corresponding technical features as the training and inference method of the policy model mentioned above, and have the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of a training and reasoning acceleration system provided by an embodiment of the present invention; Figure 2 A flowchart of a training and reasoning method for a decision model provided by an embodiment of the present invention; Figure 3 A schematic diagram of a decision model training and reasoning acceleration system based on an FPGA cluster provided in an embodiment of the present invention; Figure 4 A schematic diagram of a heterogeneous computing framework based on an FPGA cluster provided in an embodiment of the present invention; Figure 5 A schematic diagram of a decision model structure for complex decision tasks provided by an embodiment of the present invention; Figure 6 A schematic diagram of a data parallelization structure of multiple sub-nodes and a central node provided by an embodiment of the present invention; Figure 7 A schematic diagram of a model update process based on multi-child node-central node data parallelization provided by an embodiment of the present invention; Figure 8 A schematic diagram of a ring communication topology structure with decentralized node data parallelization provided by an embodiment of the present invention; Fig. 9 A schematic diagram of a multi-Actor parallel interaction structure based on FPGA provided by an embodiment of the present invention; Fig.10 A structural diagram of a single FPGA computing node for convolution computing provided by an embodiment of the present invention; Fig.11 A structural diagram of a single FPGA computing node for attention calculation provided by an embodiment of the present invention; Fig.12 A structural diagram of a single FPGA computing node for convolution computing provided by an embodiment of the present invention; Fig.13 A schematic diagram of the structure of a multi-stage cluster of accelerators provided in an embodiment of the present invention; Fig.14 A logical schematic diagram of an action cluster node provided by an embodiment of the present invention; Fig.15 A logical schematic diagram of a trainer cluster node provided by an embodiment of the present invention; Fig.16 A logical structure diagram of a state coding task cluster of an action part provided by an embodiment of the present invention; Fig.17 A logical structure diagram of a state encoding task cluster of a trainer part provided by an embodiment of the present invention; Fig.18 A logical schematic diagram of a module cluster for convolutional computing provided in the action part of an embodiment of the present invention; Fig.19 A logical schematic diagram of a module cluster for convolutional computing in a trainer part provided in an embodiment of the present invention; Fig. 20 A logical schematic diagram of a module cluster for attention calculation of the action part provided by an embodiment of the present invention; Fig.21 A logical schematic diagram of a module cluster for attention calculation in a trainer part provided in an embodiment of the present invention; Fig. 22 A logical schematic diagram of a module cluster for point cloud feature coding of an action part provided by an embodiment of the present invention; Fig.23 A logical schematic diagram of a module cluster for point cloud feature coding of a trainer part provided in an embodiment of the present invention; Fig.24 A logic diagram of a variational autoencoder decoder generation model provided by an embodiment of the present invention; Fig.25 A logical schematic diagram of a generative adversarial network generation model provided by an embodiment of the present invention; Fig.26 A logical schematic diagram of a diffusion generation model provided by an embodiment of the present invention; Fig. 27 A structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The core of the present invention is to provide a training and reasoning method, product, electronic device and medium for a decision model to solve the technical problem of unreasonable node load distribution in an FPGA cluster.

[0049] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 A schematic diagram of a training and reasoning acceleration system provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the training and reasoning acceleration system includes: a host computer and an accelerator cluster. The accelerator cluster of the training and reasoning acceleration system is a cluster structure. Figure 2 A flowchart of a training and reasoning method for a decision model provided by an embodiment of the present invention. The method includes: S10: obtaining a decision model and obtaining network features of the decision model; S11: Divide the training and reasoning process of the decision model into one or more computing tasks according to the network characteristics of the decision model, and assign different computing tasks to different cluster nodes; S12: Distribute multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes to different computing nodes, and transfer the computing tasks on the cluster nodes to the computing nodes to which the input data are distributed; S13: Determine the training inference result of the decision model according to the output characteristics of the network model on the computing node to which the input data is allocated.

[0050] There is no limitation on the decision model to be set, which is determined according to the actual situation. In order to make the decision model set more reasonable, the complexity of the decision task can be considered to establish the structure of the decision model. As the complexity of the task and the heterogeneity of the hardware increase, the coding module design method based on predefined rules will lack flexibility in a dynamic environment. In order to improve the flexibility of the decision model structure, in the implementation, the method of determining the structure of the decision model includes at least one of the following methods: Method 1: Obtaining the operating status of the hardware platform; wherein the operating status of the hardware platform at least includes hardware resource utilization, latency and power consumption; Obtain the current expected cumulative reward value under the current state and the current decision model structure; wherein the current state at least includes the current input data characteristics and the current operating state of the hardware platform; Determining an expected value according to a current expected cumulative reward and a current reward function; wherein the current reward function is determined by a current operating state of the hardware platform; Obtain the current decision model structure whose expected value meets the requirements; The current decision model structure whose expected value meets the requirements is taken as the structure of the decision model.

[0051] Method 2: A structure of multiple pre-set decision models; Obtaining hardware performance indicators under decision models of different structures, and establishing hardware constraints based on the hardware performance indicators; wherein the hardware performance indicators at least include latency, power consumption, and resource occupancy; Obtain the fitness of decision models with different structures; Under the condition of satisfying hardware constraints, obtain the decision model structure with the greatest fitness among all decision models of the structure; The decision model structure with the largest fitness is taken as the structure of the decision model.

[0052] Method 3: Pre-set structures of multiple decision models and extract features of input data; Predict the probability of selecting decision models with different structures based on the characteristics of the input data; Obtain the structure of the decision model with the highest probability among all decision models of the structure; The structure of the decision model with the highest probability is taken as the structure of the decision model.

[0053] That is, the above-mentioned method of determining the structure of the decision model realizes the adaptive selection of the model structure.

[0054] After determining the decision model, the network characteristics of the decision model are obtained. The network characteristics of the decision model include the training and reasoning stages included in the decision model, the network structure used when processing each training and reasoning stage, and the complexity of implementing each layer of the network structure. In order to realize the training and reasoning of the decision model, an accelerator cluster is established in an embodiment of the present invention. The accelerator cluster includes a first type of cluster node, a second type of cluster node, and a third type of cluster node that undertake task allocation, control and state return tasks, as well as a computing node that undertakes the implementation of computing tasks. These nodes are all accelerators. In order to improve the communication efficiency between the processor of the host computer and the first type of cluster node, in implementation, the first type of cluster node can be set in the host computer; in order to realize parallel processing, the second type of cluster node and the third type of cluster node and the computing node that undertakes the implementation of computing tasks can be set in a host connected to the host computer.

[0055] In the implementation, the training and reasoning process of the decision model is divided into multiple computing tasks according to the network characteristics of the decision model, and different computing tasks are assigned to different cluster nodes, including: Dividing the training and reasoning process of the decision model into one or more tasks according to the training and reasoning stages included in the decision model, and assigning the tasks to different cluster nodes of the first type; Divide the task into one or more subtasks according to the network structure used when processing the task, and assign different subtasks to different cluster nodes of the second type; The subtask is treated as one computing task or divided into multiple computing tasks according to the complexity of processing the subtask, and different computing tasks are allocated to different third-type cluster nodes.

[0056] Specifically, according to the complexity of processing the subtasks, the subtasks are treated as one computing task or divided into multiple computing tasks, and different computing tasks are assigned to different third-type cluster nodes, including: When it is detected that the computation amount of processing the subtask is less than the preset computation amount, the subtask is regarded as a computation task, and the computation task is allocated to the cluster node of the third type; When it is detected that the amount of computation when processing a subtask is greater than or equal to a preset amount of computation, the subtask is divided into multiple computing tasks; the network structure of the network model used when processing the computing task is classified according to the complexity of the network structure of the network model used when processing the computing task; and the computing tasks corresponding to different categories of network structures are allocated to different third-type cluster nodes.

[0057] There is no limit on the preset computing amount, which is determined according to the actual situation. Allocating tasks to different first-type cluster nodes includes: Allocating tasks to different cluster nodes of the first type through a third signal transmission method; Allocating different subtasks to different second-type cluster nodes includes: Using the first type of cluster nodes and allocating different subtasks to different second type of cluster nodes through a third signal transmission method; Allocating different computing tasks to different third-type cluster nodes includes: The second type of cluster nodes are utilized and different computing tasks are allocated to different third type of cluster nodes through a third signal transmission method.

[0058] The third signal transmission mode is such as Ethernet transmission mode. Taking the reinforcement learning model as an example, the training and reasoning stage of the reinforcement learning model includes two processes: the environment interaction stage and the model update. According to the environment interaction stage, the training and reasoning process of the decision model is divided into one or more action (Actor) tasks; according to the model update, the training and reasoning process of the decision model is divided into one or more trainer (Learner) tasks. The Actor task is assigned to the first type of cluster node, which can be called an Actor cluster node, and the Learner task is also assigned to the first type of cluster node, which can be called a Learner cluster node. It is worth noting that when there are multiple tasks, different tasks can be assigned to the first type of cluster nodes (including Actor cluster nodes or Learner cluster nodes) located on different hosts. A decision model is set on the first type of cluster node. Establishing a decision model located on the first type of cluster node includes: obtaining the complexity of the decision task; establishing decision models of different structures on the first type of cluster node according to the complexity of the decision task; wherein the complexity of the decision task is positively correlated with the complexity of the decision model structure.

[0059] After the tasks are assigned to the first type of cluster nodes, the complexity of some tasks is still relatively high. Therefore, further, the tasks are divided into one or more subtasks according to the network structure used when processing the tasks, and different subtasks are assigned to different second type cluster nodes. For example, if the network structure included in the Actor task is: state coding layer, context coding layer, decision coding layer and action coding layer, the Actor task can be divided into state coding subtasks, context coding subtasks, decision coding subtasks and action coding subtasks. Each subtask is assigned to the second type of cluster node (also called task cluster node). The subtasks corresponding to the same task can be set on the same host. After the subtasks are assigned to the second type of cluster nodes, there may still be subtasks with high task complexity. Therefore, further, according to the complexity of processing the subtasks, the subtasks are treated as one computing task or divided into multiple computing tasks, and different computing tasks are assigned to different third type cluster nodes. If the load of the state coding subtask and the context coding subtask is large, the state coding subtask and the context coding subtask can be further divided into multiple computing tasks, and then the divided computing subtasks are assigned to different third type cluster nodes (also called module cluster nodes). For example, the load of the action coding task is relatively small, so the action coding task can be directly treated as a computing task without being divided, and then the computing task is assigned to the third type of cluster nodes. The method of assigning tasks, subtasks, and computing tasks to cluster nodes described above can be called the process of model parallelization.

[0060] After assigning the computing nodes to the cluster nodes, in order to process the computing tasks and improve the efficiency of training and reasoning, the data is further processed in parallel. First, the input data corresponding to the computing nodes needs to be obtained. In implementation, obtaining the input data corresponding to the computing tasks on the cluster nodes includes: If the computing task is an initial computing task among all computing tasks, determining that the input data corresponding to the initial computing task is data sent by the processor of the host computer and received by the cluster node of the first type; If the computing task is not an initial computing task, the input data corresponding to the computing task is determined to be the output characteristics obtained after the input data corresponding to the previous computing task of the computing task is passed through the network model on the computing node.

[0061] When the training reasoning of the decision model includes an interaction phase and a model update phase; before sending data to the first type of cluster nodes, the data sent to the first type of cluster nodes can be obtained from the training reasoning process of the graphics processing unit.

[0062] After obtaining the input data corresponding to the computing task, the input data corresponding to the computing task on the cluster node is split to obtain multiple data subsets including: When it is detected that the input data corresponding to the computing task on the cluster node is image data, if the data batch size is greater than or equal to the number of input channels, the input data corresponding to the computing task on the cluster node is split along the data batch dimension to obtain multiple data subsets; if the data batch size is less than the number of input channels, the input data corresponding to the computing task on the cluster node is split along the input channel dimension to obtain multiple data subsets; When it is detected that the input data corresponding to the computing task on the cluster node is not image data, the dimension with the largest number of dimensions is obtained from the data batch dimension and multiple feature dimensions; the input data corresponding to the computing task on the cluster node is split along the dimension with the largest number of dimensions to obtain multiple data subsets.

[0063] In practice, there may be a scenario where the feature dimension cannot divide the number of computing nodes. In this scenario, in order to reasonably allocate data to the computing nodes, the input data corresponding to the computing tasks on the cluster nodes are split into multiple data subsets and allocated to different computing nodes, including: Obtaining the number of computing nodes to which the preset input data is to be distributed; Obtain the result of performing a modulo operation on the dimension with the largest number of dimensions and the number of computing nodes to be allocated to determine a first number of computing nodes; Determining a second number of computing nodes according to a difference between the number of computing nodes to be allocated and the first number; Splitting the input data corresponding to the computing task on the cluster node according to the dimension with the largest number of dimensions determines a data subset of a first data size and a data subset of a second data size; wherein the first data size is determined by multiplying the result obtained by rounding down the dimension with the largest number of dimensions and the number of computing nodes to be allocated and then adding 1, and the data size on the remaining dimensions; the second data size is determined by multiplying the result obtained by rounding down and the data size on the remaining dimensions; the remaining dimensions are dimensions other than the dimension with the largest number of dimensions among all dimensions; Data subsets of a first data size are respectively distributed to a first number of computing nodes, and data subsets of a second data size are respectively distributed to a second number of computing nodes.

[0064] In order to improve the efficiency of input data processing, the input data corresponding to the computing tasks on the cluster nodes are split into multiple data subsets, which are distributed to different computing nodes, including: Get the network layer characteristics of the computing task; Determine the target computing node based on the network layer characteristics; The multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes are distributed to different target computing nodes.

[0065] After the input data is divided into multiple data subsets, in order to realize the processing of the data subsets, the computing tasks on the cluster nodes are further transferred to the computing nodes to which the input data is assigned. Finally, the training and inference results of the decision model are determined according to the output characteristics of the network model on the computing nodes to which the input data is assigned.

[0066] In implementation, determining the training inference result of the decision model according to the output characteristics of the network model on the computing node to which the input data is distributed includes: Obtain the current output features of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node; wherein the current subtask starts from the initial subtask of the training and reasoning process of the decision model; Obtain the next subtask of the current subtask according to the first order of the subtasks in the training and reasoning process of the decision model, and use the next subtask of the current subtask as the new current subtask; Transmitting the current output feature to the second type of cluster node where the current subtask is located via the third type of cluster node where the computing task corresponding to the current subtask is located; Obtain the second type of cluster node where the new current subtask is located, transmit the current output feature to the second type of cluster node where the new current subtask is located through the second type of cluster node where the current subtask is located, so as to obtain the input data corresponding to the new current subtask, and return the current output feature after the input data of the computing task corresponding to the current subtask passes through the network model on the computing node, until the current subtask is the last subtask in the training and reasoning process of the decision model, stop returning, and determine the training and reasoning result of the decision model according to the current output feature.

[0067] Transmitting the current output feature via the second type cluster node where the current subtask is located to the second type cluster node where the new current subtask is located comprises: Controlling the communication between the second type of cluster node where the current subtask is located and the second type of cluster node where the new current subtask is located through the first signal transmission mode and obtaining communication information; wherein the communication information at least includes status monitoring information and status feedback information; When it is detected that the communication information is normal, the current output feature is transmitted via the second type of cluster node where the current subtask is located to the second type of cluster node where the new current subtask is located through the second signal transmission mode.

[0068] The first signal transmission mode is, for example, a low-voltage differential signal (LVDS) interface transmission mode, and the second signal transmission mode is, for example, a peripheral component interconnect express (PCIe) transmission mode.

[0069] The current output features of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node include: When it is detected that the current computing task is not obtained by dividing the current subtask, the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node is obtained, and the sum of the current output features is used as the current output feature of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node; When it is detected that the current subtask is divided into multiple computing tasks, the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node is obtained; the next computing task of the current subcomputing task is obtained according to the second priority order of the computing tasks in the current subtask, and the next computing task of the current computing task is used as the new current computing task; the sum of the current output features is transmitted to the third type of cluster node where the new current computing task is located via the third type of cluster node where the current computing task is located, so as to obtain the input data corresponding to the new current computing task, and return to the step of obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node, until the current computing task is the last computing task in the current subtask, stop returning, and output the sum of the current output features, and use the sum of the current output features as the current output feature of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node.

[0070] Specifically, obtaining the sum of current output features of the data subset corresponding to the current computing task on the corresponding computing node includes: Obtaining a computing node of the first type and a computing node of the second type used when processing a data subset corresponding to a current computing task; wherein a network model is set in the computing node of the first type, and a network model is not set in the computing node of the second type; and there is one computing node of the second type; Obtain output features of each data subset corresponding to the current computing task on the corresponding computing node of the first type; Sending output features on all computing nodes of the first type to computing nodes of the second type; All output features are aggregated through the second type of computing nodes to obtain the sum of current output features of the data subset corresponding to the current computing task on the corresponding computing node.

[0071] The above method of obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node can be called a model update method based on multi-subnode-central node data parallelism. Before using this method, the load situation on the computing node can be judged. Specifically, before obtaining the output features of each data subset corresponding to the current computing task on the corresponding computing node of the first type, it also includes: If it is detected that the load on the first type of computing nodes and the second type of computing nodes used to process the data subset corresponding to the current computing task is less than the preset load, the step of obtaining the output characteristics of each data subset corresponding to the current computing task on the corresponding first type of computing node is entered.

[0072] In this method, the load conditions on the first type of computing nodes and the second type of computing nodes used to process the data subset corresponding to the current computing task are judged. Only when the load condition is less than the preset load amount, the model updating method based on multi-child node-central node data parallelism is used to avoid excessive load pressure on the computing nodes.

[0073] In addition, obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node can also be achieved in the following way: Obtaining the communication order between all computing nodes processing the current computing task; wherein the last computing node among all computing nodes arranged in the communication order is connected to the first computing node in communication; Obtain data segments of each data subset corresponding to the current computing task after being passed through the network model on the corresponding computing node and store them in a sending buffer of the computing node for sending data; Determine whether the current computing node processing the current computing task has data fragments of all other nodes; wherein the other nodes are computing nodes other than the current computing node among all computing nodes processing the current computing task; If not, then send the data fragment on the current computing node from the sending buffer to the receiving buffer for receiving data of the next computing node of the current computing node in the communication order; use the next computing node of the current computing node as the new current computing node; aggregate the data of the receiving buffer of the new current computing node with the data of the sending buffer of the new current computing node and use them as the data fragment of the sending buffer of the new current computing node, and return to the step of determining whether there are data fragments of all other nodes in the current computing node processing the current computing task; If so, the data fragments on the current computing node will be sent from the sending buffer to the receiving buffer for receiving data of the next computing node of the current computing node in accordance with the communication order; the next computing node of the current computing node will be used as the new current computing node; the data in the receiving buffer of the new current computing node will be overwritten with the data in the sending buffer of the new current computing node and used as the data fragments of the sending buffer of the new current computing node, and the process will return to the step of determining whether there are data fragments of all other nodes in the current computing node processing the current computing task, and the process will stop returning after it is detected that there are data fragments of all other nodes in all computing nodes processing the current computing task.

[0074] Transmitting the sum of the current output features via the cluster node of the third type where the current computing task is located to the cluster node of the third type where the new current computing task is located comprises: Controlling the communication between the third type of cluster node where the current computing task is located and the third type of cluster node where the new current computing task is located through the first signal transmission mode and obtaining communication information; wherein the communication information at least includes status monitoring information and status feedback information; When it is detected that the communication information is normal, the sum of the current output features is transmitted through the third type of cluster node where the current computing task is located to the third type of cluster node where the new current computing task is located through the second signal transmission mode.

[0075] The method of summing up the current output features of the data subset corresponding to the current computing task on the corresponding computing node can be called a ring communication method for decentralized node data parallelization.

[0076] Before obtaining the communication order between all computing nodes processing the current computing task, it also includes: Obtaining load conditions on the first type of computing nodes and the second type of computing nodes used when processing the data subset corresponding to the current computing task; If it is detected that the load on the first type of computing nodes and the second type of computing nodes used to process the data subset corresponding to the current computing task is greater than or equal to the preset load, the step of obtaining the communication order between all computing nodes processing the current computing task is entered.

[0077] There is no limit on the preset load amount, which is determined according to the actual situation. Before using the ring communication method with decentralized node data parallelization, the load conditions on the first type of computing nodes and the second type of computing nodes are judged first. If there are computing nodes with a load greater than the preset load amount, the ring communication method with decentralized node data parallelization is used to ensure load balancing on the computing nodes.

[0078] When the decision model is a reinforcement learning model, the training and reasoning results of the decision model are determined based on the current output features, including: In the interactive phase, receiving a first current output feature outputted sequentially by a cluster node of the third type, a cluster node of the second type, and a cluster node of the first type; Storing the first current output feature in a data storage module located in the host computer; In the case where it is detected that the amount of data in the data storage module is greater than the preset amount of data, receiving, in the model updating phase, a second current output feature outputted sequentially by the cluster nodes of the third type, the cluster nodes of the second type, and the cluster nodes of the first type; Calculating a loss function based on the second current output feature to determine parameters of the decision model, so as to obtain a training inference result of the decision model; After obtaining the training and inference results of the decision model, it also includes: The parameters of the decision model are transmitted to the cluster nodes of the first type to update the parameters of the decision model located in the cluster nodes of the first type.

[0079] In order to enable those skilled in the art to better understand the training and reasoning method of the decision model described above, the above method is described in detail below by taking the decision model as a reinforcement learning model as an example.

[0080] Figure 3 A schematic diagram of a decision model training and reasoning acceleration system based on an FPGA cluster provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, it mainly includes the following parts: 1. Design the decision model network structure. According to the task and space complexity, divide the decision model into state coding layer, decision coding layer, and action coding layer. Select the network structure according to the task requirements and whether it is necessary to adopt an adaptive selection strategy for model structure design to optimize performance.

[0081] 2. Design a distributed parallel training framework for decision models, use data parallelization methods to divide the input data into multiple subsets, and form a parallel computing structure consisting of one (or more) central computing nodes and multiple sub-computing nodes. At the same time, use model parallelization methods to set up multiple data collection and model update processes, and split the decision model calculation tasks step by step to different nodes to improve equipment utilization and throughput.

[0082] 3. Design the hardware structure of a single FPGA computing node, including different node architectures for convolutional computing, attention computing, and fully connected computing to meet the data computing requirements of different network layers; configure a variety of FPGA unit node interconnection communication methods to meet the computing requirements of heterogeneous platforms.

[0083] 4. Design an FPGA cluster with a multi-level cluster structure, and use multi-level FPGA cluster nodes to build a computing framework. Each level of cluster nodes includes a master node and lower-level sub-cluster nodes. Different communication methods are used between nodes according to the data type. The master node of the lowest-level cluster node is connected to the FPGA unit node to ensure load balance of each node.

[0084] 5. Design a heterogeneous computing framework based on FPGA cluster to achieve stable interconnection between GPU environment process modules, CPU main controller, data storage module and FPGA cluster, and support efficient training and inference of decision models.

[0085] The following is a detailed description of each part of the decision model training and inference acceleration system based on FPGA cluster.

[0086] 1. First, the design of the heterogeneous computing framework based on FPGA cluster is explained.

[0087] The decision model based on reinforcement learning adopts an interactive-update iterative training mechanism and sets up two modules to implement computing tasks at different stages. The Actor module is responsible for the interaction between the decision model and the environment, which only involves forward reasoning calculations. The Learner module is responsible for updating the decision model parameters, including forward reasoning and reverse gradient updates. Due to the differences in the operation of the decision model between the two, each will maintain a local decision model.

[0088] Figure 4 A schematic diagram of a heterogeneous computing framework based on an FPGA cluster provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the decision model training reasoning computing framework based on FPGA cluster includes an environment process module created based on GPU, a CPU main controller for controlling FPGA cluster and environment process, a data storage module for storing interactive experience samples and model parameters, and an FPGA cluster for implementing Actor and Learner tasks. Among them, in order to ensure the load balancing of each FPGA computing node, the present invention adopts a multi-level cluster structure to design an FPGA cluster interconnection solution, which can effectively avoid computing delays and enhance data consistency. The CPU main controller creates an environment process through the environment simulator application programming interface (Application Programming Interface, API), selects to call the GPU according to the simulator operation requirements, and realizes the return of action, status, reward and other data through API communication. Here, in order to improve the interaction efficiency, the environment process pipeline parallel design is adopted, that is, multiple simulation environments are created in each environment process, and action execution and action reasoning are performed alternately in two groups.

[0089] During the interaction, the CPU main controller sends the interaction experience, including action, status, reward and other data, to the data storage module through the data bus for updating the model parameters of the Learner module. In terms of action reasoning, each environment process will correspond to an Actor (action) cluster node in the Actor module to achieve task parallelization. The CPU main controller sends program instructions to the FPGA cluster through PCIe to realize computing scheduling and start-stop control, and after receiving the feedback information of each environment process, it will be transmitted to the Actor cluster master node through PCIe. The Actor master node is further sent to cluster nodes at all levels through Ethernet. The master node of each level of task cluster or module cluster constitutes a tightly coupled FPGA embedded system. The DMA controller is used to map the local memory of each master node to a shared memory space through the PCIe interface, so that any master node can directly access the memory area of ​​other master nodes to achieve high-speed data exchange. After completing the model reasoning calculation, the Actor cluster master node will output the result back to the CPU main controller through PCIe, and send it to each environment process through the API to complete the action execution.

[0090] When the amount of interaction experience collected in the data storage module reaches a certain amount, the CPU main controller controls the Learner cluster node through PCIe to start the model update calculation, and sends the experience samples to each task cluster and module cluster master node through Ethernet step by step until they are distributed to each unit computing node. Each level of task cluster and module cluster master node also constitutes a tightly coupled FPGA embedded system, and high-speed data interconnection is achieved through PCIe and DMA. After completing the model update, the Learner cluster returns the updated model parameters of each node to the data storage module for backup. At the same time, the model parameters are copied to each Actor cluster node to update the local model.

[0091] 2. Explain the network structure part of the design decision model.

[0092] 1. The setting scheme of the decision model is as follows: According to the task requirements, the complexity of the state and action space, the decision model structure can be designed in many ways. According to the functional modules, the decision model can be divided into three parts: state encoding layer, decision encoding layer and action encoding layer: State encoding layer: maps the input original state space to the potential feature space; Decision encoding layer: approximates the mapping distribution between input states and output actions; Action encoding layer: maps the latent feature space after decision encoding to the action space.

[0093] In order to cope with different types of decision-making tasks, the complexity of the model structure also varies.

[0094] (1) Simple decision-making task: For decision-making tasks with low-dimensional state space, the state encoding layer, decision encoding layer, and action encoding layer can be constructed using a simple fully connected network to form an end-to-end decision model. The core calculation of this model mainly involves matrix multiplication and activation function calculation.

[0095] (2) Complex decision-making tasks: For tasks with high-dimensional state spaces (such as images or multimodal perceptual data), the structure of the decision model will be more complex. Figure 5 A schematic diagram of a decision model structure for complex decision-making tasks provided by an embodiment of the present invention, such as Figure 5 As shown: State encoding layer: For tasks with low-pixel intermediate feature images as input, the state encoding layer is usually constructed using a convolutional neural network (CNN). In order to reduce the amount of computation, the decision encoding layer and the action encoding layer still use a fully connected network. The state encoding layer can be pre-trained or trained end-to-end together with the decision encoding layer and the action encoding layer.

[0096] High-pixel image input: For high-pixel raw camera image input, the number and structure of convolutional layers in the state encoding layer will be more complex, and pre-trained deep networks (such as Residual Network (ResNet), Visual Geometry Group (VGG), and object detection algorithms) are used to implement it.

[0097] Multimodal perceptual data: For tasks involving multi-view images or multimodal perceptual data, a multi-branch perceptual coding network is usually used to preprocess different data, and then a network structure with a cross-attention mechanism (such as Transformer) is used for feature fusion and alignment. In such tasks, in addition to matrix multiplication, convolution, and activation function calculations, attention calculation operations are also required.

[0098] (3) Timing dependence and experience replay: In order to make full use of the perceptual state information and improve the decision model's ability to understand the system dynamics and environmental context, a side branch network can be added to the state encoding layer to process the serialized information of the historical state and capture the temporal dependencies. This part can be implemented through the Transformer network structure, involving loop calculations or attention calculation operations. In addition, in order to improve the utilization efficiency of training data and the training stability of the model, a generative experience replay module can be added. This module is built on a generative model (such as a variational auto-encoder (VAE), a generative adversarial network (GAN) or a diffusion model, etc.), which can enhance the model's learning of online interactive experience. The update of the generative experience replay module can be used as a bypass branch, alternating with the training process of the decision model to jointly promote the optimization of the model.

[0099] 2. Model structure adaptive selection strategy.

[0100] The following is a detailed description of the three optional adaptive design methods mentioned above: (1) Adaptive design method based on reinforcement learning: In the decision model, the call of each encoding module can be modeled as an action in reinforcement learning, and the module selection strategy can be optimized through trial and feedback. This method is applicable to: Dynamic tasks: The input data distribution or characteristics change dynamically, and the module calling strategy needs to be adjusted according to the real-time environment.

[0101] No clear rules: When module selection cannot be determined by fixed rules, the best choice needs to be found through exploration and optimization.

[0102] Online learning: The system needs to continuously improve its selection strategy during operation.

[0103] The specific implementation steps include: a) Building a reinforcement learning framework: In adaptive design based on reinforcement learning, the optimization goal is to dynamically adjust the calling strategy of each coding module in the decision model so that the system performance (such as accuracy, execution speed and resource utilization) is optimized. To achieve this goal, it is necessary to first build a reinforcement learning framework and define states, actions, rewards and environments.

[0104] state : Represents the input data characteristics of the current system and the operating status of the hardware platform. The data characteristics include the distribution and complexity (such as dimension and sparsity) of the input data, and the system status includes hardware resource utilization, latency, and power consumption.

[0105] action : Select the calling strategy of the encoding module. For example, in the state encoding layer, choose to use a convolutional neural network (CNN) or a Transformer; in the decision encoding layer, choose a model based on the attention mechanism or a lightweight network.

[0106] award : Evaluate the performance of module selection strategies, such as the weighted sum of accuracy, execution speed, and resource usage.

[0107] Environment: The environment is determined by the characteristics of the task and the hardware platform, including the dynamic changes of input data and the resource allocation of the FPGA hardware.

[0108] b) Design reward function: The reward function is the core of optimization. Its role is to guide the agent to learn the optimal module calling strategy through feedback. The set reward function needs to balance the weights of performance and resource consumption. Suppose the performance after model selection is , the execution delay is , the resource utilization is , the reward function can be expressed as: ; in, , , is a weight parameter, which is adjusted according to the application scenario. When the real-time requirement is high, increase When hardware resources are limited, increase weight to reduce resource usage.

[0109] c) Optimization objectives: The goal of reinforcement learning is to find an optimal strategy that maximizes the cumulative reward. Its optimization objective function is the optimal state-action value function : ; in, is the discount factor, controlling for the importance of future rewards; It means in the state Take Action The expected cumulative reward that can be obtained after the game, E represents the expected value.

[0110] d) Optimization process: Using Neural Network Approximation , the network structure is a fully connected network with 2 hidden layers, and the input layer accepts the state and actions The number of nodes in each hidden layer is adjusted according to the complexity of the task, and the output layer outputs The following mean square error is used as the loss function : ; The target network The expression is: ; For the current Network parameters, For the goal The network parameters are updated regularly to stabilize the training. The training steps are as follows: 1) Initialization Network and Target network; 2) Get a random initialization state from the environment ; 3) According to - Greedy strategy to choose actions : With probability Choose actions randomly, or with probability Select the largest The action corresponding to the value; 4) Execute the action , observe the reward and the new state ; 5) Update Network parameters: ,in is the learning rate, is the gradient; 6) Repeat the above process, and Network parameters are synchronized to the target through operations such as copying and accumulation Network until Network convergence.

[0111] During the execution of the decision model, the reinforcement learning agent Dynamically select the optimal network design for different encoding modules , according to real-time feedback, adjust the reward function weight to adapt to task and hardware changes. After the agent completes training, it uses the trained The network can quickly infer the optimal module calling strategy.

[0112] (2) Adaptive Neural Architecture Search: Based on the adaptive neural architecture, the optimal neural network structure is searched. By incorporating the calling strategy of the encoding module into the search space, this method can automatically optimize the module design according to the task characteristics and hardware constraints. This method is applicable to: Customized module structure is required: Specific modules such as state encoding and context encoding require different architectures such as convolutional layers and Transformer.

[0113] Multi-objective optimization: Performance and hardware limitations (such as latency and power consumption) need to be balanced.

[0114] The specific implementation steps include: a) Define the search space: Design candidate modules: Design the state encoding module, context encoding module, etc. in the decision model as searchable candidate modules: ; Parameterized Search Space : Including network depth (number of layers, such as 3, 6 or 12 layers), network width (number of neurons or channels in each layer), layer type (convolutional layer, fully connected layer, attention layer, etc.), activation function (such as Rectified Linear Unit (ReLU)), etc.), for example, a candidate module design can be expressed as: = (number of layers = 6, number of channels = 128, activation function = ReLU).

[0115] b) Design a multi-objective optimization problem: In practical applications, the optimization of neural networks usually requires a trade-off between multiple objectives. For example, seeking a balance between performance (such as accuracy) and hardware constraints (such as latency and power consumption). The optimization problem can be formalized as: ; in, Design for candidates performance goals (such as classification accuracy, target detection accuracy), Design for candidates Hardware constraints (such as execution delay, memory usage, power consumption). In this solution, the performance goal is Can be determined by the decision model evaluation index, hardware constraints It is expressed through hardware performance indicators, including latency (the time required for the model to execute once), power consumption (the energy consumed by the model running), and resource usage (FPGA logic unit or memory usage).

[0116] c) Search process: The search process uses algorithms to explore the search space , find the optimal solution between performance and hardware constraints. First, a set of initial designs are randomly generated in the search space. , as the population. Then calculate the fitness of each architecture , defined here as a comprehensive function of performance and hardware constraints : ; in, is a penalty factor used to limit designs that exceed hardware constraints. In each round of iteration, selection, crossover, and mutation operations are performed to gradually generate a better architecture until the performance or hardware requirements are met. Among them, the selection operation is used to select excellent individuals in the population according to fitness, the crossover operation is used to combine the characteristics of two designs to generate a new design, and the mutation operation is used to randomly change certain parameters of the architecture (such as the number of layers and width). Then, the generated new architecture replaces the individuals with poor performance to form the next generation of population. Repeat the above process until the performance reaches the expected goal under the condition that the hardware constraints meet the requirements.

[0117] (3) Data-driven dynamic model selection: Data-driven dynamic model selection is a flexible adaptive design method that analyzes the characteristics of input data in real time and dynamically selects the optimal encoding module to improve the adaptability and performance of the model. This method is suitable for: The data characteristics are variable: for example, image resolution and modality characteristics vary greatly; Real-time applications: need to quickly decide the calling strategy.

[0118] The specific implementation steps include: a) Extract data features: In dynamic model selection, the characteristics of input data are the core basis for selecting modules. By extracting the statistical characteristics of input data, we can provide a basis for subsequent module selection. The following two operations can be taken: Feature engineering: Extract key features using simple statistical methods or feature engineering. For example, for image data, the main data features include resolution (width and height of the input image), color distribution (average pixel value, standard deviation), and sparsity (proportion of non-zero pixels); for multimodal data, the main data features include data type (audio, image, or text, etc.), time series characteristics (periodicity or trend of time series data), and real-time applications (need to quickly decide on the calling strategy).

[0119] Use lightweight networks to extract features: In order to efficiently process complex data, lightweight neural networks can be used to extract high-dimensional features, defining the input as raw data (such as images, audio), the output is defined as a low-dimensional feature vector representation , contains the key characteristics of the data. The feature vector extraction operation can be expressed as: ; b) Set up a dynamic control network: The dynamic control network is responsible for the characteristic vector of the input data , predict each candidate module The probability of selection ; The control network is a lightweight classifier whose output is the selection probability of each module: ; Among them, the weight matrix Used to control the learnable parameters of the network and output probability distribution Represents the selection probability of each candidate module. For example, suppose there are three candidate modules: Convolutional Network (CNN), Transformer, Fully Convolutional Network (FCN), and the input feature vector After the control network calculation, we get: ; .

[0120] c) Adaptive selection: According to the probability of controlling the network output , dynamically select the optimal module design and generate the final output. First, determine the selection strategy. There are two ways: Hard selection: directly select the module with the highest probability design: ; This method has low computational overhead and is suitable for scenarios with high real-time requirements.

[0121] Soft selection: According to The weighted fusion of all module outputs is obtained: ; in It is a module The output of the design, For the final output, this method can retain the output information of multiple module designs and improve the robustness of the model.

[0122] Furthermore, the output selections designed by each encoding module are weighted and summed: ; Final Output Can be used for subsequent tasks.

[0123] d) Implementation optimization and deployment: The goal of the dynamic control network is to predict the selection probability of the optimal module design , take the following cross entropy loss to measure the predicted probability and real choice The difference between: ; in, represents the cross entropy loss, is the distribution of the real choice. During training, the parameters of the control network are updated using stochastic gradient descent or Adam optimizer. Using the trained dynamic control network, the model design can adapt to different input data characteristics in real time, thereby improving system performance and flexibility.

[0124] (III) Decision model hardware adaptation: When adapting the decision model to hardware, the final computing deployment of the decision model must take into account the resource allocation and task characteristics in the hardware acceleration environment, mainly considering the following aspects: Hardware resource utilization: FPGA hardware resources are limited. By accelerating computationally intensive tasks (such as convolution, matrix multiplication, activation functions, etc.) using large-scale parallel processing element (PE) arrays, performance can be significantly improved. When designing FPGA node computing logic, consider how to reasonably allocate computing tasks to FPGA resources, and improve overall performance by optimizing the computation graph, reducing redundant computing, and reducing memory bandwidth consumption.

[0125] Modular design: In order to better adapt to FPGA hardware, by decomposing different layers in the decision-making process (such as state coding layer, decision coding layer, and action coding layer) into independent modules, computing nodes with differentiated logic designs can be assigned to each module according to the hardware resources of the FPGA. For example, the state coding layer (mainly involving convolution operations) is more suitable for deployment on FPGA nodes that focus on convolution calculations, while the decision coding layer and action coding layer (more involving fully connected layers and activation functions) can be deployed on FPGA nodes that focus on matrix multiplication calculations.

[0126] Computational parallelization: Through appropriate data partitioning and task scheduling, multiple computing tasks can be executed simultaneously, giving full play to the advantages of multi-device parallel computing. For example, in convolutional neural networks (CNNs), convolution operations are computationally intensive and suitable for parallelization. They can be directly mapped to multiple FPGA nodes to efficiently collaborate and complete the calculation.

[0127] 3. Describe the distributed parallel training framework.

[0128] Since the decision model is trained based on trial and error exploration based on environmental feedback signals, the model needs to interact with the environment to collect online experience as training data before each parameter update. After collecting the data, the model updates the parameters through a forward calculation + back propagation, and the above process is iteratively advanced. The existing model training platform based on this interactive exploration training method mainly adopts a single environment configuration of the CPU-GPU framework, in which the CPU is responsible for environment establishment and data control, and the GPU is used for high-throughput parallel computing. Due to the single environment configuration, only one state can be predicted for action each time, and the overhead of scheduling the GPU for calculation is sometimes longer than the time for parallel computing, resulting in low computing efficiency. In response to this problem, the present invention designs a distributed parallel computing architecture for decision models, decouples tasks such as environmental interaction, model reasoning, and experience storage, and sets multiple computing nodes for parallel execution for different tasks, thereby making full use of computing resources and improving computing efficiency.

[0129] The main idea of ​​parallel computing of decision models is to decompose tasks, data and computing processes and execute them simultaneously on multiple processing units to improve computing efficiency. Two design schemes are mainly adopted here: data parallelism and model parallelism.

[0130] (1) Data parallelization.

[0131] The data parallel solution mainly divides the input data into multiple subsets and processes them in parallel on multiple processing units to improve the processing efficiency of large-scale computing data. The specific structure consists of one (or more) central computing nodes and multiple sub-computing nodes, where each computing node is built through 1 FPGA board card and completes its own computing tasks. Figure 6 A schematic diagram of a data parallelization structure of multiple sub-nodes and a central node provided by an embodiment of the present invention, such as Figure 6 As shown in the figure, the data is divided into multiple subsets and distributed to multiple sub-nodes for parallel processing, where each sub-computing node maintains a local model. By dividing the data into different processing units, computing the local results of each small batch of data in parallel, and then summarizing and updating them, the computing efficiency can be greatly improved.

[0132] In the process of decision model training, both the model forward reasoning and parameter update stages can adopt data parallel design to improve processing efficiency. Assume that the input feature is the image data, size is ,in is the data batch size, is the number of input channels, is the image size. , it is recommended to divide the data subsets along If , it can be divided according to the input channel dimension. For non-image data input, its shape can be expressed as a general form ,in Corresponding to The size of the characteristic dimension of the dimension. Generally speaking, will not exceed 3. For this input case, if the feature dimension The dimensions are independent of each other, that is, there is no correlation when calculating the data. The characteristic data of can be regarded as a characteristic individual in calculation, and the size of each individual is , the data in each dimension are interrelated, It is the dimension that can be parallelized. When dividing the data subsets, choose according to the principle of maximizing parallelism. The middle dimension with the largest number of corresponding dimensions is divided to improve resource utilization.

[0133] Assume that the data is The size of the data subset allocated to each sub-computing node is determined by the number of sub-computing nodes set by the system. Decision, among which According to the data parallelism of the system, the data size that can be allocated to each node should be Because in many cases Cannot be Therefore, the amount of data allocated to the sub-computing nodes may be inconsistent. The data size on each sub-computation node is , and the remaining The data size on each sub-computation node is , represents the remainder operation, Indicates a floor operation.

[0134] In the process of decision model training, the two stages of environmental interaction and model update are iteratively advanced. The environmental interaction stage mainly involves the forward reasoning calculation of the model. Figure 6 According to the system data parallelism, the input data is divided into multiple subsets and sent to each sub-computing node in parallel. Each computing node has the same model parameters and independently completes the model layer-by-layer reasoning calculation to obtain the corresponding output features. Then each node sends its own output to the central computing node, and the overall output features are obtained through a data aggregation operation.

[0135] In addition to the model forward reasoning calculation, the model update phase also needs to calculate the parameter gradient based on the feature results and complete the parameter update through back propagation. Figure 7 A schematic diagram of a model update process based on multi-child node-central node data parallelization provided by an embodiment of the present invention, such as Figure 7 As shown in the figure, the data parallel processing method is similar to that in the inference stage. Here, a reordering (Resort) operation is added to the data subset partitioning link. This operation will disrupt and reorder the order of each feature individual along the data partitioning dimension to achieve the function of random sampling. Since the model update stage will only be triggered when enough experience samples are collected, the number of samples in the data set is often much larger than the number of other feature dimensions. Therefore, the data partitioning here is equivalent to batch data sampling from the data set. Each sub-computing node will also maintain a local model. Each node independently completes forward propagation and back propagation to obtain the corresponding gradient. At this time, the gradients on each node are different. Then each node sends its own gradient to the central computing node, obtains the merged gradient through a data aggregation operation, and returns the merged gradient to each node. At this time, the gradients on each node are exactly the same. Finally, each node completes the local model parameter update independently.

[0136] The above scheme integrates the information of sub-computing nodes by setting data aggregation operations and central computing nodes. However, each node is prone to load imbalance during communication. When a node is found to be overloaded, the present invention proposes a data parallelization scheme of decentralized nodes to achieve it. Figure 8 A schematic diagram of a ring communication topology structure with decentralized node data parallelization provided by an embodiment of the present invention. Figure 8 As shown in the figure, this scheme is based on a ring communication topology. There is no central computing node, and all sub-computing nodes form a logical ring. First, the data to be calculated is evenly divided into multiple fragments, so that each node only needs to process and transmit a part of the data during the communication process, thereby reducing the communication volume and computational complexity. The same division strategy as the previous scheme is adopted here. Similarly, each node has a part of the complete data at the beginning, and independently completes its own local model reasoning calculation. The difference is that this scheme does not require the central computing node to assist in data aggregation operations, but completes the data integration of each sub-node through a ring communication method. During the ring communication process, each node sends its own data to the next node and receives data from the previous node. As the communication proceeds, each node gradually integrates data fragments from other nodes, and finally all nodes have complete synchronized data. The ring communication process specifically includes the following two stages: Decentralized processing stage: a. Initialization: for each node The data fragment obtained after the local calculation is completed , set the initial send buffer , and Deposit ; b. Ring communication: nodes Send buffer The content in is sent to the node ,if , then send it to node 0; at the same time, from node Receive data and store it in the receive buffer ,like , then from the node Receive data; c. Data update: Node The receive buffer Content and local data fragment Perform specific operations (such as concatenation, accumulation, etc.) to obtain new data fragments And update the data in the send buffer to the new ; d. Repeat the above steps times, so that each node gradually accumulates the partial sum of the data fragments corresponding to all nodes.

[0137] Centralized synchronization phase: a. Initialization: After the distributed processing phase, each node Have partially aggregated data fragments , set the initial send buffer ,Will Deposit ; b. Ring communication: Similar to the distributed processing stage, nodes Send buffer The content is sent to the node , while from the node Receive data into the receive buffer ; c. Data update: Node The receive buffer Content and local data fragment Perform specific operations (such as combination, overwriting, etc.) to obtain new data fragments , and update the send buffer to the new ; d. Repeat the above steps times, and finally each node has complete synchronized data.

[0138] In the decentralized node data parallelization scheme, each node fully utilizes the uplink and downlink bandwidth, has high bandwidth utilization, and has good scalability. However, as the number of nodes increases, the communication delay will gradually increase, and this ring topology requires the bandwidth and delay between nodes to be as consistent as possible. Therefore, when adopting this scheme, it is necessary to control the number of nodes and ensure that the performance of each node is similar. In addition, this scheme has the risk of error accumulation during multi-node data transmission and aggregation, and each node must have sufficient memory space to cache intermediate results. Therefore, for modules that are greatly affected by cumulative errors and occupy a large amount of memory, it is recommended to adopt a centralized data parallel scheme with multiple sub-nodes and central nodes.

[0139] (2) Model parallelization.

[0140] In order to achieve a higher throughput, the various computing tasks in the decision model training process are fully parallelized. During implementation, the Actor module is set to be responsible for collecting data, and the Learner module is responsible for updating the model. Multiple Actors are parallel to each other, and one Learner can correspond to one Actor or multiple Actors. Considering the simulator operation requirements, the multi-environment process simulation process will be implemented on multiple GPU cores, and multiple environment processes will be arranged on each GPU card. During this period, the CPU controls the environment interaction through the API interface, and realizes interconnection communication with the Actors through PCIe to control data transmission and caching. Each Actor can be responsible for the interaction of one or more simulation environments, and the model reasoning process is implemented based on FPGA. Fig. 9 A schematic diagram of a multi-Actor parallel interaction structure based on FPGA provided by an embodiment of the present invention, Fig. 9 In the process, each Actor and Learner will maintain a local decision model for action reasoning and model iterative update of environmental interaction. Considering that when processing complex decision tasks with visual images or multimodal perception data as the original state input, the network scale of the state encoding part is relatively large, under the data parallelization setting, the FPGA nodes corresponding to each Actor and Learner are required to retain complete model parameters, which will cause redundancy on the one hand, and on the other hand, when the context encoding module, experience playback module and other functional modules are introduced, the number of decision model parameters will be further expanded, especially for functional modules built on the basis of complex network structures such as Transformer and diffusion model, due to the limited memory of a single FPGA board, the decision model will be difficult to load. To solve the above problems, the present invention divides the decision model into multiple parts, and uses multiple cluster nodes to collaboratively complete the calculation tasks of the entire model, wherein each cluster node contains multiple unit calculation nodes based on a single FPGA board, and different data parallelization design schemes are applied according to the characteristics and requirements of the corresponding task modules. In the decision model division stage, the following three division methods are specifically adopted: ①Task division: The calculation process of the decision model is regarded as a pipeline operation. Different calculation tasks during model reasoning are divided into different task cluster nodes. Each task cluster node is responsible for one task. The intermediate features obtained by calculation will be transmitted in sequence between the task cluster nodes. Figure 5 Taking the decision model structure for complex decision tasks in as an example, it can be divided into state coding layer, decision coding layer, and action coding layer according to the computing task. The state coding layer contains a side network for processing historical data, which can be further split out as a context coding layer. In addition, the experience replay module is used for data enhancement during the training of the decision model. Since the training and reasoning of the generation model in this module are relatively independent, it can also be split as an independent model. Among them, the computing tasks divided by the generation model according to different network structures are slightly different. The generation model based on the VAE structure can be divided into two parts: the encoder and the decoder. The generation model based on the GAN structure can be divided into two parts: the generator and the discriminator. The generation model based on the diffusion structure can be divided into two parts: forward diffusion and reverse denoising.

[0141] After the decision model is split, each computing subtask will be assigned to a corresponding task cluster node. Each task cluster node can work independently in the stage it is responsible for, which can achieve higher equipment utilization and improve overall processing efficiency.

[0142] ②Module division: After taking the decision model task division and task cluster node setting, if the computational amount of a certain computational subtask is too large or the performance of the task cluster node is poor, the computational efficiency of the entire model will be reduced. To address this problem, the present invention further divides the modules for tasks with heavy computational burdens such as state coding and context coding, and sets corresponding module cluster nodes inside the corresponding task cluster nodes to cooperate in completing the calculations of each part. Specifically, in the state coding task, it is divided into image processing, point cloud processing, and multi-scale feature fusion according to the data processing stage. If the image processing input is a multi-view image, a multi-view image feature fusion operation is required after the image coding is completed. This can be achieved by using a network structure based on Transformer and other cross-attention computing operations; the network of the point cloud processing part can be divided into point cloud feature coding, pseudo-image coding, single multi-target detection, and feature aggregation. In the above tasks, if Figure 5In the paper, image encoding, pseudo-image encoding and single-shot multi-target detection are all implemented based on convolutional networks, point cloud feature encoding is mainly implemented through rotation transformation operations and multilayer perceptrons (MLP), multi-view image feature fusion and multi-scale feature fusion are implemented based on Transformer, and each part is assigned to different module clusters for processing. For multi-view image feature fusion, multi-scale feature fusion and context encoding, considering the complex structure of the Transformer model, the multi-head attention module and feedforward neural network module are further divided into different module clusters. The multi-head attention module is responsible for capturing the dependencies between different positions in the input sequence, with a large amount of calculation and relatively independent calculation logic; the feedforward neural network module further performs nonlinear transformation on the output of the attention module.

[0143] By dividing task clusters with heavy computational burdens into modules, we can fully utilize the computing power of different devices, improve the training and reasoning efficiency of the model, and help improve the parallel efficiency of the model. In particular, when the computational load and parameter scale of different module clusters are relatively balanced, we can better utilize the resources of multiple computing devices.

[0144] ③Network layer division: For the computing tasks of image coding, pseudo-image coding, and single-shot multi-target detection, since the computational amount of feature data and the number of model parameters of the convolutional network are relatively large, in order to ensure that the computational load of all cluster nodes is relatively balanced, a layer division method is adopted to assign the convolutional network with more layers to multiple module clusters for collaborative processing. The division basis can be network layer block group, scale or computing function, which needs to be determined according to the model structure. Taking ResNet, VGG and You Only Look Once (YOLO) (unified real-time target detection), three convolutional models commonly used in image processing, as examples, the specific division method can refer to the following scheme: ①ResNet: ResNet is mainly composed of multiple residual blocks, which include convolutional layers, batch normalization layers, and activation function layers. The network depth can reach dozens or even hundreds of layers. The following two division schemes can be adopted: a. Division by residual block group: Several consecutive residual blocks can be divided into one computing device. For example, in ResNet-50, there are 4 stages of residual blocks, and the number of residual blocks in each stage is different. The residual blocks of the first stage (for example, containing 3 residual blocks) can be placed on a module cluster node. This stage is mainly used to extract low-level features of the image, such as edges, simple textures, etc. The residual blocks of the second stage (for example, containing 4 residual blocks) are placed on another module cluster node to further extract more complex features. The residual blocks of the third and fourth stages are similar. This division helps to balance the computing load between different devices, and resources can be allocated according to the functional characteristics of the residual blocks in each stage.

[0145] b. Division of convolutional layers and shortcut connection layers: Inside each residual block, there are convolutional layers and shortcut connections (used to skip one or more layers and add directly). The convolutional layer can be placed on one module cluster node, and the shortcut connection part can be placed on another device. However, this division method may increase the communication overhead between devices because intermediate results need to be transferred for addition operations, but it may be useful in certain specific hardware architectures or optimization scenarios. For example, when testing the contribution of different layer types to performance, this division method can be used to evaluate the computational efficiency of convolutional layers and shortcut connection layers separately.

[0146] ②VGG: The VGG network is mainly composed of multiple convolutional layers and pooling layers. The network structure is relatively regular, with multiple convolutional layers stacked and then fully connected layers. Taking VGG-16 as an example, it has 13 convolutional layers and 3 fully connected layers, and the following two division schemes can be adopted: a. Divide by convolution layer group and fully connected layer: The front convolution layers (for example, the first 6 convolution layers) can be placed on a module cluster node, which is mainly used to extract image features such as edges, textures, simple shapes, etc. Then the following convolution layers and pooling layers (for example, the remaining 7 convolution layers and all pooling layers) are placed on another module cluster node to further extract high-level semantic features. Finally, the 3 fully connected layers are placed on the third module cluster node for further feature fusion based on the previously extracted features.

[0147] b. Divide by convolutional layer and pooling layer functions: According to the functions of convolutional layer and pooling layer, the layers that only perform convolution operations are placed on one device, and the layers that include pooling operations are placed on another device. In the VGG network, the pooling layer is mainly used to reduce the data dimension and extract the main features, while the convolution layer is used to mine various features in the image. This division allows each device to focus on one type of computing operation, but may increase the number of communications between devices because the pooling layer needs to receive the output of the convolution layer for operation.

[0148] ③YOLO: YOLO is a real-time target detection model that can be used for single-shot multi-target detection in point cloud coding. It mainly includes convolutional layers and fully connected layers. Taking YOLOv3 as an example, it has a Darknet-53 backbone network (mainly composed of convolutional layers) for feature extraction, followed by a detection head (including convolutional layers and fully connected layers) for predicting the category, location and confidence of the target. The following two division schemes can be adopted: a. Backbone network and detection head division: The Darknet-53 backbone network can be placed on a module cluster node, which is responsible for extracting rich features from the input image. For example, it can first extract low-level features of the image, such as edges and textures, and then extract higher-level semantic features, such as the shape and category-related features of the object as the network depth increases. The detection head part is placed on another module cluster node. The detection head uses the features extracted by the backbone network to perform object detection-related calculations, such as generating prediction boxes of different scales and predicting the object category and position.

[0149] b. Division of prediction layers of different scales: In the detection head of YOLOv3, there are layers for predicting objects of different scales. These layers for predicting objects of different scales can be divided into different devices. For example, the layer for predicting small-scale objects is placed on one module cluster node, because the detection of small-scale objects usually requires finer features and more computing resources to distinguish different small objects; the layer for predicting medium-scale and large-scale objects is placed on another module cluster node, so that objects of different scales can be detected in parallel, improving detection efficiency.

[0150] 4. Explanation of FPGA single computing node: (1) Node architecture: In order to meet the data computing requirements of different types of network layers of the decision model, the present invention abstracts the computing operations in the forward reasoning and back-propagation process of the model into multiple core computing units, and designs a single computing node hardware structure for different computing tasks based on FPGA, with the aim of enabling the unit nodes in each cluster node to allocate resources more efficiently to complete a certain type of network calculation.

[0151] ①Node architecture for convolutional computing: Fig.10 A structural diagram of a single FPGA computing node for convolution computing provided by an embodiment of the present invention, such as Fig.10As shown in the figure, it includes a communication interface, a cache unit, a scheduling kernel, a forward reasoning module, a back propagation module and a multiplexer. The communication interface includes an Ethernet interface, a peripheral component interconnect express (PCIe) interface, a low-voltage differential signaling (LVDS) interface, a double data rate synchronous dynamic random access memory interface (DDR) memory interface, and a host random access memory (RAM) memory interface, which are used to connect the host computer software, other computing nodes and the cache unit inside the node for data transmission. The cache unit consists of two parts, DDR and RAM, for storing data, among which DDRA is used to cache feature data and DDRB is used to store model parameter data. The RAM connected to DDRB is used to store data results. After receiving the data, DDRB will transmit it back to the host computer or other nodes through the communication interface. The communication interface is connected to the scheduling core. The host computer or master node can modify the internal logic and data flow of the FPGA programmatically through PIO (Programmed Input / Output) to execute data transmission or specific control commands. The scheduling core is used to send each layer of memory access, calculation control and start signal to each computing unit. After completing the calculation operation, each computing unit will return the operating status and completion signal. In addition, the scheduling core is also connected to a multiplexer for result writeback and debugging control.

[0152] The input data of the data operation unit is directly obtained from the feature buffer, and the calculation results are returned to the feature buffer. In the forward reasoning module, the general matrix multiplication calculation unit and the convolution calculation unit are used to complete the basic matrix multiplication and convolution calculation operations respectively, where the feature data is read from the feature buffer, the parameter data is read from the parameter buffer, and the bias data is directly read from the bias buffer integrated in the calculation unit. After the calculation is completed, the feature results will be passed to the parameter buffer and the multiplexer. In the back propagation module, the loss gradient calculation unit and the parameter gradient calculation unit are used to complete the error loss and gradient calculation of each network layer respectively, where the loss gradient calculation unit needs to read the feature and parameter data from the feature buffer and the parameter buffer respectively for gradient calculation, and the results will be returned to the parameter buffer and sent to the multiplexer at the same time. The parameter gradient calculation unit directly reads the network parameters and the corresponding error gradients from the parameter buffer, and sends the results to the parameter buffer and the multiplexer after completing the parameter update calculation.

[0153] The FPGA single node using this computing architecture is mainly used for the module cluster construction of the three parts of image coding, pseudo-image coding and single multi-target detection in state coding. It should be noted that if the node is assigned to the task cluster node and module cluster node of the decision model in the Actor, since the Actor only needs to perform model forward reasoning calculations, the back propagation module in the node can be omitted. In addition, if the decision model directly calls the pre-trained convolutional model to implement state coding, since the model parameters of this part will be frozen during the model update phase, there is no back propagation calculation, and the back propagation module in the corresponding node of the Learner can also be omitted.

[0154] ②Node architecture for attention calculation: Fig.11 A structural diagram of an FPGA single computing node for attention calculation provided by an embodiment of the present invention. Except for the forward reasoning module, the rest of the settings are the same as the FPGA single node architecture for convolution calculation. The FPGA node using this type of computing architecture is mainly used for the construction of module clusters of three parts: multi-view image feature fusion, multi-scale feature fusion, and context coding in state coding, focusing on processing complex networks based on Transformer. Therefore, in the forward reasoning module, it mainly includes three key computing units: general matrix multiplication, layer normalization (Layer Normalization, LN) and Softmax, where the input data of Softmax and LN are obtained from the feature buffer, and the calculation results are returned to the feature buffer. The Softmax and LN units are also connected to the multiplexer to return the results. During LN calculation, the weight data and bias data are read from the weight buffer and bias buffer respectively, but unlike the general matrix multiplication unit, these two parts are integrated in the LN unit.

[0155] ③Node architecture for fully connected computing: Fig.12 A structural diagram of a single FPGA computing node for convolution computing provided by an embodiment of the present invention. The node architecture for fully connected computing can be regarded as a simplified version of the node architecture for convolution computing. Since the fully connected layer only has parallel multiplication and accumulation operations, the forward reasoning module in the node architecture only includes a general matrix multiplication computing unit, which is equivalent to transferring all the computing resources corresponding to the convolution computing unit to the general matrix multiplication computing unit, so that it has a larger PE parallel array and higher computing density, and can achieve more efficient and dense parallel multiplication and accumulation operations.

[0156] (2) Communication method: There are many ways to interconnect and communicate between FPGA unit nodes and other devices. In the present invention, FPGA unit nodes are used to build module cluster nodes for different computing tasks, and the master nodes in each module cluster and task cluster need to undertake tasks such as data distribution, control signal / instruction transmission, and status monitoring. Therefore, FPGA boards with special management functions are used as master nodes to achieve collaboration between module clusters and task clusters. Compared with unit nodes, in addition to basic FPGA logic processing capabilities, master nodes also integrate additional control modules, which implement tasks distribution, status monitoring and other functions for the entire cluster node through Ethernet controllers or high-speed serial interfaces, and use their internal embedded processors to run management programs and send instructions to other FPGA nodes, such as starting specific hardware acceleration functions, reading node status information, etc.

[0157] In terms of interconnection, the interconnection communication between FPGA and FPGA mainly occurs between the master node and the unit node. If a module cluster node adopts a ring communication topology structure with decentralized nodes to realize data parallel processing, there is also an interconnection demand between adjacent unit nodes. In addition, the master node will involve interconnection communication with the host computer, where the host computer includes two computing devices, the CPU and the GPU. Therefore, there is communication transmission between FPGA and CPU and between FPGA and GPU. For the above three interconnection communication requirements, there are specific implementation methods as follows: ①FPGA-FPGA interconnection communication.

[0158] The interconnection communication between FPGAs is mainly achieved through three channels: PCIe, LVDS, and Ethernet. In this solution, large-scale data transmission such as characteristics and parameters between single computing nodes in each task cluster and module cluster is achieved by PCIe, and control signal / instruction transmission between nodes and status flag information feedback are achieved by LVDS. Considering the limitation of the number of FPGA interfaces and the bandwidth and communication delay problems of the expansion interface, cross-level data transmission between the main node and the lower-level sub-cluster nodes in each task cluster and module cluster is achieved by Ethernet.

[0159] ②FPGA-CPU interconnection communication.

[0160] The interconnection and communication requirements between FPGA and CPU are mainly reflected in the transmission of computing data such as features and parameters between the master nodes of each task cluster and the host computer, as well as control, debugging information and status feedback signals. PCIe or Compute Express Link (CXL) can be used to achieve this. CXL provides higher bandwidth and lower latency and supports cache consistency. Although it has advantages in performance and function, it is a relatively new technical standard. Considering the compatibility of devices, this solution chooses to use PCIe to achieve data transmission between FPGA and CPU. Through the PCIe interface, direct memory access (DMA) is used for data transmission, which can effectively reduce the burden on the CPU and improve the efficiency of data transmission. Since PCIe is backward compatible, the new version of the PCIe interface can be compatible with the old version of the device, which facilitates subsequent system upgrades and expansions.

[0161] ③FPGA-GPU interconnection communication.

[0162] In this solution, the GPU is only used to create an interactive environment. The main communication requirements involved include sending the state and reward signals from the simulation environment in the GPU to the Actor module, and the Actor module sending the inferenced decision action back to the simulation environment in the GPU to complete the execution. Since the simulation environment needs to call the API interface from the CPU side of the host computer to realize the creation and interaction on the GPU side, the FPGA-GPU direct communication method is not adopted in this solution. The FPGA master node and GPU in each task cluster and module cluster are all PCIe devices, connected to the CPU through the PCIe bus, where the FPGA transfers the data to the RAM through DMA, and then the GPU reads the data from the host RAM through DMA.

[0163] 5. The FPGA cluster with a multi-level cluster structure is described below.

[0164] Taking into account the differences in computing task requirements of each module in the parallel training architecture of the decision model, this solution will build a cluster through multiple FPGA task clusters and module clusters to implement model reasoning and update tasks, and at the same time combine CPU and GPU to build an environmental interaction framework to achieve efficient online reasoning and training of the model. Fig.13A schematic diagram of the structure of a multi-level cluster of accelerators provided by an embodiment of the present invention. Each level of cluster nodes includes a master node and a lower-level sub-cluster node, wherein the task allocation, monitoring and data transmission between the master node and each cluster node are realized through Ethernet, and each lower-level sub-cluster node also includes a master node and a lower-level sub-cluster node. The master nodes at each level are responsible for receiving and parsing the upper-level master node information management package, algorithm model package and configuration parameter package, etc., and deploying different model calculation tasks and modules to each sub-cluster node through Ethernet, and controlling the calculation start and stop inside each cluster node. At the same time, the model parameters and feature data are distributed to each cluster node according to the calculation task settings through Ethernet, thereby forming a multi-level FPGA cluster structure, which is used as an Actor or Learner module to realize model reasoning and parameter update. In addition, there is also interconnection communication between the master node and the unit node of each cluster node according to the task requirements, wherein large-scale streaming data such as model parameters and training features are realized through the PCIe interface, and signal management and status monitoring related data are transmitted between nodes through LVDS. For the lowest-level cluster node, its master node will be directly connected to each FPGA unit node, and each unit node corresponds to a single FPGA board. Similarly, data transmission and task allocation between each master node and the unit nodes within the cluster node are implemented using Ethernet, and status monitoring and data transmission between unit nodes are also implemented through PCIe and LVDS respectively.

[0165] (1) Actor / Learner cluster node logical structure: Designed according to the model structure and computing task requirements, the Actor and Learner cluster nodes have different logical structures. Fig.14 A logical schematic diagram of an action cluster node provided by an embodiment of the present invention, such as Fig.14As shown in the figure, for the Actor cluster node, its subordinate sub-cluster nodes include four parts: state coding task cluster, context coding task cluster, decision coding task cluster and action coding task cluster. In terms of data caching logic, the Actor master node and the host computer are interconnected through PCIe, and task allocation and state transmission are realized between each task cluster through Ethernet. For the network parameters of each module of the decision model, Ethernet is also used to transmit from the Actor master node to each task cluster. Among them, the state coding task cluster will also read the original state data obtained in real time from the environment interaction from the master node. The coded features obtained after processing will be further sent to the context coding task cluster through PCIe for caching and subsequent coding processing, and sent to the decision coding task cluster as input data. The coded features obtained after processing by the context coding task cluster will also be sent to the decision coding task cluster through PCIe as input data. Similarly, the coded features obtained after processing by the decision coding task cluster will be sent to the action coding task cluster through PCIe as input data. The action features obtained after processing are finally transmitted back to the master node through Ethernet and sent to the host computer to execute actions to complete the environment transfer. Since each task cluster has a certain degree of independence and timing in the process of processing data, and the data processing speed and load conditions of different task clusters may be different, in order to ensure the stable operation of the entire system and the accurate transmission of data, status monitoring and feedback are required between adjacent task clusters through LVDS. Fig.15 A logical schematic diagram of a trainer cluster node provided by an embodiment of the present invention, such as Fig.15 As shown in the figure, for the Learner cluster node, its subordinate sub-cluster node adds a generation playback task cluster based on the Actor cluster node, and is also interconnected through Ethernet. The task cluster needs to read the online collected experience samples from the master node for diffusion model training, where each experience sample contains three parts: original state data, action, and reward. Then, the virtual experience samples generated by the diffusion model are sent back to the master node to expand the training data of the decision model. Since it involves the update of decision model parameters, in the Learner cluster node, in order to realize the back propagation operation, the action encoding task cluster calculates the loss based on the action and reward data in the experience sample, and performs gradient return. Each task cluster needs to send the updated local parameters back to the master node, so the data transmission method here is bidirectional.

[0166] (2) Logical structure of task cluster nodes.

[0167] Considering that some task cluster nodes are overloaded with computation, in order to balance the resource allocation of each cluster node, this solution divides multiple module cluster nodes to construct each task cluster node. For specific module division, please refer to the model parallelization part described above. Fig.16 A logical structure diagram of a state coding task cluster of an action part provided by an embodiment of the present invention. Fig.17A logical structure diagram of a state encoding task cluster of a trainer part provided by an embodiment of the present invention. The state encoding task cluster in the Learner adds module parameters and loss gradient return operations based on the Actor part.

[0168] a) For the multi-view image coding, pseudo image coding and single multi-target detection module clusters that focus on convolution calculations, they are further divided into multiple CNN module clusters according to the network layer. Each CNN module cluster contains multiple unit computing nodes, and data parallelism of decentralized nodes is realized based on the ring topology structure.

[0169] Fig.18 The following is a logical diagram of a module cluster for convolutional computing provided in the action part of the embodiment of the present invention. Fig.18 As shown in the figure, each CNN module cluster transmits intermediate features through PCIe. Considering that some convolution calculation operations have residual connections, in addition to data interaction between adjacent CNN module clusters, there may also be data transmission across module clusters to ensure the accuracy of feature transfer and calculation involved in residual connections. Fig.19 The following is a logical diagram of a module cluster for convolutional computing in the trainer part provided in an embodiment of the present invention. Fig.19 As shown in the figure, compared with the Actor part, the return of the loss gradient is increased.

[0170] b) For the multi-view feature fusion and multi-scale feature fusion module clusters that focus on Transformer attention calculation, they are divided into multi-head attention and feedforward neural network module clusters according to their functions. Each module cluster realizes centralized data parallelism based on the structure of central node and multiple sub-nodes. Fig. 20 The following is a logical diagram of a module cluster for attention calculation in the action part provided by an embodiment of the present invention. Fig. 20 As shown, the input features are first transmitted from the main node to the multi-head attention module cluster via Ethernet for processing, and then the feedforward neural network and multi-head attention calculations are performed alternately according to the network block group settings. After completing the last feedforward neural network calculation, the output features are transmitted back to the main node via Ethernet. Therefore, there is feature mutual transmission between the multi-head attention and feedforward neural network module clusters. Fig.21 The following is a logical diagram of a module cluster for attention calculation in the trainer part provided in an embodiment of the present invention. Fig.21 As shown in the figure, compared with the Actor part, the return of the loss gradient is increased.

[0171] c) The point cloud feature encoding module cluster is divided into a rotation transformation module cluster and an MLP module cluster according to its function. Each module cluster also realizes centralized data parallelization based on the central node-multiple sub-node structure. Fig. 22The following is a logical diagram of a module cluster for point cloud feature coding of the action part provided by an embodiment of the present invention. That is, a module cluster for point cloud feature coding of the Actor part. Fig. 22 As shown in the figure, the input features are first transmitted from the main node to the rotation transformation module cluster via Ethernet for processing, and then MLP and rotation transformation calculations are performed alternately according to the network block group settings. After the last MLP calculation is completed, the output features are transmitted back to the main node via Ethernet. Therefore, similar to the module cluster design for attention calculation, in the point cloud feature encoding module cluster, there is also feature transmission between rotation transformation and MLP. Fig.23 A logical schematic diagram of a module cluster for point cloud feature coding in the trainer part provided in an embodiment of the present invention. That is, the module cluster for point cloud feature coding in the learner part has a loss gradient feedback added compared to the actor part.

[0172] d) The context coding task cluster is similar to the multi-view feature fusion and multi-scale feature fusion module clusters, both of which focus on attention calculation. Therefore, it can also be divided into multi-head attention and feedforward neural network module clusters according to function, and centralized data parallelization is realized based on the central node-multiple sub-node structure. The decision coding and action coding task clusters do not need to be further divided because of their relatively simple module structure and fewer network layers. The main node of the task cluster will be directly connected to multiple unit nodes for task allocation and data transmission. Each unit node adopts a central node-multiple sub-node structure to achieve centralized data parallelization.

[0173] e) Generate replay task clusters with different model partitioning methods, Fig.24 A logic diagram of a variational autoencoder decoder generation model provided by an embodiment of the present invention. That is, a VAE generation model. Fig.24 As shown, the VAE generation model is divided into an encoder and a decoder. Fig.25 A logical schematic diagram of a generative adversarial network generation model provided by an embodiment of the present invention, namely a GAN generation model. The GAN generation model is divided into a discriminator and a generator, and the diffusion generation model is divided into forward diffusion and reverse denoising modules.

[0174] During the training phase, for the VAE generation model, the encoder module cluster reads real experience samples from the main node via Ethernet, and transmits the processed feature data to the decoder part via PCIe. After completing the loss calculation, the decoder transmits the loss gradient back to the encoder via PCIe to complete the parameter update; for the GAN generation model, the generator module cluster uses random noise as input to generate virtual experience samples, which are sent to the discriminator via PCIe. The discriminator reads real experience samples from the main node via Ethernet to calculate the sample error, and then updates the parameters of the discriminator through the back-propagation algorithm. After the parameters of the discriminator are further fixed, the generator regenerates virtual samples and sends them to the discriminator. The discriminator then transmits the discrimination probability back to the generator to calculate the generation loss, and then updates the parameters of the generator through the back-propagation algorithm. Fig.26 A logical schematic diagram of a diffusion generation model provided by an embodiment of the present invention. For the diffusion generation model, the forward diffusion module cluster reads real experience samples from the master node via Ethernet, gradually adds a small amount of Gaussian noise to the data according to a certain step size, and then transmits the noisy sample data to the reverse denoising part via PCIe, and continuously adjusts the denoising model parameters to enable it to have denoising capabilities.

[0175] In the generation phase, the VAE decoder, GAN generator, and diffusion model reverse denoising module cluster all use randomly sampled Gaussian noise as input to generate virtual experience samples, and finally send the sample data to the master node via Ethernet. In general, experience samples should contain three parts: original state data, action, and reward. However, the original state data contains multi-view images and complex point cloud data, which is too difficult to generate. In order to ensure the quality of sample generation while saving computing resources, this scheme constructs implicit experience samples based on feature space. The sample uses the potential state features after state encoding to replace the original state data, which can greatly reduce the dimension of sample data. Only a shallow fully connected network can be used to build a generation model to achieve data enhancement. Therefore, each module cluster can adopt a central node-multiple sub-node structure to achieve centralized data parallelization. It should be noted that when using implicit experience samples to update the decision model parameters, the state encoding module part should be frozen and the back propagation calculation should not be performed.

[0176] It can be seen that in view of the problems of unreasonable resource allocation, data synchronization difficulties and hardware limitations of the existing intelligent autonomous decision-making technology based on reinforcement learning when combined with FPGA, the decision model training and reasoning acceleration system based on FPGA cluster proposed in the present invention adopts a distributed parallel computing architecture design, decouples tasks such as environmental interaction, model reasoning and experience storage, sets up multi-node parallel execution, combines data parallelization and model parallelization schemes to improve computing efficiency, data parallelization includes a variety of subset division methods and solutions to deal with load imbalance, and model parallelization improves equipment utilization through module settings. In terms of decision model structure design, multi-layer functional modules are divided according to task and spatial complexity, different network structures are adopted for different state spaces and trained flexibly, and three model structure adaptive selection strategies are proposed to optimize model design according to task scenarios. Further, for the FPGA single computing node architecture and communication mode, a node architecture for different computing tasks is designed to improve resource allocation efficiency, and a variety of interconnected communication methods are set to ensure efficient and accurate data transmission between different devices. Finally, an FPGA cluster computing framework and heterogeneous computing platform are built for decision model training and inference tasks. The computing framework is built with multi-level FPGA cluster nodes to improve hardware utilization and reduce system latency while ensuring load balancing of each computing node. The CPU and GPU are combined to achieve model reasoning and updating. The training and reasoning efficiency are improved through the collaboration of various modules, thereby enhancing the system's adaptability and flexibility.

[0177] In terms of computing efficiency, a distributed parallel computing architecture is used to improve efficiency through task decoupling and multi-node parallel execution, combined with data and model parallelism. Data parallelism reasonably divides data subsets and solves the problem of load imbalance. Model parallelism improves equipment utilization and throughput. In terms of resource allocation, the decision model structure design reasonably constructs each layer according to the characteristics of tasks and data, and distributes computing tasks to cluster nodes through various division methods to improve FPGA resource utilization and overall computing resource utilization. In terms of data synchronization, a reasonable parallelization solution is used to ensure timely data synchronization, solve the problem of distributed cluster data synchronization difficulties, and improve model accuracy and stability. In terms of model performance, the model structure design and its adaptive selection strategy will effectively improve the decision model's understanding and adaptability to system dynamics and environment, improve accuracy, and then improve system stability and generalization capabilities with reasonable resource allocation and training methods.

[0178] In the above embodiments, the training and reasoning method of the decision model is described in detail, and the present invention also provides an embodiment corresponding to the training and reasoning device of the decision model. It should be noted that the present invention describes the embodiments of the device part from two perspectives, one is based on the perspective of the functional module, and the other is based on the perspective of the hardware.

[0179] The training and reasoning device of the decision model provided in the embodiment of the present invention, based on the perspective of functional modules, includes: An acquisition module, used for acquiring a decision model and acquiring network features of the decision model; A first allocation module is used to divide the training and reasoning process of the decision model into one or more computing tasks according to the network characteristics of the decision model, and allocate different computing tasks to different cluster nodes; A second allocation module is used to allocate multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes to different computing nodes, and transfer the computing tasks on the cluster nodes to the computing nodes to which the input data are allocated; The determination module is used to determine the training and inference results of the decision model according to the output characteristics of the network model on the computing node to which the input data is allocated.

[0180] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0181] Fig. 27 This is a structural diagram of an electronic device provided by an embodiment of the present invention. This embodiment is based on the hardware perspective, such as Fig. 27 As shown, the electronic equipment includes: A memory 20, used for storing computer programs; The processor 21 is used to implement the steps of the method for training and reasoning the decision model mentioned in the above embodiment when executing the computer program.

[0182] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array FPGA, and a programmable logic array. The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0183] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the training and reasoning method of the decision model disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. Data 203 may include, but is not limited to, the data involved in the training and reasoning method of the decision model mentioned above.

[0184] In some embodiments, the electronic device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power source 25 , and a communication bus 26 .

[0185] Those skilled in the art will understand that Fig. 27 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure.

[0186] The electronic device provided by an embodiment of the present invention includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: a training and reasoning method of a decision model, with the same effect as above.

[0187] An embodiment of the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned decision model training and reasoning method when executed by a processor.

[0188] Finally, the present invention also provides an embodiment corresponding to a non-volatile storage medium. The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.

[0189] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Various media that can store program codes.

[0190] The non-volatile storage medium provided by the present invention includes the training and reasoning method of the decision model mentioned above, and the effect is the same as above.

[0191] The above is a detailed introduction to the training and reasoning method, product, electronic device and medium of a decision model provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

[0192] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A training and reasoning method for a decision model, characterized in that: Processor used for host computer in training and reasoning acceleration system; The accelerator cluster of the training and reasoning acceleration system is a cluster structure; the method comprises: Obtaining a decision model and obtaining network characteristics of the decision model; Dividing the training and reasoning process of the decision model into one or more computing tasks according to the network characteristics of the decision model, and assigning different computing tasks to different cluster nodes; Distribute the multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes to different computing nodes, and transfer the computing tasks on the cluster nodes to the computing nodes to which the input data are distributed; The training and reasoning results of the decision model are determined according to the output characteristics of the network model on the computing node to which the input data is distributed.

2. The decision model training and reasoning method according to claim 1, characterized in that: The training and reasoning process of the decision model is divided into multiple computing tasks according to the network characteristics of the decision model, and different computing tasks are assigned to different cluster nodes, including: Dividing the training and reasoning process of the decision model into one or more tasks according to the training and reasoning stages included in the decision model, and allocating the tasks to different cluster nodes of the first type; Dividing the task into one or more subtasks according to the network structure used when processing the task, and allocating different subtasks to different cluster nodes of the second type; The subtask is treated as one computing task or divided into a plurality of computing tasks according to the complexity of processing the subtask, and different computing tasks are allocated to different cluster nodes of the third type.

3. The training and reasoning method of the decision model according to claim 2, characterized in that: According to the complexity of processing the subtask, the subtask is treated as one computing task or divided into multiple computing tasks, and different computing tasks are allocated to different cluster nodes of the third type, including: When it is detected that the amount of computation when processing the subtask is less than the preset amount of computation, the subtask is regarded as a computation task, and the computation task is allocated to a cluster node of the third type; When it is detected that the amount of computation when processing the subtask is greater than or equal to the preset amount of computation, the subtask is divided into multiple computing tasks; the network structure of the network model used when processing the computing task is classified according to the complexity of the network structure of the network model used when processing the computing task; and the computing tasks corresponding to different categories of network structures are allocated to different cluster nodes of the third type.

4. The training and reasoning method of the decision model according to claim 2 is characterized in that: Obtaining the input data corresponding to the computing task on the cluster node includes: If the computing task is an initial computing task among all computing tasks, determining that the input data corresponding to the initial computing task is data sent by the processor of the host computer and received by the cluster node of the first type; If the computing task is not the initial computing task, it is determined that the input data corresponding to the computing task is the output feature obtained by computing the input data corresponding to the previous computing task of the computing task through the network model on the computing node.

5. The decision model training and reasoning method according to claim 4, characterized in that: The input data corresponding to the computing tasks on the cluster nodes are split into multiple data subsets including: When it is detected that the input data corresponding to the computing task on the cluster node is image data, if the data batch size is greater than or equal to the number of input channels, the input data corresponding to the computing task on the cluster node is split along the data batch dimension to obtain multiple data subsets; if the data batch size is less than the number of input channels, the input data corresponding to the computing task on the cluster node is split along the input channel dimension to obtain multiple data subsets; When it is detected that the input data corresponding to the computing task on the cluster node is not image data, the dimension with the largest number of dimensions is obtained from the data batch dimension and multiple feature dimensions; the input data corresponding to the computing task on the cluster node is split along the dimension with the largest number of dimensions to obtain multiple data subsets.

6. The decision model training and reasoning method according to claim 5, characterized in that: In the case where it is detected that the dimension with the largest number of dimensions is the feature dimension, the step of allocating the multiple data subsets obtained by splitting the input data corresponding to the computing task on the cluster node to different computing nodes includes: Obtaining the number of computing nodes to which the preset input data is to be distributed; Obtain the result of performing a modulo operation on the dimension with the largest number of dimensions and the number of computing nodes to be allocated to determine a first number of computing nodes; Determining a second number of computing nodes according to a difference between the number of computing nodes to be allocated and the first number; Splitting the input data corresponding to the computing task on the cluster node according to the dimension with the largest number of dimensions determines a data subset of a first data size and a data subset of a second data size; wherein the first data size is determined by multiplying the result obtained by rounding down the dimension with the largest number of dimensions and the number of computing nodes to be allocated plus 1 and the data size on the remaining dimensions; the second data size is determined by multiplying the result obtained after the rounding down operation and the data size on the remaining dimensions; the remaining dimensions are dimensions other than the dimension with the largest number of dimensions among all dimensions; The data subsets of the first data size are respectively distributed to a first number of computing nodes, and the data subsets of the second data size are respectively distributed to a second number of computing nodes.

7. The decision model training and reasoning method according to claim 6, characterized in that: Determining the training and reasoning results of the decision model according to the output characteristics of the network model on the computing node to which the input data is distributed includes: Obtaining the current output features of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node; wherein the current subtask starts from the initial subtask of the training and reasoning process of the decision model; Obtain the next subtask of the current subtask according to the first order of the subtasks in the training and reasoning process of the decision model, and use the next subtask of the current subtask as the new current subtask; Transmitting the current output feature to the second type of cluster node where the current subtask is located via the third type of cluster node where the computing task corresponding to the current subtask is located; Obtain the second type of cluster node where the new current subtask is located, transmit the current output feature to the second type of cluster node where the new current subtask is located through the second type of cluster node where the current subtask is located, so as to obtain the input data corresponding to the new current subtask, and return the current output feature after the input data of the computing task corresponding to the current subtask passes through the network model on the computing node, until the current subtask is the last subtask in the training and reasoning process of the decision model, stop returning, and determine the training and reasoning result of the decision model according to the current output feature.

8. The decision model training and reasoning method according to claim 7, characterized in that: Transmitting the current output feature to the new second type cluster node where the current subtask is located via the second type cluster node where the current subtask is located comprises: Controlling the communication between the second type of cluster node where the current subtask is located and the second type of cluster node where the new current subtask is located through the first signal transmission mode and obtaining communication information; wherein the communication information at least includes status monitoring information and status feedback information; When it is detected that the communication information is normal, the current output feature is transmitted to the second type of cluster node where the new current subtask is located through the second signal transmission mode via the second type of cluster node where the current subtask is located.

9. The decision model training and reasoning method according to claim 7, characterized in that: The current output features of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node include: In the case where it is detected that the current computing task is not obtained by dividing the current subtask, the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node is obtained, and the sum of the current output features is used as the current output feature of the computing task corresponding to the current subtask after the input data passes through the network model on the computing node; When it is detected that the current subtask is divided into multiple computing tasks, the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node is obtained; the next computing task of the current subcomputing task is obtained according to the second priority order of the computing tasks in the current subtask, and the next computing task of the current computing task is used as the new current computing task; the sum of the current output features is transmitted to the third type of cluster node where the new current computing task is located via the third type of cluster node where the current computing task is located, so as to obtain the input data corresponding to the new current computing task, and return to the step of obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node, until the current computing task is the last computing task in the current subtask, stop returning, output the sum of the current output features, and use the sum of the current output features as the current output feature of the input data of the computing task corresponding to the current subtask after passing through the network model on the computing node.

10. The decision model training and reasoning method according to claim 9, characterized in that: Obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node includes: Obtaining a computing node of a first type and a computing node of a second type used when processing a data subset corresponding to a current computing task; wherein a network model is set in the computing node of the first type, and a network model is not set in the computing node of the second type; and there is one computing node of the second type; Obtain output features of each data subset corresponding to the current computing task on the corresponding computing node of the first type; Sending output features on all computing nodes of the first type to computing nodes of the second type; An aggregation operation is performed on all output features through the computing nodes of the second type to obtain the sum of current output features of the data subset corresponding to the current computing task on the corresponding computing nodes.

11. The decision model training and reasoning method according to claim 9, characterized in that: Obtaining the sum of the current output features of the data subset corresponding to the current computing task on the corresponding computing node includes: Obtaining the communication order between all computing nodes processing the current computing task; wherein the last computing node among all computing nodes arranged in the communication order is connected to the first computing node in communication; Obtain data segments of each data subset corresponding to the current computing task after being passed through the network model on the corresponding computing node and store them in a sending buffer of the computing node for sending data; Determine whether the current computing node processing the current computing task has data fragments of all other nodes; wherein the other nodes are computing nodes other than the current computing node among all computing nodes processing the current computing task; If not, then the data fragment on the current computing node is sent from the sending buffer to the receiving buffer for receiving data of the next computing node of the current computing node in accordance with the communication order; the next computing node of the current computing node is used as the new current computing node; the data of the receiving buffer of the new current computing node is aggregated with the data of the sending buffer of the new current computing node and used as the data fragment of the sending buffer of the new current computing node, and the step of determining whether there are data fragments of all other nodes in the current computing node processing the current computing task is returned; If so, the data fragments on the current computing node will be sent from the sending buffer to the receiving buffer for receiving data of the next computing node of the current computing node in accordance with the communication order; the next computing node of the current computing node will be used as the new current computing node; the data in the receiving buffer of the new current computing node will be overwritten with the data in the sending buffer of the new current computing node and used as the data fragments of the sending buffer of the new current computing node, and the step of determining whether there are data fragments of all other nodes in the current computing node processing the current computing task will be returned until it is detected that there are data fragments of all other nodes in all computing nodes processing the current computing task and the process stops returning.

12. The decision model training and reasoning method according to claim 10, characterized in that: Transmitting the sum of the current output features to the cluster node of the third type where the new current computing task is located via the cluster node of the third type where the current computing task is located comprises: Controlling the communication between the third type of cluster node where the current computing task is located and the third type of cluster node where the new current computing task is located through the first signal transmission mode and obtaining communication information; wherein the communication information at least includes status monitoring information and status feedback information; When it is detected that the communication information is normal, the sum of the current output features is transmitted via the third type of cluster node where the current computing task is located to the third type of cluster node where a new current computing task is located through the second signal transmission mode.

13. The decision model training and reasoning method according to claim 10, characterized in that: Before obtaining the output features of each data subset corresponding to the current computing task on the corresponding computing node of the first type, the method further includes: If it is detected that the load on the first type of computing nodes and the second type of computing nodes used to process the data subset corresponding to the current computing task is less than the preset load, the step of obtaining the output characteristics of each data subset corresponding to the current computing task on the corresponding first type of computing node is entered.

14. The decision model training and reasoning method according to claim 11, characterized in that: Before obtaining the communication order between all computing nodes processing the current computing task, it also includes: Obtaining load conditions on the first type of computing nodes and the second type of computing nodes used when processing the data subset corresponding to the current computing task; If it is detected that the load on the first type of computing nodes and the second type of computing nodes used to process the data subset corresponding to the current computing task is greater than or equal to the preset load, the step of obtaining the communication order between all computing nodes processing the current computing task is entered.

15. The training and reasoning method of the decision model according to any one of claims 1 to 14, characterized in that: The computing nodes include multiple types, and different types of computing nodes are used to process different types of network layer data; The step of allocating multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes to different computing nodes includes: Obtaining network layer characteristics of the computing task; Determine the target computing node based on the network layer characteristics; The multiple data subsets obtained by splitting the input data corresponding to the computing tasks on the cluster nodes are distributed to different target computing nodes.

16. The decision model training and reasoning method according to any one of claims 7 to 14, characterized in that: The first type of cluster nodes are located on the host computer; the second type of cluster nodes assigned to the subtasks corresponding to the same task and the third type of cluster nodes assigned to the same computing task are located on the same host; the third type of cluster nodes assigned to different computing tasks are located on different hosts.

17. The decision model training and reasoning method according to claim 16, characterized in that: Allocating the tasks to different cluster nodes of the first type includes: Allocating the task to different cluster nodes of the first type by a third signal transmission method; The allocating different subtasks to different cluster nodes of the second type comprises: Using the first type of cluster nodes and allocating different subtasks to different second type of cluster nodes through the third signal transmission mode; The allocating different computing tasks to different third-type cluster nodes includes: The second type of cluster nodes are utilized and different computing tasks are allocated to different third type of cluster nodes through the third signal transmission manner.

18. The decision model training and reasoning method according to claim 17, characterized in that: The training and reasoning of the decision model includes an interaction phase and a model update phase; before sending data to the first type of cluster nodes, it also includes: Obtaining data sent to a cluster node of the first type from a process of the graphics processing unit for training inference; The training inference result of determining the decision model according to the current output feature includes: receiving, in an interactive phase, a first current output feature outputted sequentially by the cluster nodes of the third type, the cluster nodes of the second type, and the cluster nodes of the first type; Storing the first current output feature in a data storage module located in a host computer; In the case where it is detected that the amount of data in the data storage module is greater than a preset amount of data, receiving, in the model updating phase, a second current output feature outputted sequentially by the cluster nodes of the third type, the cluster nodes of the second type, and the cluster nodes of the first type; Calculating a loss function based on the second current output feature to determine parameters of a decision model, so as to obtain a training inference result of the decision model; After obtaining the training and reasoning results of the decision model, the following steps are also included: The parameters of the decision model are transmitted to the cluster nodes of the first type to update the parameters of the decision model in the cluster nodes of the first type.

19. The decision model training and reasoning method according to claim 18, characterized in that: Establishing a decision model located on the first type of cluster node includes: Capturing the complexity of the decision-making task; Decision models with different structures are established on the first type of cluster nodes according to the complexity of the decision task; wherein the complexity of the decision task is positively correlated with the complexity of the decision model structure.

20. The decision model training and reasoning method according to claim 18, characterized in that: Determining the structure of the decision model includes: Acquire the operating status of the hardware platform; wherein the operating status of the hardware platform at least includes hardware resource utilization, latency and power consumption; Obtaining the current expected cumulative reward value under the current state and the current decision model structure; wherein the current state at least includes the current input data characteristics and the current operating state of the hardware platform; Determining an expected value according to a current expected cumulative reward and a current reward function; wherein the current reward function is determined by a current operating state of the hardware platform; Obtain the current decision model structure whose expected value meets the requirements; The current decision model structure whose expected value meets the requirements is used as the structure of the decision model.

21. The decision model training and reasoning method according to claim 18, characterized in that: Determining the structure of the decision model includes: The structure of multiple pre-defined decision models; Obtaining hardware performance indicators under decision models of different structures, and establishing hardware constraints based on the hardware performance indicators; wherein the hardware performance indicators at least include latency, power consumption, and resource occupancy; Obtain the fitness of decision models with different structures; Under the condition of satisfying hardware constraints, obtain the decision model structure with the greatest fitness among all decision models of the structure; The decision model structure with the greatest fitness is taken as the structure of the decision model.

22. The decision model training and reasoning method according to claim 18, characterized in that: Determining the structure of the decision model includes: Pre-set structures of various decision models and features of extracted input data; Predicting the probability of selecting decision models of different structures according to the characteristics of the input data; Obtain the structure of the decision model with the highest probability among all decision models of the structure; The structure of the decision model with the highest probability is taken as the structure of the decision model.

23. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the training reasoning method of the decision model described in any one of claims 1 to 22 are implemented.

24. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the training and reasoning method for the decision model as described in any one of claims 1 to 22 when executing the computer program.

25. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the training and reasoning method of the decision model as described in any one of claims 1 to 22 are implemented.

Images