Data reasoning method, network device, medium, electronic device and program product

Abstract

The invention relates to a data reasoning method, network equipment, a medium, electronic equipment and a program product, belongs to the technical field of computers, and can solve bandwidth bottleneck, reduce time delay and improve system performance. A data reasoning method includes: receiving an initial attention vector from a plurality of nodes, the initial attention vector including a plurality of initial elements; determining a local attention vector which is locally stored by the network equipment and comprises a plurality of local elements; the aggregation of the local element i and the corresponding initial element is executed by j-1, j, j + 1 and j + 2 in the pipeline level of the network device, the j-1 level determines the weighted value of the initial element and stores the weighted value in the HPV, the j level reads the local element i from the memory of the j level and stores the local element i in the HPV, the j + 1 level determines the weighted value of the local element i and stores the weighted value in the HPV, and the j + 2 level reads the local element i from the memory of the j level and stores the weighted value in the HPV. And the j + 2 level aggregates the local element i and the initial element based on the weighted values of the local element i and the initial element to obtain the ith element in a target attention vector, and the target attention vector is used for data reasoning.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a data reasoning method, a network device, a medium, an electronic device, and a program product. Background Art

[0002] Large language model inference refers to the process of using a trained large language model for inference, where the user inputs prompt information into the large language model, and the large language model outputs the answer information corresponding to the prompt information. The input and output of the large language model are collectively referred to as context.

[0003] As the context length of a large language model continues to grow, the amount of computation required for reasoning with a large language model will increase linearly. At present, the reasoning process of a large language model is usually run between multiple distributed nodes, and the entire reasoning process is decomposed in a distributed memory manner. Specifically, the context sequence length dimension is divided to obtain multiple subsequences. Each distributed node is responsible for a subsequence of a finite length, and each node independently calculates the attention vector of the subsequence it is responsible for. Then, the master node among the multiple distributed nodes performs a full reduction on the attention vectors calculated by all distributed nodes to obtain a global attention vector. This requires the master node to obtain all attention vectors before performing a full reduction calculation. After the calculation is completed, the calculation results are updated to all distributed nodes. This method is likely to cause calculation and network bottlenecks at the master node, affecting system performance. Summary of the invention

[0004] The purpose of the present disclosure is to provide a data reasoning method, network equipment, medium, electronic equipment and program product, which can solve the bandwidth bottleneck problem, reduce latency and improve system performance.

[0005] In order to achieve the above object, the present disclosure provides a data reasoning method, which is applied to a network device in a data reasoning system, wherein the data reasoning system includes the network device and a plurality of nodes, and the data reasoning method includes:

[0006] Receiving initial attention vectors from the multiple nodes, wherein the initial attention vectors are obtained by each of the multiple nodes performing attention calculations on the context subsequences for which they are responsible, and include multiple initial elements;

[0007] Determine a local attention vector stored locally by the network device and corresponding to the initial attention vector, wherein the local attention vector includes a plurality of local elements;

[0008] The j-1th, jth, j+1th and j+2th stages of the pipeline stage for performing vector aggregation of the network device perform aggregation of the local element i and the corresponding initial element, wherein the local element i is stored in the memory of the jth stage, the j-1th stage determines the weighted value of the initial element and stores the weighted value of the initial element in the packet header vector of the network device, the jth stage reads the local element i from the memory of the stage and stores it in the packet header vector, the j+1th stage obtains the local element i from the packet header vector, determines the weighted value of the local element i and stores the weighted value of the local element i in the packet header vector, the j+2th stage aggregates the local element i and the initial element based on the weighted value of the local element i in the packet header vector and the weighted value of the initial element to obtain the i-th element in the target attention vector, wherein the target attention vector is used by a master node among the multiple nodes for data reasoning.

[0009] The present disclosure further provides a network device, which is applied to a data reasoning system. The data reasoning system includes the network device and a plurality of nodes. The network device includes:

[0010] A receiving module, configured to receive initial attention vectors from the multiple nodes, wherein the initial attention vectors are obtained by each of the multiple nodes performing attention calculations on the context subsequences for which they are responsible, and include multiple initial elements;

[0011] A determination module, configured to determine a local attention vector corresponding to the initial attention vector and stored locally by the network device, wherein the local attention vector includes a plurality of local elements;

[0012] A pipeline stage for performing vector aggregation, wherein the j-1th stage, the jth stage, the j+1th stage and the j+2th stage in the pipeline stage for performing vector aggregation perform aggregation of a local element i with a corresponding initial element, wherein the local element i is stored in a memory of the jth stage, the j-1th stage determines a weighted value of the initial element and stores the weighted value of the initial element in a packet header vector of the network device, the jth stage reads the local element i from the memory of the stage and stores it in the packet header vector, the j+1th stage obtains the local element i from the packet header vector, determines the weighted value of the local element i and stores the weighted value of the local element i in the packet header vector, the j+2th stage aggregates the local element i with the initial element based on the weighted value of the local element i in the packet header vector and the weighted value of the initial element to obtain the i-th element in a target attention vector, wherein the target attention vector is used by a master node among the multiple nodes for data reasoning.

[0013] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any method in the present disclosure when the program is executed by a processor.

[0014] The present disclosure also provides an electronic device, comprising:

[0015] a memory having a computer program stored thereon;

[0016] A processor is used to execute a computer program in a memory to implement the steps of any method in the present disclosure.

[0017] The present disclosure also provides a computer program product, comprising a computer program, which implements the steps of any method in the present disclosure when executed by a processor.

[0018] By adopting the above technical solution, the initial attention vectors of each node are aggregated through the network device to obtain the target attention vector. This aggregation is an online aggregation. Therefore, on the one hand, since the network device is the physical central node in the data reasoning system, its original bandwidth is relatively large and it is physically closer to other nodes. On the other hand, since online aggregation is adopted, the bandwidth bottleneck problem is solved, the latency is reduced, and the system performance is improved. In addition, since the attention aggregation is performed on the network device, the node side does not need to perform attention aggregation, thereby saving storage and computing resources on the node side, and can also make full use of the high bandwidth and low latency advantages of the network device, significantly improving the efficiency of data reasoning. Furthermore, since there is no need for two-to-two communication between nodes, the bandwidth requirement for data transmission is reduced. In addition, in the related art, each level in the hardware pipeline level of the RMT switch has fixed and private computing resources, and memory data cannot be directly shared between different pipeline levels, which limits the implementation of complex operations. By adopting the above-mentioned technical solution, it is possible to meet complex distributed computing needs, implement complex aggregation operations, support in-network computing in more scenarios, and implement cross-level pipeline state reading and writing operations, thereby improving the efficiency of aggregation.

[0019] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0021] Figure 1 It is a flow chart of a data reasoning method according to an embodiment of the present disclosure.

[0022] Figure 2 It is a schematic diagram of pipeline aggregation according to an embodiment of the present disclosure.

[0023] Figure 3 2 is a schematic diagram of a change trend of the number of loops according to an embodiment of the present disclosure.

[0024] Figure 4 is a schematic diagram of batch processing of multiple vector elements in each stage according to an embodiment of the present disclosure.

[0025] Figure 5 It is a schematic diagram of a horizontal expansion aggregation operation according to an embodiment of the present disclosure.

[0026] Figure 6 is a schematic block diagram of a network device according to an embodiment of the present disclosure.

[0027] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0029] Figure 1 The data inference method according to an embodiment of the present disclosure is a flow chart. The data inference method can be applied to a network device in a data inference system, and the data inference system includes a network device and multiple nodes.

[0030] A network device refers to a convergence point in a data reasoning system, or it can also be called a physical central node. For example, in a data reasoning system, multiple nodes communicate through a switch, so the network device is the switch. The switch can be a programmable switch based on a pipelined reconfigurable match-action table (RMT) architecture.

[0031] like Figure 1 As shown, the data inference method may include the following steps S11 to S13.

[0032] In step S11, initial attention vectors are received from multiple nodes, wherein the initial attention vectors are obtained by each of the multiple nodes performing attention calculations on the context subsequences they are responsible for and include multiple initial elements.

[0033] For attention calculation, taking the data reasoning system using the Transformer model as the reasoning model as an example, each node will perform attention calculation on the context subsequence it is responsible for based on the query vector (q), the locally cached key vector (K) and the value vector (V) to obtain the initial attention vector. The context subsequence refers to the subsequence obtained by splitting along the sequence length dimension of the context. The input and output of the reasoning model are collectively referred to as the context.

[0034] In addition, the initial attention vector will include multiple elements, which are called initial elements.

[0035] In step S12, a local attention vector corresponding to the initial attention vector and stored locally in the network device is determined, wherein the local attention vector includes a plurality of local elements.

[0036] For example, multiple nodes may send the initial attention vector to the network device in the form of a data packet, and the data packet may carry the corresponding ID. The network device may determine the local attention vector corresponding to the initial attention vector according to the ID carried in the data packet sent by each node.

[0037] In addition, if the local attention vector does not exist locally in the network device before, the network device can directly save the first received initial attention vector locally as the local attention vector.

[0038] In addition, the local attention vector will include multiple elements, which are called local elements.

[0039] In step S13, the j-1th level, jth level, j+1th level and j+2th level in the pipeline level for performing vector aggregation of the network device perform aggregation of the local element i and the corresponding initial element, wherein the local element i is stored in the memory of the jth level, the j-1th level determines the weighted value of the initial element and stores the weighted value of the initial element in the header vector of the network device, the jth level reads the local element i from the memory of the level and stores it in the header vector, the j+1th level obtains the local element i from the header vector, determines the weighted value of the local element i and stores the weighted value of the local element i in the header vector, and the j+2th level aggregates the local element i with the initial element based on the weighted value of the local element i in the header vector and the weighted value of the initial element to obtain the i-th element in the target attention vector. By aggregating each local element with each initial element in this way, all elements in the target attention vector can be obtained. The target attention vector is used by the master node among the multiple nodes for data reasoning.

[0040] Figure 2is a schematic diagram of pipeline aggregation according to an embodiment of the present disclosure. Assume that the local attention vector a = {a0, a1, ..., a3} and the initial attention vector a ′ ={a0',a1',…,a3'} for aggregation. The aggregation operation adopts weighted average, and the weights are w and w' respectively. Then the aggregation operation is:

[0041] a i ←wa i +w'a i ' (1)

[0042] Assume a0 is located at s j Level, then in s j-1 The first stage obtains w'a0' by looking up the table and saves w'a0' in the Packet Header Vector (PHV) (this operation is called pre-calculation). j Level, from s j a0 is read from the memory of the first stage and stored in the PHV (this operation is called fetch), and then j+1 At the s level, wa0 is obtained by looking up the table and stored in PHV (this operation is called calculation), and then j+2 Level, perform arithmetic summation on w'a0' and wa0 carried in PHV, and write the summation result into s j+2 The local attention vector a = {a0, a1, ..., a3} and the initial attention vector a are updated. ′ ={a0',a1',…,a3'}. In addition, through pre-calculation, extraction, calculation, and update, the vector aggregation operation of MEM_R (memory read) -> TBL_LKP (flow table lookup) -> ALU_EXE (simple arithmetic operation) -> MEM_W (memory write) is realized.

[0043] Figure 2 In the aggregation operation, a0 is transferred from s j Level shift to s j+2 In particular, when s j-1 Level, s j Level, s j+1 Level, s j+2 When one of the stages reaches the last stage of the pipeline stage for performing vector aggregation in the pipeline stage of the network device, the state in the PHV can be looped back to the initial stage of the pipeline stage for performing vector aggregation in the pipeline stage by a loopback (recirculate) method.init For example, if the nth to n+mth stages in the pipeline are used to perform vector aggregation, then the nth stage is the initial stage s init , where n and m are integers greater than or equal to 1. Figure 2 As shown, s j+2 If stage s is the last stage of the pipeline stage for performing vector aggregation, the update of a2 is looped back to the initial stage s of the pipeline stage for performing vector aggregation. init , and finally write the updated result of a2 into s init+1 Level. Note that s init Is fixed and refers to the initial stage of the pipeline stage used to perform vector aggregation. Loopback is a technology used in programmable network devices (such as programmable switches) that allows packets to go back and re-enter the switch pipeline for additional processing after passing through certain processing stages of the switch. This process is usually used to address the problem of limited computing power or resources at each stage in the switch pipeline to implement complex network functions or operations.

[0044] By adopting the above pipeline operation, the aggregation operation of multiple elements in a single vector can be completely pipelined. Figure 2 As shown, when updating, from s j-1 Aggregate a0 from level s j The first stage starts to aggregate a1... From the perspective of the pipeline stage used to perform vector aggregation, under full load, each stage can perform pre-calculation, extraction, calculation and update at the same time, but for different elements. In addition, in the pipeline structure switch, a hardware stage only processes one data packet at a time, and loopback will not cause blocking, so it can ensure line-speed data packet processing.

[0045] In addition, for multiple initial attention vectors, the starting level for aggregating each initial attention vector can be determined in the following manner, that is, obtaining the starting level in the pipeline level for aggregating the current initial attention vector, and using the second level after the starting level as the starting level in the pipeline level for aggregating the next initial attention vector. For example, the aggregation of initial attention vector k is from level s j Starting from level s, the aggregation of the initial attention vector k+1 is j+2 start.

[0046] By adopting the above technical solution, the initial attention vectors of each node are aggregated through the network device to obtain the target attention vector. This aggregation is an online aggregation. Therefore, on the one hand, since the network device is the physical central node in the data reasoning system, its original bandwidth is relatively large and it is physically closer to other nodes. On the other hand, since online aggregation is adopted, the bandwidth bottleneck problem is solved, the latency is reduced, and the system performance is improved. In addition, since the attention aggregation is performed on the network device, the node side does not need to perform attention aggregation, thereby saving storage and computing resources on the node side, and can also make full use of the high bandwidth and low latency advantages of the network device, significantly improving the efficiency of data reasoning. Furthermore, since there is no need for two-to-two communication between nodes, the bandwidth requirement for data transmission is reduced. In addition, in the related art, each level in the hardware pipeline level of the RMT switch has fixed and private computing resources, and memory data cannot be directly shared between different pipeline levels, which limits the implementation of complex operations. By adopting the above-mentioned technical solution, it is possible to meet complex distributed computing needs, implement complex aggregation operations, support in-network computing in more scenarios, and implement cross-level pipeline state reading and writing operations, thereby improving the efficiency of aggregation.

[0047] In some embodiments, the initial attention vectors of multiple nodes may be aggregated on multiple network devices, for example, one network device aggregates the initial attention vectors of some nodes. Then, the data reasoning method according to the embodiment of the present disclosure may also include: setting the number of aggregations performed on each network device according to the flat interval of the number of loops.

[0048] Assume that n agg Indicates the number of aggregation operations performed on the network device, n stage Indicates the number of stages used for vector aggregation in the hardware pipeline stage, n a Indicates the number of levels occupied by the vector a to be aggregated. If n a <n stage , then use the update method described above, n agg In theory, it can be infinite. But in practice, as n agg The increase of the number of loops n recircle It is not a linear growth, but a Figure 3 Therefore, when planning the number of aggregation operations performed by each network device, n agg fall into Figure 3 The flat intervals shown, especially the first stable interval, do not trigger loops, that is, the throughput and latency of network devices will not be affected. In addition, by reducing n a , the length of the flat interval can be extended.

[0049] By adopting the above technical solution, the number of loops can be reduced, the aggregation delay can be reduced, and the aggregation performance can be improved.

[0050] In some embodiments, the data reasoning method according to the embodiment of the present disclosure may further include: aggregating the multiple local elements and the multiple initial elements in parallel in a pipeline stage for performing vector aggregation. The specific process of aggregation is the same as the process described above.

[0051] That is, if each level has sufficient table lookup and logic resources, the level can support multiple parallel update operations, so vertical expansion can be achieved by batch processing multiple vector elements in each level. Figure 4 As shown, four groups of pre-calculation, extraction, calculation and update operations are performed simultaneously in each stage.

[0052] By adopting the above technical solution, it is possible to increase the system capacity through vertical expansion, that is, it is possible to support the aggregation of longer vectors and achieve the scalability of aggregation.

[0053] In some embodiments, the data inference method according to the embodiment of the present disclosure may further include: dividing the initial attention vector and the local attention vector into vector segments of preset lengths respectively; and sequentially aggregating the local elements and the initial elements in each vector segment using a pipeline stage for performing vector aggregation.

[0054] The above-mentioned embodiment is to expand in the vertical direction, but due to the hardware resources of the network equipment, the size of each batch processing cannot be infinite. Therefore, when the length of the vector exceeds the capacity of a single pipeline, horizontal expansion can be achieved through looping.

[0055] For example, both the initial attention vector and the local attention vector can be divided into several vector segments of fixed length, each vector segment containing a number of elements. In addition, the number of vector segments into which the initial attention vector is divided is the same as the number of vector segments into which the local attention vector is divided, and the number of elements in the vector segments of the initial attention vector is the same as the number of elements in the vector segments of the local attention vector. Each stage can uniquely identify the elements in the vector by the vector ID and the vector segment index. For example, each stage in the pipeline stage used to perform vector aggregation uniquely identifies the local elements and the initial elements by the vector IDs of the initial attention vector and the local attention vector and the segment index of the vector segment, thereby ensuring that the elements in the memory are read and written correctly. When performing an aggregation operation, the same as the process of the aggregation operation described above is used to first determine the starting level s of the first element of the 0th vector segment. start , then from s startThe pipeline starts to perform the aggregation of vector segment 0 at the stage. If it reaches the end of the pipeline stage used to perform vector aggregation, it will loop back. After updating vector segment 0, it will immediately update vector segment 1 until all vector segments are updated. Figure 5 As shown in the figure, the network device can support vector aggregation of length 16 by only four horizontal loops. In addition, Figure 5 It is assumed that start =s init .

[0056] By adopting the above technical solution, the system capacity can be improved through horizontal expansion, that is, it can support the aggregation of longer vectors and achieve the scalability of aggregation.

[0057] Figure 6 The present invention is a schematic block diagram of a network device according to an embodiment of the present invention. The network device can be applied to a data reasoning system, which includes the network device and multiple nodes.

[0058] like Figure 6 As shown, the network device 60 may include: a receiving module 61, used to receive an initial attention vector from multiple nodes, wherein the initial attention vector is obtained by each of the multiple nodes performing attention calculation on the context subsequence for which they are responsible, and includes multiple initial elements; a determining module 62, used to determine a local attention vector corresponding to the initial attention vector stored locally in the network device, wherein the local attention vector includes multiple local elements; a pipeline stage 63 for performing vector aggregation, wherein the j-1th stage, the jth stage, the j+1th stage and the j+2th stage of the pipeline stage for performing vector aggregation perform aggregation of the local element i with the corresponding initial element, wherein the local element i The j-1st level determines the weighted value of the initial element and stores the weighted value of the initial element in the packet header vector of the network device. The j-1st level reads the local element i from the memory of the level and stores it in the packet header vector. The j+1st level obtains the local element i from the packet header vector, determines the weighted value of the local element i and stores the weighted value of the local element i in the packet header vector. The j+2nd level aggregates the local element i and the initial element based on the weighted value of the local element i in the packet header vector and the weighted value of the initial element to obtain the i-th element in the target attention vector, wherein the target attention vector is used by the main node among the multiple nodes for data reasoning.

[0059] By adopting the above technical solution, the initial attention vectors of each node are aggregated through the network device to obtain the target attention vector. This aggregation is an online aggregation. Therefore, on the one hand, since the network device is the physical central node in the data reasoning system, its original bandwidth is relatively large and it is physically closer to other nodes. On the other hand, since online aggregation is adopted, the bandwidth bottleneck problem is solved, the latency is reduced, and the system performance is improved. In addition, since the attention aggregation is performed on the network device, the node side does not need to perform attention aggregation, thereby saving storage and computing resources on the node side, and can also make full use of the high bandwidth and low latency advantages of the network device, significantly improving the efficiency of data reasoning. Furthermore, since there is no need for two-to-two communication between nodes, the bandwidth requirement for data transmission is reduced. In addition, in the related art, each level in the hardware pipeline level of the RMT switch has fixed and private computing resources, and memory data cannot be directly shared between different pipeline levels, which limits the implementation of complex operations. By adopting the above-mentioned technical solution, it is possible to meet complex distributed computing needs, implement complex aggregation operations, support in-network computing in more scenarios, and implement cross-level pipeline state reading and writing operations, thereby improving the efficiency of aggregation.

[0060] Optionally, if any one of the j-1th stage, the jth stage, the j+1th stage and the j+2th stage is the last stage of the pipeline stage for performing vector aggregation, the state in the packet header vector is looped back to the initial stage of the pipeline stage for performing vector aggregation by means of a loopback.

[0061] Optionally, the plurality of local elements and the plurality of original elements are aggregated in parallel in the pipeline stage 63 for performing vector aggregation.

[0062] Optionally, the pipeline stage 63 for performing vector aggregation is further configured to:

[0063] The initial attention vector and the local attention vector are respectively divided into vector segments of preset lengths;

[0064] The local elements and the original elements in each vector segment are sequentially aggregated using the pipeline stage for performing vector aggregation.

[0065] Optionally, each stage in the pipeline stage for performing vector aggregation uniquely determines the local element and the initial element by the vector IDs of the initial attention vector and the local attention vector and the segment index of the vector segment.

[0066] Optionally, the initial attention vector is aggregated on a plurality of the network devices, and the network device 60 further includes a setting module, configured to:

[0067] The number of times the aggregation is performed on each of the network devices is set according to a flat interval of the number of loopbacks.

[0068] The specific implementation manner of the operations performed by each module in the network device according to the embodiment of the present disclosure has been described in detail in the relevant method and will not be repeated here.

[0069] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any method in the present disclosure when the program is executed by a processor.

[0070] The present disclosure also provides an electronic device, comprising:

[0071] a memory having a computer program stored thereon;

[0072] A processor is used to execute a computer program in a memory to implement the steps of any method in the present disclosure.

[0073] The present disclosure also provides a computer program product, comprising a computer program, which implements the steps of any method in the present disclosure when executed by a processor.

[0074] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0075] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned data reasoning method. The memory 702 is used to store various types of data to support the operation of the electronic device 700. For example, these data may include instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, messages sent and received, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent via the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, so the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module.

[0076] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned data reasoning method.

[0077] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned data reasoning method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned data reasoning method.

[0078] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a processor. When the computer program is executed by the processor, the steps of the above-mentioned data reasoning method are implemented.

[0079] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0080] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0081] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A data reasoning method, characterized in that: A network device applied to a data reasoning system, wherein the data reasoning system comprises the network device and a plurality of nodes, and the data reasoning method comprises: Receiving initial attention vectors from the multiple nodes, wherein the initial attention vectors are obtained by each of the multiple nodes performing attention calculations on the context subsequences for which they are responsible, and include multiple initial elements; Determine a local attention vector stored locally by the network device and corresponding to the initial attention vector, wherein the local attention vector includes a plurality of local elements; The j-1th, jth, j+1th and j+2th stages of the pipeline stage for performing vector aggregation of the network device perform aggregation of the local element i and the corresponding initial element, wherein the local element i is stored in the memory of the jth stage, the j-1th stage determines the weighted value of the initial element and stores the weighted value of the initial element in the packet header vector of the network device, the jth stage reads the local element i from the memory of the stage and stores it in the packet header vector, the j+1th stage obtains the local element i from the packet header vector, determines the weighted value of the local element i and stores the weighted value of the local element i in the packet header vector, the j+2th stage aggregates the local element i and the initial element based on the weighted value of the local element i in the packet header vector and the weighted value of the initial element to obtain the i-th element in the target attention vector, wherein the target attention vector is used by a master node among the multiple nodes for data reasoning.

2. The method according to claim 1, characterized in that The method further comprises: If any one of the j-1th stage, the jth stage, the j+1th stage and the j+2th stage is the last stage of the pipeline stage for performing vector aggregation, the state in the packet header vector is looped back to the initial stage of the pipeline stage for performing vector aggregation in a loop-back manner.

3. The method according to claim 1, characterized in that The method further comprises: The plurality of local elements and the plurality of original elements are aggregated in parallel in the pipeline stage for performing vector aggregation.

4. The method according to claim 1, characterized in that: The method further comprises: The initial attention vector and the local attention vector are respectively divided into vector segments of preset lengths; The local elements and the original elements in each vector segment are sequentially aggregated using the pipeline stage for performing vector aggregation.

5. The method according to claim 4, characterized in that The method further comprises: Each stage in the pipeline stage for performing vector aggregation uniquely determines the local element and the initial element through the vector IDs of the initial attention vector and the local attention vector and the segment index of the vector segment.

6. The method according to claim 1, characterized in that Aggregating the initial attention vector on a plurality of the network devices, the method further comprises: The number of times the aggregation is performed on each of the network devices is set according to a flat interval of the number of loopbacks.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The starting stage in the pipeline stage for aggregating the current initial attention vector is obtained, and the second stage after the starting stage is used as the starting stage in the pipeline stage for aggregating the next initial attention vector.

8. A network device, characterized in that: Applied to a data reasoning system, the data reasoning system comprises the network device and a plurality of nodes, the network device comprises: A receiving module, configured to receive initial attention vectors from the multiple nodes, wherein the initial attention vectors are obtained by each of the multiple nodes performing attention calculations on the context subsequences for which they are responsible, and include multiple initial elements; A determination module, configured to determine a local attention vector corresponding to the initial attention vector and stored locally by the network device, wherein the local attention vector includes a plurality of local elements; A pipeline stage for performing vector aggregation, wherein the j-1th stage, the jth stage, the j+1th stage and the j+2th stage in the pipeline stage for performing vector aggregation perform aggregation of a local element i with a corresponding initial element, wherein the local element i is stored in a memory of the jth stage, the j-1th stage determines a weighted value of the initial element and stores the weighted value of the initial element in a packet header vector of the network device, the jth stage reads the local element i from the memory of the stage and stores it in the packet header vector, the j+1th stage obtains the local element i from the packet header vector, determines the weighted value of the local element i and stores the weighted value of the local element i in the packet header vector, the j+2th stage aggregates the local element i with the initial element based on the weighted value of the local element i in the packet header vector and the weighted value of the initial element to obtain the i-th element in a target attention vector, wherein the target attention vector is used by a master node among the multiple nodes for data reasoning.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images