Method and device for fusing calculation and communication of AI chip and medium

Abstract

The invention discloses an AI chip calculation and communication fusion method and device and a medium, and relates to the technical field of calculation and communication fusion, and the method comprises the steps: collecting on-chip link occupation data, reading a calculation graph, determining fusion edges, carrying out unified fixed-point representation, forming a fusion edge list, and generating an in-band instruction frame, a frame field template and a pure forwarding reference path; according to the fusion edge list, the frame field template and the pure forwarding reference path, interconnecting and constructing a tree-shaped path on the AI chip, arranging jump points, generating and issuing a matching and convergence rule table, and obtaining an intermediate result when the first jump point processes the in-band instruction frame; before a crossbar switch of the AI chip, in-band instruction frame data and the intermediate result are fused in transit at each jump point according to the matching and convergence rule table, the fusion result serves as a new intermediate result and continues to be forwarded along the tree-shaped path to a convergence position to complete convergence and format recovery, and an operation mark is recorded. And the passing efficiency and the energy consumption performance are improved.

Description

Technical Field

[0001] This invention relates to the field of computing and communication convergence technology, and in particular to a method, device and medium for the convergence of computing and communication in an AI chip. Background Technology

[0002] Artificial intelligence chips execute parallel operator graphs. A common approach is to use hierarchical routing on the on-chip interconnect to carry tensor transmission. Element-wise operations and format conversions are performed on the operator side. On the communication side, shortest path-based path orchestration, port arbitration, and backpressure control are used. Data is carried back and forth between the source and the target in message frames. If necessary, merging and format restoration are performed at the target end. To facilitate compilation and runtime management, the conventional process first reads the computation graph, constructs transmission requirements according to the correlation between operations and data, and then evaluates link occupancy under fixed observation periods or typical loads to form a baseline for subsequent scheduling. Numerical representation often adopts a unified fixed-point or mixed-precision scheme. Path selection is generally based on topological distance and deterministic rules. Reference paths are generated for different data streams when necessary for experimental reproduction and comparison. Inside the on-chip router, data traverses frame by frame through the cross-interconnect matrix. The frame header field usually carries identifiers such as operation type, target set, and sequence number for service forwarding control. Merging is mostly completed on the target side or near the storage side, and consistency comparisons are performed with the reference data stream to maintain functional semantics and temporal stability.

[0003] Under large-scale multi-source aggregation loads, the granularity of online consistency verification and reversible start-stop operations for target-side merging processes is limited. It is difficult to quickly locate differences and perform local rollbacks during runtime. At the same time, when computation and communication are relatively separated, intermediate data on the path needs to be injected and popped multiple times, and transmission overhead and energy consumption are easy to accumulate. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for integrating computing and communication in AI chips to solve the problems of difficulty in quickly verifying online consistency under large-scale multi-source aggregation and insufficient granularity of reversible start-stop according to tree path.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for integrating computing and communication in an AI chip, comprising, Collect on-chip link occupancy data and read the computation graph, determine the fusion edges and unify the fixed point representation, construct the fusion edge list, and generate in-band instruction frames, frame field templates and pure forwarding reference paths; Based on the fusion edge list, frame field template and pure forwarding reference path, a tree-like path is constructed on the on-chip interconnect of the AI ​​chip and jump points are arranged. A matching and convergence rule table is generated and distributed. When the first jump point processes the in-band instruction frame, the intermediate result is obtained. Before the cross switch of the AI ​​chip, the in-band instruction frame data and intermediate results are fused in transit at each hop point according to the matching and convergence rule table. The fused result is forwarded along the tree path as a new intermediate result until the convergence position is reached to complete convergence and format restoration, and the running marker is recorded. The fusion result of the format restoration and the running mark are compared for consistency on the pure forwarding reference path. If the consistency is not met, the in-transit fusion is suspended. If the consistency is met, the in-transit fusion is resumed in sequence according to the tree path.

[0007] As a preferred embodiment of the fusion method for computing and communication in the AI ​​chip of the present invention, the steps of collecting on-chip link occupancy data and reading the computation graph, determining fusion edges and unifying fixed-point representations to form a fusion edge list are as follows: Set a fixed observation period on the on-chip interconnect of the AI ​​chip, start the occupancy counter of all links, and sample occupancy data at fixed time intervals to obtain a unified occupancy baseline; Within the same observation period, read the directed connectivity of the computation graph, retain the operation types that satisfy element-by-element processing and do not change the geometric meaning of the tensor, and determine the input edges that all point to the same operation as fusion edges. For each fusion edge, record the source node, the pointing node, the operation type, and the baseline association information between the fusion edge and the observation period to form a fusion edge list. The tensor numerical representations involved in the fusion edge list are unified into fixed fixed-point representations. Real numbers are transformed to the target precision according to a fixed scale, and saturation pruning is performed within the allowable range. The truncation exceeding the allowable range is the maximum representable value, and the truncation below the allowable range is the minimum representable value.

[0008] As a preferred embodiment of the fusion method for computing and communication in the AI ​​chip described in this invention, the specific steps for generating the in-band instruction frame, frame field template, and pure forwarding reference path are as follows: Based on the fusion edge list, an in-band instruction frame is generated for each fusion edge, and a globally reusable frame field template is generated at the same time. The in-band instruction frame consists of a field part and a data part. The field part includes an operation type field, a fusion identifier field, a number field to be reached, a sequence number field, a target set field, and a fixed point label field. The data part is written with tensor data encoded according to a fixed fixed point representation. On-chip interconnection adopts a fixed shortest routing strategy. When there are several paths of equal length, the pure forwarding reference path is uniquely selected in lexicographical order with row coordinates as the priority and column coordinates as the second priority.

[0009] As a preferred embodiment of the method for integrating computing and communication in the AI ​​chip described in this invention, the specific steps of constructing a tree-like path and arranging jump points on the on-chip interconnect of the AI ​​chip are as follows: According to the fusion edge list, fusion edges that point to the same node are grouped into a fusion domain. From the target set field of the in-band instruction frame, a unique coordinate is selected as the convergence position of the fusion domain in lexicographical order with row coordinates as the priority and column coordinates as the second priority. The pure forwarding reference path is used as the baseline path of the fusion domain. Within the on-chip interconnect coordinate system of the AI ​​chip, collect the coordinates of all source nodes and convergence positions in the fusion domain. Take the median coordinate for both row and column coordinates. Use the median coordinate and the connected position of the median coordinate as candidate positions. Calculate the sum of the shortest route hops between each candidate position and all source nodes. Take the candidate position with the smallest sum of shortest route hops as the first hop point. Taking the first hop as the current convergence location, sort the source nodes that have not yet been connected from near to far according to the shortest route distance to the current convergence location and connect them. For each source node, connect it to the current convergence location along the fixed shortest route. If there are candidate routes of equal length, select them uniquely according to the lexicographical order of the pure forwarding reference path. After the current source node is connected, keep the current convergence location unchanged and continue to process source nodes until all source nodes in the fusion domain are connected, and obtain a tree-like path from each source node to the convergence location.

[0010] As a preferred embodiment of the fusion method for computation and communication of the AI ​​chip described in this invention, the specific steps for generating and distributing the matching and convergence rule table, and obtaining intermediate results when processing the in-band instruction frame at the first jump point, are as follows: For each hop on the tree path, generate matching and convergence rule entries for the fusion domain based on the frame field template, including matching conditions, convergence conditions, output behavior, and forwarding targets; When the first hop receives an in-band instruction frame belonging to the fusion domain for the first time, it performs the processing specified by the operation type field on each element of the arriving data according to the matching and convergence rule table, outputs intermediate results, and registers the arrival record and sequence number at the hop.

[0011] As a preferred embodiment of the computation and communication fusion method of the AI ​​chip described in this invention, the steps of fusing in-band instruction frame data and intermediate results at each hop point according to the matching and convergence rule table, and then forwarding the fused result as a new intermediate result along the tree-like path, are as follows: Each hop receives an in-band instruction frame before the cross switch. It checks the operation type field and the fusion identifier field according to the matching and convergence rule table. If the in-band instruction frame does not match, it forwards the data directly according to the fixed shortest route without triggering in-transit fusion. If the in-band instruction frame matches, it reads the intermediate results registered by the hop in the fusion domain and the data that has arrived this time, and registers and compares them according to the sequence number field. If the sequence number has been registered, the data is discarded. If the sequence number has not been registered, it is recorded as a valid arrival. Before the cross switch, perform element-wise processing, using the registered intermediate result as the left input and the data part of the in-band instruction frame as the right input. Calculate the temporary result element by element according to the operation type field, and then convert the temporary result into a unified fixed fixed-point representation and limit it within the allowed range to obtain a new intermediate result that overwrites the original registration. The number of valid arrivals is compared with the number of data segments to be arrived. When the number of valid arrivals reaches the number shown in the number of data segments to be arrived, and the sequence number field has no duplicates in the fusion domain, the current jump point is determined to have converged. At this time, only an in-band instruction frame is constructed and output. The field part retains the original operation type field and fusion identifier field, and the data part is written with the new intermediate result. It is forwarded according to the next coordinate of the tree path. If the current jump point has not converged, it continues to wait for an in-band instruction frame and repeats the process.

[0012] As a preferred embodiment of the fusion method for computation and communication of the AI ​​chip described in this invention, the steps of completing convergence and format recovery at the convergence position and recording the running marker are as follows: When the tree path delivers a single new intermediate result to the convergence position, the convergence position converges, and then format restoration is performed. Based on the fixed scale, the fixed point representation is restored to the target representation required downstream as the fusion result. At the same time, the running flag is recorded. The fusion result and the running flag are saved and output together.

[0013] As a preferred embodiment of the fusion method for computing and communication in the AI ​​chip described in this invention, the specific steps of performing a consistency comparison between the fusion result of format recovery and the running flag on the pure forwarding reference path are as follows: For in-band instruction frames that belong to the same fusion identifier field and sequence number field as the fusion result, forward them once along the pure forwarding reference path, obtain the reference output at the convergence position, and align the reference output with the format-restored fusion result according to the fusion identifier field and sequence number field. For all aligned reference outputs and format restoration fusion results in this round, calculate the consistency result for this round, and compare the consistency result of this round with the consistency result of the same fusion identifier field archived in the previous round.

[0014] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the fusion method of computing and communication of the AI ​​chip as described in the first aspect of the present invention.

[0015] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the fusion method of computing and communication of the AI ​​chip as described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: by performing in-band instruction frame data and intermediate results in each hop according to the matching and convergence rule table, on-site element-by-element processing and single intermediate results are achieved along the way, improving throughput and energy consumption. By performing consistency comparison between the fused results of format restoration and the running flag on the pure forwarding reference path, consistency comparison and reversible start and stop according to the tree path are achieved, thus achieving the beneficial effect of balancing correctness and throughput under dynamic load. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the method for integrating computing and communication in AI chips.

[0019] Figure 2 A flowchart for generating in-band instruction frames, frame field templates, and pure forwarding reference paths.

[0020] Figure 3 A flowchart for constructing and arranging jump points for a tree-like path.

[0021] Figure 4 This is a flowchart for merging during the jump point. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a method for integrating computing and communication in an AI chip, comprising the following steps: S1. Collect on-chip link occupancy data and read the computation graph, determine the fusion edges and unify the fixed-point representation, form a fusion edge list, and generate in-band instruction frames, frame field templates and pure forwarding reference paths.

[0026] A fixed observation period is set on the on-chip interconnect of the AI ​​chip, and the occupancy counters of all links are started. Occupancy data is sampled at fixed time intervals. The arithmetic average of all occupancy data of each link within the observation period is calculated to obtain the time average occupancy of each link. The arithmetic average of the time average occupancy of all links is calculated to obtain a unified occupancy baseline.

[0027] A fixed observation period refers to a continuous time segment defined by the global clock of the AI ​​chip on the on-chip interconnect.

[0028] Within the same observation period, the directed connections of the computation graph are read. Operation types that satisfy element-by-element processing and do not change the geometric meaning of the tensor are retained, including addition, minimum value, maximum value, right shift quantization, format conversion, and corrected linear truncation. Input edges that point to the same subsequent operation are identified as fusion edges, and directed connections that change the read / write order or shape are excluded. For each fusion edge, the source node, the pointing node, the operation type, and the baseline occupancy information corresponding to the observation period are recorded to form a fusion edge list.

[0029] The tensor values ​​involved in the fusion edge list are unified into fixed-point representations, and the real numbers are transformed to the target precision according to a fixed scale. Then, saturation pruning is performed within the allowable range. The truncation exceeding the allowable range is the maximum representable value, and the truncation below the allowable range is the minimum representable value. The fixed-point labels are written into the corresponding items of the fusion edge list.

[0030] Tensor numerical representation refers to the specific encoding method used when storing and transmitting each tensor element on the fused edge on-chip.

[0031] Fixed-point representation refers to the uniform adoption of the signed Q3.12 fixed-point format (total bit width of sixteen bits: one sign bit, three integer bits, and twelve decimal bits), with a fixed allowable range. .

[0032] A fixed scale refers to the smallest increment that is uniquely determined by twelve decimal places, namely 1 / 4096.

[0033] Target precision refers to the discrete precision after projecting the original real number onto a fixed scale.

[0034] Fixed-point labels refer to the tag fields written into the fusion edge list and frame field template.

[0035] Based on the fusion edge list, an in-band instruction frame is generated for each fusion edge, and a globally reusable frame field template is generated at the same time. The in-band instruction frame consists of a field part and a data part. The field part includes an operation type field, a fusion identifier field, a number field to be reached, a sequence number field, a target set field, and a fixed-point label field. The data part contains tensor data encoded according to a fixed fixed-point representation.

[0036] The operation type field is directly mapped to the specific name of one of the six operation types.

[0037] The fusion identifier field consists of a logical identifier pointing to the node (a unique and stable number within the same compilation) combined with a unique code pointing to the fusion edge topology order of the node.

[0038] The number of merged edges should be obtained from the data segment pointing to the node.

[0039] The serial number field monotonically increases according to the topological order.

[0040] The target set field refers to the set of coordinates to be written to the node.

[0041] The fixed-point label field refers to writing a unified, fixed-point representation description.

[0042] On-chip interconnection adopts a fixed shortest routing strategy. When multiple paths of equal length exist, the pure forwarding reference path is uniquely selected in lexicographical order with row coordinates as the priority and column coordinates as the second priority. The occupied baseline, fusion edge list, in-band instruction frames, frame field templates and pure forwarding reference paths are archived in groups.

[0043] S2. Based on the fusion edge list, frame field template and pure forwarding reference path, construct a tree path and arrange jump points on the on-chip interconnect of the AI ​​chip, generate and issue matching and convergence rule tables, and obtain intermediate results when the first jump point processes the in-band instruction frame.

[0044] According to the fusion edge list, fusion edges that point to the same node are grouped into a fusion domain. From the target set field of the in-band instruction frame, a unique coordinate is selected as the convergence position of the fusion domain in lexicographical order with row coordinates as the priority and column coordinates as the second priority. The pure forwarding reference path is used as the baseline path of the fusion domain.

[0045] Within the on-chip interconnect coordinate system of the AI ​​chip, collect the coordinates of all source nodes and convergence positions in the fusion domain. Take the median coordinate for both row and column coordinates. Use the median coordinate and the connected position of the median coordinate as candidate positions. Calculate the sum of the shortest route hops between each candidate position and all source nodes. Take the candidate position with the smallest sum of shortest route hops as the first hop point. If the sum of shortest route hops is tied, select the earlier candidate position according to the lexicographical order of row coordinate first and column coordinate second best. Write the first hop point into the starting point of the tree path in the fusion domain.

[0046] Taking the first hop as the current aggregation location, sort the source nodes that have not yet been connected from near to far according to the shortest route distance to the current aggregation location, and connect them in the order of sorting. For each source node, connect it to the current aggregation location along the fixed shortest route. If there are candidate routes of equal length, select them uniquely according to the lexicographical order of the pure forwarding reference path. After the current source node is connected, keep the current aggregation location unchanged and continue to process source nodes until all source nodes in the fusion domain are connected, and obtain a unique acyclic tree path from each source node to the convergence location.

[0047] For each hop on the tree path, a matching and convergence rule table entry is generated for the fusion domain based on the frame field template, including matching conditions, convergence conditions (convergence conditions for any hop in the fusion domain), output behavior, and forwarding target.

[0048] The matching condition is that the operation type field and the fusion identifier field are consistent.

[0049] The convergence condition is that the number of data segments that should be reached has been met and the sequence number field is not repeated in the fusion domain.

[0050] The output behavior refers to processing the data of the in-band instruction frame and the data already recorded at the jump point element by element according to the operation type field to form an intermediate result.

[0051] The forwarding target is the next coordinate on the tree path.

[0052] The matching and convergence rule entries, along with the corresponding in-band instruction frames, are sent to the forwarder.

[0053] When the first hop receives an in-band instruction frame belonging to the fusion domain for the first time, it performs the processing specified by the operation type field on the arriving data element by element according to the matching and convergence rule table, outputs intermediate results, and registers the arrival record and sequence number at the hop. Subsequently, for each frame received by the hop, it processes the arriving data element by element with the registered intermediate results and the same operation type field to obtain updated intermediate results and overwrite the registration. After each update, the current intermediate results are used as the payload of the in-band instruction frame and forwarded along the next coordinate of the tree path until the convergence position is reached.

[0054] S3. Before the cross switch of the AI ​​chip, the in-band instruction frame data and intermediate results are fused in transit at each jump point according to the matching and convergence rule table. The fused result is forwarded along the tree path as a new intermediate result until the convergence position is reached to complete the convergence and format restoration, and the running mark is recorded.

[0055] Each hop receives an in-band instruction frame before the cross switch. It checks the operation type field and the fusion identifier field against the matching and convergence rule table. If the in-band instruction frame does not match, it forwards the data directly according to the fixed shortest route without triggering in-transit fusion. If the in-band instruction frame matches, it reads the intermediate results registered by the hop in the fusion domain and the data that has arrived this time, and registers and compares them according to the sequence number field. If the sequence number has been registered, the data is discarded. If the sequence number has not been registered, it is recorded as a valid arrival.

[0056] A cross switch refers to a port cross interconnect matrix within an AI chip's on-chip interconnect router, used to establish a one-to-one connection between input ports and output ports within one clock cycle based on the arbitration result. The term "cross switch" refers to the logical processing position before data traverses the cross interconnect matrix upon entering the port.

[0057] Before the cross switch, perform element-wise processing, using the registered intermediate result as the left input and the data part of the in-band instruction frame as the right input. Calculate the temporary result element by element according to the operation type field, and then convert the temporary result into a unified fixed fixed-point representation and limit it to the allowed range to obtain a new intermediate result that overwrites the original registration.

[0058] The number of valid arrivals is compared with the number of segments to be arrived. When the number of valid arrivals reaches the number shown in the number of segments to be arrived, and the sequence number field has no duplicates in the fusion domain, the current jump point is considered to have converged. At this time, only one in-band instruction frame is constructed and output. The field part retains the original operation type field and fusion identifier field, and the data part is written with the new intermediate result. It is forwarded according to the next coordinate of the tree path. If the current jump point has not converged, it continues to wait for subsequent in-band instruction frames and repeats the process.

[0059] When the tree path delivers a single new intermediate result to the convergence position, the convergence position converges according to the same rules as the jump point. Then, format restoration is performed, and the fixed-point representation is restored to the target representation required downstream according to the fixed scale as the fusion result. At the same time, the running flag is recorded. The fusion result and the running flag are saved and output together.

[0060] Runtime markers refer to the objective record of the runtime status of each fusion domain along the tree-like path, including the arrival record overview, the deduplication overview of the sequence number field, and the quantization saturation occurrence overview of fixed point representation.

[0061] S4. Perform a consistency comparison between the format-restored fusion result and the running flag on the pure forwarding reference path. If the consistency is not met, pause the in-transit fusion. If the consistency is met, resume the in-transit fusion in sequence according to the tree path.

[0062] For in-band instruction frames that belong to the same fusion identifier field and sequence number field as the fusion result, forward them once along the pure forwarding reference path, obtain the reference output at the convergence position, and align the reference output with the format-restored fusion result according to the corresponding fusion identifier field and sequence number field.

[0063] For each pair of aligned reference outputs and format restorations, the absolute value of the difference between the values ​​of the reference outputs and format restorations is taken. The larger value between the absolute value of the reference output and the smallest non-zero amplitude corresponding to the fixed scale is used as the normalization denominator to obtain the normalized deviation of the fusion results of the reference outputs and format restorations. For all the aligned reference outputs and format restorations in this round, the arithmetic mean of the normalized deviations is taken to obtain the consistency result of this round.

[0064] The consistency results of this round are compared with the consistency results of the same fusion identifier field archived in the previous round. If the consistency results of this round are not lower than the consistency results archived in the previous round, or if the sequence number field of the running flag is repeated or the quantization saturation increases, then the fusion identifier field is paused for in-transit fusion. The hop point that is closest to the convergence position and is in the in-transit fusion state is selected according to the tree path. The hop point is changed to only forward and not perform element-by-element processing according to the matching and convergence rule table. The other hop points remain unchanged.

[0065] If the consistency result of this round is lower than the consistency result of the previous round and the running flag does not show repeated increases in the display sequence number field or increases in quantization saturation, then restore the in-transit fusion in sequence according to the tree path. From the convergence position to the source node, select the nearest jump point that is in a paused state, restore the element-by-element processing of the jump point before the cross switch, and perform quantization and saturation pruning according to the fixed fixed point representation. The remaining jump points remain unchanged.

[0066] After pausing or resuming in-transit fusion, record the general overview of the consistency results and the running markers for this round, and record the current status of hop-only forwarding or in-transit fusion. This record, along with the fusion edge list, in-band instruction frames, and frame field templates, forms an archive for continued use in the next fixed observation period.

[0067] This embodiment also provides a computer device applicable to the fusion method of computing and communication of AI chips, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the fusion method of computing and communication of AI chips as proposed in the above embodiment.

[0068] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0069] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the fusion method for computing and communication of an AI chip as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0070] In summary, this invention improves throughput and energy consumption by: fusing in-band instruction frame data and intermediate results at each hop point according to the matching and convergence rule table, achieving on-site element-by-element processing and single intermediate result transmission along the route; and by performing consistency comparison between the fused results with format restoration and the running flag on the pure forwarding reference path, achieving consistency comparison and reversible start-stop according to the tree path, thus achieving the beneficial effect of balancing correctness and throughput under dynamic load.

[0071] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A fusion method of calculation and communication of an AI chip, characterized in that: comprise, Collecting on-chip link occupation data and reading a computation graph, determining fusion edges and uniform fixed-point representation, constituting a fusion edge list, generating an in-band instruction frame, a frame field template and a pure forwarding reference path; According to the fusion edge list, the frame field template and the pure forwarding reference path, a tree-shaped path is constructed on the on-chip interconnection of the AI chip and the jump points are arranged, a matching and convergence rule table is generated and delivered, and an intermediate result is obtained when the first jump point processes the in-band instruction frame; Before the crossbar switch of the AI chip, the in-band instruction frame data and the intermediate result are fused on the way at each jump point according to the matching and convergence rule table, the fusion result is taken as a new intermediate result and continues to be forwarded along the tree-shaped path, convergence and format recovery are completed at the convergence position, and a running mark is recorded; The format-recovered fusion result and the running mark are compared for consistency on the pure forwarding reference path, and if the comparison does not meet the requirements, the on-the-way fusion is suspended, and if the comparison meets the requirements, the on-the-way fusion is restored in sequence according to the tree-shaped path. 2.The AI chip's computing and communication fusion method of claim 1, wherein: The specific steps of collecting on-chip link occupation data and reading a computation graph, determining fusion edges and uniform fixed-point representation, and constituting a fusion edge list are as follows, A fixed observation period is set on the on-chip interconnection of the AI chip, and the occupation counters of all links are started, and the occupation data is sampled at a fixed time interval to obtain a uniform occupation baseline; The directed connection relationship of the computation graph is read within the same observation period, the operation type that meets the element-by-element processing and does not change the geometric meaning of the tensor is retained, each input edge that points to the same operation is determined as a fusion edge, for each fusion edge, the source node, the pointing node, the operation type and the association information of the fusion edge and the occupation baseline of the observation period are recorded to constitute a fusion edge list; The tensor numerical representation involved in the fusion edge list is unified to a fixed fixed-point representation, the real number is converted to the target precision according to the fixed scale, and saturation clipping is performed within the allowed range, and if the allowed range is exceeded, the maximum representable value is cut off, and if the allowed range is below, the minimum representable value is cut off. 3.The AI chip's computing and communication fusion method of claim 2, wherein: The specific steps of generating an in-band instruction frame, a frame field template and a pure forwarding reference path are as follows, Based on the fusion edge list, an in-band instruction frame is generated for each fusion edge, and a globally reusable frame field template is also generated, the in-band instruction frame is composed of a field part and a data part, the field part includes an operation type field, a fusion identification field, a should-arrive-part field, a serial number field, a target set field and a fixed-point label field, and the data part writes the tensor data encoded in fixed fixed-point representation; On the on-chip interconnection, a fixed shortest routing strategy is used, when there are several equal-length paths, a pure forwarding reference path is uniquely selected in lexicographical order according to the row coordinate priority and the column coordinate secondary priority. 4.The AI chip's computing and communication fusion method of claim 3, wherein: The specific steps of constructing a tree-shaped path and arranging jump points on the on-chip interconnection of the AI chip are as follows, According to the fusion edge list, the fusion edges that point to the same pointing node are grouped into a fusion domain, and from the target set field of the in-band instruction frame, a unique coordinate is selected in lexicographical order according to the row coordinate priority and the column coordinate secondary priority as the convergence position of the fusion domain, and the pure forwarding reference path is taken as the baseline path of the fusion domain; In the on-chip interconnection coordinate system of the AI chip, all source node coordinates and convergence position coordinates of the fusion domain are collected, and the median coordinates are taken for the row coordinates and the column coordinates respectively. The median coordinates and the connected positions one hop away from the median coordinates are taken as candidate positions, the shortest route hop counts between each candidate position and all source nodes are calculated, and the first hop point is taken as the candidate position with the minimum shortest route hop count; The first hop point is taken as the current convergence position, the source nodes not yet connected are sorted according to the shortest route distance from the current convergence position from near to far and connected, for each source node, the shortest route is fixed to the current convergence position, if there is an equal length candidate route, the pure forwarding reference path is selected according to the dictionary order, after the current source node is connected, the current convergence position is kept unchanged, the source nodes are processed continuously until all source nodes of the fusion domain are connected, and the tree-shaped path from each source node to the convergence position is obtained. 5.The AI chip's computing and communication fusion method of claim 4, wherein: The matching and convergence rule table is generated and issued, and the intermediate result is obtained when the first hop point processes the in-band instruction frame, and the specific steps are, For each hop point on the tree-shaped path, the matching and convergence rule table item is generated for the fusion domain according to the frame field template, including the matching condition, the convergence condition, the output behavior and the forwarding target; When the first hop point receives the in-band instruction frame belonging to the fusion domain for the first time, the data arriving according to the matching and convergence rule table is processed according to the processing specified by the operation type field, the intermediate result is output, and the arrival record and the serial number are registered at the hop point. 6.The AI chip's computing and communication fusion method of claim 5, wherein: The in-band instruction frame data and the intermediate result are fused at each hop point according to the matching and convergence rule table, and the fusion result is taken as a new intermediate result and continues to be forwarded along the tree-shaped path, and the specific steps are, Each hop point receives the in-band instruction frame before the crossbar switch, checks the operation type field and the fusion identification field according to the matching and convergence rule table, if the in-band instruction frame does not match, the in-band instruction frame is directly forwarded according to the fixed shortest route and does not trigger the in-route fusion, if the in-band instruction frame matches, the intermediate result registered in the fusion domain and the data arriving this time are read, and the registration and comparison are performed according to the serial number field, if the serial number is registered, the data this time is discarded, if the serial number is not registered, the data this time is taken as an effective arrival; The element-by-element processing is performed before the crossbar switch, the registered intermediate result is taken as the left input, the data part of the in-band instruction frame is taken as the right input, the temporary result is calculated according to the operation type field, then the temporary result is converted into a unified fixed point representation and limited in the allowed range, a new intermediate result is obtained, and the original registration is overwritten. The number of effective arrivals is compared with the number of arrival shares, when the number of effective arrivals reaches the number shown by the number of arrival shares field, and the serial number field has no repetition in the fusion domain, it is judged that the current hop point converges, at this time, only the in-band instruction frame is constructed and output, the field part uses the original operation type field and the fusion identification field, the data part writes the new intermediate result, and is forwarded according to the next coordinate of the tree-shaped path, if the current hop point does not converge, the in-band instruction frame is continued to be waited for and repeated processing. 7.The AI chip's computing and communication fusion method of claim 6, wherein: The convergence to the convergence position is completed and the format is recovered, and the running mark is recorded, and the specific steps are, When the tree path delivers the single new intermediate result to the convergence position, the convergence position performs convergence, then performs format recovery, restores the fixed point representation to the target representation required by the downstream according to the fixed scale, as the fusion result, records the running mark, and saves and outputs the fusion result and the running mark together. 8.The AI chip's computing and communication fusion method of claim 7, wherein: The format-recovered fusion result and the running mark are compared for consistency along the pure forwarding reference path, and the specific steps are as follows, For the in-band instruction frame with the same fusion identifier field and sequence number field as the fusion result, the in-band instruction frame is forwarded once along the pure forwarding reference path, and the reference output is obtained at the convergence position; and the reference output is aligned with the format-recovered fusion result according to the fusion identifier field and the sequence number field. For all the aligned reference outputs and the format-recovered fusion results in the current round, the consistency result of the current round is calculated, and the consistency result of the same fusion identifier field of the previous round is compared with the consistency result of the same fusion identifier field of the previous round. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The computer program is executed by the processor to implement the steps of the fusion method of the calculation and communication of the AI chip according to any one of claims 1-8.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the fusion method of the calculation and communication of the AI chip according to any one of claims 1-8.

Images