Method, computing device, medium and program product for performing vector calculations
By generating small-grained sub-requests and performing vector calculations in the calculation unit, the bus pressure and delay problems caused by multiple data requests in the traditional method are solved, and efficient processing of the size of big data is achieved.
Patent Information
- Application Number
- CN202410653029.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-24
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-05-24
AI Technical Summary
Traditional methods for performing vector computing result in increased bus pressure and significant increase in computational delay when the processor initiates multiple data requests, especially when the source data size is large.
By generating a smaller granularity sub-request, sending the sub-request to the memory and computing unit based on the original request, reading data and performing vector calculations, avoiding multiple sending of data requests, reducing bus pressure and reducing delays.
Effectively reduces bus pressure and significantly reduces calculation delay, especially suitable for situations where source data size is large.
Smart Images

Figure CN118276942B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention generally relate to the field of artificial intelligence technology, and more specifically to a method, a computing device, a computer-readable storage medium, and a computer program product for performing vector calculations. Background Art
[0002] In vector or Single Instruction Multiple Data (SIMD) parallel processors, computing modules are added near the memory to achieve parallel computing near the memory, thereby reducing the delay caused by memory access.
[0003] Conventional methods for performing vector calculations include, for example, assigning source data to a bus and sending it to a control module near a memory, the control module reading the data in the memory as another set of source data, and writing the two sets of data to a computing module near the memory to complete the calculation, and then writing them back to the memory to complete a near-memory operation. In the above conventional methods, the bandwidth of the interconnect bus in the processor is limited. If the size of the vector data used for calculation is relatively small, the source data used for calculation can be assigned to the bus and transmitted to the computing module via the bus to complete the calculation. However, if the size of the vector data used for calculation is relatively large, the processor needs to initiate multiple requests for sending data to "assign the source data directly to the bus", thereby affecting other tasks. In addition, multiple requests to send data will also increase bus pressure. Furthermore, since multiple requests are required to complete the vector operation, the latency of the calculation is greatly increased.
[0004] In summary, the disadvantage of the conventional method for performing vector calculations is that when the source data size for calculation is large, the processor needs to initiate multiple requests to send data, which increases bus pressure and significantly increases calculation delays. Summary of the invention
[0005] The present invention provides a method, a computing device, a computer-readable storage medium and a computer program product for performing vector computing, which can alleviate bus pressure and reduce computing delay even when the source data size is large.
[0006] According to a first aspect of the present invention, a method for performing vector calculation is provided. The method comprises: generating an original request for vector calculation, wherein the original request at least indicates a source address, a vector length, and an operand of a vector to be used for calculation; generating multiple sub-requests based on the original request so as to send the sub-requests to a memory and a computing unit, wherein the granularity of the sub-requests is smaller than the granularity of the original request; reading data in the memory based on the sub-requests to send the data to the computing unit as source data; and performing vector calculation based on the source data and the sub-requests via the computing unit.
[0007] According to a second aspect of the present invention, a computing device is also provided. The computing device comprises: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the computing device can perform the method of the first aspect of the present invention.
[0008] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a machine, the method of the first aspect of the present invention is executed.
[0009] According to a fourth aspect of the present invention, there is further provided a computer program product, comprising a computer program, wherein when the computer program is executed by a machine, the method of the first aspect of the present invention is performed.
[0010] In some embodiments, the source address of the vector is the source address of the vector obtained by each sub-request, the vector length is the length of the vector obtained by each sub-request, the operand indicates the computing operation in which the sub-request participates, and the sub-request at least indicates the source address of the obtained vector, the length of the obtained vector and the destination address of the calculation result.
[0011] In some embodiments, generating the original request for the vector computation includes: a processor generating the original request for the vector computation, the original request being provided to a buffer controller.
[0012] In some embodiments, the plurality of sub-requests are generated by a cache controller, the computing unit is adjacent to the cache controller, and the cache controller and the computing unit are included in the cache.
[0013] In some embodiments, the method for performing vector calculations also includes: in response to determining that the last calculation of multiple calculations associated with the current original request is completed, writing the calculation result to the cache controller based on the destination address; and in response to determining that the calculation result is written to the cache controller, the cache controller notifies the current processor to complete the calculation associated with the current original request, and the current processor is the processor that receives the current calculation task.
[0014] In some embodiments, generating an original request for vector calculation includes: writing data carried by multiple direct memory accesses, or calculation results of a previous processor into a memory, where the previous processor is a processor that receives a previous computing task, and the previous processor is different from the current processor; in response to confirming that the writing of the memory is completed, notifying the current processor; and the current processor generates an original request for vector calculation based on the current computing task, the address of the received vector, and the vector length, where the received vector includes data carried by multiple direct memory accesses, calculation results of the previous processor, or calculation results of the current processor regarding the previous computing task.
[0015] In some embodiments, based on the original request, multiple sub-requests are generated so that the sub-requests are sent to the memory and the computing unit, including: after the cache controller receives the original request, based on the source address or destination address of the vector indicated in the original request, the vector length, and the operand, the original request is split into sub-requests for the memory; and the cache controller sends the generated sub-requests to the memory and the computing unit.
[0016] In some embodiments, the cache controller sending the generated sub-request to the memory and the computing unit includes: the cache controller sending the generated sub-request to the memory; obtaining miss information; and sending the sub-request associated with the miss information to the computing unit.
[0017] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements.
[0019] Figure 1 The figure schematically shows a schematic diagram of a computing device implementing a method for performing vector computing according to an embodiment of the present invention.
[0020] Figure 2 A flow chart of a method for performing vector calculation according to an embodiment of the present invention is shown.
[0021] Figure 3 A schematic diagram of a method for performing vector calculation according to an embodiment of the present invention is shown.
[0022] Figure 4A flowchart of a method 400 for generating an original request according to an embodiment of the present invention is shown.
[0023] Figure 5 A schematic diagram of a method for generating an original request and performing vector calculation according to an embodiment of the present invention is shown.
[0024] Figure 6 A flow chart of a method for performing vector calculation according to an embodiment of the present invention is shown.
[0025] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION
[0026] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.
[0027] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects.
[0028] As described above, the conventional method for performing vector calculations has the disadvantage that when the source data size for calculation is large, the processor needs to initiate multiple requests to send data, which increases bus pressure and significantly increases calculation delays.
[0029] In order to at least partially solve one or more of the above problems and other potential problems, an exemplary embodiment of the present invention proposes a method for performing vector calculation. In this method, multiple sub-requests of smaller granularity are generated based on the original request for vector calculation obtained (the original request at least indicates the source address, vector length, and operand of multiple vectors); the original request of the present invention does not directly give vector data, but gives the source address and length of multiple vectors, and generates sub-requests based on the original request. Since the length of the source address, vector length, and operand information of the vector is limited, a single sub-request can carry more vector information, thereby, the present invention can effectively reduce the number of requests required for a calculation, thereby reducing the bus pressure. In addition, the present invention sends an original request once, and reads data in the memory based on multiple sub-requests generated by the original request to send it to the calculation unit as source data; and performs vector calculation based on the source data and sub-requests via the calculation unit; the present invention avoids initiating multiple requests to send data during the process of completing the vector calculation, so the present invention can significantly reduce the delay of a calculation. Therefore, even for the case where the source data size is large, the present invention can reduce the bus pressure and reduce the delay of the calculation.
[0030] Figure 1 A schematic diagram of a computing device 100 for implementing a method for performing vector calculation according to an embodiment of the present invention is shown schematically. Figure 1 As shown, the computing device 100 may have one or more processing units, including special processing units such as a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a general-purpose graphics processing unit (GPGPU), and a general-purpose processing unit such as a CPU. The computing device 100 also includes at least: an original request generation unit 102, a sub-request generation and sending unit 104, a memory data reading unit 106. A vector calculation unit 108.
[0031] The original request generating unit 102 is used to generate an original request for vector calculation, where the original request at least indicates a source address, a vector length, and an operand of a vector involved in the calculation.
[0032] The sub-request generating and sending unit 104 is used to generate multiple sub-requests based on the original request so as to send the sub-requests to the memory and the computing unit. The granularity of the sub-requests is smaller than the granularity of the original request.
[0033] The memory data reading unit 106 is used to read data in the memory based on a sub-request, so as to send the data to the computing unit as source data.
[0034] The vector calculation unit 108 is used to perform vector calculations based on source data and sub-requests via the calculation unit.
[0035] The following will be combined Figure 2 and Figure 3 A method 200 for performing vector calculations according to an embodiment of the present invention is described. Figure 2 A flow chart of a method 200 for performing vector calculations according to an embodiment of the present invention is shown. Figure 3 FIG. 2 is a schematic diagram of a method for performing vector calculation according to an embodiment of the present invention. It should be understood that the method 200 may be used, for example, in Figure 1 The method 200 is executed at the described computing device 100. The method 200 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.
[0036] At step 202 , the computing device 100 generates an original request for vector calculation, where the original request at least indicates a source address, a vector length, and operands of a vector to be used in the calculation.
[0037] Regarding the original request, it indicates, for example, the associated calculations and the required source data length. For example, the original request at least indicates (or carries) the source address (source address) or destination address (destination address), vector length (vector len), and operand (opcode) of the vector used to participate in the calculation. In some embodiments, the original request also indicates other control information. The source address indicates the starting address of the vector used to participate in the calculation. Regarding other control information, it is, for example, whether a reply is required. The destination address is, for example, an address indicating the calculation result. In some embodiments, the source address of the vector is the source address of the vector obtained by each sub-request, the vector length is the length of the vector obtained by each sub-request, and the operand indicates the calculation operation in which the sub-request participates.
[0038] For example, the processor generates an original request for vector calculation. For example, the original request includes source addresses or destination addresses of 5 vectors, vector lengths, and 2 operands. Among them, the first vector to the third vector, for example, belong to the first operand (for example, op0), and the fourth vector to the fifth vector, for example, belong to the second operand (for example, op1).
[0039] At step 204, the computing device 100 generates multiple sub-requests based on the original request so as to send the sub-requests to the memory and the computing unit, wherein the granularity of the sub-requests is smaller than the granularity of the original request.
[0040] Regarding the sub-request, it indicates at least the source address of the acquired vector, the length of the acquired vector and the destination address of the calculation result. In some embodiments, the plurality of sub-requests are generated by a memory controller (Mem controller) or a cache controller (cache controller).
[0041] Regarding the method of sending a sub-request to a memory and a computing unit, it includes, for example, a cache controller sending the generated sub-request to the memory; obtaining miss information, and then sending the sub-request associated with the miss information to the computing unit. Miss represents a miss in the cache node, and data needs to be requested back to the source. By adopting the above means, the present invention is particularly conducive to reducing the delay of cross-device communication.
[0042] In some embodiments, after receiving the original request, the cache controller generates multiple sub-requests corresponding to the vectors suitable for processing based on the original request, so as to send the multiple sub-requests to the memory and the computing unit (ALU) in sequence. For example, after receiving the original request including 5 vectors and 2 operands mentioned in step 202 above, the cache controller generates 5 sub-requests corresponding to the vectors. The 5 sub-requests are, for example, Figure 5 Shown are a first sub-request (sub req0), a second sub-request (sub req1), a third sub-request (sub req2), a fourth sub-request (sub req3), and a fifth sub-request (sub req4).
[0043] Regarding the buffer controller 312, as Figure 3 As shown, it is configured to be included by a cache 310 (ie, Cache). The cache 310 further includes a calculation unit 318 , which is arranged adjacent to the cache controller 312 .
[0044] At step 206, the computing device 100 reads data in the memory based on the sub-request to send to the computing unit as source data.
[0045] Regarding memory, Figure 3 Indicated by the mark 320 in FIG. The memory 320 is, for example but not limited to, a high bandwidth memory (High Bandwidth Memory, HBM).
[0046] The data in the memory may include at least: data transferred by multiple direct memory accesses, or calculation results of the last processor written into the memory. In some embodiments, the data in the memory may also include: calculation results of the last calculation task of the current processor.
[0047] At step 208 , vector calculations are performed based on the source data and the sub-requests via a computing unit in the computing device 100 .
[0048] The calculation unit 318 is, for example, a calculation unit 318 adjacent to the buffer controller 312. For example, the calculation unit 318 is an ALU in the buffer 310.
[0049] In some implementations, the computing unit performs vector calculations based on operands indicated by the obtained sub-request and source data obtained from the memory 324 to obtain a calculation result.
[0050] In some embodiments, the calculation unit 318 sends the calculation result to the buffer controller 312. Specifically, the method of sending the calculation result to the buffer controller includes, for example: if it is determined that the last calculation of the multiple calculations associated with the current original request is completed, the calculation result is written to the buffer controller based on the destination address; and if it is determined that the calculation result is written to the buffer controller, the buffer controller notifies the current processor to complete the calculation associated with the current original request, and the current processor is the processor receiving the current calculation task.
[0051] For example, the computing unit completes the corresponding calculations for the first operands (e.g., op0) of the first vector (vec0), the second vector (vec1), and the third vector (vec2), and the corresponding calculations for the second operands (e.g., op1) of the fourth vector (vec3) and the fifth vector (vec4); the computing unit writes the calculation results to the destination address indicated in the original request. If the computing unit determines that the last calculation of the multiple calculations associated with the current original request is completed (e.g., the corresponding calculation of the second operand (e.g., op1) is completed), and after writing the calculation results to the cache controller based on the destination address, the cache controller returns a signal (e.g., a done signal) indicating that the calculation associated with the current original request is completed, so as to notify the processor to complete the current original request, such as Figure 3 Indicated by mark 306 in .
[0052] In the above scheme, the original request of the present invention does not directly give the vector data in the original request, but gives the source address and length of multiple vectors, and generates sub-requests based on the original request. Since the source address, vector length, and operand information of the vector are limited in length, a single sub-request can carry more vector information. Therefore, the present invention can effectively reduce the number of requests required for a calculation, thereby reducing the bus pressure. In addition, the present invention sends an original request once, and reads the data in the memory as source data to send to the computing unit based on the multiple sub-requests generated by the original request; and performs vector calculations based on the source data and sub-requests via the computing unit; the present invention avoids initiating multiple requests to send data during the process of completing the vector calculation, so the present invention can significantly reduce the delay of a calculation. Therefore, even for the case where the source data size is large, the present invention can reduce the bus pressure and reduce the delay of the calculation.
[0053] The following will be combined Figure 3 , Figure 4 and Figure 5 A method 400 for generating an original request according to an embodiment of the present invention is described. Figure 4 A flowchart of a method 400 for generating an original request according to an embodiment of the present invention is shown. Figure 5 FIG. 4 is a schematic diagram showing a method for generating an original request and performing vector calculation according to an embodiment of the present invention. It should be understood that the method 400 may be used, for example, in Figure 1 The method 400 is executed at the described computing device 100. The method 400 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.
[0054] At step 402, the computing device 100 writes data carried by multiple direct memory accesses or calculation results of a previous processor into the memory, where the previous processor is a processor that receives a previous computing task, and the previous processor is different from the current processor.
[0055] Regarding multiple direct memory access (DMA), for example Figure 5 The three DMAs in Figure 5 DMA-0, DMA-1, and DMA-2 are shown.
[0056] Regarding the data carried by multiple direct memory accesses, for example, Figure 5The first data 502 carried by the first direct memory access (i.e., DMA-0), the second data 504 carried by the second direct memory access (i.e., DMA-1), and the third data 506 carried by the third direct memory access (i.e., DMA-2) are shown. Each of the carried data includes, for example, a data address (i.e., data ptr) and a data length (i.e., data len).
[0057] Regarding the calculation result of the previous processor, it is, for example, a calculation result generated by the previous processor performing calculations, and the calculation result can be further performed by the current processor (a processor different from the previous processor) based on the calculation result. It should be understood that by adopting the above means, the present invention facilitates the use of the calculation result of the previous processor to continue the calculation.
[0058] Regarding memory (e.g. Figure 3 In some embodiments, the memory is, for example, but not limited to, a high bandwidth memory (HBM).
[0059] like Figure 3 As shown, data is written into the memory 320 via a direct memory access (DMA) indicated by a mark 330 or a previous processor. The data written into the memory 320 is, for example: data moved by a plurality of DMAs, or a calculation result of a previous processor.
[0060] At step 404, if the computing device 100 confirms that the writing of the memory is completed, the current processor is notified.
[0061] For example, if it is confirmed that the writing of the data moved by the DMA of the memory 320 or the calculation result of the previous processor is completed, the current processor 302 is notified, as indicated by the mark 332.
[0062] At step 406, the current processor of the computing device 100 generates an original request for vector calculation based on the current computing task, the address of the received data, and the vector length, the received data including data moved by multiple direct memory accesses, the calculation results of the previous processor, or the calculation results of the current processor regarding the previous computing task.
[0063] For example, Figure 5 As shown, an original request 512 is generated based on the computing task 510 of the current processor, and the addresses of the data in the first data 502, the second data 504, and the third data 506 respectively transported by DMA-0, DMA-1, and DMA-2, and the vector length.
[0064] Regarding the original request, it indicates the source address of the vector obtained by each sub-request, the length of the vector obtained by each sub-request, and the operand (which indicates the computing operation involved in the sub-request). In some embodiments, the original request also includes grouping information of the sub-request.
[0065] The source address of the vector obtained by each sub-request is, for example, Figure 5 The source address (vec0 ptr) of the vector obtained by the first sub-request (sub req0) indicated by the mark 520, the source address (vec1 ptr) of the vector obtained by the second sub-request (sub req1), the source address (vec2 ptr) of the vector obtained by the third sub-request (sub req2), the source address (vec3 ptr) of the vector obtained by the fourth sub-request (sub req3), and the source address (vec4 ptr) of the vector obtained by the fifth sub-request (sub req4).
[0066] Regarding the grouping information of the sub-requests, for example Figure 5 As shown: the first sub-request (sub req0), the second sub-request (sub req1) and the third sub-request (sub req2) are a group, and the sub-requests in this group participate in the corresponding calculation of the first operand (i.e., op0, as indicated by the mark 530). The fourth sub-request (sub req3) and the fifth sub-request (sub req4) are a group, and the sub-requests in this group participate in the corresponding calculation of the second operand (i.e., op1, as indicated by the mark 532).
[0067] The calculation involved in each sub-request is indicated by an operand, for example. For example, the corresponding calculation of the first operand (for example, op0) associated with the first vector (vec0), the second vector (vec1), and the third vector (vec2) is to perform an accumulation calculation. The corresponding calculation of the second operand (for example, op1) associated with the fourth vector (vec3) and the fifth vector (vec4) is a multiplication calculation.
[0068] For example, Figure 3 As shown, the current processor 302 generates an original request for vector calculation based on the calculation task, the address of the received vector, and the vector length, and sends the generated original request to the buffer controller 312 in the buffer 310 , as indicated by mark 304 .
[0069] By adopting the above means, the original request generated by the present invention may not directly provide vector data for vector calculation, but may provide source addresses and lengths of multiple vectors, and the generated original request can be easily split into multiple sub-requests.
[0070] The following will be combined Figure 3 , Figure 5 and Figure 6 Method 600 for performing vector calculations according to some other embodiments of the present invention is described. Figure 6 FIG. 6 is a flowchart of a method 600 for performing vector calculation according to an embodiment of the present invention. It should be understood that the method 600 may be implemented in Figure 1 The method 600 is executed at the described computing device 100. The method 600 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.
[0071] At step 602, after receiving the original request, the cache controller of the computing device 100 splits the original request into sub-requests for the memory based on the source address or destination address, vector length, and operand of the vector indicated in the original request.
[0072] For example, after receiving the original request generated by the current processor 302, the cache controller 312 splits the original request into multiple sub-requests for the memory 320 based on the source address of the vector obtained by each sub-request indicated in the original request, the length of the vector obtained by each sub-request, and the operand. Figure 5 As shown, the original request is split into a first sub-request (sub req0), a second sub-request (sub req1), a third sub-request (sub req2), a fourth sub-request (sub req3) and a fifth sub-request (sub req4). Each sub-request at least indicates the source address of the vector obtained by the sub-request, the length of the obtained vector and the destination address of the calculation result.
[0073] At step 604, the cache controller of the computing device 100 sends the generated sub-request to the computing unit and the memory.
[0074] For example, the cache controller 312 sends the generated sub-request to the computing unit 318 as indicated by mark 314 ; and sends the generated sub-request to the memory 320 as indicated by mark 322 .
[0075] At step 606, the computing device 100 reads data in the memory based on the sub-request to send to the computing unit as source data.
[0076] For example, based on the source address and vector length of the obtained vector in the sub-request, vector data of the source address and corresponding vector length in the memory 320 is read, and the read vector data is sent to the calculation unit 318 as source data for vector calculation, as indicated by mark 324.
[0077] At step 608 , the computing device 100 performs vector calculations based on the source data and the sub-requests via the computing unit to send the calculation results to the buffer controller.
[0078] For example, the calculation unit 318 performs vector calculation based on the sub-request sent by the buffer controller 312 and the source data from the memory 320 to generate a calculation result of the vector calculation (such as Figure 5 As shown, the mark 534 indicates the first operation calculation result, and the mark 536 indicates the second operation calculation result); and the calculation result of the vector calculation is sent to the buffer controller 312, as indicated by the mark 316. The starting address where the first operation calculation result 534 is stored is, for example Figure 5 The address of the first operation calculation result is shown. The starting address of the second operation calculation result 536 is stored, for example Figure 5 The second operation shown calculates the result address.
[0079] In the above scheme, the current processor only needs to send an original request once, and the computing unit can perform multiple calculations based on multiple sub-requests split by the cache controller, using the source data obtained from the memory, without multiple source data interactions between the current processor and the cache or memory. Therefore, the present invention can significantly reduce the number of access data requests required for a single operation, and can release the processor, so that the current processor can be released to perform other tasks when the computing unit performs vector calculations. Furthermore, after the processor sends an original request once, the computing unit does not need to call the processor again when performing calculations, and only the cache needs to participate, so it is helpful to reduce power consumption.
[0080] The various processes and processing described above, such as methods 200 and 400, may be performed at a computing device. The computing device, for example, includes: at least one processor (at least one graphics processor and at least one central processing unit); and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor. In some embodiments, methods 200 and 400 may be implemented as a computer software program or program product, which is tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on a computing device via a read-only memory (ROM) and / or a communication unit. When the computer program is loaded into a random-access memory (RAM) and executed by a GPU and a CPU, one or more actions of methods 200 and 400 described above may be performed.
[0081] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for executing various aspects of the present invention. The computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.
[0082] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each box of the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram, can be implemented by computer-readable program instructions.
[0083] These computer-readable program instructions can be provided to a central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the central processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0084] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the function involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.
[0085] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps loaded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0086] The above specific implementations do not constitute a limitation on the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made according to design requirements and other factors.
Claims
1. A method for performing vector calculations, characterized in that The method comprises: The processor generates an original request for vector calculation, the original request is provided to the buffer controller, the original request at least indicates a source address, a vector length, and an operand for a vector involved in the calculation; Based on the original request, a plurality of sub-requests are generated by the cache controller so as to send the sub-requests to the memory and the computing unit, wherein the granularity of the sub-requests is smaller than the granularity of the original request; Based on the sub-request, reading data in the memory to send as source data to the computing unit; and Vector calculations are performed based on source data and sub-requests via the calculation unit.
2. The method according to claim 1, characterized in that The source address of the vector is the source address of the vector obtained by each sub-request, the vector length is the length of the vector obtained by each sub-request, the operand indicates the computing operation in which the sub-request participates, and the sub-request at least indicates the source address of the obtained vector, the length of the obtained vector and the destination address of the calculation result.
3. The method according to claim 1, characterized in that The calculation unit is adjacent to a buffer controller, and the buffer controller and the calculation unit are included in a buffer.
4. The method according to claim 3, characterized in that Also includes: In response to determining that a last calculation of the plurality of calculations associated with the current original request is completed, writing the calculation result into the buffer controller based on the destination address; as well as In response to determining that the calculation result is written into the cache controller, the cache controller notifies the current processor to complete the calculation associated with the current original request, and the current processor is the processor receiving the current calculation task.
5. The method according to claim 4, characterized in that The original request generated by the processor for vector calculation includes: Writing data carried by multiple direct memory accesses or calculation results of a previous processor into a memory, where the previous processor is a processor that receives a previous calculation task, and the previous processor is different from the current processor; In response to confirming that the writing of the memory is completed, notifying the current processor; and The current processor generates an original request for vector calculation based on the current calculation task, the address of the received vector, and the vector length. The received vector includes data carried by multiple direct memory accesses, the calculation result of the previous processor, or the calculation result of the current processor on the previous calculation task.
6. The method according to claim 1, characterized in that Based on the original request, multiple sub-requests are generated so that sending the sub-requests to the memory and the computing unit includes: After receiving the original request, the cache controller splits the original request into sub-requests for the memory based on the source address or destination address, vector length, and operand of the vector indicated in the original request; and The cache controller sends the generated sub-requests to the memory and the computing unit.
7. The method according to claim 1, characterized in that The cache controller sends the generated sub-request to the memory and the computing unit including: The cache controller sends the generated sub-request to the memory; Obtaining miss information; and The sub-request associated with the miss information is sent to the computing unit.
8. A computing device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a machine, the method according to any one of claims 1 to 7 is performed.
10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a machine, performs the method according to any one of claims 1 to 7.
