Method for performing intra-SIMD data movement for lane-wise operated single instruction, multiple data (SIMD) coprocessors

Abstract

A processing system includes a first single instruction multiple data (SIMD) coprocessor including a first plurality of SIMD registers. Each of the first plurality of SIMD registers including a first plurality of SIMD lanes, the first SIMD coprocessor configured to: move data from a SIMD register of the plurality of SIMD registers to a memory independent of a cross-lane operation occurring between the first plurality of SIMD lanes of the SIMD register. The processing system includes a second SIMD coprocessor including a second plurality of SIMD registers. Each of the second plurality of SIMD registers including a second plurality of SIMD lanes, the second SIMD coprocessor configured to: move the data from the memory to a SIMD register of the second plurality of SIMD registers; perform movement operations on the data that involve cross-lane operations between the second plurality of SIMD lanes of the SIMD register.

Description

QUALCOMM Ref. No.: 2500613U2WO1METHOD FOR PERFORMING INTRA-SIMD DATA MOVEMENT FOR LANE-WISE OPERATED SINGLE INSTRUCTION, MULTIPLE DATA (SIMD)COPROCESSORS CROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to US Patent Application No. 19 / 067,654, filed February 28, 2025, and US Patent Application No. 19 / 067,670, filed February 28, 2025, which are hereby incorporated by reference herein.TECHNICAL FIELD

[0002] Aspects of the present disclosure generally relate to coprocessors and, more particularly, to a lane-wise operated SIMD coprocessor.BACKGROUND

[0003] Coprocessors (e.g., hardware accelerators) may be used to support a main processor (e.g., central processing unit) of a processing system. For example, the main processor may offload computationally intensive tasks (e.g., matrix multiplication, vector processor) or applications to the coprocessor so that the main processor may perform more general-purpose processing. Coprocessors can perform tasks in parallel with the main processor, allowing for concurrent processing and faster execution of certain workloads (e.g., machine learning algorithms).

[0004] SIMD coprocessors have a parallel processing architecture that allows an operation to be simultaneously performed on multiple data elements. Thus, the parallel processing architecture allows SIMD coprocessors to efficiently process large data sets. SIMD coprocessors can, for example, include specialized instructions that may be optimized for vector operations (e.g., addition, multiplication).BRIEF SUMMARY

[0005] In one aspect, a processing system is provided. The processing system typically includes: a first single instruction multiple data (SIMD) coprocessor including a first plurality of SIMD registers, each of the first plurality of SIMD registers includingP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO2a first plurality of SIMD lanes, the first SIMD coprocessor configured to: move data from a SIMD register of the plurality of SIMD registers to a memory independent of a crosslane operation occurring between the first plurality of SIMD lanes of the SIMD register; a second SIMD coprocessor including a second plurality of SIMD registers, each of the second plurality of SIMD registers including a second plurality of SIMD lanes, the second SIMD coprocessor configured to: move the data from the memory to a SIMD register of the second plurality of SIMD registers; perform movement operations on the data that involve cross-lane operations between the second plurality of SIMD lanes of the SIMD register.

[0006] In another aspect, a method performable by a processing system comprising a first single instruction multiple data (SIMD) coprocessor and a second single instruction multiple data (SIMD) coprocessor is provided. The method generally includes: moving, by the first SIMD coprocessor, data from a first SIMD lane of a lane-wise SIMD register of the first SIMD coprocessor to a first segment of a memory; retrieving, by the second SIMD coprocessor, the data from the first segment of the memory and storing the data in a first SIMD lane of a SIMD register of the second SIMD coprocessor that supports intra-SIMD data movement; executing, by the second SIMD coprocessor, one or more instructions to move the data from the first SIMD lane of the SIMD register of the second SIMD coprocessor to a second SIMD lane of the SIMD register of the second SIMD coprocessor; loading, by the second SIMD coprocessor, the data stored in the second SIMD lane of the SIMD register of the second SIMD coprocessor onto a second segment of the memory; and retrieving, by the first SIMD coprocessor, the data stored on the second segment of the memory and storing the data on a second SIMD lane of the lanewise SIMD register of the first SIMD coprocessor.

[0007] In yet another aspect, an apparatus is provided. The apparatus includes: a first single instruction multiple data (SIMD) coprocessor including a first plurality of SIMD registers, each of the first plurality of SIMD registers including a first plurality of SIMD lanes, the first SIMD coprocessor configured to: move data from a SIMD register of the plurality of SIMD registers to a memory independent of a cross-lane operation occurring between the first plurality of SIMD lanes of the SIMD register; a second SIMD coprocessor including a second plurality of SIMD registers, each of the second plurality of SIMD registers including a second plurality of SIMD lanes, the second SIMDP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO3coprocessor configured to: move the data from the memory to a SIMD register of the second plurality of SIMD registers; perform movement operations on the data that involve cross-lane operations between the second plurality of SIMD lanes of the SIMD register.

[0008] The following description and the related drawings set forth in detail certain illustrative features of one or more aspects.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The appended figures depict certain features of one or more aspects of the present disclosure and are therefore not to be considered limiting of the scope of this disclosure.

[0010] FIG. 1 depicts a heterogeneous computing system including a main processor and a coprocessor according to various aspects of the present disclosure.

[0011] FIG. 2 depicts a SIMD coprocessor according to various aspects of the present disclosure.

[0012] FIG. 3 depicts a lane-wise SIMD coprocessor according to various aspects of the present disclosure.

[0013] FIG. 4 depicts connections between a memory device and SIMD lanes of a lane-wise SIMD register included in a lane-wise SIMD coprocessor according to various aspects of the present disclosure.

[0014] FIG. 5 depicts a system for performing intra-SIMD data movement for a lanewise SIMD coprocessor according to various aspects of the present disclosure.

[0015] FIG. 6 depicts a method for performing intra-SIMD data movement for a lanewise SIMD coprocessor according to various aspects of the present disclosure.

[0016] FIG. 7 depicts an example processing system in which a heterogeneous computing system may be included according to various aspects of the present disclosure.P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO4

[0017] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the drawings. It is contemplated that elements and features of one aspect may be beneficially incorporated in other aspects without further recitation.DETAILED DESCRIPTION

[0018] Aspects of the present disclosure provide a lane-wise operated SIMD coprocessor.

[0019] SIMD coprocessors typically include multiple SIMD registers, which are a type of processor register that can hold multiple data elements of the same data type. In this manner, SIMD instructions can simultaneously perform the same operation (e.g., addition, multiplication, comparison, etc.) on all the data elements in a given SIMD register. SIMD registers typically hold the multiple data elements in a vector-like structure. Each of the data elements included in the vector-like structure may be referred to as a lane.

[0020] A SIMD register typically supports cross-lane operations which, for example, may involve movement of data elements within the SIMD register. To facilitate such cross-lane operations, the SIMD register may include hardware (e.g., data buses) that allows the different lanes of the SIMD register to be in data communication with one another. The hardware is typically complex and presents challenges associated with scaling the SIMD register so that SIMD coprocessors can accommodate larger data sets that may be associated with more complex tasks (e.g,. executing machine learning algorithms).

[0021] Example aspects of the present disclosure are directed to a lane-wise operated SIMD coprocessor. The lane-wise operated SIMD coprocessor includes multiple lanewise SIMD registers. However, in contrast to SIMD registers included in traditional SIMD coprocessors, the different lanes within each respective lane-wise SIMD register are decoupled from one another. In this manner, each respective lane-wise SIMD register does not support cross-lane operations and, as a result, does not include the hardware (e.g., multiple data buses) that is included in conventional SIMD registers to facilitate such cross-lane operations. Thus, the lane-wise operated SIMD coprocessor according toP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO5the present disclosure may be scaled more efficiently compared to traditional SIMD coprocessors. More specifically, the number of lanes that are included in each respective lane-wise SIMD register of the lane-wise SIMD coprocessor may be scaled (e.g., increased) more easily compared to conventional SIMD registers, because each lane-wise SIMD register of the lane-wise operated SIMD coprocessor disclosed herein does not support intra-SIMD movement (that is, movement of data from one SIMD lane to another SIMD lane) and therefore do not include the additional hardware (e.g,. data buses and interconnects) and complexity associated with implementing such hardware for intra-SIMD movement. Furthermore, since each of the lane-wise SIMD registers is substantially wider (e.g,. includes many multiples the number of SIMD lanes) compared to traditional SIMD registers, the lane-wise SIMD coprocessor may have a greater compute density (e.g., about 8 times) than traditional SIMD coprocessors.Example Heterogeneous Computing System

[0022] FIG. 1 depicts a heterogeneous computing system 100 according to aspects of the present disclosure. The heterogeneous computing system 100 may be used in a variety of different apparatuses (e.g., smartphones, tablets) and may be used for a variety of different applications (e.g., machine learning, digital signal processing, graphics processing). The heterogeneous computing system 100 includes a main processor 110 and a coprocessor 120. The main processor 110, which in some aspects may be a central processing unit (CPU), handles general -purpose computing tasks. The main processor 110 delegates computationally-intensive tasks (e.g., matrix multiplication, vector processing) to the coprocessor 120. Examples of the coprocessor s) may include, without limitation, a graphics processing unit (GPU), a neural processing unit (NPU), and a tensor processing unit (TPU).

[0023] As illustrated, the main processor 110 may send a request 112 to the coprocessor 120. The request 112 may, for example, be for the coprocessor 120 to perform a computationally intensive task on data associated with a particular application (e.g., machine learning, digital signal processing) being executed by the heterogeneous computing system 100. The coprocessor 120 may communicate a result 114 of the computationally intensive task to the main processor 110.P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO6Example SIMD Coprocessor with SIMD Register Supporting Intra-SIMD Movement

[0024] FIG. 2 depicts a SIMD coprocessor 200 according to aspects of the present disclosure.

[0025] The SIMD coprocessor 200 may include one or more SIMD registers 202 (one SIMD register 202 is shown in FIG. 2). Each of the SIMD registers 202 may include a plurality of SIMD processing units 204 (e.g., labeled “PU”). As illustrated, the plurality of SIMD processing units 204 may be arranged in an array (e.g., a vector arranged as columns and rows). Each respective column of the array may be referred to as a SIMD lane. In some aspects, each of the SIMD registers 202 may include four columns and four rows. In other aspects, each of the SIMD registers 202 may include more (or fewer) columns and rows. For instance, as illustrated in FIG. 2, each of the SIMD registers 202 may include a first SIMD lane, a second SIMD lane, a third SIMD lane, and a fourth SIMD lane.

[0026] In some aspects, the SIMD register 202 may be configured to receive an instruction 206 and data 208. More specifically, the instruction 206 may be provided as an input to each of the plurality of SIMD processing units 204. Additionally, a plurality of instances of data 208 may be provided as an input to each of the plurality of SIMD processing units 204.

[0027] The instruction 206 may correspond to an operation (e.g., multiply, add, etc.) that the SIMD registers 202 may simultaneously perform on the multiple instances of data 208. In some aspects, the operation may involve moving data between two respective SIMD lanes of the plurality of SIMD lanes included in the SIMD register 202. Such an operation may be referred to as a “cross-lane” operation. Furthermore, such movement of data (e.g., from one SIMD lane to another SIMD lane) may be referred to as “intra-SIMD” movement.

[0028] In some aspects, the SIMD register 202 may include hardware, such as dedicated data buses or interconnects, to facilitate intra-SIMD movement associated with cross-lane operations. As previously mentioned, SIMD registers, such as the SIMD register 202 of FIG. 2, that support cross-lane operations involving intra-SIMD movement may be difficult to scale. More specifically, the hardware associated withP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO7supporting cross-lane operations may become increasingly complex as the size (e.g., number of bits) of data increases. As will now be discussed with reference to FIG. 3, example aspects of the present disclosure are directed to a lane-wise SIMD coprocessor having a SIMD register that is configured for lane-wise only operations and therefore does not include the above-mentioned hardware for intra-SIMD movement and therefore can scale (e.g., to accommodate even larger data sets) more easily compared conventional SIMD coprocessors, such as the SIMD coprocessor 200 of FIG. 2.Example Lane -Wise SIMD Coprocessor with Lane-Wise SIMD Register Having No Intra-SIMD Movement

[0029] FIG. 3 depicts a lane-wise SIMD coprocessor 300 according to some aspects of the present disclosure.

[0030] The lane-wise SIMD coprocessor 300 may include one or more lane-wise SIMD registers 302 (one lane-wise SIMD register 302 is shown in FIG. 3). Each of the lane-wise SIMD registers 302 may include a plurality of SIMD processing units 304 (e.g., labeled “PU”). As illustrated, the plurality of SIMD processing units 304 may be arranged in an array (e.g., a vector arranged as columns and rows). Each respective column of the array may be referred to as a SIMD lane. In some aspects, each of the SIMD registers 302 may include four columns and four rows. In other aspects, each of the SIMD registers 302 may include more (or fewer) columns and rows. For instance, as illustrated in FIG.3, each of the SIMD registers 302 may include a first SIMD lane, a second SIMD lane, a third SIMD lane, and a fourth SIMD lane. In other aspects, the lane-wise SIMD register 302 may include more or fewer SIMD lanes.

[0031] In some aspects, the lane-wise SIMD register 302 may be configured to receive a lane-wise instruction 306 and data 308. More specifically, the lane-wise instruction 306 may be provided as an input to each of the plurality of SIMD processing units 304. Additionally, a plurality of instances of data 308 may be provided as an input to each of the plurality of SIMD processing units 304.

[0032] The lane-wise instruction 306 may involve lane-wise operations that do not involve multiple SIMD lanes and therefore do not require intra-SIMD movement within the lane-wise SIMD register 302. It should be appreciated, however, that lane-wiseP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO8operations may still include movement of data within a particular SIMD lane of the lanewise SIMD register 302.

[0033] In contrast to the SIMD register 202 discussed above with reference to FIG.2, the lane-wise SIMD register 302 in FIG.3 does not include hardware, such as dedicated data buses or interconnects, to facilitate intra-SIMD movement associated with cross-lane operations. This is generally denoted by including a rectangular box around each of the plurality of SIMD lanes (e.g., First SIMD Lane, Second SIMD Lane, Third SIMD Lane, and Fourth SIMD Lane). In this manner, the lane-wise SIMD coprocessor 300, specifically the lane-wise SIMD register 302 thereof, may include less hardware compared to conventional SIMD coprocessors, such as the SIMD coprocessor 200 of FIG. 2, and therefore may scale (e.g., to accommodate even larger data sets) more easily compared conventional SIMD coprocessors.

[0034] The total number of SIMD lanes included in the lane-wise SIMD register 302 of FIG. 3 may be greater than the total number of SIMD lanes included in the SIMD register 202 of FIG. 2. For instance, in some aspects, the total number of SIMD lanes included in the lane-wise SIMD register 302 may be many multiples (e.g., at least 4 times as many) of the total number of SIMD lanes included in the SIMD register 202 of FIG.2. For instance, in some aspects, the lane-wise SIMD register 302 may include as many as 1024 SIMD lanes. In this manner, the lane-wise SIMD coprocessor 300 may considered an ultra-wide SIMD coprocessor having no intra-SIMD movement within each of the lane-wise SIMD registers (e.g., lane-wise SIMD register 302) that are included in the lane-wise SIMD coprocessor 300.

[0035] As used herein, executing an instruction independent of a cross-lane operation occurring between the plurality of SIMD lanes of the lane-wise SIMD register 302 means that the instruction is executed within a single SIMD lane of the lane-wise SIMD register 302 and does not involve moving data from the single SIMD lane of the lane-wise SIMD register 302, nor does it involve retrieving data from another SIMD lane of the lane-wise SIMD register 302.Example Connections Between Memory and SIMD Lanes of Lane-Wise SIMD RegisterP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO9

[0036] FIG. 4 depicts connections between memory 400 (e.g., main memory) and each of a plurality of SIMD lanes of a lane-wise SIMD register, such as the lane-wise SIMD register 302 of the lane-wise SIMD coprocessor 300 discussed above with reference to FIG. 3.

[0037] As illustrated, each of a first SIMD lane 402, a second SIMD lane 408, a third SIMD lane 414, and a fourth SIMD lane 420 may be in data communication with the memory 400 via independent data paths (e.g., denoted by double arrows extending between a respective SIMD lane and a respective segment of the memory 400). In some aspects, the data paths are decoupled from one another such that there is no data communication between two data paths extending between respective SIMD lanes and respective segments of the memory.

[0038] As illustrated, the first SIMD lane 402 may be in data communication with a first segment 404 of the memory 400 via a first data path 406. The second SIMD lane 408 may be in data communication with a second segment 410 of the memory 400 via a second data path 412 that is independent of the first data path 406. The third SIMD lane 414 may be in data communication with a third segment 416 of the memory 400 via a third data path 418 that is independent of the first data path 406 and the second data path 412. The fourth SIMD lane 420 may be in data communication with a fourth segment 422 of the memory 400 via a fourth data path 424 that is independent of each of the first data path 406, the second data path 412, and the third data path 418.

[0039] In some aspects, the different segments (e.g., first segment 404, second segment 410, third segment 416, and fourth segment 422) of the memory 400 may correspond to different address ranges of the memory 400. For instance, the first segment 404 of the memory 400 may correspond to a first subset of addresses of the memory 400, the second segment 410 may correspond to a second subset of addresses of the memory 400, the third segment 416 may correspond to a third subset of addresses of the memory 400, and the fourth segment 422 may correspond to a fourth subset of addresses of the memory. Furthermore, in some aspects, the different subsets of addresses (e.g., first subset, second subset, third subset, fourth subset) may be mutually exclusive. Stated another way, addresses included in one subset (e.g., first subset) of addresses of the memory 400 may not be included in any of the other subsets (e.g., second, third, fourthP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO10subsets) of the memory 400. In this manner, the different SIMD lanes (e.g., first SIMD lane 402, second SIMD lane 408, third SIMD lane 414, fourth SIMD lane 420) may be decoupled from one another (that is, not in data communication with one another) all the way to the memory 400.

[0040] Although FIG. 4 depicts four separate SIMD lanes, it should be appreciated that the connections between the memory 400 and a lane- wise SIMD register, such as the lane-wise SIMD register 302 of FIG.3, may accommodate any suitable number of SIMD lanes. For instance, in some aspects, the lane-wise SIMD register may include 1024 different SIMD lanes. In such aspects, there may be 1024 separate data paths between the memory 400 and the lane-wise SIMD register to accommodate decoupling of the 1024 SIMD lanes all the way to the memory 400.Example System for Performing Data Movement for Lane-Wise SIMD Coprocessors

[0041] FIG. 5 depicts an example system 500 for performing data movement for lanewise SIMD coprocessors according to some aspects of the present disclosure. For simplicity, the system 500 may include the lane-wise SIMD coprocessor 300 discussed above with reference to FIG.3, the conventional SIMD coprocessor 200 discussed above with reference to FIG. 2, and the memory 400 discussed above with reference to FIG. 4.

[0042] In some aspects, intra-SIMD movement for the lane-wise SIMD register 302 of the lane-wise SIMD coprocessor 300 may be accomplished by moving data from the lane-wise SIMD coprocessor 300, which does not support intra-SIMD movement, to SIMD coprocessor 200, which does support intra-SIMD movement. As an example, to swap data included in the first SIMD lane 402 of the lane-wise SIMD register 302 with data included in the fourth SIMD lane 420 of the lane-wise SIMD register 302, data included in the first SIMD lane 402 may be communicated to the first segment 404 of the memory 400 via the first data path 406. Additionally, the data included in the fourth SIMD lane 420 of the lane-wise SIMD register 302 may be communicated to the fourth segment 422 of the memory 400 via the fourth data path 424.

[0043] As illustrated, the SIMD coprocessor 200, which does support intra-SIMD movement, may retrieve the data from the memory 400. For instance, the SIMD coprocessor 200 may retrieve data stored in the first segment 404 of the memory 400 asP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO11well as data stored in the fourth segment 422 of the memory 400. In some aspects, the SIMD coprocessor 200 may be in data communication with the memory 400 via a single data path 502. In other aspects, the SIMD coprocessor 200 may include multiple data paths between the memory 400 and respective SIMD lanes (e.g., first SIMD lane 504, second SIMD lane 506, third SIMD lane 508, and fourth SIMD lane 510) of the SIMD register 202 of the SIMD coprocessor 200.

[0044] In some aspects, the data retrieved from the first segment 404 of the memory 400 may be stored in the first SIMD lane 504 of the SIMD register 202. Additionally, the data retrieved from the fourth segment 422 of the memory 400 may be stored in the fourth SIMD lane 510 of the SIMD coprocessor 200.

[0045] In some aspects, the SIMD coprocessor 200 may be configured to execute one or more instructions associated with swapping the data stored in the first SIMD lane 504 of the SIMD register 202 with the data stored in the fourth SIMD lane 510 of the SIMD register 202. For instance, the SIMD coprocessor 200 may be configured to execute an instruction included in an instruction set architecture that supports intra-SIMD movement. It should be appreciated that the instruction executed by the SIMD coprocessor 200 to swap the data stored in the first SIMD lane 504 of the SIMD register 202 with the data stored in the fourth SIMD lane 510 of the SIMD register 202 may not be included in an instruction set architecture for the lane-wise SIMD coprocessor 300.

[0046] After executing the instruction to swap the data stored in the first SIMD lane 504 of the SIMD register 202 and the data stored in the fourth SIMD lane 510 of the SIMD register 202, the SIMD coprocessor 200 may be configured to return the data to the memory. For instance, the data now stored in the first SIMD lane 504 of the SIMD register 202 may be communicated to the memory 400 via data path 502 and ultimately stored in the first segment 404 of the memory 400. Additionally, the data now stored in the fourth SIMD lane 510 of the SIMD register 202 may be communicated to the memory 400 and ultimately stored in the fourth segment 422 of the memory 400.

[0047] With the data now loaded onto the memory 400, the lane-wise SIMD coprocessor 300 may execute one or more instructions included in an instruction set architecture for the lane-wise SIMD coprocessor 300. More specifically, the lane-wiseP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO12SIMD coprocessor 300 may execute the instruct! on(s) to retrieve the data stored in the first segment 404 of the memory 400 and the data stored in the fourth segment 422 of the memory 400. More specifically, the data stored in the first segment 404 of the memory 400 may be communicated to the first SIMD lane 402 of the lane-wise SIMD register 302 via the first data path 406. Additionally, the data stored in the fourth segment 422 of the memory 400 may be communicated to the fourth SIMD lane 420 of the lane-wise SIMD register 302 via the fourth data path 424.

[0048] It should be appreciated that the data now stored in the first SIMD lane 402 of the lane-wise SIMD register 302 is the data that was previously stored in the fourth SIMD lane 420 of the lane-wise SIMD register 302. It should also be appreciated that the data now stored in the fourth SIMD lane 420 of the lane-wise SIMD register 302 is the data that was previously stored in the first SIMD lane 402 of the lane-wise SIMD register 302. Thus, the disclosed system 500 provides for intra-SIMD movement of data stored on the lane-wise SIMD register 302 of the lane-wise SIMD coprocessor 300 even though such movement is not supported by the lane-wise SIMD coprocessor 300.Example Technique for Performing Data Movement for Lane-Wise SIMD Coprocessors

[0049] FIG. 6 is a diagram depicting an example method 600 of performing intra-SIMD data movement on lane-wise SIMD coprocessors that do not support intra-SIMD data movement according to various aspects of the present disclosure. For example, the method 600 may be performed by the system 500 of FIG. 5. Furthermore, although FIG.6 depicts steps performed in a particular order for purposes of illustration and discussion, the method 600 discussed herein is not intended to be limited to any particular order or arrangement. One skilled in the art, using the disclosure provided herein, will appreciate that various steps of the method 600 can be omitted, rearranged, combined and / or adapted in various ways without deviating from the scope of the present disclosure.

[0050] Operation 602 may include moving data from a first SIMD lane of a lane-wise SIMD register included in a lane-wise SIMD coprocessor to a first segment of a memory.

[0051] Operation 604 may include retrieving the data from the first segment of the memory and storing the data in a first SIMD lane of a SIMD register included in a SIMD coprocessor that supports intra-SIMD data movement. For instance, the SIMD register ofP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO13the SIMD coprocessor may include hardware (e.g., data buses and interfaces) that facilitates intra-SIMD data movement within the SIMD register of the SIMD coprocessor.

[0052] Operation 606 may include executing one or more instructions associated with moving the data from the first SIMD lane of the SIMD register in the SIMD coprocessor to a second SIMD lane of the SIMD register in the SIMD coprocessor. For instance, the instruction(s) may be included in an instruction set architecture for the SIMD coprocessor that supports intra-SIMD data movement.

[0053] Operation 608 may include loading the data stored in the second SIMD lane of the SIMD register in the SIMD coprocessor on a second segment of the memory. For instance, the SIMD coprocessor may communicate the data stored in the second SIMD lane of the SIMD register to the second segment of the memory via the same data path by which the data stored in the first segment of the memory was retrieved. In alternative aspects, the data stored in the second lane of the SIMD register in the SIMD coprocessor may be uploaded to the second segment of the memory via a different data path.

[0054] Operation 610 may include retrieving the data stored in the second segment of the memory using the lane-wise SIMD coprocessor. For instance, the lane-wise SIMD coprocessor may execute one or more instructions included in an instruction set architecture for the lane-wise SIMD coprocessor to retrieve the data from the second segment of the memory. Furthermore, since the second segment of the memory is in data communication with a second SIMD lane of the lane-wise SIMD register of the lane-wise SIMD coprocessor, the data the lane-wise SIMD coprocessor retrieves from the second segment of the memory may be stored on the second SIMD lane of the lane-wise SIMD register.

[0055] The method 600 described above allows lane-wise SIMD coprocessors, such as the lane-wise coprocessor of FIG. 3, to perform intra-SIMD data movement even though such SIMD coprocessors lack the hardware (e.g., data buses and interconnects) needed for performing such intra-SIMD data movement. Thus, the present disclosure allows for lane-wise SIMD coprocessors having ultra-wide SIMD processing lanes as well as for intra-SIMD data movement in such SIMD coprocessors. Accordingly, the present disclosure provides for lane-wise SIMD coprocessors that can support even largerP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO14data sets associated with computationally intensive tasks (e.g., softmax calculations in activation layers of machine learning models) while still allowing for intra-SIMD data movement within such SIMD coprocessors.Example Processing System

[0056] In some aspects, the heterogeneous computing system 100 discussed above with reference to FIG. 1 may be included in a device or processing system. FIG. 7 depicts an example processing system 700. Although depicted as a single system for conceptual clarity, in some aspects, as discussed above, the operations described below with respect to the processing system 700 may be distributed across any number of devices or systems.

[0057] The processing system 700 includes a central processing unit (CPU) 702. Instructions executed at the CPU 702 may be loaded, for example, from a memory 724 associated with the CPU 702.

[0058] The processing system 700 also includes additional processing components tailored to specific functions, such as a graphics processing unit (GPU) 704, a digital signal processor (DSP) 706, a neural processing unit (NPU) 708, a multimedia component 710 (e.g., a multimedia processing unit), and a wireless connectivity component 712.

[0059] An NPU, such as NPU 708, is generally a specialized circuit configured for implementing the control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), and the like. An NPU may sometimes alternatively be referred to as a neural signal processor (NSP), tensor processing unit (TPU), neural network processor (NNP), intelligence processing unit (IPU), vision processing unit (VPU), or graph processing unit.

[0060] NPUs, such as the NPU 708, are configured to accelerate the performance of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, a plurality of NPUs may be instantiated on a single chip, such as a SoC, while in other examples the NPUs may be part of a dedicated neural-network accelerator.P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO15

[0061] NPUs may be optimized for training or inference, or in some cases configured to balance performance between both. For NPUs that are capable of performing both training and inference, the two tasks may still generally be performed independently.

[0062] NPUs designed to accelerate training are generally configured to accelerate the optimization of new models, which is a highly compute-intensive operation that involves inputting an existing dataset (often labeled or tagged), iterating over the dataset, and then adjusting model parameters, such as weights and biases, in order to improve model performance. Generally, optimizing based on a wrong prediction involves propagating back through the layers of the model and determining gradients to reduce the prediction error.

[0063] NPUs designed to accelerate inference are generally configured to operate on complete models. Such NPUs may thus be configured to input a new piece of data and rapidly process this piece of data through an already trained model to generate a model output (e.g., an inference).

[0064] In some implementations, the NPU 708 is a part of one or more of the CPU 702, the GPU 704, and / or the DSP 706.

[0065] In some examples, the wireless connectivity component 712 may include subcomponents, for example, for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G Long-Term Evolution (LTE)), fifth generation connectivity (e.g., 5G or New Radio (NR)), Wi-Fi connectivity, Bluetooth connectivity, and / or other wireless data transmission standards. The wireless connectivity component 712 is further coupled to one or more antennas 714.

[0066] The processing system 700 may also include one or more sensor processing units 716 associated with any manner of sensor, one or more image signal processors (ISPs) 718 associated with any manner of image sensor, and / or a navigation processor 720, which may include satellite-based positioning system components (e.g., GPS or GLONASS), as well as inertial positioning system components.P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO16

[0067] The processing system 700 may also include one or more input and / or output devices 722, such as screens, touch-sensitive surfaces (including touch-sensitive displays), physical buttons, speakers, microphones, and the like.

[0068] In some examples, one or more of the processors of the processing system 700 may be based on an ARM or RISC-V instruction set.

[0069] The processing system 700 also includes the memory 724, which is representative of one or more static and / or dynamic memories, such as a dynamic random access memory, a flash-based static memory, and the like. In this example, the memory 724 includes computer-executable components, which may be executed by one or more of the aforementioned processors of the processing system 700.

[0070] Generally, the processing system 700 and / or components thereof may be configured to perform the methods described herein.

[0071] Notably, in other aspects, elements of the processing system 700 may be omitted, such as where the processing system 700 is a server computer or the like. For example, the multimedia component 710, the wireless connectivity component 712, the sensor processing units 716, the ISPs 718, and / or the navigation processor 720 may be omitted in other aspects. Further, aspects of the processing system 700 may be distributed between multiple devices.Example Clauses

[0072] In addition to the various aspects described above, specific combinations of aspects are within the scope of the disclosure, some of which are detailed below:

[0073] Aspect 1 : A processing system, comprising: a first single instruction multiple data (SIMD) coprocessor including a first plurality of SIMD registers, each of the first plurality of SIMD registers including a first plurality of SIMD lanes, the first SIMD coprocessor configured to: move data from a SIMD register of the plurality of SIMD registers to a memory independent of a cross-lane operation occurring between the first plurality of SIMD lanes of the SIMD register; a second SIMD coprocessor including a second plurality of SIMD registers, each of the second plurality of SIMD registers including a second plurality of SIMD lanes, the second SIMD coprocessor configured to:P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO17move the data from the memory to a SIMD register of the second plurality of SIMD registers; perform movement operations on the data that involve cross-lane operations between the second plurality of SIMD lanes of the SIMD register.

[0074] Aspect 2: The processing system of Aspect 1, wherein: to move the data to the memory, the first SIMD coprocessor is configured to execute an instruction included in a first instruction set architecture; and to perform movement operations on the data, the second SIMD coprocessor is configured to execute one or more instructions included in a second instruction set architecture that is different from the first instruction set architecture.

[0075] Aspect 3: The processing system of Aspect 2, wherein: the first instruction set architecture includes lane-wise only instruction.

[0076] Aspect 4: The processing system of Aspect 2, wherein the second instruction set architecture includes cross-lane instructions.

[0077] Aspect 5: The processing system of Aspect 1, wherein: subsequent to performing the movement operations on the data, the second SIMD coprocessor is configured to move the data to the memory; and the first SIMD coprocessor is configured to move the data from the memory to the SIMD register of the first plurality of SIMD registers.

[0078] Aspect 6: The processing system of Aspect 1, wherein the second SIMD coprocessor is configured with limited access rights to the first plurality of SIMD registers of the first SIMD coprocessor.

[0079] Aspect 7 : The processing system of Aspect 6, wherein the limited access rights include read-only access or no read / write access.

[0080] Aspect 8: The processing system of Aspect 1, wherein a bit width of each of the first plurality of SIMD registers is at least 4 times greater than a bit width of each of the second plurality of SIMD registers.

[0081] Aspect 9: A method performable by a processing system comprising a first single instruction multiple data (SIMD) coprocessor and a second single instructionP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO18multiple data (SIMD) coprocessor, the method comprising: moving, by the first SIMD coprocessor, data from a first SIMD lane of a lane-wise SIMD register of the first SIMD coprocessor to a first segment of a memory; retrieving, by the second SIMD coprocessor, the data from the first segment of the memory and storing the data in a first SIMD lane of a SIMD register of the second SIMD coprocessor that supports intra-SIMD data movement; executing, by the second SIMD coprocessor, one or more instructions to move the data from the first SIMD lane of the SIMD register of the second SIMD coprocessor to a second SIMD lane of the SIMD register of the second SIMD coprocessor; loading, by the second SIMD coprocessor, the data stored in the second SIMD lane of the SIMD register of the second SIMD coprocessor onto a second segment of the memory; and retrieving, by the first SIMD coprocessor, the data stored on the second segment of the memory and storing the data on a second SIMD lane of the lane-wise SIMD register of the first SIMD coprocessor.

[0082] Aspect 10: The method of Aspect 9, wherein moving the data to the first segment of the memory comprises executing, by the first SIMD coprocessor, an instruction included in a first instruction set architecture.

[0083] Aspect 11 : The method of Aspect 10, wherein the one or more instructions to move the data from the first SIMD lane of the SIMD register of the second SIMD coprocessor to a second SIMD lane of the SIMD register of the second SIMD coprocessor are included in a second instruction set architecture.

[0084] Aspect 12: The method of Aspect 11, wherein first instruction set architecture includes lane-wise only instruction.

[0085] Aspect 13: The method of Aspect 11, wherein the second instruction set architecture includes cross-lane instructions.

[0086] Aspect 14: The method of Aspect 9, wherein the second SIMD coprocessor is configured with limited access rights to the lane-wise SIMD register of the first SIMD coprocessor.

[0087] Aspect 15 : The method of Aspect 14, wherein the limited access rights include read-only access or no read / write access.P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO19

[0088] Aspect 16: The method of Aspect 9, wherein a bit width of the lane-wise SIMD register of the first SIMD coprocessor is at least 4 times greater than a bit width of the SIMD register of the second SIMD coprocessor.

[0089] Aspect 17: An apparatus, comprising: a first single instruction multiple data (SIMD) coprocessor including a first plurality of SIMD registers, each of the first plurality of SIMD registers including a first plurality of SIMD lanes, the first SIMD coprocessor configured to: move data from a SIMD register of the plurality of SIMD registers to a memory independent of a cross-lane operation occurring between the first plurality of SIMD lanes of the SIMD register; a second SIMD coprocessor including a second plurality of SIMD registers, each of the second plurality of SIMD registers including a second plurality of SIMD lanes, the second SIMD coprocessor configured to: move the data from the memory to a SIMD register of the second plurality of SIMD registers; perform movement operations on the data that involve cross-lane operations between the second plurality of SIMD lanes of the SIMD register.

[0090] Aspect 18: The apparatus of Aspect 17, wherein: to move the data to the memory, the first SIMD coprocessor is configured to execute an instruction included in a first instruction set architecture; and to perform movement operations on the data, the second SIMD coprocessor is configured to execute one or more instructions included in a second instruction set architecture that is different from the first instruction set architecture.

[0091] Aspect 19: The apparatus of Aspect 18, wherein: the first instruction set architecture includes lane-wise only instruction.

[0092] Aspect 20: The apparatus of Aspect 19, wherein the second instruction set architecture includes cross-lane instructions.Additional Considerations

[0093] The various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software components(s) module(s), including, but not limited to a circuit or processor. Generally, where there are operations illustrated inP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO20figures, those operations may have corresponding counterpart means-plus-function components with similar numbering.

[0094] The preceding description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein are not limiting of the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. For example, changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For instance, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such an apparatus or method that is practiced using other structure, functionality, or structure and functionality in addition to, or other than, the various aspects of the disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0095] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0096] As used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. As an example, “at least one of a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).

[0097] As used herein, the term “determining” encompasses a wide variety of actions. For example, “determining” may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure),P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO21ascertaining, and the like. Also, “determining” may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Also, “determining” may include resolving, selecting, choosing, establishing, and the like.

[0098] The methods disclosed herein comprise one or more steps or actions for achieving the methods. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Further, the various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software component(s) and / or module(s), including, but not limited to a circuit, an application specific integrated circuit (ASIC), or processor. Generally, where there are operations illustrated in figures, those operations may have corresponding counterpart means-plus-function components with similar numbering.

[0099] The following claims are not intended to be limited to the aspects shown herein, but are to be accorded the full scope consistent with the language of the claims. Within a claim, reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. No claim element is to be construed under the provisions of 35 U.S.C. § 112(f) unless the element is expressly recited using the phrase “means for” or, in the case of a method claim, the element is recited using the phrase “step for.” All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims.P+S Ref. No.: QUAL / 2500613PC02

Claims

QUALCOMM Ref. No.: 2500613U2WO22WHAT IS CLAIMED IS:

1. A processing system, comprising:a first single instruction multiple data (SIMD) coprocessor including a first plurality of SIMD registers, each of the first plurality of SIMD registers including a first plurality of SIMD lanes, the first SIMD coprocessor configured to:move data from a SIMD register of the plurality of SIMD registers to a memory independent of a cross-lane operation occurring between the first plurality of SIMD lanes of the SIMD register;a second SIMD coprocessor including a second plurality of SIMD registers, each of the second plurality of SIMD registers including a second plurality of SIMD lanes, the second SIMD coprocessor configured to:move the data from the memory to a SIMD register of the second plurality of SIMD registers;perform movement operations on the data that involve cross-lane operations between the second plurality of SIMD lanes of the SIMD register.

2. The processing system of Claim 1, wherein:to move the data to the memory, the first SIMD coprocessor is configured to execute an instruction included in a first instruction set architecture; andto perform movement operations on the data, the second SIMD coprocessor is configured to execute one or more instructions included in a second instruction set architecture that is different from the first instruction set architecture.

3. The processing system of Claim 2, wherein the first instruction set architecture includes lane-wise only instruction.

4. The processing system of Claim 2, wherein the second instruction set architecture includes cross-lane instructions.

5. The processing system of Claim 1, wherein:subsequent to performing the movement operations on the data, the second SIMD coprocessor is configured to move the data to the memory; andthe first SIMD coprocessor is configured to move the data from the memory to the SIMD register of the first plurality of SIMD registers.P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO236. The processing system of Claim 1, wherein the second SIMD coprocessor is configured with limited access rights to the first plurality of SIMD registers of the first SIMD coprocessor.

7. The processing system of Claim 6, wherein the limited access rights include read-only access or no read / write access.

8. The processing system of Claim 1, wherein a bit width of each of the first plurality of SIMD registers is at least 4 times greater than a bit width of each of the second plurality of SIMD registers.

9. A method performable by a processing system comprising a first single instruction multiple data (SIMD) coprocessor and a second single instruction multiple data (SIMD) coprocessor, the method comprising:moving, by the first SIMD coprocessor, data from a first SIMD lane of a lanewise SIMD register of the first SIMD coprocessor to a first segment of a memory; retrieving, by the second SIMD coprocessor, the data from the first segment of the memory and storing the data in a first SIMD lane of a SIMD register of the second SIMD coprocessor that supports intra-SIMD data movement;executing, by the second SIMD coprocessor, one or more instructions to move the data from the first SIMD lane of the SIMD register of the second SIMD coprocessor to a second SIMD lane of the SIMD register of the second SIMD coprocessor;loading, by the second SIMD coprocessor, the data stored in the second SIMD lane of the SIMD register of the second SIMD coprocessor onto a second segment of the memory; andretrieving, by the first SIMD coprocessor, the data stored on the second segment of the memory and storing the data on a second SIMD lane of the lane-wise SIMD register of the first SIMD coprocessor.

10. The method of Claim 9, wherein moving the data to the first segment of the memory comprises executing, by the first SIMD coprocessor, an instruction included in a first instruction set architecture.

11. The method of Claim 10, wherein the one or more instructions to move the data from the first SIMD lane of the SIMD register of the second SIMD coprocessor to aP+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO24second SIMD lane of the SIMD register of the second SIMD coprocessor are included in a second instruction set architecture.

12. The method of Claim 11, the first instruction set architecture includes lane-wise only instruction.

13. The method of Claim 11, wherein the second instruction set architecture includes cross-lane instructions.

14. The method of Claim 9, wherein the second SIMD coprocessor is configured with limited access rights to the lane-wise SIMD register of the first SIMD coprocessor.

15. The method of Claim 14, wherein the limited access rights include read-only access or no read / write access.

16. The method of Claim 9, wherein a bit width of the lane-wise SIMD register of the first SIMD coprocessor is at least 4 times greater than a bit width of the SIMD register of the second SIMD coprocessor.

17. An apparatus, comprisinga first single instruction multiple data (SIMD) coprocessor including a first plurality of SIMD registers, each of the first plurality of SIMD registers including a first plurality of SIMD lanes, the first SIMD coprocessor configured to:move data from a SIMD register of the plurality of SIMD registers to a memory independent of a cross-lane operation occurring between the first plurality of SIMD lanes of the SIMD register;a second SIMD coprocessor including a second plurality of SIMD registers, each of the second plurality of SIMD registers including a second plurality of SIMD lanes, the second SIMD coprocessor configured to:move the data from the memory to a SIMD register of the second plurality of SIMD registers;perform movement operations on the data that involve cross-lane operations between the second plurality of SIMD lanes of the SIMD register.

18. The apparatus of Claim 17, wherein:P+S Ref. No.: QUAL / 2500613PC02QUALCOMM Ref. No.: 2500613U2WO25to move the data to the memory, the first SIMD coprocessor is configured to execute an instruction included in a first instruction set architecture; andto perform movement operations on the data, the second SIMD coprocessor is configured to execute one or more instructions included in a second instruction set architecture that is different from the first instruction set architecture.

19. The apparatus of Claim 18, wherein the first instruction set architecture includes lane-wise only instruction.

20. The apparatus of Claim 19, wherein the second instruction set architecture includes cross-lane instructions.P+S Ref. No.: QUAL / 2500613PC02

Images