Data reduction methods, apparatus, devices, and storage media based on vector processors
By dividing and processing data in parallel within a multi-core DSP vector processor, the problem of low reduction efficiency when dealing with large amounts of data is solved, achieving high-performance reduction computation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAT UNIV OF DEFENSE TECH
- Filing Date
- 2023-05-31
- Publication Date
- 2026-07-17
AI Technical Summary
In multi-core DSP vector processors, existing technologies require all data to be loaded into the on-chip storage space of a single core during data reduction. This results in multiple loading operations when the data volume is too large, leading to low reduction efficiency and an inability to fully utilize the DSP's performance.
By identifying the first DSP core to move the data in each reduction operation, the data is moved to the global shared space and divided into multiple sub-data that have a one-to-one correspondence with the second DSP core that performs the reduction operation. The reduction operation is then performed in parallel by multiple second DSP cores to obtain the reduction result data. The result data is then used as the data to be reduced in the next reduction operation, and the process continues.
It enables parallel execution of reduction operations across multiple cores under large data volumes, improving reduction efficiency and fully utilizing the performance of multi-core processors.
Smart Images
Figure CN116627896B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and more specifically, to a data reduction method, apparatus, device, and storage medium based on a vector processor. Background Technology
[0002] A general-purpose multi-core DSP (Digital Signal Processing) vector processor has N different DSP cores, each with its own on-chip vector memory (AM). Multiple DSP cores share a single global shared memory (GSM). In traditional solutions, when multiple cores on a DSP vector processor perform on-chip data reduction, both data segments to be reduced must be loaded into the AM space of a single core before the reduction calculation begins. However, if the total storage space occupied by these two data segments exceeds the on-chip memory space of each core, multiple data loading operations are typically required, resulting in low reduction efficiency.
[0003] Therefore, how to improve the reduction efficiency of multi-core DSP vector processors is a problem that needs to be solved by those skilled in the art. Summary of the Invention
[0004] The purpose of this invention is to provide a data reduction method, apparatus, device, and storage medium based on a vector processor to improve the reduction efficiency of a multi-core DSP vector processor.
[0005] To achieve the above objectives, the present invention provides a data reduction method based on a vector processor, comprising:
[0006] Determine the first DSP core from which the data to be moved is to be transferred in each reduction operation;
[0007] During each reduction operation, the data to be reduced of each first DSP core is moved to the global shared space, and the data to be reduced in the global shared space is divided into multiple sub-data that have a one-to-one correspondence with the second DSP core performing the reduction operation.
[0008] Multiple sub-data that have a one-to-one correspondence with the same second DSP core in the global shared space are moved to the same second DSP core so that each second DSP core can perform reduction operations on the multiple stored sub-data in parallel to obtain reduction result data;
[0009] All reduction result data is moved to the global shared space and used as the data to be reduced in the next reduction operation, and the next reduction operation is then performed.
[0010] Optionally, determining the first DSP core for the data to be moved in each reduction operation includes:
[0011] The maximum number is calculated based on the first capacity of the global shared space and the second capacity of the on-chip vector storage space of each DSP core; wherein, the global shared space can store the reduction data of the maximum number of DSP cores at most;
[0012] Based on the total number of DSP cores participating in the reduction operation and the maximum number mentioned above, determine the number of the first DSP cores to be moved in each reduction operation, as well as the total number of reduction operations.
[0013] Optionally, determining the number of first DSP cores to be transferred in each reduction operation, and the total number of reduction operations, based on the total number of DSP cores participating in the reduction operation and the maximum number, includes:
[0014] The maximum number is taken as the first number of the first DSP cores to be moved in the first reduction operation;
[0015] Calculate the difference between the total number of DSP cores participating in the reduction operation and the first number;
[0016] Determine whether the difference is greater than the quantity obtained by subtracting one from the maximum quantity;
[0017] If so, the quantity obtained by subtracting one from the maximum quantity is used as the second quantity of the first DSP cores to be moved in the intermediate reduction operation, and the quotient of the difference and the second quantity is used as the number of times the intermediate reduction operation is executed, and the remainder of the difference and the second quantity is used as the third quantity of the first DSP cores to be moved in the last reduction operation.
[0018] If not, the difference will be used as the third quantity of data to be moved in the last reduction operation;
[0019] The total number of the first reduction operation, intermediate reduction operations, and the last reduction operation is taken as the total number of reduction operations executed.
[0020] Optionally, after determining the number of first DSP cores to be moved in each reduction operation and the total number of executions of the reduction operation based on the total number of DSP cores participating in the reduction operation and the maximum number, the method further includes: determining the number of second DSP cores that perform the reduction operation in each reduction operation.
[0021] Optionally, determining the number of second DSP cores performing the reduction operation in each reduction operation includes: using the maximum number as the number of second DSP cores performing the reduction operation in each reduction operation.
[0022] Optionally, determining the number of second DSP cores performing the reduction operation in each reduction operation includes: determining the number of second DSP cores performing the reduction operation in each reduction operation based on the number of idle DSP cores; wherein the idle DSP cores are DSP cores that do not store data to be reduced.
[0023] Optionally, before determining the target number of DSP cores participating in each reduction operation, the method further includes:
[0024] The total capacity is calculated based on the total number of DSP cores participating in the protocol operation and the second capacity of the on-chip vector storage space of each DSP core.
[0025] Determine whether the total capacity is greater than the first capacity of the global shared space;
[0026] If so, proceed to the step of determining the target number of DSP cores participating in each reduction operation.
[0027] To achieve the above objectives, the present invention further provides a data reduction device based on a vector processor, comprising:
[0028] The determination module is used to determine the first DSP core for data to be moved in each reduction operation;
[0029] The first transfer module is used to transfer the data to be reduced from each first DSP core to the global shared space when performing each reduction operation;
[0030] The partitioning module is used to divide the data to be reduced in the global shared space into multiple sub-data that have a one-to-one correspondence with the second DSP core that performs the reduction operation;
[0031] The second transfer module is used to transfer multiple sub-data that have a one-to-one correspondence with the same second DSP core in the global shared space to the same second DSP core, so that each second DSP core can perform reduction operations on the multiple stored sub-data in parallel to obtain reduction result data;
[0032] The third migration module is used to migrate all reduction result data to the global shared space and use it as the data to be reduced in the next reduction operation, so that the next reduction operation can continue to be performed.
[0033] To achieve the above objectives, the present invention further provides an electronic device, comprising:
[0034] Memory, used to store computer programs;
[0035] A processor is used to implement the steps of the above-described vector processor-based data reduction method when executing the computer program.
[0036] To achieve the above objectives, the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data reduction method based on a vector processor.
[0037] As can be seen from the above scheme, the present invention provides a data reduction method, apparatus, device, and storage medium based on a vector processor. In this scheme, it is first necessary to determine the first DSP core to be moved in each reduction operation; when performing each reduction operation, the data to be reduced of each first DSP core is moved to a global shared space, and the data to be reduced in the global shared space is divided into multiple sub-data that have a one-to-one correspondence with the second DSP core performing the reduction operation; the multiple sub-data that have a one-to-one correspondence with the same second DSP core in the global shared space are moved to the same second DSP core, so that each second DSP core performs reduction operations on the multiple stored sub-data in parallel to obtain reduction result data; all reduction result data is moved to the global shared space and used as the data to be reduced in the next reduction operation to continue to perform the next reduction operation.
[0038] As can be seen, when performing data reduction using this scheme, each core can be responsible for only a portion of the data reduction task, thereby enabling high-performance reduction calculations of data of different sizes on the chip. When the amount of data to be reduced is large, the reduction operation can be executed in parallel by multiple cores, thus improving the reduction efficiency. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 This is a schematic diagram of data transmission in a general-purpose multi-core DSP vector processor in the prior art.
[0041] Figure 2 This is a schematic flowchart of a data reduction method based on a vector processor disclosed in an embodiment of the present invention;
[0042] Figure 3 This is a schematic diagram of data transfer according to an embodiment of the present invention;
[0043] Figure 4 This is another data transfer diagram disclosed in an embodiment of the present invention;
[0044] Figure 5 This is another data transfer diagram disclosed in an embodiment of the present invention;
[0045] Figure 6 This is another data transfer diagram disclosed in an embodiment of the present invention;
[0046] Figure 7 This is another data transfer diagram disclosed in an embodiment of the present invention;
[0047] Figure 8 This is a schematic diagram of a data reduction device based on a vector processor disclosed in an embodiment of the present invention;
[0048] Figure 9 This is a schematic diagram of an electronic device structure disclosed in an embodiment of the present invention. Detailed Implementation
[0049] See Figure 1 This is a schematic diagram of data transmission in a general-purpose multi-core DSP vector processor in existing technology. Figure 1 As can be seen, this multi-core DSP vector processor includes N DSP cores, each with an on-chip vector storage space (AM). Multiple DSP cores share a global shared storage space (GSM), and DDR (Double Data Rate) is off-chip memory. In traditional solutions, when performing data reduction on data stored in the AM of DSP0 and the AM of DSP1, the data stored in the AM of DSP1 needs to be moved to the AM of DSP0 via the GSM so that DSP0 can perform reduction calculations on these two data segments. However, if the amount of data to be reduced is large, the following two problems arise: First, storing both data segments in the AM space of a single core exceeds the maximum range of the AM space; second, the GSM space cannot accommodate all the data at once. In traditional solutions, to solve this problem, the amount of data transferred each time can only be reduced. However, this method leads to frequent on-chip data transfers, wasting a lot of time, resulting in low data reduction efficiency and failing to fully utilize the DSP's true performance.
[0050] Therefore, this solution discloses a data reduction method, apparatus, device, and storage medium based on a vector processor to improve the reduction efficiency of a multi-core DSP vector processor. Specifically, this solution provides a reduction function in a multi-core environment. After taking the most basic parameters as input, the reduction function can execute the data reduction method based on the vector processor to obtain the reduced data.
[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0052] See Figure 2 This invention provides a schematic flowchart of a data reduction method based on a vector processor; through Figure 2 As can be seen, this solution specifically includes the following steps:
[0053] S101: Determine the first DSP core for data to be transferred in each protocol operation;
[0054] In this embodiment, before determining the target number of DSP cores participating in each reduction operation, the method further includes: calculating the total capacity based on the total number of DSP cores participating in the reduction operation and the second capacity of the on-chip vector storage space of each DSP core; determining whether the total capacity is greater than the first capacity of the global shared space; if so, continuing to execute the step of determining the target number of DSP cores participating in each reduction operation.
[0055] It should be noted that in a multi-core environment, the data to be reduced needs to be prepared into the AM space of the corresponding core. Then, it is determined whether the total capacity of the data to be reduced in the AM space of all cores exceeds the maximum value of the GSM space. If it does not exceed the maximum value, the data of all cores can be calculated at once. If it exceeds the maximum value, the data to be reduced needs to be processed in segments and loops. This segmented loop processing means that the data to be reduced is divided into multiple parts, and each part is reduced through multiple reduction operations. After multiple loops, the final reduction result is obtained.
[0056] Furthermore, when determining the first DSP core to be moved in each reduction operation through S101, it is also necessary to calculate the maximum number based on the first capacity of the global shared space and the second capacity of the on-chip vector storage space of each DSP core; the global shared space can store the data to be reduced of the maximum number of DSP cores; then, based on the total number of DSP cores participating in the reduction operation and the maximum number, the number of the first DSP core to be moved in each reduction operation and the total number of executions of the reduction operation are determined.
[0057] Specifically, this solution calculates the maximum number of GSM_coremax using the formula: GSM_coremax = GSM_speacemax / AM_speacemax. Here, GSM_coremax represents the first capacity of the global shared memory, and AM_speacemax represents the second capacity of the on-chip vector memory. Furthermore, this solution calculates the number of first DSP cores to be moved in each reduction operation, as well as the total number of reduction operations, using the following process:
[0058] The maximum quantity is used as the first quantity of the first DSP cores to be transferred in the first reduction operation; the difference between the total number of DSP cores participating in the reduction operation and the first quantity is calculated; it is determined whether the difference is greater than the quantity obtained by subtracting one from the maximum quantity; if so, the quantity obtained by subtracting one from the maximum quantity is used as the second quantity of the first DSP cores to be transferred in the intermediate reduction operation, and the quotient of the difference and the second quantity is used as the number of times the intermediate reduction operation is executed, and the remainder of the difference and the second quantity is used as the third quantity of the first DSP cores to be transferred in the last reduction operation; if not, the difference is used as the third quantity of the data to be transferred in the last reduction operation; the total number of the first reduction operation, intermediate reduction operation and last reduction operation is used as the total number of times the reduction operation is executed.
[0059] For example: if the second capacity AM_speacemax of the on-chip vector memory space for each DSP core is 768K, the first capacity GSM_speacemax of the global shared space is 6144K, and the total number N of DSP cores participating in the reduction operation is 20, then by using the formula...
[0060] GSM_coremax = GSM_speacemax / AM_speacemax can be calculated to have a maximum number of GSM_coremax, which is 8. This means that the global shared space can store a maximum of 8 DSP cores' data to be reduced. Therefore, 8 is taken as the first number of DSP cores to be moved in the first reduction operation. At this time, the difference between the total number of DSP cores participating in the reduction operation N(20) and the first number (8) is 12. Since 12 is greater than 7, 7 is taken as the second number of DSP cores to be moved in the intermediate reduction operation. The quotient of 12 and 7, 1, is taken as the number of times the intermediate reduction operation is executed. The remainder of 12 and 7, 5, is taken as the third number of DSP cores to be moved in the last reduction operation.
[0061] As can be seen from the above process, three reduction operations are required. The first reduction operation requires moving the data to be reduced from 8 DSP cores to the global shared space. The second reduction operation requires moving the data to be reduced from 7 DSP cores to the global shared space. The last reduction operation requires moving the data to be reduced from 5 DSP cores to the global shared space. Through these three operations, the calculation of the data to be reduced in 20 DSP cores can be completed.
[0062] It should be noted that, in order to distinguish between the DSP cores that transfer data and those that perform reduction operations, this embodiment defines the DSP core that transfers data as the first DSP core and the DSP core that performs reduction operations as the second DSP core. Furthermore, in this embodiment, the first and second DSP cores can be determined based on the core ID (IdentityDocument, unique code) of each DSP core. For example, if the total number N of DSP cores participating in the reduction operation is 20, then the core IDs of these 20 DSP cores are set as core 0, core 1, core 2, core 3...core 19. In this case, when selecting the first DSP core, it can be selected in ascending order of the core IDs. For example, in the first reduction operation, cores 0 to 7 are selected as the first DSP core; in the intermediate reduction operations, cores 8 to 14 are selected as the first DSP core; and in the last reduction operation, cores 15 to 19 can be selected as the first DSP core.
[0063] S102: When performing each reduction operation, the data to be reduced of each first DSP core is moved to the global shared space, and the data to be reduced in the global shared space is divided into multiple sub-data that have a one-to-one correspondence with the second DSP core performing the reduction operation.
[0064] In this embodiment, when performing each reduction operation, it is necessary to determine the number of first DSP cores whose data needs to be moved in this reduction operation. As shown above, if the first reduction operation requires moving the data to be reduced from 8 DSP cores to the global shared space, then the data to be reduced from the 8 first DSP cores needs to be moved to the global shared space. Then, according to the number of second DSP cores performing the reduction operation, the data to be reduced in the global shared space is divided into multiple sub-data. The number of sub-data is the same as the number of second DSP cores, and each sub-data has a one-to-one relationship with the corresponding second DSP core. The correspondence is as follows: It should be noted that the data to be reduced includes the data moved from each first DSP core and the reduction result data of the previous reduction operation. That is, this solution needs to divide each data to be reduced moved from the first DSP core in the global shared space into multiple sub-data that have a one-to-one correspondence with the second DSP core that performed the reduction operation. If the global shared space includes the reduction result data of the previous reduction operation, then the reduction result data also needs to be used as the data to be reduced and divided into multiple sub-data that have a one-to-one correspondence with the second DSP core that performed the reduction operation.
[0065] It should be noted that the timing for determining the second DSP core to perform the reduction operation in this scheme can be determined before each reduction operation or when the total number of reduction operations is determined. However, to improve reduction efficiency, the second DSP core can be determined after the total number of reduction operations is determined.
[0066] In this embodiment, the following two methods are provided for determining the second DSP core:
[0067] The first method involves using the maximum number of DSP cores as the number of second DSP cores to perform the reduction operation in each reduction operation. The second DSP cores determined in this way can be the first number of first DSP cores whose data is to be moved in the first reduction operation. For example, if the global shared space can store the data to be reduced on a maximum of 8 DSP cores, and the data to be reduced on 8 DSP cores (core 0 to core 7) is moved to the global shared space in the first reduction operation, then cores 0 to 7 are used as the second DSP cores to perform the reduction operation. In each reduction operation, each data to be reduced is divided into 8 sub-data that have a one-to-one correspondence with cores 0 to 7, so that the reduction operation can be performed in parallel by cores 0 to 7.
[0068] The second method: Determine the number of second DSP cores to perform the reduction operation in each reduction operation based on the number of idle DSP cores; these idle DSP cores are those that do not store data to be reduced. For example, if the global shared space can store data to be reduced from a maximum of 8 DSP cores, and during the first reduction operation, the data to be reduced from 8 DSP cores (core 0 to core 7) is moved to the global shared space, then cores 0 to 7 are idle DSP cores, and the number of idle DSP cores is 8. These 8 idle DSP cores are then all used as second DSP cores. During the reduction operation, the data to be reduced from each first DSP core is divided into 8 sub-data that have a one-to-one correspondence with cores 0 to 7, so that core 0... Cores 0-14 perform reduction operations in parallel. In the next reduction operation, if the data to be reduced from cores 8-14 is moved to the global shared space, then cores 0-14 become idle DSP cores, with a total of 15 idle DSP cores. These 15 idle DSP cores are then used as second DSP cores. During the reduction operation, the data to be reduced from each first DSP core can be divided into 15 sub-data segments, each corresponding one-to-one with cores 0-14, allowing the reduction operation to be performed in parallel by cores 0-14. It should be noted that when determining the number of second DSP cores based on the number of idle DSP cores, all idle DSP cores can be used as second DSP cores, or only a portion of the DSP cores can be used; there is no specific limitation here.
[0069] S103: Move multiple sub-data that have a one-to-one correspondence with the same second DSP core in the global shared space to the same second DSP core, so that each second DSP core can perform reduction operations on the multiple stored sub-data in parallel to obtain the reduction result data;
[0070] In this embodiment, multiple sub-data items that have a one-to-one correspondence with the same second DSP core in the global shared space need to be moved to the same second DSP core. This way, each second DSP core stores a portion of the data to be reduced from each first DSP core. By having multiple second DSP cores simultaneously perform reduction operations on the stored data, parallel reduction processing of the data to be reduced from multiple first DSP cores can be achieved, improving processing efficiency. It should be noted that the reduction operation performed on the data in this embodiment can be addition, finding the maximum value, finding the minimum value, etc., and is not specifically limited here.
[0071] S104: Move all reduction result data to the global shared space and use it as the data to be reduced in the next reduction operation, and continue to execute the next reduction operation.
[0072] It should be noted that each loop operation is executed through S102-S104. The difference is that in the first reduction operation, no data is stored in the GSM space. Therefore, in the first reduction operation, the data to be reduced only includes the data to be reduced moved from the first DSP core. In each subsequent reduction operation, the reduction result data of the previous reduction operation is stored. Therefore, in each subsequent reduction operation, the data to be reduced includes the reduction result data of the previous reduction operation, as well as the data moved from multiple first DSP cores when performing the current reduction operation. After multiple reduction operations, the final reduction result data can be obtained.
[0073] In summary, this solution discloses an on-chip adaptive parallel reduction method for vector processors. It fully utilizes the concept of multi-core segmentation, allowing each core to handle only a small portion of the data segment's reduction task, thus enabling high-performance reduction computation on-chip for data of varying sizes. This solution improves reduction efficiency when dealing with large datasets by executing reduction operations in parallel across multiple cores.
[0074] To clearly illustrate this solution, a specific embodiment is provided to explain the execution process of this solution.
[0075] See Figure 3 This is a schematic diagram of data transfer according to an embodiment of the present invention. Figure 3 In this scheme, the total number of DSP cores participating in the reduction operation is N, which is 20. The second capacity of the on-chip vector storage space of each DSP core is AM_speacemax, which is 768K. The first capacity of the global shared space is GSM_speacemax, which is 6144K. The total capacity is determined by multiplying N by AM_speacemax. After calculation, the total capacity is 20 * 768K = 15360K. Since 15360K is greater than GSM_speacemax, the data to be reduced from the 20 DSP cores cannot be moved to the GSM space at once. Therefore, this scheme can be used for fast data reduction.
[0076] The maximum number of GSM_coremax is calculated to be 8 using the formula GSM_coremax = GSM_speacemax / AM_speacemax. This means that the global shared space can store a maximum of 8 DSP cores' data to be reduced. Therefore, 8 is taken as the first number of DSP cores to be moved in the first reduction operation. At this time, the difference between the total number of DSP cores participating in the reduction operation N(20) and the first number (8) is 12. Since 12 is greater than 7, 7 is taken as the second number of DSP cores to be moved in the intermediate reduction operation. The quotient of 12 and 7, 1, is taken as the number of times the intermediate reduction operation is executed. The remainder of 12 and 7, 5, is taken as the third number of DSP cores to be moved in the last reduction operation. In other words, in this embodiment, three reduction operations need to be performed. The first reduction operation requires moving the data to be reduced from 8 DSP cores (core 0 to core 7) to the global shared space. The second reduction operation requires moving the data to be reduced from 7 DSP cores (core 8 to core 14) to the global shared space. The last reduction operation requires moving the data to be reduced from 5 DSP cores (core 15 to core 19) to the global shared space. Through these three operations, the calculation of the data to be reduced in 20 DSP cores can be completed.
[0077] See Figure 4 This is another data transfer diagram disclosed in an embodiment of the present invention, through... Figure 4 It can be seen that during the initial protocol operation, the data to be protocolized from cores 0 to 7 needs to be moved to the global shared space. For ease of explanation, the data to be protocolized from core 0 corresponding to layer 0 of GSM is 0-0 to 0-7, and the data to be protocolized from core 1 corresponding to layer 1 of GSM is 1-0 to 1-7, and so on. In this embodiment, the maximum number of 8 is used as the number of second DSP cores that perform the reduction operation in each reduction operation, and these 8 second DSP cores are: core 0 to core 7. At this time, the data of each layer of GSM needs to be divided into 8 sub-data. For example, the 0th layer of GSM is divided into: sub-data 0-0, sub-data 0-1, sub-data 0-2, sub-data 0-3, sub-data 0-4, sub-data 0-5, sub-data 0-6, and sub-data 0-7. Sub-data 0-0 has a one-to-one correspondence with core 0, sub-data 0-1 has a one-to-one correspondence with core 1, sub-data 0-2 has a one-to-one correspondence with core 2, and so on. The data of each layer of GSM is divided into 8 sub-data, and the 8 sub-data have a one-to-one correspondence with 8 second DSP cores.
[0078] See Figure 5This is another data migration diagram disclosed in an embodiment of the present invention. During the initial reduction operation, the data to be reduced from cores 0 to 7 is moved to the global shared space. The data to be reduced from each core in the global shared space is divided into eight sub-data items with a one-to-one correspondence with cores 0 to 7. There are eight sub-data items with a one-to-one correspondence with core 0: sub-data 0-0 in core 0, sub-data 1-0 in core 1, 2-0 in core 2, 3-0 in core 3, 4-0 in core 4, 5-0 in core 5, 6-0 in core 6, and 7-0 in core 7. All eight sub-data items with a one-to-one correspondence with core 0 are moved to core 0. At this point, core 0 stores a portion of the original data to be reduced from cores 0 to 7: sub-data 0-0 to sub-data 7-0. Similarly, other sub-data items are moved to their corresponding cores according to their one-to-one correspondence with cores 1 to 7. Details of the data stored in each core after the migration are provided below. Figure 5 AM space. For example... Figure 5 As shown, when core 0 performs the reduction operation, it can calculate the sum of subdata 0-0 + subdata 1-0 + ... + subdata 7-0, and use the final calculated value as the reduction result data. Here, the reduction result data calculated by core 0 is called core_num0. Similarly, the reduction result data of cores 1 to 7 are calculated as core_num1 to core_num7.
[0079] See Figure 6 This is another data transfer diagram disclosed in an embodiment of the present invention. See [link / reference]. Figure 6 In the AM space, data 0-0 to data 0-7 represent the reduction result data core_num0 to core_num7, respectively. In this embodiment, since the final reduction result needs to be obtained multiple times, the reduction result data needs to be moved to the global shared space and used as the data to be reduced in the next reduction operation. Figure 6 As shown, after the first reduction operation, the obtained reduction result data 0-0 to 0-7 is moved to layer 0 of the GSM space, and the next reduction operation is performed.
[0080] See Figure 7 This is another data transfer diagram disclosed in an embodiment of the present invention, such as... Figure 7 As shown, when performing the next reduction operation, the 0th layer of the GSM space has already stored the data to be reduced. This data to be reduced is the reduction result data calculated by the previous reduction operation. The data to be reduced of the 7 DSP cores (core 8 to core 14) that need to be moved in this reduction operation is moved to the 1st to 7th layers of the global shared space. Then, the reduction operation is continued to be performed according to the steps of the previous cycle until the last reduction operation is completed, and the final reduction result data is obtained.
[0081] In summary, each loop of this scheme uses the GSM space as an intermediate medium, treating cores 0-7, 8-14, and 15-19 as a whole. Data is transmitted into and out of the GSM space, and then processed on fixed cores according to their core IDs. This approach improves code readability and reusability. Furthermore, this scheme utilizes the characteristics of the GSM space to allow data exchange between multiple cores, thus placing data from different cores on the same core for processing. This approach fully leverages the concept of multi-core parallelism, significantly improving the performance of the DSP cores. Moreover, this scheme can obtain the final reduced result data by executing the above process through a reduction function, making the reduction method simpler and more convenient than conventional reduction methods.
[0082] The data reduction apparatus, device, and storage medium provided in the embodiments of the present invention will be described below. The data reduction apparatus, device, and storage medium described below can be referred to in conjunction with the data reduction method described above, and will not be repeated here.
[0083] See Figure 8 The present invention provides a schematic diagram of a data reduction device based on a vector processor, the device comprising:
[0084] Module 11 is used to determine the first DSP core to be moved in each reduction operation;
[0085] The first transfer module 12 is used to transfer the data to be reduced of each first DSP core to the global shared space when performing each reduction operation;
[0086] The partitioning module 13 is used to divide the data to be reduced in the global shared space into multiple sub-data that have a one-to-one correspondence with the second DSP core that performs the reduction operation;
[0087] The second transfer module 14 is used to transfer multiple sub-data that have a one-to-one correspondence with the same second DSP core in the global shared space to the same second DSP core, so that each second DSP core can perform reduction operations on the multiple stored sub-data in parallel to obtain reduction result data;
[0088] The third transfer module 15 is used to transfer all reduction result data to the global shared space and use it as the data to be reduced in the next reduction operation, so as to continue to execute the next reduction operation.
[0089] The determining module includes:
[0090] The calculation unit is used to calculate the maximum number based on the first capacity of the global shared space and the second capacity of the on-chip vector storage space of each DSP core; wherein the global shared space can store the reduction data of the maximum number of DSP cores at most;
[0091] The first determining unit is used to determine the number of first DSP cores to be moved in each reduction operation, and the total number of executions of the reduction operation, based on the total number of DSP cores participating in the reduction operation and the maximum number thereon.
[0092] The determining unit includes:
[0093] The first determining subunit is used to take the maximum number as the first number of the first DSP cores of the data to be moved in the first reduction operation;
[0094] A calculation subunit is used to calculate the difference between the total number of DSP cores participating in the reduction operation and the first number.
[0095] The determination subunit is used to determine whether the difference is greater than the quantity obtained by subtracting one from the maximum quantity; if yes, the second determination subunit is triggered; if no, the third determination subunit is triggered.
[0096] The second determining subunit is used to take the quantity obtained by subtracting one from the maximum quantity as the second quantity of the first DSP cores to be moved in the intermediate reduction operation, and to take the quotient of the difference and the second quantity as the number of times the intermediate reduction operation is executed, and to take the remainder of the difference and the second quantity as the third quantity of the first DSP cores to be moved in the last reduction operation.
[0097] The third determining subunit is used to use the difference as the third quantity of data to be moved in the last reduction operation;
[0098] The fourth determination subunit is used to take the total number of the first reduction operation, the intermediate reduction operations and the last reduction operation as the total number of reduction operations executed.
[0099] The determining module also includes:
[0100] The second determining unit is used to determine the number of second DSP cores that perform the reduction operation in each reduction operation.
[0101] Specifically, the second determining unit is used to: use the maximum number as the number of second DSP cores that perform the reduction operation in each reduction operation.
[0102] Specifically, the second determining unit is used to: determine the number of second DSP cores that perform the reduction operation in each reduction operation based on the number of idle DSP cores; the idle DSP cores are DSP cores that do not store data to be reduced.
[0103] The device further includes:
[0104] The calculation module is used to calculate the total capacity based on the total number of DSP cores participating in the reduction operation and the second capacity of the on-chip vector storage space of each DSP core.
[0105] The judgment module is used to determine whether the total capacity is greater than the first capacity of the global shared space; if so, the determination module is triggered.
[0106] See Figure 9 The present invention provides a schematic diagram of an electronic device structure, comprising:
[0107] Memory 21 is used to store computer programs;
[0108] The processor 22 is used to implement the steps of the data reduction method based on the vector processor described in any of the above method embodiments when executing the computer program.
[0109] In this embodiment, the device can be a PC (Personal Computer), or a terminal device such as a smartphone, tablet computer, handheld computer, or portable computer.
[0110] The device may include a memory 21, a processor 22, and a bus 23.
[0111] The memory 21 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 21 can be an internal storage unit of the device, such as the hard disk of the device. In other embodiments, the memory 21 can also be an external storage device of the device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 21 can include both internal and external storage units of the device. The memory 21 can be used not only to store application software and various types of data installed on the device, such as program code executing data specification methods, but also to temporarily store data that has been output or will be output.
[0112] In some embodiments, processor 22 may be a central processing unit (CPU), controller, microcontroller, microprocessor or other data processing chip, used to run program code stored in memory 21 or process data, such as program code for executing data reduction methods.
[0113] This bus 23 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0114] Furthermore, the device may also include a network interface 24, which may optionally include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), typically used to establish communication connections between the device and other electronic devices.
[0115] Optionally, the device may further include a user interface 25, which may include a display, an input unit such as a keyboard, and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the device and to display a visual user interface.
[0116] Figure 9 Only devices with components 21-25 are shown; those skilled in the art will understand that... Figure 9 The structure shown does not constitute a limitation on the device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0117] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data reduction method based on a vector processor as described in any of the above method embodiments.
[0118] The storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0120] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data reduction method based on a vector processor, characterized in that, include: Determine the first DSP core from which the data to be moved is to be transferred in each protocol operation; During each reduction operation, the data to be reduced of each first DSP core is moved to the global shared space, and the data to be reduced in the global shared space is divided into multiple sub-data that have a one-to-one correspondence with the second DSP core performing the reduction operation. Multiple sub-data that have a one-to-one correspondence with the same second DSP core in the global shared space are moved to the same second DSP core so that each second DSP core can perform reduction operations on the multiple stored sub-data in parallel to obtain reduction result data; All reduction result data is moved to the global shared space and used as the data to be reduced in the next reduction operation, and the next reduction operation is performed. The step of determining the first DSP core for data to be moved in each reduction operation includes: The maximum number is calculated based on the first capacity of the global shared space and the second capacity of the on-chip vector storage space of each DSP core; wherein, the global shared space can store the reduction data of the maximum number of DSP cores at most; Based on the total number of DSP cores participating in the reduction operation and the maximum number mentioned above, determine the number of the first DSP cores to be moved in each reduction operation, as well as the total number of executions of the reduction operation; The step of determining the number of first DSP cores to be moved in each reduction operation, based on the total number of DSP cores participating in the reduction operation and the maximum number of DSP cores, and the total number of executions of the reduction operation, includes: The maximum number is taken as the first number of the first DSP cores to be moved in the first reduction operation; Calculate the difference between the total number of DSP cores participating in the reduction operation and the first number; Determine whether the difference is greater than the quantity obtained by subtracting one from the maximum quantity; If so, the quantity obtained by subtracting one from the maximum quantity is used as the second quantity of the first DSP cores to be moved in the intermediate reduction operation, and the quotient of the difference and the second quantity is used as the number of times the intermediate reduction operation is executed, and the remainder of the difference and the second quantity is used as the third quantity of the first DSP cores to be moved in the last reduction operation. If not, the difference will be used as the third quantity of data to be moved in the last reduction operation; The total number of the first reduction operation, intermediate reduction operations, and the last reduction operation is taken as the total number of reduction operations executed.
2. The data reduction method according to claim 1, characterized in that, After determining the number of first DSP cores to be moved in each reduction operation based on the total number of DSP cores participating in the reduction operation and the maximum number, and the total number of reduction operations, the method further includes: Determine the number of second DSP cores that perform the reduction operation in each reduction operation.
3. The data reduction method according to claim 2, characterized in that, Determining the number of second DSP cores performing the reduction operation in each reduction operation includes: The maximum number is used as the number of second DSP cores that perform the reduction operation in each reduction operation.
4. The data reduction method according to claim 2, characterized in that, Determining the number of second DSP cores performing the reduction operation in each reduction operation includes: The number of second DSP cores performing the reduction operation in each reduction operation is determined based on the number of idle DSP cores; the idle DSP cores are DSP cores that do not store data to be reduced.
5. The data reduction method according to any one of claims 1 to 4, characterized in that, Before determining the target number of DSP cores participating in each reduction operation, the following steps are also included: The total capacity is calculated based on the total number of DSP cores participating in the protocol operation and the second capacity of the on-chip vector storage space of each DSP core. Determine whether the total capacity is greater than the first capacity of the global shared space; If so, proceed to the step of determining the target number of DSP cores participating in each reduction operation.
6. A data reduction device based on a vector processor, characterized in that, include: The determination module is used to determine the first DSP core for data to be moved in each reduction operation; The first transfer module is used to transfer the data to be reduced from each first DSP core to the global shared space when performing each reduction operation; The partitioning module is used to divide the data to be reduced in the global shared space into multiple sub-data that have a one-to-one correspondence with the second DSP core that performs the reduction operation; The second transfer module is used to transfer multiple sub-data that have a one-to-one correspondence with the same second DSP core in the global shared space to the same second DSP core, so that each second DSP core can perform reduction operations on the multiple stored sub-data in parallel to obtain reduction result data; The third migration module is used to migrate all reduction result data to the global shared space and use it as the data to be reduced in the next reduction operation, so that the next reduction operation can continue to be performed. The determining module includes: The calculation unit is used to calculate the maximum number based on the first capacity of the global shared space and the second capacity of the on-chip vector storage space of each DSP core; wherein the global shared space can store the reduction data of the maximum number of DSP cores at most; The first determining unit is used to determine the number of first DSP cores to be moved in each reduction operation, and the total number of executions of the reduction operation, based on the total number of DSP cores participating in the reduction operation and the maximum number thereon. The first determining unit includes: The first determining subunit is used to take the maximum number as the first number of the first DSP cores of the data to be moved in the first reduction operation; A calculation subunit is used to calculate the difference between the total number of DSP cores participating in the reduction operation and the first number. The determination subunit is used to determine whether the difference is greater than the quantity obtained by subtracting one from the maximum quantity; if yes, the second determination subunit is triggered; if no, the third determination subunit is triggered. The second determining subunit is used to take the quantity obtained by subtracting one from the maximum quantity as the second quantity of the first DSP cores to be moved in the intermediate reduction operation, and to take the quotient of the difference and the second quantity as the number of times the intermediate reduction operation is executed, and to take the remainder of the difference and the second quantity as the third quantity of the first DSP cores to be moved in the last reduction operation. The third determining subunit is used to use the difference as the third quantity of data to be moved in the last reduction operation; The fourth determination subunit is used to take the total number of the first reduction operation, the intermediate reduction operations and the last reduction operation as the total number of reduction operations executed.
7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data reduction method based on a vector processor as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data reduction method based on a vector processor as described in any one of claims 1 to 5.