An astronomical certificate authentication computing access optimization method for a CPU-DCU architecture

Abstract

The application discloses a CPU-DCU architecture-oriented astronomical identification calculation memory access optimization method, and steps are as follows: S1, performing grid processing on celestial body data of to-be-identified star catalog A and star catalog B, and respectively vectorizing the star catalog A and the star catalog B in a star catalog-strip-grid hierarchical manner, and recording boundary information of each grid; S2, according to the maximum thread concurrency number of each DCU, dividing the celestial body data in the star catalog A and the star catalog B into a plurality of transmission sub-tasks corresponding to each DCU in a strip unit; S3, setting a calculation task of a single thread in the DCU; S4, setting a kernel function, so that each thread of the DCU completes identification calculation by executing the kernel function; S5, using the method in step S4 to perform identification calculation between celestial bodies in the star catalog A and the star catalog B; the method can improve the memory access performance of astronomical grid calculation on the DCU while ensuring calculation accuracy and integrity, and realizes the improvement of overall calculation performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of high-performance computing technology in astronomy and computer science, in particular to an astronomical identification calculation memory access optimization method for CPU-DCU architecture. BACKGROUND

[0002] The cross-identification of astronomical catalogs is the basis for a series of astronomical researches, and requires large-scale computing resources to achieve an efficient identification process.

[0003] The existing astronomical cross-identification system maps the data to the grid divided on the celestial sphere according to the right ascension and declination of the astronomical data, limits the identification calculation to the grid, avoids the cyclic identification of the entire catalog, and accelerates the cross-identification process on a homogeneous or heterogeneous computing cluster. One problem caused by dividing the grid is that the data at the grid boundary may have cross-grid identification. In the existing system, either the data that may have cross-grid identification is redundantly mapped to the corresponding grid through data redundancy, or cross-grid identification is introduced.

[0004] Currently, the data generated by astronomical telescopes and sky survey projects is increasingly large, and the constructed astronomical data is gradually increasing. The entries of the catalog often reach the level of one billion. The cross-identification of the catalog of the above scale requires memory and computing power beyond the limit of the existing astronomical cross-identification system. In order to achieve cross-identification within a specified time, high-performance computing technology needs to be improved in algorithm and computing power.

[0005] Heterogeneous computing nodes usually contain CPU and co-processor. DCU, as a co-processor, has excellent computing power and can efficiently process computationally intensive tasks.

[0006] On a heterogeneous computing cluster using DCU as a co-processor, the CPU multi-core performance can be better utilized and the efficiency of cross-identification can be improved by designing corresponding data processing strategies. However, through performance analysis of the identification calculation process on DCU, there is still a bottleneck in the memory access process between the global memory and the computing components of DCU. Therefore, how to further improve the data access efficiency of identification calculation on DCU and improve the performance of the astronomical cross-identification algorithm is a technical problem to be solved. SUMMARY

[0007] The purpose of the present application is to provide an astronomical identification calculation memory access optimization method for CPU-DCU architecture to solve the above technical problems.

[0008] To this end, the technical solution of the present application is as follows:

[0009] An astronomical identification calculation memory access optimization method for CPU-DCU architecture, the steps are as follows:

[0010] S1, the celestial data of the star catalog A and the star catalog B to be identified is grid processed, and is vectorized in the hierarchical vector of the star catalog-strip-grid, and the boundary information of each grid is recorded;

[0011] S2, according to the maximum thread concurrency of each DCU, the celestial data in the star catalog A and the star catalog B is divided into a plurality of transmission sub-tasks corresponding to each DCU in units of strips, and a transmission mechanism is set, in which the CPU controls the transmission of the celestial data corresponding to each transmission sub-task from the host memory to the DCU global memory, that is, when the DCU performs the identification calculation of the celestial data of the current transmission sub-task, the CPU can control the transmission of the celestial data corresponding to the next transmission sub-task from the host memory to the DCU global memory;

[0012] S3, the calculation task of a single thread in the DCU is set as: one thread is responsible for the identification calculation of each celestial body in the star catalog A and all celestial bodies in the same grid and all adjacent grids in the star catalog B;

[0013] S4, the kernel function is set, so that each thread of the DCU completes the identification calculation by executing the kernel function; the processing process of the kernel function on the celestial data is set as:

[0014] 1) a shared memory for storing the celestial data in the star catalog B is created, so that the threads in the thread bundle reuse the same data in the shared memory;

[0015] 2) according to the grid to which the celestial body in the star catalog A assigned to the first thread in the thread bundle belongs, the left boundary of the to-be-called grid in the star catalog B is determined, and according to the grid to which the celestial body in the star catalog A assigned to the last thread in the thread bundle belongs, the right boundary of the to-be-called grid in the star catalog B is determined;

[0016] 3) a double buffering mechanism is set, that is, while the celestial data in the star catalog B determined in step S402 is loaded from the global memory of the DCU to the shared memory in batches, the identification calculation of the celestial data already loaded to the shared memory is performed;

[0017] S5, the identification calculation mechanism set in step S4 is used to complete the identification calculation between the celestial bodies in the star catalog A and the star catalog B.

[0018] Further, in step S1, the grid processing method for the celestial data in the star catalog is:

[0019] (1) the strip height h is set, and the strip index zone of each celestial body is calculated according to the declination Dec of the celestial body id :zone id = Dec / h, so as to divide the celestial bodies into strips first;

[0020] (2) Set the error radius r, calculate whether the calculated belonging to the strip when the declination of each celestial body is Dec+r and Dec-r is the same as the belonging to the strip calculated by the declination Dec:

[0021] 1) If the belonging to the strip calculated based on Dec+r or Dec-r is the same as the original belonging to the strip, no source supplement is needed;

[0022] 2) If any of the belonging to the strips calculated based on Dec+r or Dec-r is different, source supplement is needed; the source supplement method is to add the original celestial body data to the different belonging to the strips at the same time;

[0023] (3) For the catalog data in each strip obtained from step (2), calculate the grid to which each celestial body belongs according to the right ascension RA of each celestial body id : Grid id = RA / h, to divide each strip into grids.

[0024] Further, in step S1, the vectorization processing step of the celestial body data in each catalog is:

[0025] (1) The position data of each celestial body is represented by a structure composed of two double type data that store the right ascension and declination in turn, and the identity information of each celestial body is represented by an int64_t type data

[0026] (2) The celestial body data in the catalog is divided into strips, and arranged in the order of the divided strips; then the celestial body data in each strip is divided into grids according to the grid division method described in the above grid method, and arranged in the order of the divided grids; then the celestial body data in each grid is arranged in turn, and each catalog forms an array composed of several double type data and an array composed of several int64_t type data.

[0027] (3) Record the boundary information of each grid in the celestial body data array, that is, the array subscript corresponding to the first celestial body in each grid and the array subscript corresponding to the last celestial body in the array storing the identity information; for example, if a celestial body in the array corresponds to subscript i, the right ascension of the celestial body in the array storing the position information corresponds to subscript 2i, and the declination corresponds to subscript 2i+1;

[0028] Further, in step S2, the division step of the plurality of transmission sub-tasks of each DCU is:

[0029] S201, determine the maximum thread concurrency thread of each DCU max , the calculation formula is:

[0030] thread max = num CU × num SIMD × num wavefront × X,

[0031] where num CU is the number of computing units, num SIMD is the number of SIMD in each computing unit, num wavefront is the number of vector computing units, and X is the size of vector computing units.

[0032] S202, according to the number n dcu of DCUs and the maximum thread concurrency thread max , the data required for DCU calculation is divided;

[0033] S203, for the multiple strips assigned to each DCU, according to the set subtask size, the multiple strips are sequentially divided into multiple transmission subtasks according to the arrangement order of the strips, to be transmitted to the DCU in batches.

[0034] Further, the specific implementation steps of step S202 are:

[0035] S2021, determine the number num i of celestial bodies in each strip in the star catalog, the sum num sum of the number of celestial bodies in all strips, and the number num c of strips, and the maximum thread concurrency S j of each DCU, the sum S sum of the thread concurrency of all DCUs, and the number n dcu of DCUs, wherein the maximum thread concurrency S j of each DCU is substituted into the result thread max calculated by step S301;

[0036] S2022, allocate the strips from high to low according to the maximum thread concurrency S j of each DCU, until the first time the condition is met:

[0037] 0≤start<end<num c , i=start,...,end,

[0038] where start and end are the serial numbers of the first strip and the last strip in the multiple strips assigned to the DCU, respectively.

[0039] Further, in step S4, the setting of the double buffering mechanism is:

[0040] 1) The shared memory is divided into two memory regions, and the two regions are alternately used as a calculation buffer and a data loading buffer; the memory region used as the calculation buffer is used to store the celestial body data loaded into the shared memory, so as to facilitate the thread to call the celestial body data in the shared memory; the memory region used as the data loading buffer is used to load the next batch of data for identification calculation from the global memory of the DCU;

[0041] 2) In each functional role iteration of the two memory regions, the thread reads the celestial body data from the current calculation buffer of the shared memory for identification calculation while loading the next batch of data from the global memory of the DCU into the current data loading buffer;

[0042] 3) The thread is synchronized at each time the functional roles of the two memory regions are alternated.

[0043] Further, in step S5, the thread temporarily stores the single identification result between the corresponding celestial body in the star catalog A and one celestial body in the star catalog B in the register, and when the identification calculation of the star catalog A celestial body and all related celestial bodies in the star catalog B is completed, the thread writes the identification result back to the global memory of the DCU.

[0044] Further, the astronomical identification calculation memory optimization method for the CPU-DCU architecture further includes step S6: when the identification calculation of all celestial bodies in the star catalog A and the celestial bodies in the star catalog B is completed, the identification result is transmitted from the global memory of the DCU back to the host memory, and the CPU controls the identification result to be converted into the ID of each pair of celestial bodies in the star catalog A and the star catalog B, and the conversion process is set as:

[0045] 1) The DCU transmits an array, and the subscript of the array corresponds to the subscript of the star catalog A storage identity information array, and the value corresponding to the subscript is the subscript of the star catalog B storage identity information array.

[0046] 2) Assuming that the array transmitted by the DCU is named cm_result, the array storing the identity information of the A star catalog is a, the array storing the identity information of the B star catalog is b, a certain identification result information is cm_result[i], when each celestial body record has an identification result, the identity information of the star catalog A celestial body corresponding to the star catalog A is ID=a[i], and the identity information of the star catalog B celestial body is ID=b[cm_result[i]], thus, an identification result is converted into an ID pair, corresponding to the identified celestial bodies in the star catalog A and the star catalog B.

[0047] Compared with the prior art, the CPU-DCU architecture-oriented astronomical witness recognition computing memory optimization method can improve the memory efficiency of witness recognition computing on the DCU while ensuring the computing accuracy and integrity, and realizes the improvement of the overall computing performance of astronomical cross-witness recognition. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 A flowchart of the CPU-DCU architecture-oriented astronomical witness recognition computing memory optimization method of the application;

[0049] Figure 2 A data structure diagram of the data after the gridization and vectorization processing of the celestial body data in the star catalog in step S1 of the CPU-DCU architecture-oriented astronomical witness recognition computing memory optimization method of the application,

[0050] Figure 3 A diagram of the data loading mode between a thread bundle and shared memory in step S402 of the CPU-DCU architecture-oriented astronomical witness recognition computing memory optimization method of the application.

[0051] Figure 4 A diagram of the data change in the star catalog division process in steps S1 and S2 of the CPU-DCU architecture-oriented astronomical witness recognition computing memory optimization method of the application. DETAILED DESCRIPTION

[0052] The application will be further described below in combination with the drawings and specific embodiments, but the following embodiments are by no means any limitation on the application.

[0053] Reference Figure 1 The specific implementation steps of the CPU-DCU architecture-oriented astronomical gridization computing memory optimization strategy are as follows:

[0054] S1, gridizing the celestial body data of the star catalog A and the star catalog B to be recognized, and vectorizing in the level of star catalog-strip-grid, and recording the boundary information of each grid;

[0055] Each star catalog to be recognized contains a plurality of celestial bodies, and each celestial body corresponds to a plurality of celestial body data; in this application, the celestial body data involved in the application of each celestial body is: celestial body identity information, ID, and celestial body position information, right ascension RA and declination Dec.

[0056] In this step S1, the gridization processing method of the celestial body data in the star catalog to be recognized is: first, dividing the celestial body data in the star catalog into strips according to the declination and supplementing the missing sources, and then dividing each strip into grids in parallel according to the right ascension.

[0057] In this step S1, the step of dividing the celestial body data into strips according to the declination and supplementing the missing sources is:

[0058] (1) Set the strip height h, calculate the index of the strip to which each celestial body belongs zone id ,

[0059] The specific calculation formula of the strip zone id is:

[0060] zone id = Dec / h,

[0061] In the formula, Dec is the declination of the celestial body, and h is the strip height, which is set to 1 / 60 in this embodiment;

[0062] (2) Set the error radius r, calculate the strip to which the celestial body belongs when the declination is Dec+r and Dec-r, respectively, using the same calculation method as above, and judge whether the calculated strip is the same as the strip corresponding to the declination Dec calculated in step S101:

[0063] 1) If the strip based on Dec+r or Dec-r is the same as the original strip, no missing source supplement is needed;

[0064] 2) If any of the strips based on Dec+r or Dec-r is different, missing source supplement is needed; the missing source supplement method is to add the original celestial body data (i.e. ID, Dec and RA) to different strips at the same time, to solve the problem of missing sources in the declination direction.

[0065] Wherein, the error radius r can be calculated according to the formula: , in which R_A and R_B are the accuracy of the star catalog A and the accuracy of the star catalog B, respectively, and their specific values are: 0.001388889 / 5 (decimal form of degree, the value represents 1 second angle, 5 second angle corresponds to the resolution of the observation equipment, here it represents the accuracy of cross identification relative to the observation accuracy).

[0066] In this step S1, based on the pre-strip division, the step of grid division for the celestial body data in each strip is:

[0067] For the celestial body data in each strip, according to the right ascension RA of each celestial body, calculate the grid Grid id to which each celestial body belongs, and the calculation formula is:

[0068] Grid id = RA / h,

[0069] In the formula, RA is the right ascension of the celestial body, and h is the strip height.

[0070] In step S1, based on the above-described gridding method, the data of celestial objects within each catalog undergoes a hierarchical vectorization process of catalog-strip-grid as follows:

[0071] (1) The declination and right ascension of celestial bodies are stored using double-type data, with each double-type data occupying 8 bytes; furthermore, the position data of each celestial body consists of two doubles storing right ascension and declination in sequence, which can be represented as "double N double N+1 "N≥0; The identity information ID of the celestial body is stored in int64_t type data, and each int64_t type data occupies 8 bytes.

[0072] (2) For the celestial object data in the star catalog, divide it into strips according to the strip division method described in the above gridding method, and divide it into zones. id Arrange them in order; then group each strip into zones. id The celestial data within the data were divided into grids according to the gridding method described above, and then processed according to the divided grid. id Arrange in order; then arrange each grid. id The celestial data within are arranged sequentially, with each star catalog forming an array consisting of several double-type data and an array consisting of several int64_t-type data.

[0073] by Figure 2 As shown in the example, after the vectorization process in step (2), the new celestial body data of all celestial bodies is first presented in strips (e.g., Figure 2 zone0…zone l ) are arranged in order, and then celestial bodies within the same zone are arranged according to their grid (e.g. Figure 2 Within zone0, Grid0…Grid h Arrange them in order, and finally process the celestial data (e.g., ...) within each grid. Figure 3 Inside Grid0, double0, double1, ..., double h Arrange them.

[0074] (3) Record the boundary information of each grid in sequence, namely the array index corresponding to the first celestial body in each grid and the array index corresponding to the last celestial body. This array is an array for storing identity information.

[0075] For example, if a celestial body in the identity information array corresponds to subscript i, the right ascension of the celestial body in the array storing the position information corresponds to subscript 2i, and the declination corresponds to subscript 2i+1; the subscripts are assigned in sequence from number 0 according to the data arrangement order; and then, when the data is called subsequently, the right ascension and the declination of all the celestial bodies in the specified strip and in the specified grid can be quickly called through the subscript.

[0076] In this step S1, by closely storing the right ascension and the declination of each celestial body and closely vectorizing the storage of the above two kinds of data of all the celestial bodies, the data is arranged in advance in a strategic manner. The adjustment of the celestial body data layout can avoid accessing different memory areas when continuously accessing, to a certain extent, reduce memory access transactions; and can ensure memory alignment and improve the efficiency of memory merge access.

[0077] S2, according to the maximum thread concurrency of each DCU, the celestial body data of the two star catalogs processed by step S1 is divided into a plurality of transmission sub-tasks corresponding to each DCU in units of strips, and a transmission mechanism is set in which each transmission sub-task is controlled by the CPU and independently transmitted from the host memory to the DCU global memory;

[0078] The specific processing steps of this step S2 are as follows:

[0079] S201, determine the maximum thread concurrency thread of each DCU max , the calculation formula is:

[0080] thread max =num CU ×num SIMD ×num wavefront ×X,

[0081] In the formula, num CU is the number of computing units, num SIMD is the number of SIMDs in each computing unit, num wavefront is the number of vector computing units, and X is the size of the vector computing unit, similar to the warp_size of the thread bundle (warp) in the GPU structure of Nvidia;

[0082] Among them, the Chinese name of DCU is deep computing unit, in this embodiment, DCU uses a coprocessor produced by Haiguang Information Technology Co., Ltd. (Haiguang), DCU contains a plurality of computing units (CU), and each computing unit contains a plurality of single instruction multiple data processors (SIMD), and the processor is divided into a plurality of vector computing units (wavefront); the size X of the vector computing unit is 64;

[0083] S202、According to the number n of DCUs dcu and the maximum thread concurrency number thread max , the data required for the DCU calculation is divided into each DCU in units of strips to balance the calculation time of each processor;

[0084] The specific steps of data division in this step S202 are:

[0085] S2021, determine the number of celestial bodies num in each strip in the star catalog i , the sum num of the number of celestial bodies in all strips sum and the number of strips num c , and the maximum thread concurrency number S of each DCU j , the sum S of the thread concurrency numbers of all DCUs sum , and the number n of DCUs dcu , wherein the maximum thread concurrency number S of each DCU j is substituted into the result thread max calculated by step S301;

[0086] S2022, according to the maximum thread number S of each DCU j , the strips are allocated from high to low until the first time the condition is met:

[0087] 0≤start<end<num c , i=start,...,end,

[0088] In the formula, start and end are the serial numbers of the first strip and the last strip in the multiple strips allocated to the DCU, respectively;

[0089] S203, for the multiple strips allocated to each DCU, according to the set subtask size, the multiple strips are divided into multiple transmission subtasks in the order of strip arrangement to be transmitted to the DCU in batches, specifically in the DCU global memory.

[0090] Among them, the transmission mechanism of each transmission subtask is set as follows: after the CPU completes the vectorization of the celestial body data in the current transmission subtask and the transmission of the celestial body data from the host memory to the DCU global memory, the DCU can control the authentication calculation of the celestial body data in the current transmission subtask. The CPU can control the transmission of the celestial body data of the next transmission subtask from the host memory to the DCU global memory.

[0091] In this step S2, the celestial body data in the star catalog A and the star catalog B are task-allocated according to the processing capacity of each DCU, while achieving the allocation of the celestial body data in the same band in the two star catalogs to the same DCU, the calculation time of each processor is effectively balanced, and the data transmission efficiency is maximized by setting the transmission mechanism of the subtasks.

[0092] S3, the calculation task of a single thread in the DCU is set as: one thread corresponding to each celestial body in the star catalog A, and the thread is responsible for the identification calculation of the celestial body in the star catalog A and all celestial bodies in the same grid and all adjacent grids in the star catalog B.

[0093] Specifically, based on the maximum thread concurrency thread max in step S2, the calculation formula is that each DCU has num CU calculation units, each calculation unit has num SIMD SIMD, and each SIMD has num wavefront vector calculation units. A group of threads running on the vector calculation unit is called a thread bundle, and the size X of the vector calculation unit is also the size of the thread bundle. In this embodiment, the value of X is 64, that is, each thread bundle contains 64 threads. Therefore, each thread bundle can be allocated at most 64 celestial bodies in the star catalog A in sequence.

[0094] For a single thread, one thread specifies a celestial body in the star catalog A and obtains the right ascension and declination of the celestial body and the grid Grid id to which the celestial body belongs; then, the thread obtains the right ascension and declination information of all celestial bodies in the grid Grid id , Grid id ( the previous grid of the grid Grid id—1 ), and Grid id ( the next grid of the grid Grid id+1 ) in the star catalog B according to the grid Grid id to which the celestial body belongs, so as to sequentially perform identification calculation on the celestial body in the star catalog A and all celestial bodies in the three grids in the star catalog B.

[0095] In this step S3, the task allocation mode of the calculation task is different from the method commonly used in the existing identification calculation. Specifically, compared with the existing method, the calculation task of the present application uses the number of celestial bodies instead of the grid calculation amount as the basis for dividing the calculation task, so that one thread is responsible for the identification calculation between one celestial body in the star catalog A and all related celestial bodies in the star catalog B, which reduces the calculation task amount of each thread on the DCU and helps to improve the calculation performance on the DCU.

[0096] S4, set the kernel function, so that each thread of the DCU completes the authentication calculation by executing the kernel function.

[0097] In this step S4, the processing procedure implemented by the kernel function is set as follows.

[0098] S401, create a shared memory for storing the celestial body data in the star table B, so that the threads inside the warp reuse the same data in the shared memory; specifically, the celestial body data in the star table B is loaded from the DCU global memory into the shared memory.

[0099] In this embodiment, the shared memory applied for each block is a space capable of storing 512 double-type data, totaling 4KB, which can be used to store the right ascension and declination of a maximum of 256 celestial bodies. The size of the block is usually an integer multiple of the warp_size, which is set to 64 in this embodiment. All threads in the block use the same shared memory. The creation of the shared memory in the DCU, the loading of data into the shared memory, and the reuse of data in the shared memory all adopt known methods.

[0100] In this step S401, the created shared memory is pre-loaded with data from the global memory. During loading, the mechanism of combining memory access by multiple threads in the warp can be used to improve data memory efficiency. During calculation on the DCU, the threads directly obtain data from the shared memory, which has lower latency and faster speed than obtaining data from the global memory.

[0101] Specifically, using the shared memory to store the celestial body data of the star table B can ensure the reuse of data by multiple threads in the warp. In the corresponding thread calculation task of the present application, multiple threads will simultaneously access all celestial body data in the same grid in the star table B. When using the shared memory, due to the existence of the broadcast mechanism, a single memory request can make all threads obtain data. When directly obtaining data from the global memory, if it cannot be ensured that all threads in the warp simultaneously access the same address, multiple memory access requests may be generated, and the memory access transaction cannot be combined into one. Therefore, the shared memory can improve the efficiency of thread access to celestial body data in the star table B.

[0102] S402, determine the left boundary of the to-be-called grid in the star table B according to the grid to which the celestial body in the star table A assigned to the first thread in the warp belongs, and determine the right boundary of the to-be-called grid in the star table B according to the grid to which the celestial body in the star table A assigned to the last thread in the warp belongs, so as to load the required celestial body data in the star table B into the shared memory according to the determined left boundary and right boundary of the to-be-called grid.

[0103] For example, Figure 1One thread bundle of the embodiment is taken as an example. Since the thread bundle size of the DCU is 64, each thread bundle can process the identification calculation of 64 celestial bodies in the star table A. The 64 celestial bodies are sequentially arranged and can come from the same grid or different grids. Based on this, the left boundary of the grid to be called in the star table B is determined according to the grid to which the celestial body in the star table A assigned to the first thread in the thread bundle belongs i , that is, the grid Grid i-1 to be called in the star table B. The right boundary of the grid to be called in the star table B is determined according to the grid to which the celestial body in the star table A assigned to the last thread in the thread bundle belongs j , that is, the grid Grid j+1 to be called in the star table B. Then, all the celestial body data in the star table B from the grid Grid i-1 to the grid Grid j+1 are loaded into the shared memory according to the left boundary and the right boundary of the grid to be called, so that all the related celestial bodies in the star table B required by the identification calculation of all the celestial bodies in the star table A in the thread bundle are loaded into the shared memory.

[0104] S403, setting a double buffering mechanism, loading the celestial body data in the star table B determined in step S402 from the global memory of the DCU into the shared memory in batches, and synchronously performing the identification calculation on the celestial body data loaded into the shared memory.

[0105] In this step S403, the setting of the double buffering mechanism is as follows:

[0106] S4031, dividing the shared memory created based on step S401 into two memory areas and alternately using the two memory areas as a calculation buffer and a data loading buffer, that is, during the thread running, the memory area used as the calculation buffer is used to store the celestial body data loaded into the shared memory, so that the thread can call the celestial body data in the shared memory for the identification calculation, and the memory area used as the data loading buffer is used to store the next batch of data loaded from the global memory for the identification calculation;

[0107] S4032, in each iteration of the function role of the two memory areas, the thread reads the celestial body data from the current calculation buffer of the shared memory for the identification calculation, and at the same time, loads the next batch of data from the global memory into the current data loading buffer. Then, the roles of the two buffers are exchanged, that is, the alternation of the calculation and the data loading is continued in the next iteration.

[0108] S4033, synchronizing the threads at each time of alternation of the function roles of the two memory areas to avoid data competition and incorrect access.

[0109] The advantage of the double buffering mechanism of the two memory regions alternately serving as the calculation buffer and the data loading buffer in this step S403 is that the data access and calculation can be parallelized at the thread level, the overlapping data access time and calculation time can hide the delay of memory access, the memory access operation can be performed in parallel with the calculation operation, the time of the thread waiting for data loading can be reduced, and thus the overall calculation efficiency is improved. The specific implementation method of the above two buffers and thread synchronization can be known methods.

[0110] Further, in this step S4, in the intercelestial identification calculation process of the DCU, a variety of strategies of creating shared memory and setting double buffering mechanism to overlap data access and calculation are sequentially adopted to improve the data access efficiency of the thread calculation, and thus the calculation performance of the DCU is effectively improved.

[0111] S5, using the identification calculation mechanism set in step S4, completing the identification calculation between the celestial bodies in the star catalog A and the star catalog B;

[0112] In the identification calculation process, the thread temporarily stores the single identification result of the corresponding celestial body in the star catalog A and one celestial body in the star catalog B in the register, and then writes the identification result back to the DCU global memory when the identification calculation of the star catalog A celestial body and all related celestial bodies in the star catalog B is completed; this data transmission method can avoid multiple modifications of the data in the global memory, and improve the access efficiency of recording and modifying the identification calculation result.

[0113] The specific identification calculation method between two celestial bodies is a known calculation method; since there is a case of identification of a single celestial body with multiple celestial bodies, in this embodiment, three times the space is allocated for the identification result of each celestial body during the identification calculation, that is, up to three identification results can be recorded (the number can be changed, and three is only an example here). In the identification calculation process, three pairs of star bodies corresponding to the three calculation results with the smallest spherical distance d ≤ error radius r are selected as the identification selected results, and are temporarily stored in the register or update the current temporary storage result of the register (when there are less than three successful identification celestial bodies, only the information of the successful identification celestial bodies is recorded);

[0114] The calculation formula of the spherical distance d is:

[0115]

[0116] In the formula, RA1 and RA2 are the right ascensions of the two celestial bodies, and Dec1 and Dec2 are the declinations of the two celestial bodies.

[0117] S6. After all objects in catalog A have completed their verification calculations with objects in catalog B, the verification results are transferred from the DCU's global memory back to the host memory. The CPU then controls the conversion of the verification results into IDs for each pair of objects in catalog A and catalog B, respectively. This conversion process is set as follows:

[0118] 1) The DCU returns an array whose index corresponds to the index of the identity information array stored in star table A, and the value corresponding to the index is the index of the identity information array stored in star table B.

[0119] 2) Let the array returned by the DCU be named cm_result, the array storing the identity information of star catalog A be a, the array storing the identity information of star catalog B be b, and the authentication result information be cm_result[i]. Since each celestial body corresponds to a maximum of three authentication results in this embodiment, the identity information of the celestial body in star catalog A is ID = a[i / 3], and the identity information of the celestial body in star catalog B is ID = b[cm_result[i]] which remains unchanged. Thus, an authentication result is converted into an ID pair, corresponding to the celestial body authenticated in star catalog A and star catalog B.

[0120] To further demonstrate the superiority of the astronomical authentication computation memory access optimization method for CPU-DCU architecture proposed in this application over existing authentication algorithms, comparative experiments are conducted below.

[0121] Specifically, the comparative experiment was conducted on a server with two 8-core CPUs and two Z100L model DCU computing cards. The experiment performed cross-validation on one-eighth of the data in the SDSS dataset and compared the results.

[0122] like Figure 1 As shown, kernel_v2 corresponds to the existing authentication algorithm, while kernel_v3 corresponds to the astronomical authentication computation memory access optimization method of this application; the technical differences between the two algorithms are as follows: Figure 1 The blue box indicates the following: 1) In the kernel_v2 experiment, the right ascension and declination of celestial data in each star catalog are stored separately and called separately in subsequent verification; while in the kernel_v3 experiment, the celestial data in each star catalog is stored in a compact vectorized manner through continuous gridding processing; 2) In the kernel_v2 experiment, the threads on the DCU computing card, corresponding to the verification of a single celestial region, do not use shared memory and double buffering mechanisms; while in the kernel_v3 experiment of this application, the threads, corresponding to the verification of a single celestial body and a single celestial region, use shared memory and double buffering mechanisms to optimize the memory access process of grid computing.

[0123] In addition, the cross-identification running results of the CPU version are added to prove the acceleration effect of using the DCU calculation on the process, the parameters of the grid division in the CPU version code are consistent with the two versions of the DCU code, and the multi-threaded acceleration calculation is used. Since there is no data rearrangement process in the CPU version, only the certification time is compared with the above two experiments.

[0124] The specific experimental results of the three experiments are shown in Table 1.

[0125] Table 1:

[0126]

[0127] In Table 1, the data rearrangement time refers to the time for the CPU to control the gridization and vectorization of the star catalog data in the host memory.

[0128] The host memory to device memory data transfer time refers to the time for the CPU to control the transfer of the vectorized star catalog data from the host memory to the global memory of the DCU.

[0129] The certification calculation time refers to the time for completing the certification calculation of the celestial bodies in the grid, and the kernel_v2 and kernel_v3 are the calculation of this step in the DCU, and the CPU version code uses the CPU multi-thread parallel to perform the calculation of this step.

[0130] From the comparison results in Table 1, it can be seen that under the similar calculation process, the method corresponding to the present application using the DCU has more than 3 times the efficiency improvement compared with the cross-identification using only the CPU. The astronomical certification calculation memory optimization method for the CPU-DCU architecture of the present application in the certification process takes slightly more time than the data rearrangement time in the prior art in the data rearrangement stage, but the data rearrangement details of the present application are associated with the subsequent certification calculation, and based on the new data rearrangement of the present application, the subsequent certification calculation process is significantly lower than the certification calculation process of the prior art; under the condition that the certification task amount is unchanged, the kernel calculation time corresponding to the method of the present application is significantly reduced, proving the effectiveness of the shared memory and double buffering mechanism, and the astronomical grid calculation memory optimization strategy for the CPU-DCU architecture of the present application is more excellent than the existing certification scheme.

[0131] In summary, compared with the existing certification scheme, the astronomical certification calculation memory optimization method for the CPU-DCU architecture of the present application can improve the memory performance of the astronomical grid calculation on the DCU and realize the improvement of the overall calculation performance.

Claims

1. A memory access optimization method for astronomical verification computation in CPU-DCU architecture, characterized in that, The steps are as follows: S1. The celestial data of star catalogs A and B to be verified are processed into grids, and vectorized into star catalog-strip-grid hierarchy respectively, and the boundary information of each grid is recorded. S2. Based on the maximum number of concurrent threads in each DCU, the celestial data in star catalog A and star catalog B are divided into multiple transmission subtasks corresponding to each DCU, using stripes as the unit. Each transmission subtask is controlled by the CPU, and the transmission mechanism from the host memory to the DCU's global memory is set as follows: when the DCU performs the authentication calculation of the celestial data of the current transmission subtask, the CPU can control the transmission of the celestial data corresponding to the next transmission subtask from the host memory to the DCU's global memory. The steps for dividing the multiple transmission subtasks for each DCU are as follows: S201. Determine the maximum number of concurrent threads for each DCU. max The calculation formula is: thread max =num CU ×num SIMD ×num wavefront ×X, In the formula, num CU num is the number of computational units. SIMD The number of SIMDs in each computing unit, num wavefront X represents the number of vector computing units, and X represents the size of the vector computing unit, i.e., the number of threads running on the vector computing unit. S202, based on the number n of DCUs dcu and maximum number of concurrent threads max The data that needs to be fed into the DCU for computation is divided into parts: S2021. Determine the number of celestial objects (num) in each band of the star catalog. i The sum of the number of celestial bodies within all bands, num sum and the number of stripes num c And the maximum number of concurrent threads S per DCU j The sum of the number of concurrent threads in all DCUs, S sum The number of DCUs n dcu Wherein, the maximum number of concurrent threads S per DCU j Substitute the result thread obtained from step S301 max ; S2022, based on the maximum thread concurrency S of each DCU j Distribute strips from high to low until the condition is met for the first time: In the formula, start and end are the sequence numbers of the first and last stripes among the multiple stripes assigned to the DCU, respectively; S203. For multiple stripes allocated to each DCU, according to the set subtask size, the multiple stripes are divided into multiple transmission subtasks in the order of the stripes, so as to transmit them to the DCU in batches. S3. Set the computation task of a single thread in the DCU as follows: each celestial body in star catalog A corresponds to one thread, so that the thread is responsible for the verification calculation of all celestial bodies in star catalog A and all celestial bodies in star catalog B that are located in the same grid and all adjacent grids. S4. Define the kernel function so that each thread of the DCU completes the authentication calculation by executing the kernel function; the kernel function's processing procedure for celestial data is defined as follows: 1) Create a shared memory to store celestial object data in catalog B, so that threads within the thread bundle can reuse the same data in the shared memory; 2) Determine the left boundary of the grid to be called in star list B based on the grid to which the celestial body in star list A belongs, and determine the right boundary of the grid to be called in star list B based on the grid to which the celestial body in star list A belongs, as assigned by the last thread in the thread bundle. 3) Set up a double buffering mechanism to load the celestial data determined in step 2) from the global memory of the DCU into the shared memory in batches in the star catalog B, while performing verification calculations on the celestial data already loaded into the shared memory. S5. Using the verification calculation mechanism set in step S4, complete the verification calculation between celestial bodies in star catalog A and star catalog B.

2. The method for optimizing memory access in astronomical verification computation for CPU-DCU architecture according to claim 1, characterized in that, In step S1, the method for gridding the celestial object data within the star catalog is as follows: (1) Set the strip height h, and calculate the zone index of each celestial body according to its declination Dec. id :zone id =Dec / h, to first divide celestial bodies into stripes; (2) Set the error radius r, and calculate whether the calculated band is the same as the band calculated from the declination Dec when the declination of each celestial body is Dec+r and Dec-r: 1) If the band calculated based on Dec+r or Dec-r is the same as the original band, then no missing source supplementation is required; 2) If any band calculated based on Dec+r or Dec-r is different, then missing source supplementation is required; the missing source supplementation method is to add the original celestial body data to different bands at the same time; (3) For the star catalog data within each strip obtained from step (2), calculate the grid to which each celestial body belongs based on its right ascension RA. id Grid id =RA / h, to divide each strip into a grid.

3. The method for optimizing memory access in astronomical verification computation for CPU-DCU architecture according to claim 2, characterized in that, In step S1, the data of celestial objects within each catalog undergoes a hierarchical vectorization process of catalog-strip-grid as follows: (1) The position data of each celestial body is represented by a structure consisting of two double-type data that store right ascension and declination in sequence, and the identity information of each celestial body is represented by an int64_t type data; (2) Divide the celestial data in the star catalog into stripes and arrange them in the order of the strips; then divide the celestial data in each strip into grids according to the grid division method described above and arrange them in the order of the grids; then arrange the celestial data in each grid in sequence, so that each star catalog forms an array consisting of several double type data and an array consisting of several int64_t type data; (3) Record the boundary information of each grid in the celestial data array, namely the array index corresponding to the first celestial body in each grid and the array index corresponding to the last celestial body. This array is an array for storing identity information.

4. The astronomical verification computation memory access optimization method for CPU-DCU architecture according to claim 1, characterized in that, In step S4, the double buffering mechanism is configured as follows: 1) Divide the shared memory into two memory regions and alternate between them as a computation buffer and a data loading buffer; the memory region serving as the computation buffer is used to store celestial data that has been loaded into the shared memory so that threads can retrieve the celestial data in the shared memory; the memory region serving as the data loading buffer is used to load the next batch of data for verification calculation from the DCU's global memory. 2) In each functional role iteration of the two memory regions, while the thread reads celestial data from the current computation buffer of the shared memory for verification calculation, it loads the next batch of data from the global memory of the DCU into the current data loading buffer. 3) Perform thread synchronization each time the functional roles of the two memory regions are switched.

5. The astronomical verification computation memory access optimization method for CPU-DCU architecture according to claim 1, characterized in that, In step S5, the thread temporarily stores the single verification result between the corresponding celestial body in star catalog A and a celestial body in star catalog B in a register. After the verification calculation between the celestial body in star catalog A and all related celestial bodies in star catalog B is completed, the thread writes the verification result back to the DCU global memory from the register.

6. The astronomical verification computation memory access optimization method for CPU-DCU architecture according to claim 1, characterized in that, It also includes step S6: After all celestial bodies in catalog A have completed the verification calculation with celestial bodies in catalog B, the verification results are transmitted from the global memory of the DCU back to the host memory. The CPU controls the conversion of the verification results into the IDs of each pair of celestial bodies in catalog A and catalog B, respectively. This conversion process is set as follows: 1) For an array returned by the DCU, the array index corresponds to the index of the identity information array stored in star table A, and the value corresponding to the index is the index of the identity information array stored in star table B; 2) Let the array returned by the DCU be named cm_result, the array storing the identity information of star catalog A be a, the array storing the identity information of star catalog B be b, and the authentication result information be cm_result[i]. When each celestial body records an authentication result, the corresponding identity information of the celestial body in star catalog A is ID = a[i], and the identity information of the celestial body in star catalog B is ID = b[cm_result[i]]. Thus, an authentication result is converted into an ID pair, corresponding to the celestial body authenticated in star catalog A and star catalog B.

Images