Hbm-based near-memory computing bandwidth optimization method and system

Abstract

This invention relates to the field of near-memory computing technology, and discloses a method and system for near-memory computing bandwidth optimization based on HBM. The method includes: decoding and parsing instructions for access tasks on the host side to obtain the storage address and the required operation type; constructing metadata for the access tasks to obtain a scheduling context, and pre-selecting target processing elements in high-bandwidth memory; performing data storage access on the high-bandwidth memory to obtain data blocks to be processed, and performing information dimensionality reduction mapping on the data blocks to obtain partial result data; performing multi-path fusion and reduction on the partial result data to obtain final result data, and returning the final result data to the host side; while performing multi-path fusion and reduction, pre-fetching scheduling is performed on the access tasks to be processed on the host side to establish a scheduling context for the access tasks to be processed; this invention can improve the efficiency of near-memory computing bandwidth optimization based on HBM.

Description

Technical Field

[0001] This invention relates to the field of near-memory computing technology, and in particular to a near-memory computing bandwidth optimization method and system based on HBM. Background Technology

[0002] In existing near-memory computing architectures, the instruction parsing for host-side access tasks lacks standardized adaptation logic, and the pre-selection of metadata construction and processing components does not take into account the characteristics of the storage layer space, resulting in a timing disconnect between data access and computation operations. Furthermore, the lack of dimensional optimization in the data processing stage leads to redundant feature dimensions, directly increasing the resource consumption for data transmission and computation.

[0003] The data reduction process does not build an efficient topology based on the physical layout of processing components. The lengthy reduction path leads to low data aggregation efficiency. Task execution and prefetching scheduling do not form a pipeline collaboration, and HBM bandwidth resources cannot be fully released. Therefore, how to improve the bandwidth utilization efficiency of near-memory computing based on HBM has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a near-memory computing bandwidth optimization method and system based on HBM to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a near-memory computing bandwidth optimization method based on HBM, comprising: The access task on the host side is decoded and parsed to obtain the storage address and required operation type of the access task; Based on the storage address and the required operation type, metadata is constructed for the access task to obtain the scheduling context of the access task, and the target processing element in the high-bandwidth memory is pre-selected based on the scheduling context. Based on the target processing element, data storage access is performed on the high-bandwidth memory to obtain the data block to be processed in the high-bandwidth memory, and information dimensionality reduction mapping is performed on the data block to be processed to obtain partial result data of the data block to be processed. The partial result data is fused and reduced through multiple channels to obtain the final result data of the data block to be processed, and the final result data is returned to the host. While performing the multi-path fusion and reduction, the pending access tasks on the host side are prefetched and scheduled to establish the scheduling context of the pending access tasks.

[0006] In a preferred embodiment, the step of decoding and parsing the access task on the host side to obtain the storage address and required operation type of the access task includes: Receive access tasks sent by the host through the physical interface of the high-bandwidth memory; The opcodes in the access task are decoded and parsed to obtain the required operation type of the access task; The address code in the access task is decomposed into address fields to obtain the storage layer number, storage bank selection signal and storage cell row address of the access task; The storage layer number, the storage bank selection signal, and the storage cell row address are recombined and integrated to obtain the storage address of the access task.

[0007] In a preferred embodiment, the step of constructing metadata for the access task based on the storage address and the required operation type to obtain the scheduling context of the access task includes: Assign a context identifier to the access task, and associate and bind the storage address, the required operation type, and the context identifier to obtain the initial context record of the access task; Based on the storage layer number in the storage address, determine the processing element identifier corresponding to the storage layer number, and add the processing element identifier to the initial context record; Configure the index space for the initial context record to obtain the buffer index field of the initial context record; The scheduling context of the access task is obtained by aggregating and encapsulating the complete information of the context identifier, the storage address, the required operation type, the processing element identifier, and the buffer index field.

[0008] In a preferred embodiment, the step of pre-selecting target processing elements in the high-bandwidth memory based on the scheduling context includes: Based on the storage layer number in the scheduling context, the spatial location of the processing element in the high-bandwidth memory and the storage layer in the high-bandwidth memory is resolved to obtain the proximity relationship between the processing element and the storage layer; Based on the proximity relationship, the processing elements are initially selected to obtain candidate processing elements for the high-bandwidth memory; Processing elements whose current load state value is lower than a preset load threshold are selected as target processing elements for the high-bandwidth memory.

[0009] In a preferred embodiment, the step of performing data storage access on the high-bandwidth memory based on the target processing element to obtain the data block to be processed in the high-bandwidth memory includes: Based on the target processing element, extract the buffer address from the scheduling context; A data read request is sent to the temporary storage space pointed to by the buffer address through the internal data path of the logical base layer in the high-bandwidth memory. The data read request is parsed using the bus protocol to obtain the workflow identifier of the data read request; Based on the workflow identifier, query the mapping relationship between the workflow identifier and the data block storage location in the high-bandwidth memory to confirm the storage location index of the data read request; Based on the storage location index, the high-bandwidth memory is addressed and located to obtain the data block to be processed in the high-bandwidth memory.

[0010] In a preferred embodiment, the step of performing information dimensionality reduction mapping on the data block to be processed to obtain partial result data of the data block to be processed includes: The data block to be processed is mapped to zero mean to obtain a centered data matrix of the data block to be processed. Based on the centralized data matrix, calculate the covariance matrix between the original feature dimensions in the data block to be processed; The covariance matrix is ​​decomposed into eigenvalues, and the eigenvectors are reassembled in order of the numerical values ​​of the decomposed eigenvalues ​​to obtain the projection matrix of the decomposed eigenvectors. Perform a matrix inner product between the centralized data matrix and the projection matrix to obtain partial result data of the data block to be processed.

[0011] In a preferred embodiment, the covariance matrix is ​​calculated using the following formula: ; In the formula, Let be the covariance matrix. Here, represents the number of data samples contained in the data block to be processed, and represents the number of original feature dimensions of the data samples in the data block to be processed. For the first data block to be processed One data sample, The mean vector of the feature dimensions of the samples in the data block to be processed. This is for the transpose operation.

[0012] In a preferred embodiment, the step of performing multi-path fusion and reduction on the partial result data to obtain the final result data of the data block to be processed includes: Based on the physical location distribution of the target processing elements, a reduced topology of the high-bandwidth memory is constructed; Based on the reduction topology, the partial result data is merged at multiple levels to obtain the fused intermediate state of the partial result data. The fused intermediate state is subjected to top-level convergence aggregation to obtain the final result data of the data block to be processed.

[0013] In a preferred embodiment, constructing the reduced topology of the high-bandwidth memory based on the physical location distribution of the target processing elements includes: The layout parameters of the target processing element are analyzed to obtain the physical coordinate information of the target processing element; Based on the physical coordinate information, the Manhattan distance is measured on the target processing element to generate a spatial proximity matrix of the target processing element; Hierarchical clustering analysis is performed on the spatial proximity matrix to divide the target processing element into different reduction levels; Based on the membership relationships between the reduction levels, node path connections are made to the reduction levels to obtain the reduction topology of the high-bandwidth memory.

[0014] To address the above problems, this invention also provides a near-memory computing bandwidth optimization system based on HBM, the system comprising: The task parsing module is used to decode and parse the access tasks on the host side to obtain the storage address and required operation type of the access task. The scheduling pre-selection module is used to construct metadata for the access task based on the storage address and the required operation type, obtain the scheduling context of the access task, and pre-select the target processing element in the high-bandwidth memory based on the scheduling context. The memory access and dimensionality reduction module is used to access the high-bandwidth memory based on the target processing element, obtain the data block to be processed in the high-bandwidth memory, and perform information dimensionality reduction mapping on the data block to be processed to obtain partial result data of the data block to be processed. The reduction return module is used to perform multi-channel fusion reduction on the partial result data to obtain the final result data of the data block to be processed, and return the final result data to the host. The pipeline prefetch module is used to prefetch and schedule the pending access tasks on the host side while the multi-path fusion and reduction is performed, so as to establish the scheduling context of the pending access tasks.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention rapidly generates a scheduling context and pre-selects target processing elements through precise instruction decoding and metadata construction, optimizing the spatial matching relationship between processing elements and the storage layer, and improving the accuracy of task scheduling and resource allocation. This invention also reduces data feature dimensionality redundancy through information dimensionality reduction mapping, decreasing data transmission volume and computational overhead, and accelerating data processing response speed.

[0016] 2. This invention constructs a reduction topology based on the physical layout of processing elements to achieve efficient fusion and reduction of multi-path data, thereby improving the aggregation efficiency of the result data. This invention prefetches and schedules the synchronous execution tasks of data reduction, forming a pipelined processing mechanism that fully utilizes HBM bandwidth resources and continuously improves the overall efficiency of near-memory computing bandwidth optimization based on HBM. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a near-memory computing bandwidth optimization method based on HBM provided in an embodiment of the present invention. Figure 2 A functional block diagram of a near-memory computing bandwidth optimization system based on HBM provided in an embodiment of the present invention; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This application provides a near-memory computing bandwidth optimization method based on HBM. The execution entity of the HBM-based near-memory computing bandwidth optimization method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the HBM-based near-memory computing bandwidth optimization method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a near-memory computing bandwidth optimization method based on HBM according to an embodiment of the present invention. In this embodiment, the near-memory computing bandwidth optimization method based on HBM includes: The access task on the host side is decoded and parsed to obtain the storage address and required operation type of the access task; In this embodiment of the invention, the step of decoding and parsing the access task on the host side to obtain the storage address and required operation type of the access task includes: Receive access tasks sent by the host through the physical interface of the high-bandwidth memory; The opcodes in the access task are decoded and parsed to obtain the required operation type of the access task; The address code in the access task is decomposed into address fields to obtain the storage layer number, storage bank selection signal and storage cell row address of the access task; The storage layer number, the storage bank selection signal, and the storage cell row address are recombined and integrated to obtain the storage address of the access task.

[0021] The physical interface of the high-bandwidth memory is electrically connected to the corresponding interface on the host side through preset pin definitions. The physical interface and the host side complete signal matching according to preset level standards. Both parties confirm that the data transmission path is ready through fixed handshake signals. The host side converts the access task into a continuous digital signal that conforms to the transmission standard according to the timing requirements and frame structure definition in the preset bus transmission specification. The digital signal is sent serially through the transmission path. The physical interface of the high-bandwidth memory captures the digital signal cycle by cycle under the drive of the synchronous clock. The captured digital signal is stably latched by the internal latch. Data integrity verification is completed during the latching process. After ensuring that no signal is lost, the complete reception of the access task is completed.

[0022] After receiving an access task, the dedicated instruction decoding unit inside the high-bandwidth memory locates the fixed field position corresponding to the opcode in the access task. Under the control of the synchronous clock, it reads and parses the opcode digital signal bit by bit. The instruction decoding unit matches the parsed opcode with the preset opcode and operation type correspondence in the internal fixed register bit by bit. If the match is successful, it directly outputs the unique corresponding operation type code, which is the required operation type of the access task.

[0023] The address field parsing unit inside the high-bandwidth memory extracts the complete digital signal corresponding to the address code in the access task. It segments the address code digital signal according to the preset total address bit width and fixed segmentation rules. The segmentation rules are as follows: the signal segment with a fixed number of bits is defined as the storage layer number, the signal segment with a fixed number of bits following it is defined as the storage bank selection signal, and the signal segment with the remaining bits is defined as the storage cell row address. After completing the segmentation, the address field parsing unit directly outputs three independent signals, namely the storage layer number of the access task, the storage bank selection signal of the access task, and the storage cell row address of the access task.

[0024] The address combination unit inside the high-bandwidth memory receives the split storage layer number, storage bank selection signal, and storage cell row address. It performs bit width alignment and timing synchronization of the three signals according to a preset fixed splicing order. The splicing order is that the storage layer number is in the highest bit segment of the address, the storage bank selection signal is in the middle bit segment of the address, and the storage cell row address is in the lowest bit segment of the address. There is no signal misalignment or missing during the splicing process. The continuous address digital signal formed after integration is the storage address of the access task.

[0025] The beneficial effects include standardized electrical connections and signal transmission through physical interfaces, combined with synchronous clock capture and latching, and integrity verification to ensure the accuracy and integrity of access task reception. The dedicated instruction decoding unit accurately parses the opcode through fixed field positioning and bit-by-bit matching, directly outputting the unique required operation type and avoiding operation type identification errors. The address field parsing unit splits the address code according to preset fixed rules, accurately generating three types of address information: memory layer number, memory bank selection signal, and memory cell row address. The address combination unit completes signal splicing in a fixed order to generate standardized memory addresses. The entire instruction decoding and parsing process has clear steps and unified execution standards, with no information deviation or data loss. It provides accurate and complete basic data for subsequent scheduling context construction and processing element pre-selection, effectively improving the stability and accuracy of the instruction parsing link in the HBM-based near-memory computing system, and ensuring the orderly conduct of subsequent memory access and computation operations.

[0026] Based on the storage address and the required operation type, metadata is constructed for the access task to obtain the scheduling context of the access task, and the target processing element in the high-bandwidth memory is pre-selected based on the scheduling context. In this embodiment of the invention, based on the storage address and the required operation type, metadata is constructed for the access task to obtain the scheduling context of the access task, including: Assign a context identifier to the access task, and associate and bind the storage address, the required operation type, and the context identifier to obtain the initial context record of the access task; Based on the storage layer number in the storage address, determine the processing element identifier corresponding to the storage layer number, and add the processing element identifier to the initial context record; Configure the index space for the initial context record to obtain the buffer index field of the initial context record; The scheduling context of the access task is obtained by aggregating and encapsulating the complete information of the context identifier, the storage address, the required operation type, the processing element identifier, and the buffer index field.

[0027] The step of pre-selecting target processing elements in high-bandwidth memory based on the scheduling context includes: Based on the storage layer number in the scheduling context, the spatial location of the processing element in the high-bandwidth memory and the storage layer in the high-bandwidth memory is resolved to obtain the proximity relationship between the processing element and the storage layer; Based on the proximity relationship, the processing elements are initially selected to obtain candidate processing elements for the high-bandwidth memory; Processing elements whose current load state value is lower than a preset load threshold are selected as target processing elements for the high-bandwidth memory.

[0028] The context identifier allocation unit inside the high-bandwidth memory assigns a unique and non-repeating numerical identifier to the current access task. This numerical identifier is the context identifier of the access task. The context identifier allocation unit binds the generated context identifier to the parsed storage address and the required operation type in a one-to-one fixed association. During the binding process, a corresponding mapping relationship is established between the three. The mapping relationship is stored in a dedicated storage area inside the high-bandwidth memory. The complete record information formed after the binding is completed is the initial context record of the access task.

[0029] The component matching unit inside the high-bandwidth memory extracts the storage layer number contained in the storage address. Based on the fixed correspondence between the storage layer number and the processing component identifier that is preset internally, the component matching unit retrieves and determines the processing component identifier that uniquely corresponds to the storage layer number. The component matching unit directly writes the determined processing component identifier into the specified field position of the initial context record to complete the addition operation of the processing component identifier.

[0030] The index configuration unit inside the high-bandwidth memory calls the internal preset buffer space allocation rules to plan the temporary storage location required for the initial context record. The index configuration unit allocates a dedicated index position for the initial context record according to the fixed address segment division method of the buffer space. The identification information corresponding to the index position is the buffer index field of the initial context record.

[0031] The field encapsulation unit inside the high-bandwidth memory integrates the context identifier, storage address, required operation type, processing element identifier, and buffer index field according to a preset fixed field arrangement order. During the integration process, the fields are time-aligned and bit-width matched. The structured data information formed after encapsulation is the scheduling context of the access task.

[0032] The spatial resolution unit inside the high-bandwidth memory extracts the storage layer number from the scheduling context. The spatial resolution unit reads the physical location information of the processing element and the physical location information of the storage layer stored inside the high-bandwidth memory. The spatial resolution unit compares the physical location of the processing element with the corresponding physical location of the storage layer one by one. The positional association state between the processing element and the storage layer determined after the comparison is the proximity relationship between the processing element and the storage layer.

[0033] The component selection unit inside the high-bandwidth memory selects processing components that are directly related to the storage layer based on the determination of proximity. During the selection process, processing components that are not directly related to the storage layer are eliminated. The set of processing components obtained after the selection is the candidate processing components of the high-bandwidth memory.

[0034] The load detection unit inside the high-bandwidth memory collects the current load status value of each candidate processing element in real time. The load detection unit compares the collected current load status value with the internally preset fixed load threshold. Candidate processing elements whose current load status value is less than the preset load threshold are directly identified as the target processing element of the high-bandwidth memory.

[0035] The beneficial effects include: generating initial context records through unique context identifier allocation and multi-information binding, ensuring the integrity and uniqueness of access task information; accurately matching processing element identifiers based on storage layer numbers and improving record content; generating dedicated buffer index fields by combining index space configuration; and aggregating and encapsulating these fields to form a standardized scheduling context, providing standardized data support for processing element pre-selection. Spatial location resolution clarifies the proximity relationship between processing elements and storage layers, accurately selecting candidate processing elements to narrow the pre-selection range. Combined with preset load thresholds, low-load processing elements are screened to ensure that target processing elements have stable execution capabilities. The entire metadata construction and processing element pre-selection process has clear logic and unified execution standards, effectively improving the accuracy of task scheduling and the rationality of resource allocation, providing reliable hardware resource guarantees for subsequent data access and computation operations.

[0036] Based on the target processing element, data storage access is performed on the high-bandwidth memory to obtain the data block to be processed in the high-bandwidth memory, and information dimensionality reduction mapping is performed on the data block to be processed to obtain partial result data of the data block to be processed. In this embodiment of the invention, the step of performing data storage access on the high-bandwidth memory based on the target processing element to obtain the data block to be processed in the high-bandwidth memory includes: Based on the target processing element, extract the buffer address from the scheduling context; A data read request is sent to the temporary storage space pointed to by the buffer address through the internal data path of the logical base layer in the high-bandwidth memory. The data read request is parsed using the bus protocol to obtain the workflow identifier of the data read request; Based on the workflow identifier, query the mapping relationship between the workflow identifier and the data block storage location in the high-bandwidth memory to confirm the storage location index of the data read request; Based on the storage location index, the high-bandwidth memory is addressed and located to obtain the data block to be processed in the high-bandwidth memory.

[0037] The step of performing information dimensionality reduction mapping on the data block to be processed to obtain partial result data of the data block to be processed includes: The data block to be processed is mapped to zero mean to obtain a centered data matrix of the data block to be processed. Based on the centralized data matrix, calculate the covariance matrix between the original feature dimensions in the data block to be processed; The covariance matrix is ​​decomposed into eigenvalues, and the eigenvectors are reassembled in order of the numerical values ​​of the decomposed eigenvalues ​​to obtain the projection matrix of the decomposed eigenvectors. Perform a matrix inner product between the centralized data matrix and the projection matrix to obtain partial result data of the data block to be processed.

[0038] The formula for calculating the covariance matrix is ​​as follows: ; In the formula, Let be the covariance matrix. Here, represents the number of data samples contained in the data block to be processed, and represents the number of original feature dimensions of the data samples in the data block to be processed. For the first data block to be processed One data sample, The mean vector of the feature dimensions of the samples in the data block to be processed. This is for the transpose operation.

[0039] The target processing element activates its internal address extraction logic, locates the fixed storage field corresponding to the buffer address in the scheduling context data structure, and continuously reads the complete address signal in that field. The address signal obtained after reading is the buffer address in the scheduling context.

[0040] The logic base layer of the high-bandwidth memory is connected to an internal dedicated data path, which directly connects the target processing element and the on-chip temporary storage area. According to the preset bus transmission timing standard, the data read request carrying the buffer address information is stably sent to the temporary storage space pointed to by the buffer address, and the timing is kept synchronized without interruption during the transmission of the request signal.

[0041] The bus protocol parsing unit inside the high-bandwidth memory splits the complete signal of the data read request into fields according to the preset bus protocol frame structure division rules, and accurately extracts the fixed signal segment used to identify the workflow in the request signal. This signal segment is the workflow identifier of the data read request.

[0042] The mapping relationship query unit inside the high-bandwidth memory calls the on-chip fixed workflow identifier and data block storage location mapping table, and matches the extracted workflow identifier with each entry in the mapping table bit by bit. After a successful match, the corresponding storage location code is locked, which is the storage location index of the data read request.

[0043] The target processing element sends a precise addressing instruction to the high-bandwidth memory's storage array based on the storage location index. The storage array reads data address by address according to the physical storage unit pointed to by the index. The complete set of data after reading is summarized is the data block to be processed in the high-bandwidth memory.

[0044] For each original feature value of each data sample in the data block to be processed, the mean value corresponding to that original feature dimension is subtracted. All the feature values ​​after mean adjustment are combined according to the original data sample order and the original feature dimension arrangement rules to form a regular matrix structure, which is the centralized data matrix of the data block to be processed.

[0045] First, count the total number of data samples in the data block to be processed and the number of original feature dimensions of a single data sample. Then, calculate the correlation between each pair of original feature dimensions based on the centralized data matrix. Arrange all the calculation results according to the correspondence of the original feature dimensions. Combine the data sample count to complete the normalization adjustment and form the matrix, which is the covariance matrix between the original feature dimensions in the data block to be processed.

[0046] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and eigenvectors corresponding to the matrix. Arrange all eigenvalues ​​in descending order of magnitude. Reorganize the corresponding eigenvectors according to the sorted eigenvalues. The combined structure of the reorganized eigenvectors is the projection matrix of the decomposed eigenvectors.

[0047] The data in each row of the centralized data matrix is ​​multiplied by the corresponding items in each column of the projection matrix, and then the sums are accumulated. All the sums are then arranged and combined according to the original matrix arrangement rules. The resulting data set is the partial result data of the data block to be processed.

[0048] The beneficial effects are as follows: the buffer address is accurately extracted by the target processing element and the read request is sent by relying on the internal data path of the logical base layer. The accurate location of the storage location is achieved by combining bus protocol parsing and mapping relationship query, which ensures the accuracy and integrity of the data block to be processed. The centralized data matrix is ​​generated by zero-mean mapping to eliminate the influence of numerical offset of feature dimension. The covariance matrix is ​​calculated based on the number of samples and the number of feature dimensions to accurately reflect the feature correlation characteristics. The projection matrix is ​​constructed by feature decomposition and ordered recombination to achieve effective dimension filtering. Partial result data is obtained by matrix inner product operation to complete information dimensionality reduction. The entire set of memory access and dimensionality reduction process steps are standard and reproducible, which effectively reduces the resource consumption of data transmission and computation and improves the efficiency and accuracy of data processing in near-memory computing.

[0049] The partial result data is fused and reduced through multiple channels to obtain the final result data of the data block to be processed, and the final result data is returned to the host. In this embodiment of the invention, the step of performing multi-path fusion and reduction on the partial result data to obtain the final result data of the data block to be processed includes: Based on the physical location distribution of the target processing elements, a reduced topology of the high-bandwidth memory is constructed; Based on the reduction topology, the partial result data is merged at multiple levels to obtain the fused intermediate state of the partial result data. The fused intermediate state is subjected to top-level convergence aggregation to obtain the final result data of the data block to be processed.

[0050] The step of constructing the reduced topology of the high-bandwidth memory based on the physical location distribution of the target processing elements includes: The layout parameters of the target processing element are analyzed to obtain the physical coordinate information of the target processing element; Based on the physical coordinate information, the Manhattan distance is measured on the target processing element to generate a spatial proximity matrix of the target processing element; Hierarchical clustering analysis is performed on the spatial proximity matrix to divide the target processing element into different reduction levels; Based on the membership relationships between the reduction levels, node path connections are made to the reduction levels to obtain the reduction topology of the high-bandwidth memory.

[0051] The layout parsing unit inside the high-bandwidth memory reads the location configuration register corresponding to the target processing element inside the chip through hardware control signals. The register pre-stores the horizontal and vertical position parameters of the target processing element in the chip layout. After the layout parsing unit extracts these two parameters completely, it combines them in an orderly manner according to the preset coordinate encoding rules inside the chip. The unique position code formed by the combination is the physical coordinate information of the target processing element. This coordinate information can accurately characterize the actual physical placement position of the target processing element inside the high-bandwidth memory.

[0052] The distance calculation unit inside the high-bandwidth memory retrieves the physical coordinate information of all target processing elements. For each different combination of target processing elements, it calculates the difference between their lateral position parameters and takes the absolute value to obtain the lateral distance component. It also calculates the difference between their longitudinal position parameters and takes the absolute value to obtain the longitudinal distance component. The lateral distance component and the longitudinal distance component are added together to obtain the spatial distance value of the combination of elements. The distance calculation unit arranges the spatial distance values ​​of all element combinations into a two-dimensional data structure according to the numbering order and correspondence of the target processing elements. This structure is the spatial proximity matrix of the target processing elements. Each value in the matrix corresponds to the actual spatial distance of a pair of target processing elements.

[0053] The clustering analysis unit inside the high-bandwidth memory loads all spatial distance values ​​in the spatial proximity matrix and compares all distance values ​​with a preset proximity threshold. The proximity threshold is the effective interconnection distance standard for components determined during the chip design stage. The clustering analysis unit prioritizes grouping target processing components with distance values ​​less than or equal to the proximity threshold into one group, and then gradually integrates groups with similar spatial locations through iterative merging. Finally, all target processing components are divided into multi-level grouping structures, with each grouping structure serving as an independent reduction level. The division of reduction levels is entirely based on the spatial proximity relationship of the target processing components.

[0054] The topology building unit inside the high-bandwidth memory resolves the hierarchical relationship between each reduction level. The bottom reduction level is a basic element group, and the upper reduction level is an aggregation group. The topology building unit assigns an independent data transmission node to each target processing element in the bottom reduction level and a dedicated aggregation node to each aggregation group in the upper reduction level. Then, a fixed hardware data transmission path is established between each bottom data transmission node and the corresponding upper aggregation node. After completing the path connection of all nodes layer by layer, the complete hierarchical data transmission architecture is the reduction topology of the high-bandwidth memory.

[0055] The multi-level merging units inside the high-bandwidth memory control the lower-level target processing elements to send the generated partial result data to the upper-level aggregation nodes through the corresponding data transmission paths, according to the hierarchical order of the reduction topology. Each aggregation node simultaneously receives partial result data transmitted from multiple lower-level nodes. After format alignment and timing synchronization of the multiple data streams, a merging and integration operation is performed. The integration process retains the valid calculation results of the data and removes redundant information. The data is transmitted upwards layer by layer and the merging and integration operation is repeated. The transitional data state formed after the integration of all levels is completed is the fusion intermediate state of the partial result data.

[0056] The top-level aggregation unit inside the high-bandwidth memory receives all the fused intermediate data transmitted from the top layer of the reduced topology. It performs unified format calibration and integrity verification on all fused intermediate data. The verification standard is the data validity judgment rule preset inside the high-bandwidth memory. After the verification is passed, all fused intermediate data are summarized and integrated as a whole. The complete and unique data set formed is the final result data of the data block to be processed.

[0057] The physical interface of the high-bandwidth memory converts the final result data into digital signals that meet the transmission requirements according to the bus transmission specifications agreed upon with the host. The digital signals are then continuously sent to the host through a pre-established stable electrical transmission path. After the host completes the signal reception and verification, it sends back an acknowledgment signal to complete the return operation of the final result data.

[0058] The beneficial effects are that a spatial proximity matrix is ​​constructed based on the physical coordinates of the target processing element, the reduction levels are divided according to a fixed proximity threshold and a reduction topology structure that fits the hardware layout is built, multi-level reduction and top-level aggregation are used to efficiently complete data integration, and the final result data is stably transmitted back to the host, thereby improving data reduction efficiency and transmission stability.

[0059] In this embodiment of the invention, while performing multi-path fusion and reduction, the pending access tasks on the host side are prefetched and scheduled to establish the scheduling context of the pending access tasks.

[0060] The pipelined prefetch unit inside the high-bandwidth memory collects the operating status signals of the multi-channel fusion and reduction module in real time through hardware monitoring lines. When it detects that the multi-channel fusion and reduction module has started to perform multi-level merging and top-level aggregation operations on some result data, the pipelined prefetch unit immediately starts the prefetch scheduling process synchronously. It sends a task prefetch enable signal to the host through the fixed electrical transmission path between the high-bandwidth memory and the host. The transmission timing of this enable signal is completely synchronized with the execution timing of the multi-channel fusion and reduction.

[0061] After receiving the task prefetch enable signal, the host extracts the pending access task from the internal task cache queue. It converts the pending access task into a stable digital signal according to the preset bus transmission specification and sends the pending access task to the physical interface of the high-bandwidth memory through a dedicated transmission channel connected to the high-bandwidth memory. The physical interface performs timing latching and integrity verification on the received digital signal to ensure that the pending access task is received by the high-bandwidth memory completely and without error.

[0062] The pipeline prefetching unit calls the pre-defined task parsing logic to decode and parse the received completed access tasks. The parsing process strictly follows the pre-determined operation code and address code parsing rules to accurately extract the storage address and required operation type corresponding to the access task. The generation standard of the parsing result is completely consistent with the parsing standard of the processed access task, ensuring the uniformity of data format and information type.

[0063] Based on the extracted storage address and the required operation type, the pipeline prefetching unit initiates the metadata construction process, assigns a unique and non-repeating context identifier to the pending access task, binds the context identifier to the storage address and the required operation type one-to-one, matches the corresponding processing element identifier according to the storage layer number in the storage address and adds it to the record information, and configures a dedicated buffer index field for the pending access task.

[0064] The pipelined prefetching unit aggregates and encapsulates the context identifier, storage address, required operation type, processing element identifier, and buffer index field in a fixed field order. The structured data formed after encapsulation is the scheduling context of the access task to be processed. The pipelined prefetching unit stores the scheduling context in a dedicated scheduling cache area inside the high-bandwidth memory, waiting for subsequent processing element pre-selection and data access processes to be called.

[0065] The beneficial effects are that prefetching scheduling and multi-path fusion reduction are executed synchronously, making full use of hardware operation gaps to complete the context construction of the task to be processed, shortening the overall task processing cycle, and improving the pipeline efficiency of near-memory computing in high-bandwidth memory.

[0066] like Figure 2 The diagram shown is a functional block diagram of a near-memory computing bandwidth optimization system based on HBM provided in an embodiment of the present invention.

[0067] The HBM-based near-memory computing bandwidth optimization system 100 described in this invention can be installed in an electronic device. Depending on the functions implemented, the HBM-based near-memory computing bandwidth optimization system 100 may include a task parsing module 101, a scheduling pre-selection module 102, a memory access dimensionality reduction module 103, a reduction return module 104, and a pipelined prefetching module 105. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0068] In this embodiment, the functions of each module / unit are as follows: The task parsing module 101 is used to decode and parse the access task on the host side to obtain the storage address and required operation type of the access task. The scheduling preselection module 102 is used to construct metadata for the access task based on the storage address and the required operation type, obtain the scheduling context of the access task, and preselect the target processing element in the high-bandwidth memory based on the scheduling context. The memory access dimensionality reduction module 103 is used to perform data storage access on the high-bandwidth memory based on the target processing element, obtain the data block to be processed in the high-bandwidth memory, and perform information dimensionality reduction mapping on the data block to be processed to obtain partial result data of the data block to be processed. The reduction return module 104 is used to perform multi-channel fusion reduction on the partial result data to obtain the final result data of the data block to be processed, and return the final result data to the host. The pipeline prefetch module 105 is used to prefetch and schedule the pending access tasks on the host side while performing multi-path fusion and reduction, so as to establish the scheduling context of the pending access tasks.

[0069] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0070] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0071] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0072] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0073] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A near-memory computing bandwidth optimization method based on HBM, characterized in that, The method includes: The access task on the host side is decoded and parsed to obtain the storage address and required operation type of the access task; Based on the storage address and the required operation type, metadata is constructed for the access task to obtain the scheduling context of the access task, and the target processing element in the high-bandwidth memory is pre-selected based on the scheduling context. Based on the target processing element, data storage access is performed on the high-bandwidth memory to obtain the data block to be processed in the high-bandwidth memory, and information dimensionality reduction mapping is performed on the data block to be processed to obtain partial result data of the data block to be processed. The partial result data is fused and reduced through multiple channels to obtain the final result data of the data block to be processed, and the final result data is returned to the host. While performing the multi-path fusion and reduction, the pending access tasks on the host side are prefetched and scheduled to establish the scheduling context of the pending access tasks.

2. The near-memory computing bandwidth optimization method based on HBM as described in claim 1, characterized in that, The process of decoding and parsing the access task on the host side to obtain the storage address and required operation type of the access task includes: Receive access tasks sent by the host through the physical interface of the high-bandwidth memory; The opcodes in the access task are decoded and parsed to obtain the required operation type of the access task; The address code in the access task is decomposed into address fields to obtain the storage layer number, storage bank selection signal and storage cell row address of the access task; The storage layer number, the storage bank selection signal, and the storage cell row address are recombined and integrated to obtain the storage address of the access task.

3. The near-memory computing bandwidth optimization method based on HBM as described in claim 1, characterized in that, The step of constructing metadata for the access task based on the storage address and the required operation type to obtain the scheduling context of the access task includes: Assign a context identifier to the access task, and associate and bind the storage address, the required operation type, and the context identifier to obtain the initial context record of the access task; Based on the storage layer number in the storage address, determine the processing element identifier corresponding to the storage layer number, and add the processing element identifier to the initial context record; Configure the index space for the initial context record to obtain the buffer index field of the initial context record; The scheduling context of the access task is obtained by aggregating and encapsulating the complete information of the context identifier, the storage address, the required operation type, the processing element identifier, and the buffer index field.

4. The near-memory computing bandwidth optimization method based on HBM as described in claim 1, characterized in that, The step of pre-selecting target processing elements in high-bandwidth memory based on the scheduling context includes: Based on the storage layer number in the scheduling context, the spatial location of the processing element in the high-bandwidth memory and the storage layer in the high-bandwidth memory is resolved to obtain the proximity relationship between the processing element and the storage layer; Based on the proximity relationship, the processing elements are initially selected to obtain candidate processing elements for the high-bandwidth memory; Processing elements whose current load state value is lower than a preset load threshold are selected as target processing elements for the high-bandwidth memory.

5. The near-memory computing bandwidth optimization method based on HBM as described in claim 1, characterized in that, The step of accessing the high-bandwidth memory based on the target processing element to obtain the data block to be processed in the high-bandwidth memory includes: Based on the target processing element, extract the buffer address from the scheduling context; A data read request is sent to the temporary storage space pointed to by the buffer address through the internal data path of the logical base layer in the high-bandwidth memory. The data read request is parsed using the bus protocol to obtain the workflow identifier of the data read request; Based on the workflow identifier, query the mapping relationship between the workflow identifier and the data block storage location in the high-bandwidth memory to confirm the storage location index of the data read request; Based on the storage location index, the high-bandwidth memory is addressed and located to obtain the data block to be processed in the high-bandwidth memory.

6. The near-memory computing bandwidth optimization method based on HBM as described in claim 1, characterized in that, The step of performing information dimensionality reduction mapping on the data block to be processed to obtain partial result data of the data block to be processed includes: The data block to be processed is mapped to zero mean to obtain a centered data matrix of the data block to be processed. Based on the centralized data matrix, calculate the covariance matrix between the original feature dimensions in the data block to be processed; The covariance matrix is ​​decomposed into eigenvalues, and the eigenvectors are reassembled in order of the numerical values ​​of the decomposed eigenvalues ​​to obtain the projection matrix of the decomposed eigenvectors. Perform a matrix inner product between the centralized data matrix and the projection matrix to obtain partial result data of the data block to be processed.

7. The near-memory computing bandwidth optimization method based on HBM as described in claim 6, characterized in that, The formula for calculating the covariance matrix is ​​as follows: ； In the formula, Let be the covariance matrix. Here, represents the number of data samples contained in the data block to be processed, and represents the number of original feature dimensions of the data samples in the data block to be processed. For the first data block to be processed One data sample, The mean vector of the feature dimensions of the samples in the data block to be processed. This is for the transpose operation.

8. The near-memory computing bandwidth optimization method based on HBM as described in claim 1, characterized in that, The step of performing multi-path fusion and reduction on the partial result data to obtain the final result data of the data block to be processed includes: Based on the physical location distribution of the target processing elements, a reduced topology of the high-bandwidth memory is constructed; Based on the reduction topology, the partial result data is merged at multiple levels to obtain the fused intermediate state of the partial result data. The fused intermediate state is subjected to top-level convergence aggregation to obtain the final result data of the data block to be processed.

9. The near-memory computing bandwidth optimization method based on HBM as described in claim 8, characterized in that, The step of constructing the reduced topology of the high-bandwidth memory based on the physical location distribution of the target processing elements includes: The layout parameters of the target processing element are analyzed to obtain the physical coordinate information of the target processing element; Based on the physical coordinate information, the Manhattan distance is measured on the target processing element to generate a spatial proximity matrix of the target processing element; Hierarchical clustering analysis is performed on the spatial proximity matrix to divide the target processing element into different reduction levels; Based on the membership relationships between the reduction levels, node path connections are made to the reduction levels to obtain the reduction topology of the high-bandwidth memory.

10. A near-memory computing bandwidth optimization system based on HBM, characterized in that, The system for implementing the HBM-based near-memory computing bandwidth optimization method of claim 1 includes: The task parsing module is used to decode and parse the access tasks on the host side to obtain the storage address and required operation type of the access task. The scheduling pre-selection module is used to construct metadata for the access task based on the storage address and the required operation type, obtain the scheduling context of the access task, and pre-select the target processing element in the high-bandwidth memory based on the scheduling context. The memory access and dimensionality reduction module is used to access the high-bandwidth memory based on the target processing element, obtain the data block to be processed in the high-bandwidth memory, and perform information dimensionality reduction mapping on the data block to be processed to obtain partial result data of the data block to be processed. The reduction return module is used to perform multi-channel fusion reduction on the partial result data to obtain the final result data of the data block to be processed, and return the final result data to the host. The pipeline prefetch module is used to prefetch and schedule the pending access tasks on the host side while the multi-path fusion and reduction is performed, so as to establish the scheduling context of the pending access tasks.

Images