Storage and calculation integrated parallel processing system and method

Abstract

The invention relates to the technical field of data processing, in particular to a storage and calculation integrated parallel processing system and method. The method comprises the following steps of obtaining original stored data and performing topological skeleton projection, generating a projection skeleton structure to realize characteristic hierarchical integration, determining a weight relation matrix based on multi-scale space fusion data, realizing matching of a memory and a calculation unit through gradient optimization adjustment, and obtaining a multi-scale space fusion matrix. And the mining storage and calculation unit activates the chain type interaction relationship of the mapping data and performs recursive self-calibration, generates self-calibration update data, predicts a heterogeneous access channel and reconstructs a data access link, and finally performs task recombination and pipeline scheduling and eliminates parallel conflicts, thereby obtaining an efficient parallel processing result. According to the method, high efficiency and intelligentization of storage and calculation are realized, and reliable support is provided for execution of complex calculation tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to an in-memory computing integrated parallel processing system and method. Background Art

[0002] The architecture of separating storage from computing leads to high latency between data storage and computing. Existing methods often face the problem of low efficiency when dealing with complex data. Traditional storage systems cannot meet the requirements of real-time computing. Data is prone to bottlenecks during transmission, affecting the overall performance and response speed. Especially when dealing with multi-scale data, there is a lack of effective feature fusion and optimization strategies, which limits the accuracy and efficiency of data analysis. In addition, existing technologies have deficiencies in dynamic data self-calibration and chain interaction mining, and cannot make full use of the interconnection relationship between in-memory computing units, resulting in the ineffective mining of the correlation between data. The implementation of recursive self-calibration is difficult, lacking a flexible adjustment mechanism, and it is difficult to adapt to a rapidly changing data environment. Especially in multi-level data access, traditional methods often cannot achieve efficient flow control and cache integration, resulting in access latency and resource waste. Summary of the Invention

[0003] Based on this, it is necessary to provide an in-memory computing integrated parallel processing system and method to solve at least one of the above technical problems.

[0004] To achieve the above object, the in-memory computing integrated parallel processing method includes the following steps: Step S1: Obtain the original stored data; perform topological skeleton projection on the original stored data to obtain a projection skeleton structure; perform feature hierarchical integration on the original stored data according to the projection skeleton structure to generate multi-scale spatial fusion data; Step S2: Determine the weight relationship matrix based on the multi-scale spatial fusion data; perform gradient optimization adjustment on the multi-scale spatial fusion data through the weight relationship matrix, and perform memory-computing unit matching to obtain activation mapping data of multiple in-memory computing units; Step S3: Mine the chain interaction correlation data between the activation mapping data of the in-memory computing units; perform recursive self-calibration on the original stored data according to the chain interaction correlation data to generate self-calibration update data; Step S4: Predict heterogeneous access channels based on the self-calibration update data, and reconstruct each hierarchical data access link through the predicted heterogeneous access channels; perform multi-level collaborative flow control cache integration according to the reconstructed hierarchical data access links to obtain access latency elimination optimization data; Step S5: Recombine the dependent path tasks of the access latency elimination optimization data to generate task recombination parallel processing data; perform pipeline scheduling on the task recombination parallel processing data to execute parallel conflict elimination to obtain parallel conflict decoupling data.

[0005] By obtaining the original stored data and performing topological skeleton projection, the generated projection skeleton structure provides a clear spatial framework for subsequent data processing. The feature hierarchical integration realizes the multi-dimensional analysis of the original data. The generated multi-scale spatial fusion data improves the processability and accuracy of the data while retaining important features. The weight relationship matrix determined based on the multi-scale spatial fusion data provides a scientific basis for data optimization. The gradient optimization adjustment makes the matching between calculation and storage more efficient. The generation of activation mapping data of multiple memory-calculation units lays the foundation for parallel processing. Mining the chain interaction correlation data between the activation mapping data of memory-calculation units further improves the correlation and processing efficiency of the data. The recursive self-calibration can dynamically adjust the original stored data, and the generated self-calibration updated data ensures the consistency and accuracy of the data. Predicting the heterogeneous access channels based on the self-calibration updated data and reconstructing each hierarchical data access link optimize the data flow path. The reconstructed multi-layer cooperative flow control cache integration effectively eliminates the access latency. The generated access latency elimination optimized data reduces resource waste while improving the access efficiency. The dependency path task reorganization creates conditions for the parallel processing of tasks. The generated task reorganization parallel processing data improves the system response speed during execution. The pipeline scheduling mechanism effectively eliminates conflicts in parallel processing. The finally obtained parallel conflict decoupled data improves the overall processing capacity of the system. The overall method realizes the high-efficiency and intelligent data processing under the framework of memory-calculation integration, provides reliable support for the execution of complex computing tasks, promotes the development of new memory-calculation architectures, enhances the flexibility and adaptability of the system, promotes the improvement of data processing capabilities in high-performance computing environments, optimizes the utilization of storage and computing resources, and provides new ideas and technical paths for future multi-task parallel processing.

[0006] The present invention also provides a memory-calculation integrated parallel processing system for executing the memory-calculation integrated parallel processing method as described above. The memory-calculation integrated parallel processing system includes: A multi-scale feature fusion module, configured to obtain the original stored data; perform topological skeleton projection on the original stored data to obtain a projection skeleton structure; and perform feature hierarchical integration on the original stored data according to the projection skeleton structure, thereby generating multi-scale spatial fusion data; A weight matching activation module, configured to determine a weight relationship matrix based on the multi-scale spatial fusion data; perform gradient optimization adjustment on the multi-scale spatial fusion data through the weight relationship matrix, and perform memory-computation unit matching, thereby obtaining activation mapping data of multiple memory-calculation units; An interaction chain self-calibration module, configured to mine the chain interaction correlation data between the activation mapping data of memory-calculation units; perform recursive self-calibration on the original stored data according to the chain interaction correlation data, and generate self-calibration updated data; A channel reconstruction flow control integration module is used to predict heterogeneous access channels based on self-calibrated updated data, and reconstruct each hierarchical data access link through the predicted heterogeneous access channels; perform multi-layer collaborative flow control cache integration according to the reconstructed hierarchical data access links, so as to obtain access latency elimination and optimized data. A task decoupling and scheduling module is used to reorganize the dependent path tasks of the access latency elimination and optimized data to generate task reorganization parallel processing data; perform pipelining scheduling on the task reorganization parallel processing data to execute and eliminate parallel conflicts, so as to obtain parallel conflict decoupled data.

[0007] In the present invention, through a multi-scale feature fusion module, the acquisition of the original stored data is combined with the topological skeleton projection to form a clear projection skeleton structure. Feature hierarchical integration realizes multi-dimensional analysis of data on this basis. The generated multi-scale spatial fusion data improves the efficiency of data processing while retaining key features. The weight matching activation module provides a scientific basis for the gradient optimization of data based on the weight relationship matrix generated from the multi-scale spatial fusion data. The effective matching of the memory and the computing unit improves the resource utilization rate. The activation mapping data of multiple memory and computing units realizes the basis of parallel processing through optimization and adjustment. The interactive chain self-calibration module enhances the correlation between data by mining the chain interaction correlation data between the activation mapping data of the memory and computing units. The updated data generated by recursive self-calibration ensures the consistency and accuracy of the original data. The channel reconstruction flow control integration module predicts heterogeneous access channels based on the self-calibrated updated data. The reconstructed hierarchical data access links optimize the data flow path. The multi-layer collaborative flow control cache integration formed after reconstruction effectively eliminates access latency. The generated optimized data significantly improves the access efficiency. The task decoupling and scheduling module creates conditions for parallel processing by reorganizing the dependent path tasks of the access latency elimination and optimized data. The generated task reorganization parallel processing data improves the system response speed during execution. The pipelining scheduling mechanism effectively eliminates conflicts in parallel processing. The finally obtained parallel conflict decoupled data improves the overall system processing ability. The entire system realizes efficient and intelligent data processing within the framework of memory-computation integration, provides reliable support for the execution of complex computing tasks, promotes the development of new memory-computation architectures, enhances the flexibility and adaptability of the system, optimizes the utilization of storage and computing resources, provides an innovative technical path for future multi-task parallel processing. The high efficiency and accuracy of the overall process lay a solid foundation for the application in large-scale data processing scenarios, promote the improvement of data processing capabilities in high-performance computing environments, and drive the wide application and development of memory-computation integration technology. Description of the Drawings

[0008] Figure 1 It is a schematic diagram of the step flow of a memory-computation integration parallel processing method; Figure 2It is a schematic diagram of the detailed implementation steps of step S2; The realization of the object of the present invention, its functional characteristics and advantages will be further described with reference to the embodiments and the accompanying drawings. Specific embodiments

[0009] The technical method of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0010] In addition, the accompanying drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0011] It should be understood that although terms such as "first" and "second" may be used here to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit can be called the second unit, and similarly the second unit can be called the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed related items.

[0012] To achieve the above object, please refer to Figures 1 to 2 , an in-memory computing integrated parallel processing method, including the following steps: Step S1: Obtain the original stored data; perform topological skeleton projection on the original stored data to obtain a projection skeleton structure; perform feature hierarchical integration on the original stored data according to the projection skeleton structure to generate multi-scale spatial fusion data; Step S2: Determine the weight relationship matrix based on the multi-scale spatial fusion data; perform gradient optimization adjustment on the multi-scale spatial fusion data through the weight relationship matrix, and perform memory-computation unit matching to obtain multiple in-memory computing unit activation mapping data; Step S3: Mine the chained interaction correlation data between the in-memory computing unit activation mapping data; perform recursive self-calibration on the original stored data according to the chained interaction correlation data to generate self-calibration update data; Step S4: Predict heterogeneous access channels based on self-calibrated update data, and reconstruct each hierarchical data access link through the predicted heterogeneous access channels; perform multi-level collaborative flow control cache integration according to the reconstructed hierarchical data access links to obtain access latency elimination optimized data; Step S5: Reorganize the dependent path tasks of the access latency elimination optimized data to generate task reorganization parallel processing data; perform pipelining scheduling on the task reorganization parallel processing data to execute and eliminate parallel conflicts, thereby obtaining parallel conflict decoupled data.

[0013] In the present invention, by obtaining the original stored data and performing topological skeleton projection, the generated projection skeleton structure provides a clear spatial framework for subsequent data processing. Feature hierarchical integration realizes multi-dimensional analysis of the original data. The generated multi-scale spatial fusion data improves the processability and accuracy of the data while retaining important features. The weight relationship matrix determined based on the multi-scale spatial fusion data provides a scientific basis for data optimization. Gradient optimization adjustment makes the matching between calculation and storage more efficient. The generation of activation mapping data of multiple memory and computing units lays the foundation for parallel processing. Mining the chain interaction correlation data between the activation mapping data of memory and computing units further improves the correlation and processing efficiency of the data. Recursive self-calibration can dynamically adjust the original stored data, and the generated self-calibrated update data ensures the consistency and accuracy of the data. Predicting heterogeneous access channels based on the self-calibrated update data and reconstructing each hierarchical data access link optimize the data flow path. The reconstructed multi-level collaborative flow control cache integration effectively eliminates access latency. The generated access latency elimination optimized data reduces resource waste while improving access efficiency. Dependent path task reorganization creates conditions for parallel processing of tasks. The generated task reorganization parallel processing data improves the system's response speed during execution. The pipelining scheduling mechanism effectively eliminates conflicts in parallel processing. The finally obtained parallel conflict decoupled data improves the overall processing ability of the system. The overall method realizes the high-efficiency and intelligent data processing under the framework of memory and computing integration, provides reliable support for the execution of complex computing tasks, promotes the development of new memory and computing architectures, enhances the flexibility and adaptability of the system, promotes the improvement of data processing ability in a high-performance computing environment, optimizes the utilization rate of storage and computing resources, and provides new ideas and technical paths for future multi-task parallel processing.

[0014] In the embodiment of the present invention, the memory and computing integrated parallel processing method includes the following steps: Step S1: Obtain the original stored data; perform topological skeleton projection on the original stored data to obtain a projection skeleton structure; perform feature hierarchical integration on the original stored data according to the projection skeleton structure to generate multi-scale spatial fusion data; In this embodiment, when acquiring the original stored data, a high-speed acquisition interface based on the DDR4 bus communication protocol is used to access the data stream input by the heterogeneous device into the FPGA (Field Programmable Gate Array) buffer control module. The data sampling period is set to 5 nanoseconds, and batch data collection is performed through a 64-bit wide bus. The collected data is organized into original data blocks in units of 128 bytes and accessed into the internal buffer of the FPGA using the AXI bus protocol. The buffer is configured as a dual-port FIFO structure to maintain the stability of continuous data input. When performing topological skeleton projection on the data blocks in the buffer, the first principal axis direction of each data block in the three-dimensional feature space is extracted through three-dimensional principal component analysis (PCA), and this principal axis direction is used as the projection principal vector. All data points are projected along this direction and their two-dimensional mappings on the orthogonal plane are recorded to form an initial skeleton graph structure. Based on this skeleton structure, a feature hierarchy is constructed. The original projection map is downsampled 3 times in space using the multi-scale Gaussian pyramid method, with a downsampling factor of 2 for each layer, and local geometric feature points are extracted at each scale. The feature maps at each scale are combined into a three-dimensional feature tensor by constructing local extreme points in the scale space. Finally, this tensor is point-to-point mapped with the original data block to construct multi-scale space fusion data, and this data structure records the spatial positions, relative skeleton distances, scale levels where the feature points are located, and the index relationship with the original data according to the index respectively.

[0015] Step S2: Determine the weight relationship matrix based on the multi-scale space fusion data; perform gradient optimization adjustment on the multi-scale space fusion data through the weight relationship matrix, and perform memory-computation unit matching, so as to obtain activation mapping data of multiple memory-computation units; In this embodiment, when determining the weight relationship matrix based on the multi-scale spatial fusion data constructed above, the sparse tensor coding method is used to encode the feature point information at each scale into sparse vectors, and the similarity score is calculated through a dual index based on the spatial angle between points and the relative scale difference. The similarity score formula uses the Euclidean distance plus the scale ratio to form a mixed weight function, where the weighting coefficient of the distance term is set to 0.7 and the weighting coefficient of the scale difference is set to 0.3. Finally, the mixed weight values between all pairs of feature points are recorded in the sparse matrix to form the weight relationship matrix. The dimension of this matrix is N×N, where N is the total number of all valid feature points in the fusion data. When performing gradient optimization adjustment on the fusion data based on this matrix, the L-BFGS (Limited-memory Broyden-Fletcher-Goldfarb-Shanno) algorithm is used for constrained optimization. The objective function is set to minimize the sum of the squares of the feature point reconstruction errors. The initial value of the gradient step size is set to 0.01, and the maximum number of iterations is set to 200 times. Through iterative convergence, the optimized multi-scale fusion tensor is obtained. When performing the memory-computation unit matching operation on this tensor, the fusion data is grouped and sorted according to the spatial density and access frequency. The sorting basis is the number of times each feature point appears in the original data index plus its hierarchical depth value in the skeleton structure. Finally, the top K high-weight points are selected and mapped to the SRAM. The value of K is set to 128, and the corresponding mapping operation is completed using the DMA controller. After the SRAM is bound to the FPGA computing core, all the mapped data is converted into the active state, and the active mapping data of the memory-computation unit is generated, which includes the active index table, the data access frequency table, and the tensor mapping pointer structure.

[0016] Step S3: Mine the chained interaction correlation data between the active mapping data of the memory-computation unit; perform recursive self-calibration on the originally stored data according to the chained interaction correlation data to generate self-calibration update data; In this embodiment, when mining the chained interaction correlation data based on the activation mapping data of the memory computing unit, a graph convolutional network (GCN) structure is used for processing. First, all nodes in the activation mapping data are constructed into an undirected graph structure. The edge weights between nodes are set according to the non-zero terms in the weight relationship matrix generated in the previous stage. The number of interactions attached to each edge is obtained through a time-series access simulation experiment. The simulation process is set to continuously write and read 128 times, record the access switching frequency between adjacent nodes, and use it as the interaction intensity parameter of the edge. Based on this graph structure, a 3-layer GCN network is built, with the output dimension of each layer being 64. The ReLU (Rectified Linear Unit) is used as the activation function. By training the graph network model, a high-interaction-frequency chain structure is identified. The chain structure is defined as a sequence of nodes with continuous access dependency relationships. After the GCN output, all node sequences in the interaction chain are extracted as chained interaction correlation data, and at the same time, their interaction directions, access frequencies, and node-to-node time delays are recorded. When performing recursive self-calibration on the original stored data based on this interaction correlation structure, a multi-round reverse mapping strategy is adopted to map the chain sequence back to the original data storage structure, and the arrangement order of the data in the storage block is readjusted according to the interaction direction. Consistency verification is performed for each adjustment. The verification criterion is that the access interval of three consecutive groups of data in the same interaction chain does not exceed two memory access cycles. If not, a data exchange operation is performed to complete the rearrangement within the entire data block, and the updated data structure is marked as self-calibrated updated data. Each data block is appended with an interaction chain sequence index and a calibration timestamp.

[0017] Step S4: Predict heterogeneous access channels based on the self-calibrated updated data, and reconstruct each hierarchical data access link through the predicted heterogeneous access channels; perform multi-layer collaborative flow control cache integration according to the reconstructed hierarchical data access links, so as to obtain access delay elimination optimized data; In this embodiment, when constructing a heterogeneous access channel prediction model based on the generated self-calibration update data, the long short-term memory network (LSTM) is used to learn the access history sequence of the self-calibration data. The input sequence is set as the access interval sequence of each 128-byte data block and the type label of the interaction chain. The LSTM structure is set with two hidden layers, each with 128 neurons, trained using the RMSprop optimizer, and the mean squared error is used as the loss function. The training data is sourced from 512 groups of typical self-calibration sequences and their historical access paths. The prediction result is the heterogeneous channel type number to be assigned to each data block, and the type numbers include the high-throughput SRAM channel, the low-latency DRAM channel, and the large-capacity Flash channel. According to the prediction result, the original hierarchical data access link is reconstructed, and the data blocks are respectively assigned to the corresponding access paths according to the predicted channel types. When constructing the new link structure, a hash index mechanism is used to append the channel type identifier and the channel start address to each data block. After the link reconstruction is completed, a multi-level collaborative flow control cache integration operation is performed to impose flow control consistency constraints on each cache layer in the reconstructed access chain. Each cache uses the LRU (Least Recently Used) algorithm for page table management, the cache page size is fixed at 4KB, and the write-back policy between cache layers is set such that the write-back latency does not exceed two scheduling cycles. Finally, the access structure after cache integration is generated and marked as access latency elimination optimization data. This data structure includes a channel allocation table, an access latency distribution matrix, and a cache synchronization status record.

[0018] Step S5: Reorganize the dependent path tasks for the access latency elimination optimization data to generate task reorganization parallel processing data; perform pipelining scheduling on the task reorganization parallel processing data to execute the elimination of parallel conflicts, thereby obtaining parallel conflict decoupled data.

[0019] In this embodiment, when performing the dependent path task reorganization on the access latency elimination optimized data, first, the channel mapping order and cache synchronization status in the optimized data are extracted to construct a directed graph structure for task execution. Each node in the graph represents the calculation task of a data block, and the edge represents its dependency relationship and channel transmission order. The Topological Sort algorithm is used to rearrange the graph structure to ensure that all data blocks are reordered according to the shortest latency path. After reorganization, task reorganization parallel processing data is generated, and this data structure is saved in the form of an execution flow array. Each array element contains the calculation task index, the identifier of the dependent task, the required channel number, and the task trigger latency. Subsequently, a pipelining scheduling operation is performed on this parallel processing data. The scheduling process adopts a priority-based polling strategy, where the priority is determined by the criticality of the path where the task is located (i.e., the node with the maximum path latency). One scheduling window is allocated per clock cycle, and the window size is 8 tasks. The pipelining scheduler sequentially schedules non-conflicting tasks and sends them to the parallel computing array. If two tasks access the same channel or the dependency relationship is not satisfied, they are postponed to the next cycle. All scheduling conflict situations are recorded in the conflict matrix, and resource decoupling is performed according to the conflict matrix. The decoupling operation is to introduce an intermediate task buffer and recalibrate the task trigger conditions. After completion, the final parallel conflict decoupled data is output, and the data format is the task execution sequence, the scheduling cycle comparison table, and the resource isolation flag bit index table.

[0020] Preferably, step S1 includes the following steps: Step S11: Perform channel domain partitioning processing on the originally stored data to obtain channel distribution segment data, where the channel partitioning granularity range is set to the pixel block dimension from 8×8 to 64×64; Step S12: Perform local topological tensor mapping processing on the channel distribution segment data to generate basic topological mapping data, and the mapping dimension is set to a 3rd-order tensor structure, and the element dimension range is 16 - 512; Step S13: Perform skeletonized structure projection on the basic topological mapping data to obtain a projected skeleton structure, where the structure connectivity threshold is set to 0.35 - 0.65, and the skeleton sparsity rate is controlled at 15% - 40%; Step S14: Perform scale weight coupling according to the projected skeleton structure and nest it into a skeleton scale label; Step S15: Perform hierarchical feature compression and fusion on the originally stored data according to the skeleton scale label to generate hierarchical feature integration data, where the fusion depth is set to 4 to 12 layers, and the compression ratio is controlled at 1:4 - 1:12; Step S16: Perform spatial redundancy regularization processing on the hierarchical feature integration data to obtain multi-scale spatial fusion data, where the redundancy regularization window size is set to 5×5 to 11×11, and the residual filtering threshold is set to 0.02 - 0.12.

[0021] In this embodiment, the operation process of performing channel domain division processing on the originally stored data is as follows. First, the originally stored data is loaded into the data acquisition unit, and the data is expressed in the form of a three-dimensional matrix with a dimension size of 256×256×128. The first two dimensions correspond to the spatial range of the image, and the third dimension is the number of channels. During the division process, a fixed sliding window operation is used to set each scanned block as a pixel block of 32×32. The channel division is sliced according to 16 channels per group, and the segmentation cutting in the spatial and channel dimensions is completed in sequence, thereby obtaining channel distribution segment data. A total of 64 spatial region blocks are generated, each region corresponding to 8 groups of channel segments, and the data dimension of each group of channel segments is 32×32×16. By using the region aggregation method based on channel energy sorting, channels with similar signal-to-noise ratios and intensity distributions are combined into the same segment, and the average channel density distribution matrix is recorded on each group of segments as the subsequent mapping basis. The channel distribution segment data is input into the tensor processing module, and a tensor representation is constructed for each segment using a three-dimensional tensor reconstruction engine. The tensor dimension is set to a third-order structure, that is, it contains information in three directions: horizontal, vertical, and channel. The unit value of each tensor is derived from the normalization result of the original channel pixel values. The minimum value of each dimension in the tensor is set to 16, and the maximum value does not exceed 512. After performing tensor assembly, a homogeneous filling operation is performed on the tensor boundary through the structure regularization module to ensure the consistency of the form and size between each group of data. The unified output size is set to 32×32×64, and this structure constitutes the basic topological mapping data. The basic topological mapping data is input into the skeletonization structure module, and the main connected path in the tensor structure is extracted using the structure path extraction method. First, a channel structure diagram is generated through the adjacency degree calculation method, and then according to the set structure connectivity threshold of 0.5. Delete all weak connection paths, only retain the connection relationships between strongly connected channels. Further set the skeleton sparsity rate to 25%. During the entire sparse processing process, sort the path priorities according to the pixel gradient intensity inside each tensor block, and only retain the top 75% of the connection nodes in the gradient sorting. After deleting the remaining paths, a sparse skeleton structure is formed. This structure is output in a two-dimensional projection manner, and the dimension is maintained within the same spatial range as the input tensor. Send the obtained projected skeleton structure into the scale coupling module for scale weight annotation processing, use a multi-scale fusion strategy to construct skeleton scale labels, and superimpose the corresponding structure complexity labels on each skeleton node. This label is derived from the channel activity statistics at different resolutions. Set three groups of scale ranges corresponding to small-scale, medium-scale, and large-scale structures respectively, and determine the label weights of each node according to the probability distribution of the skeleton appearing in these three types of structures. Finally, reorganize the skeleton structure into a nested structure according to the node scale labels, forming output data of the skeleton scale label containing three types of weight information. Send the skeleton scale label into the feature fusion module, combine the original stored data to construct a multi-layer fusion structure, use a hierarchical convolution compression processing method, and group and compress the original data channels step by step according to the skeleton scale label. The fusion process is set to 8 layers deep. The first layer processes 128-channel original data, and each subsequent layer further aggregates its compression result. The compression ratio of each layer is successively 1:4, 1:5, 1:6 until 1:12. After each layer of compression, use a multi-channel attention mechanism to weighted integrate the outputs of each compression layer to generate the final hierarchical feature integration data. The dimension of this data is 128×128×48, retaining the main feature channel information under different scale skeletons. Send the hierarchical feature integration data into the redundancy regularization module for multi-scale spatial regularization operations. Use a sliding window method to scan each region in the spatial dimension. Set the window size to 9×9 and the sliding step size to 3 each time. Calculate the residual redundancy value within each window region, mark the redundant part lower than the set threshold of 0.08 as an invalid region, and use a low-rank filling strategy to restore and compress this region, retaining the backbone feature data with lower redundancy. After completing the redundancy regularization process, output the final multi-scale spatial fusion data. This data serves as the input basic data for subsequent weight relationship matrix extraction and activation mapping calculation, with the dimension remaining 128×128×48 and the redundancy controlled below 5% of the total feature volume.

[0022] Preferably, step S2 includes the following steps: Step S21: Perform memory mapping expansion on the multi-scale spatial fusion data to obtain memory mapping structure expansion data; Step S22: Identify the patterns of the memory mapping structure expansion data and classify the identified patterns according to similarity to obtain classified spatial feature patterns; Step S23: Construct a weight matrix based on the classified spatial feature pattern to generate a weight relationship matrix; perform gradient optimization adjustment on the weight relationship matrix and conduct structured sparsity projection to obtain structured sparse mapping data; Step S24: Perform memory addressing optimization based on the structured sparse mapping data and calculate the unit activation mapping process based on the addressed optimization data to obtain multiple memory-computation unit activation mapping data.

[0023] In this embodiment, the specific operation of memory mapping expansion for multi-scale spatial fusion data is as follows: load the multi-scale spatial fusion data output from the previous step into the mapping control module. The dimension of the fusion data is 128×128×48. During the mapping expansion process, by setting the memory linear expansion rule, the three-dimensional tensor structure is flattened into a one-dimensional continuous vector stream in row-major order. The fixed-length segment coding strategy is adopted to set the length of each expansion unit group to 6144 floating-point data units. Each data unit occupies 4 bytes of memory space, and a total of 1024 groups of memory mapping segments are formed. During the expansion process, a direct mapping structure is used to delimit the mapping address range as a 32-bit address space. The address mapping strategy uses in-page offset combined with high-order address switching to ensure that the expansion form of the high-dimensional feature dimension in the memory structure maintains the original spatial continuity. At the same time, the partitioned loop cache mechanism (cache loop partitioning) is adopted to cache the feature maps of different channels in the four-way parallel cache area respectively, so as to form the expanded data of the memory mapping structure. Send the expanded data of the memory mapping structure into the pattern recognition and classification unit, and use the spatial clustering method to identify the repeated structure patterns existing in the data. First, calculate the vector cosine similarity of the adjacent area for each group of expanded data using the local self-similarity measure. The similarity threshold is set to 0.85, and the data segments with a cosine angle less than 30 degrees are identified as the initial similarity pattern set. Then, use the density-based spatial clustering algorithm (DBSCAN) to perform clustering operations on the initial pattern set. Set the minimum sample number to 6 and the maximum distance threshold to 1.2. After clustering, a spatial feature pattern set with different similarity levels is obtained, which is divided into three categories: A, B, and C. Among them, the A category pattern is a full structure match, the B category pattern is a channel vector form match, and the C category is a high-frequency area texture similarity match. The three categories of patterns are respectively output as classified spatial feature patterns with different numbers, and mark their starting offset and length range in the memory address. Import the classified spatial feature patterns into the weight construction module, and use the vector weighted integration method to construct the weight matrix. Set each category of spatial feature pattern as a core node in the weight matrix, and construct the edge connection weight value for the pattern segments around each node through the adjacent weight mapping. The edge weight value is obtained from the aforementioned similarity score and combined with the channel cooperation coefficient calculation. After the weight matrix is constructed, perform gradient optimization operations on the matrix. Use a three-round loop iteration method to adjust the edge weight value of each node. In the first round, perform linear scaling adjustment, in the second round, perform boundary fitting adjustment, and in the third round, perform direction consistency adjustment. Subsequently, perform structured sparsity projection processing on the optimized matrix. During this process, set the sparsity to 65%, retain the highest weight connection edges, and delete those below the set threshold 0.The edge connection relationship of 2 outputs structured sparse mapping data, which is expressed in the form of an index matrix, recording the effective connection paths and activation weight values of each feature pattern. The structured sparse mapping data is passed into the memory addressing optimization unit, and the physical addressing strategy is readjusted according to the sparse connection relationship. First, the access addresses of consecutive feature patterns are reconstructed into a shared addressing area through a compressed address mapping table, and the displacement address mapping mechanism is used to bind the same cache mapping segment to multiple activation nodes to avoid repeated access. During the adjustment process, 1024 bytes are used as the minimum address remapping unit, and a three-level page table structure is adopted to record the compressed mapping relationship, and a reverse addressing index table is added at the end of the structure for quickly locating the original mapping offset. After the addressing optimization is completed, the mapping result is sent into the activation mapping control module, and the mapping control engine is used to perform activation processing on the data segments in each valid address area for the storage and computing unit. It is set that there are 128 parallel processing units for the activation processing, and each processing unit corresponds to activating a sparse structure node. The control engine uniformly distributes the processing tasks and starts the matching activation tasks in the physical array. Each activation process uses 4-channel data input for parallel expansion, and finally outputs multiple activation mapping data for the storage and computing unit. The dimension of the activation data is 128×48, and each group of data contains the complete activation path and the corresponding processing unit number information.

[0024] Preferably, mining the chain interaction correlation data between the activation mapping data of the storage and computing unit in step S3 includes: Monitoring the interaction channels of the activation mapping data of the storage and computing unit to obtain interaction channel probe data; Extracting the activation coupling degree in the interaction channel probe data to generate activation coupling degree correlation data; Performing chain fusion mapping according to the activation coupling degree correlation data to obtain chain interaction correlation data.

[0025] Especially importantly, extracting the activation coupling degree in the interaction channel probe data includes: Sifting the excitation frequency between channels of the interaction channel probe data to obtain channel excitation frequency data; Based on the channel excitation frequency data, performing synchronous activation interval registration to generate registered activation window data; Performing coupling strength quantization mapping on the registered activation window data to generate activation coupling degree mapping data; Performing threshold partition normalization according to the activation coupling degree mapping data to obtain activation coupling degree correlation data.

[0026] In this embodiment, the operation process of monitoring the interaction channels of the activation mapping data of the memory-computation unit is as follows: load the activation mapping data of the memory-computation unit generated in the previous stage into the channel probe control unit. The dimension of this activation data is 128×48, and each dimension corresponds to an independent computing unit status signal. In the channel monitoring operation, set every 4 consecutive activation units as a group of interaction unit clusters. Each group of clusters is divided into 32 groups and accessed in parallel to the interaction channel listening link. Use a capacitive coupling probe to record the signal switching frequency and data transfer sequence between channels in real time. The listening period is set to 64 clock units, and each unit length is 12.5 nanoseconds. Sample the channel switching signals of all interaction clusters within each period, establish a binary state encoding for each signal rising and falling edge change, and finally output an interaction status matrix. This matrix takes the time axis as the horizontal axis and the interaction cluster number as the vertical axis, recording the activation status encoding of each channel in each cluster at each time node. The obtained interaction status matrix is the interaction channel probe data. The operation of screening the excitation frequency between channels for the interaction channel probe data includes: first, input the interaction status matrix into the frequency analysis unit, count the activation times of each channel in each group of interaction clusters within 512 consecutive periods, use a sliding window method with a 64-period unit sliding window, and a sliding step of 16 periods each time. Calculate the excitation frequency value of each channel in each window. The excitation frequency value is expressed as the number of activations per unit time divided by the total length of the window, and the result is floating-point percentage data. Mark the channels with a frequency value higher than the set threshold of 0.65 as strongly activated channels, and the rest as weakly activated channels. At the same time, based on the co-occurrence times of the activation of channel pairs, establish a channel excitation co-occurrence frequency table. The operation of synchronizing the activation interval registration based on the channel excitation frequency data includes: for all strongly activated channels that satisfy the co-occurrence frequency greater than 0.The channels of 45 are extracted, and registration operations are performed based on their activation timestamp vectors. The activation window width is set to 16 cycles, and the activation time vectors of each pair of channels are synchronously detected by the phase alignment method. The phase alignment method adopts the Euclidean distance matching plus time offset compensation strategy, with a maximum allowable offset of ±3 cycles. Finally, the synchronous activation registration windows of each pair of channels in each time period are output. The registration window data is stored in the form of triples, including the channel pair number, start time, and end time. Multiple synchronous registration activation window sequences are constructed in the entire data sequence. The operation of performing coupled strength quantization mapping on the registration activation window data includes inputting all the registration window data into the coupled strength calculation module. First, the active time overlap degree is calculated for each registration window. This overlap degree is defined as the ratio of the effective time length of the two channels within the synchronous activation interval to the total registration window length. Then, the channel weight coefficient adjustment value is introduced, and this value is weighted and corrected based on the previous activation frequency value. Finally, the coupled strength value is calculated by multiplying the overlap degree by the weight coefficient. The coupled strength values of all registration windows are organized into a coupled strength mapping matrix. This matrix uses the channel pair as the row index and the registration window number as the column index, and the element is the coupled strength value. The data format is fixed-point floating-point representation with a precision of 32 bits. The operation of performing threshold partitioning and normalization based on the coupled strength mapping data includes setting the coupled strength threshold level partitioning rules. Specifically, above 0.8 is the high coupling area, 0.5 to 0.8 is the medium coupling area, 0.3 to 0.5 is the low coupling area, and below 0.3 is the weak coupling area. Each element in the coupled strength mapping matrix is labeled with a level to form a level label matrix, and the channel pairs with the same level are clustered into a group to construct an activation coupling degree association set. This set is represented by a structured vector, and each structure unit contains the channel pair number, activation time period index, coupling level label, and synchronous window number. After completing the construction of the activation coupling degree association data, the process of performing the chained fusion mapping operation is as follows: The topological path construction method is adopted to search for the channel sequences with continuous coupling paths in the coupling degree association set. For each path, the transfer coupling direction and strength gradient direction between nodes are calculated, and a chained interactive channel graph is constructed based on the path continuity. The channel graph is encoded in the form of a sparse graph and embedded into the activation structure of the current memory and computing unit as a parallel computing reference structure. Finally, the chained interactive association data is output. The chained data format is represented by a structured multi-directional graph model, and each node contains the activation channel number, coupling level, channel position offset, and channel coupling direction identifier. Each edge records the coupling path code number and edge coupling strength parameter.

[0027] Preferably, the recursive self-calibration of the original stored data according to the chained interactive association data in step S3 includes: Inferring the interactive pulse data according to the chained interactive association data; Performing timing reconstruction based on the interactive pulse data to generate timing interactive reconstruction data; Reconstruct the data verification correlation context through timing interaction to obtain the verification interaction correlation data; Perform feature residual decoupling on the verification interaction correlation data to obtain the residual decoupling data; Perform autoregressive feature recursion on the residual decoupling data to generate the recursive feature data; Determine the attenuation state of the verification interaction correlation data according to the recursive feature data; Perform synchronous gain adjustment on the verification interaction correlation data based on the attenuation state to generate the self-calibration update data.

[0028] In this embodiment, the operation of inferring interactive pulse data based on chained interactive correlation data includes loading the chained interactive correlation data into an interactive structure decoding module for inferring the logic flow. This module extracts the activation states between channels segment by segment in the time domain based on the interactive path order identified in the chained graph, and extracts the state change mapping between the starting activation node and the ending response node of the channels by identifying the continuous activation conduction structure between the channels. For each path, neural structure analogy deconstruction is performed in the order of channel activation, and whether there are inferable interactive pulses is judged by identifying the mutation segments of the activation states. The channel pulse indication label is output according to the cumulative distribution trend of the activation state differences between adjacent channels in the path. The paths with continuous activation responses and meeting the mutation trigger characteristics are marked as strong response paths, and the interactive pulse data structure is constructed accordingly. Each structural unit includes a path number, a channel sequence, an inferred pulse type identifier, a time index label, and a description of the activation sequence summary features. The operation of performing time series reconstruction based on the interactive pulse data includes inputting the interactive pulse data into an index-driven synchronous rearrangement module, and performing start-stop synchronous scanning processing on the activation segments of all pulse activation paths included. First, the channels in each path are numbered in chronological order, and the original activation signal segments corresponding to each number are extracted. The original activation sequence is sliced uniformly using a time window with a fixed width, and the length of the time window is set using preset parameters according to the event period corresponding to the inferred pulse. Each activation segment is divided into multiple small segments and attached with relative time tags. After aligning all segment data at a unified reference time, a time series activation matrix structure driven by the channel sequence is generated. Subsequently, the cross nodes between adjacent paths are identified as interactive points, and time marking signals are inserted into the activation matrix to form a multi-path synchronous interaction structure. The final output result is time series interaction reconstruction data organized by path, and the structure includes a channel index, a time period number, a summary of the signal segment features, and an interaction mark index. The operation of verifying the associated context through the time series interaction reconstruction data includes inputting the reconstructed data into a verification module for path consistency verification. The module performs context backtracking processing based on the activation time order of the channels in each path, identifies the activation delay relationship between consecutive channels in the path, and judges whether it conforms to the conduction feature logic defined in the previous chained interactive correlation data. By comparing the start and end times of the activation time periods and the expected interaction intervals in each path, if there is a path where the delay time between channels significantly deviates from the range defined in the preset conduction logic model, then the path is marked as an inconsistent path, otherwise it is marked as a consistent path. At the same time, a path consistency matrix is constructed for the verification results of all paths. The matrix includes a path number, a verification status label, a deviation interval time period, a deviation node number, and an activation delay index feature. The final output structured verification data is the verified interactive correlation data. The operation of performing feature residual decoupling on the verified interactive correlation data includes,Map the verification status of each path to its actual activation time distribution in the reconstructed data. Use the pre-stored standard activation template as a control model to compare the timing differences between the actual activation structure of the path and the standard template. Extract the activation start point, end point, duration, and activation sequence number of each channel to form a feature vector. Calculate the residual information through the differences between the vectors, and organize it into a residual vector group in units of paths. Deconstruct the vector group through dimensional decomposition, convert the cross-coupling residuals between all channels into several independent channel response residual sets, and perform structured storage on the response residual sets respectively. Each residual set corresponding to a channel contains activation duration offset, activation sorting offset, delay interval offset, and synchronization mismatch metrics. After structural integration, the output is residual decoupled data. The operation of autoregressive feature recursion on the residual decoupled data includes dividing the residual data of each channel into continuous time series according to its time period label, performing a residual trend scan operation on each segment of data according to a time sliding window of a fixed length, identifying whether there is a phenomenon of continuous offset accumulation in each segment, and judging the trend direction by comparing whether there is a gradually increasing trend in the offset degree distribution in the front and back windows. Then construct a residual recursion trajectory according to the trend direction. The trajectory contains the time period number, offset direction, offset acceleration label, historical cumulative residual amplitude classification identifier, and activation frequency change summary. Integrate the trajectories of all channels to generate recursion feature data. Each record unit in this data contains channel identification, recursion direction, time index, and recursion trend classification code. The operation of determining the attenuation state of the verification interaction correlation data according to the recursion feature data includes inputting the recursion feature data into the channel state evaluation module. First, classify the channels involved in each path according to their recursion direction identifiers. Mark the channels with negative continuous change in recursion direction as the decreasing category, mark the channels with positive continuous change as the increasing category, and mark the channels with multiple consecutive time periods marked as stable as the stable category. Summarize the marking information of all channels to generate a path-level status index table. This index table records the path number, channel number, status label, corresponding time period label, residual level category, and the conclusion of the synchronization status comparison between channels. At the same time, identify the channel combinations that are frequently marked as the decreasing category in the coupling relationship and mark them as potential attenuation coupling paths. Finally, all the mapping results of the status and path relationships form an attenuation state data set. The operation of synchronously adjusting the gain of the verification interaction correlation data based on the attenuation state includes inputting the attenuation state data set into the gain control engine module, performing enhancement instruction allocation processing on all the path channels in the decreasing state, assigning a predefined gain enhancement factor to the decreasing channels in each path, and applying this factor to the activation amplitude field of the corresponding channel in the path structure. At the same time, extract the channel pairs marked as attenuation coupling in all the coupling paths, and use the dual-channel synchronization strategy to set the enhancement ratio of their interaction amplitudes, and perform synchronous ratio adjustment on the amplitude field while ensuring that the synchronous activation intervals are consistent.Complete parameter replacement for the general reconstruction module of all enhanced channels, and repackage the adjusted structure into self-calibrated updated data. In each update node structure, record the activation amplitude, adjustment parameters, channel number, path number, and time period label before and after the update. Organize all self-calibrated updated data in a chained structure for storage.

[0029] Preferably, predicting heterogeneous access channels based on the self-calibrated updated data in step S4, and reconstructing each hierarchical data access link through the predicted heterogeneous access channels includes: Extract the multi-domain channel attribute spectrum from the self-calibrated updated data; Construct a channel cluster grouping structure based on the multi-domain channel attribute spectrum; Sort out the corresponding relationship between channel clusters and data blocks according to the storage-computation hierarchy based on the channel cluster grouping structure; Decompose the corresponding relationship between channel clusters and data blocks into multiple dynamically reconfigurable links; Predict heterogeneous access channels according to the dynamically reconfigurable links; Based on the preset storage capacity, adaptively reconstruct the predicted heterogeneous access channels to generate node adaptation links; Simulate the cache limit extension boundary through the node adaptation link, and determine the optimal cache layout based on the simulated cache limit extension boundary; Perform hierarchical link reconstruction and integration according to the optimal cache layout to obtain the reconstructed hierarchical data access links.

[0030] In this embodiment, extract the multi-domain channel attribute spectrum from the self-calibrated updated data, use the multi-source data fusion technology to integrate access data from different sources, including key attributes such as device identifiers, storage types, current bandwidth, historical latency, error rates, etc. Normalize the multi-dimensional attribute data using the Z-score standardization method so that the mean of each attribute is 0 and the standard deviation is 1. Construct a multi-domain channel attribute matrix, where the matrix dimension is the number of devices multiplied by the number of attributes. Extract the top K principal components through principal component analysis, and the value of K is determined according to the principle that the cumulative contribution rate exceeds 85%. Use the K-means clustering algorithm to divide the attribute spectrum into three categories: basic storage clusters, cache clusters, and computing acceleration clusters according to device types and storage media. Calculate the weighted average attribute value for each cluster, and the weight is determined according to the device online duration. Finally, save the result in the binary attribute spectrum file format, including a file header description and a data segment. The file header defines the attribute type and unit, and the data segment organizes the attribute values by cluster. Construct a channel cluster grouping structure based on the multi-domain channel attribute spectrum, establish a hierarchical clustering tree for various channels in the attribute spectrum file, calculate the inter-cluster distance using the Ward minimum variance method, and stop merging when the increase in intra-cluster variance exceeds the threshold When; The value is 1.5 times of the initial intra-cluster variance. The final clusters are divided into three categories according to the storage-computing architecture: high-performance computing clusters, large-capacity storage clusters, and balanced clusters. A globally unique identifier GUID is assigned to each type of cluster, which is generated by a combination of timestamp, machine ID, and serial number. An adjacency matrix is ​​established for the channels in each cluster. The matrix elements are calculated using the Jaccard similarity coefficient, and the threshold is set to 0.6. The threshold is determined from the interval [0.5, 0.7] by the elbow rule to form a weighted directed graph structure. The structure is stored in an adjacency list, and the table entries contain the pointed nodes, weight values, and connection type tags. According to the channel cluster grouping structure, the channel cluster-data block correspondence is sorted out according to the storage-computing level, and a storage-computing level mapping table is constructed. The table items include data block identifiers, hot and cold attribute tags, access frequency counters and priority levels. The LRU (Least Recently Used) strategy is used to update the access frequency counter. The time window is set to 5 minutes. The hot and cold tags are divided according to the median access frequency. The tags above half of the value are hot, otherwise they are cold. For each data block, the three channel clusters with the highest correlation are calculated. The correlation is calculated using TF-IDF weighted calculation. The word frequency is counted by access records. The inverse document frequency is the ratio of the total number of data blocks in the cluster to the total number of data blocks in the system. The data block-channel cluster association matrix is ​​generated. The matrix is ​​stored in CSR (Compressed Sparse Row) format to save space. An association log is created to record change operations in timestamp order. Each log contains the operation type, the data block involved, the cluster identifier and timestamp associated with the operation before and after. The channel cluster-data block correspondence relationship is decomposed into multiple dynamically reorganized links. The graph traversal algorithm is implemented according to the association relationship matrix. The improved Dijkstra algorithm is used to calculate the optimal data flow path. The edge weight is determined by the bandwidth-delay product. The delay is the moving average of the last 10 measurements. The bandwidth is a real-time detection value. The path calculation result is stored as a link descriptor object. The object contains the path sequence, estimated delay and bandwidth guarantee level. The health of the channels in the path is evaluated. The indicators include error rate, load rate and temperature. The upper limit of the health threshold is set. The channel exceeding the upper limit triggers the rerouting mechanism. The backup path pool mechanism is used to calculate and maintain three backup paths for each link in advance. The path with the best health is selected during rerouting. The link descriptors are stored according to the storage layer to which they belong. The computing layer link uses the metadata area of ​​the high-performance storage medium, and the storage layer link uses the log area of ​​the SSD. According to the dynamically reorganizable link, the heterogeneous access channel is predicted. The time series prediction method is used to analyze the historical traffic data of the link. The Holt-Winters three-parameter exponential smoothing model is selected. The smoothing coefficient Set them to 0.3, 0.2, and 0.1 respectively. Fit the parameters from historical data using the least squares method to predict the traffic distribution within a future time window. The size of the time window is set to 15 minutes. Adjust the link bandwidth allocation ratio according to the prediction results. Adopt a dynamic weight adjustment algorithm to prioritize ensuring the link resources with predicted traffic growth exceeding 20%. The bandwidth adjustment step size is set to 10 Mbps, and the cooling period after each adjustment is 30 seconds to prevent frequent fluctuations. Synchronously update the bandwidth parameters in the link descriptor. Recover the resources for the links with predicted traffic less than 50% of the current allocation, and transfer the idle bandwidth to the high-load links, with the transfer amount not exceeding 75% of the idle bandwidth. Adaptively reconstruct the predicted heterogeneous access channels based on the preset storage capacity to generate node-adapted links. Read the node hardware configuration parameters, including the number of CPU cores, memory capacity, storage capacity, and network bandwidth, and establish a resource capacity matrix. The matrix elements are the proportions of each resource in the total resources. Calculate the resource gap based on the resource matrix and the predicted traffic. Set the resource over-allocation coefficient to 1.2 to generate a set of reconstruction strategies. The strategies use a greedy algorithm to prioritize meeting the requirements of critical links, and the critical links are defined by grading according to the business importance. During the reconstruction process, keep the bandwidth ratio between the storage layer and the computing layer between 3:1 and 5:1. Generate a node-adapted link configuration file, organized in JSON format, containing the link ID, source node, target node, allocated bandwidth, QoS level, and priority label. The configuration file is stored in the shared memory area for access by each module. Simulate the cache limit extension boundary through the node-adapted link, and determine the optimal cache layout based on the simulated cache limit extension boundary. Establish a cache simulation environment, use a pseudo-random number generator to generate access patterns, and the access frequency distribution conforms to the Zipf's law with the parameter α set to 0.85, indicating the intensity of the long-tail effect. Gradually increase the simulated cache capacity from 100 MB to 10 GB, with an increase of 1 GB at each level, and record the cache hit rate change curve at different capacities. Use the binary search method to locate the capacity corresponding to the 95th percentile of the hit rate as the cache limit boundary. When optimizing the cache layout, classify the data blocks by temperature. The hot data is managed using the LRU strategy, and the cold data uses the LFU (Least Frequently Used) strategy. The cache block size is set to 4 KB, aligned to the memory page boundary. The distribution strategy uses the consistent hashing algorithm, and the number of virtual nodes is set to 200 times the number of physical nodes to reduce the data migration amount when the nodes change.Hierarchical link reconstruction and integration are performed according to the preferred cache layout to obtain reconstructed hierarchical data access links. The cache hierarchy is divided into three levels: L1, L2, and L3. The L1 cache is adjacent to the computing node, with a capacity of 10% of the total cache. The L2 is the intermediate layer, with a capacity of 50%. The L3 is the remote layer, with a capacity of 40%. Data prefetch thresholds between levels of caches are set based on the preferred layout. The threshold from L1 to L2 is set at 80% utilization rate, and from L2 to L3 is set at 70%. When reconstructing the hierarchical links, priority control is implemented for cross-level transmissions. The lower limit of the exclusive bandwidth ratio for high-priority links is 25%, and the upper limit is 50%. When establishing a link, end-to-end delay measurement is performed, using the three-way handshake protocol and RTT (Round-Trip Time) estimation. If the delay exceeds the threshold, path reconstruction is triggered, and the threshold is set at 1.5 times the average delay. The information of the reconstructed link is stored in a distributed key-value database, where the key is the global identifier of the link, and the value is the serialized object of the link parameter configuration.

[0031] Preferably, the multi-layer collaborative flow control cache integration according to the reconstructed hierarchical data access links in step S4 includes: Perform node priority reconstruction on the reconstructed hierarchical data access links to generate reconstructed data access links; Perform multi-source scheduling coupling processing on the reconstructed data access links to generate a collaborative scheduling link, where the range of the channel bandwidth balance coefficient for the multi-source scheduling coupling processing is set to 0.4 - 0.8; Perform cache path cascading integration on the data of the collaborative scheduling link to obtain flow control cache integration data. During the cache path cascading integration process, the cache block granularity is set to 512KB to 2MB, and the hierarchical cascading does not exceed a 4-layer structure; Suppress node queuing jitter based on the flow control cache integration data to generate a stable output cache, where the jitter suppression operation is performed based on the rule that the maximum transient queuing delay does not exceed 5ms; Perform delay trajectory difference cancellation processing through the stable output cache data to obtain access delay elimination and optimization data.

[0032] In this embodiment, historical bandwidth utilization data of each node is collected. The bandwidth usage in the past hour is obtained through the monitoring system. The bandwidth peak value and average utilization rate of each node are recorded. At the same time, the current load status of the node is monitored, including CPU usage rate, memory occupancy rate, and cache hit rate. These data are collected in real time by the distributed monitoring probe and updated every 10 seconds. According to these metrics, the comprehensive priority score of each node is calculated. Using the method of weighted summation, the bandwidth utilization rate accounts for 40% weight, the current load status accounts for 30% weight, and the cache hit rate accounts for 30% weight. The calculated priority scores are sorted. The top 20% of the nodes are marked as high-priority, the middle 60% are marked as medium-priority, and the bottom 20% are marked as low-priority. The nodes in the reconstructed data access link are marked with priorities. High-priority nodes are preferentially allocated high-speed channel resources, medium-priority nodes are allocated medium-speed channel resources, and low-priority nodes are allocated low-speed channel resources. The reconstructed data access link information with priority markings is generated. A channel bandwidth monitoring mechanism is established. The real-time bandwidth usage data of each channel, including the used bandwidth and the remaining bandwidth, is collected every 50 milliseconds. The bandwidth difference rate between channels is calculated. When the difference rate exceeds 15%, the multi-source scheduling coupling operation is triggered. The initial balance coefficient is 0.6. The optimal value is searched between 0.4 and 0.8 through the dynamic adjustment mechanism. During the adjustment process, the change in the overall throughput of the monitoring system is monitored. After each adjustment, wait for 10 milliseconds to observe the effect. Record the balance coefficient with the highest throughput as the current optimal value. According to the determined balance coefficient, the bandwidth resources of each channel are allocated. High-priority links are allocated a larger proportion of bandwidth, medium-priority links are allocated a medium proportion, and low-priority links are allocated a smaller proportion. However, ensure that the minimum service bandwidth of each link is not less than 100 Mbps. The collaborative scheduling link metadata containing the balance coefficient and the corresponding bandwidth allocation scheme is generated. The collaborative scheduling link data is implemented with cache path cascading integration to obtain the flow control cache integration data. During the cache path cascading integration process, the cache block granularity is set to 512 KB to 2 MB, and the hierarchical cascade does not exceed 4-layer structure. A multi-level cache path structure is constructed. The first layer is the storage layer cache, the second layer is the computing layer distributed cache, and the third layer is the node local cache. According to the data access heat, the data is allocated to different levels of caches. The heat grading mechanism is adopted. Data with an access frequency higher than the set threshold is marked as hot data and allocated to the first layer cache. Data with a moderate access frequency is marked as medium-temperature data and allocated to the second layer cache. Data with a lower access frequency is marked as cold data and allocated to the third layer cache. The maximum capacity of each layer of cache is set. The capacity of the first layer cache accounts for 50% of the total capacity, the second layer accounts for 30%, and the third layer accounts for 20%. The LRU replacement strategy is adopted to manage the cache blocks within each layer of cache. When the number of cache blocks exceeds the limit, the least recently used data block is eliminated. Different differential compression strategies are implemented for the cache data at different levels. Hot data is not compressed to ensure the access speed, and medium-temperature data uses a fast compression algorithm.Cold data uses a high compression ratio algorithm to generate a flow control cache integrated data object containing the cache path structure and parameters, suppresses node queuing jitter based on the flow control cache integrated data, and thus generates a stable output cache. Among them, the jitter suppression operation is executed based on the rule that the maximum transient queuing delay does not exceed 5 ms. A node queuing state monitoring mechanism is established, and the queue length and average service time data are collected every 10 milliseconds to calculate the transient queuing delay. When it is detected that the transient queuing delay exceeds 5 ms, the jitter suppression mechanism is started. A dynamic window adjustment strategy is adopted, and the optimal service window size is calculated according to the current queue length and service time. The adjustment range of the window size is limited within ±15% of the current value. Group scheduling is implemented for queues exceeding the threshold, and they are divided into several groups according to the packet arrival time. The maximum service volume of each group does not exceed 20% of the total queue length. The polling method is used to process each group of data in turn to ensure relatively balanced service time for each packet, generate a stable output cache index table, record the physical location and service timestamp of each data block in the buffer, perform delay trajectory difference cancellation processing on the stable output cache data to obtain access delay elimination and optimization data, establish a delay trajectory prediction model, collect the data block arrival time series of the first 10 time windows as the model input, predict the data block arrival time at the next moment, compare the predicted arrival time with the actual arrival time, calculate the time difference, and when the difference exceeds the set threshold, perform a delay compensation operation. Different compensation strategies are selected according to the size of the difference. When the difference is small, the prefetch strategy is adopted to read the data 1 millisecond in advance. When the difference is moderate, the dual-cache mechanism is used to load the data in parallel. When the difference is large, the fast channel switching mechanism is triggered to re-obtain the data from the standby high-speed channel, and an optimized access delay log is generated, recording the original delay, compensated delay, and final access timestamp of each data block.

[0033] Preferably, the task reorganization of the dependency path for the access delay elimination and optimization data in step S5 includes: Tracking the data flow path of the access delay elimination and optimization data, and analyzing the dependency path characteristics in the access delay elimination and optimization data based on the data flow path; Identifying bottleneck nodes according to the dependency path characteristics; Determining the critical dependency path based on the bottleneck nodes; Decoupling and reorganizing the task units of the critical dependency path to obtain decoupled and reorganized task data; Performing dependency grouping and aggregation on the decoupled and reorganized task data to generate dependency grouping and aggregation data; Performing parallel mapping and distribution based on the dependency grouping and aggregation data to obtain task reorganization parallel processing data.

[0034] In this embodiment, the data flow path of the access latency elimination optimization data is traced, and the dependency path features in the access latency elimination optimization data are analyzed based on the data flow path. The timestamps and node identifiers of the access latency elimination optimization data when flowing in the system are collected. The node sequence and time consumption passed by each data packet are recorded through a distributed tracing system. The timestamp information is embedded in the data packet header with a precision reaching the nanosecond level. The unique identifier of the data packet is calculated using a hash function, and the SHA-256 hash algorithm is adopted to ensure the uniqueness of the identifier. A data flow path graph is constructed, where the nodes represent processing units, the edges represent the data flow direction, and the edge weights are the transmission delays. 5000 data flow samples are collected, and features such as path length, node hops, and transmission delays are analyzed. The key path dependency patterns are extracted, and the importance of nodes is calculated using an improved PageRank algorithm. The number of iterations is set to 20 times, and the damping factor is set to 0.85. The long dependency paths and cyclic dependencies are identified. The data flow path graph is stored in an adjacency list structure, and each table entry records the target node, transmission delay, and data volume size. The bottleneck nodes are identified according to the dependency path features. By analyzing the node delay and throughput metrics in the data flow path graph, the bottleneck node identification criteria are set, and the average node delay exceeds the system average delay by 1.Nodes with a throughput less than one-fifth of the average are determined as potential bottleneck nodes. Collect the performance metrics of each node, including average latency, maximum latency, throughput, and packet loss rate. Calculate the weighted score for each node, with the weight distribution being 60% for latency and 40% for throughput. The score threshold is dynamically adjusted based on historical data. Use a sliding window algorithm to calculate the statistical metrics for the last 1000 data packets, with a window sliding step of 50. Conduct a secondary verification on nodes that exceed the threshold, confirm the bottleneck status by increasing probe packets, mark the confirmed bottleneck nodes, and record their IDs, types, and processing capabilities. Store the results in a distributed key-value database, with the key being the node ID and the value being a structure containing performance metrics and processing suggestions. Determine the critical dependency paths based on the bottleneck nodes. Extract all paths containing the identified bottleneck nodes from the data flow path graph. Use a depth-first search algorithm to traverse the path graph, with a depth limit set to 10 levels. Collect paths with a length exceeding 5 hops as candidate critical paths. Calculate the comprehensive weight for each candidate path, which is determined by the total path latency, the number of bottleneck nodes, and the data volume size. Latency accounts for 50%, the number of nodes accounts for 30%, and the data volume accounts for 20%. Sort the candidate paths and select the top 20 with the highest weights as the critical dependency paths. Mark the determined critical paths, record the path IDs, start and end nodes, and the included bottleneck nodes, and generate a critical path metadata descriptor. The descriptor contains the path topology structure, node attributes, and weight calculation parameters. Store the descriptor in an in-memory database, with an expiration time set to 24 hours and recalculate after timeout. Decouple and reorganize the task units on the critical dependency paths to obtain decoupled and reorganized task data. Divide the tasks on the critical dependency paths into independent units according to the processing logic. Use a control flow graph analysis method to identify the task unit boundaries, with the division granularity set to a single function call or data processing stage. Establish a task unit dependency relationship matrix, where the matrix elements are the data dependency strengths between task units, and the strengths are divided into three levels: strong, medium, and weak. Reorganize the task units according to the dependency relationship matrix. Task units with strong dependency relationships are retained in the same processing unit, and task units with medium and weak dependency relationships are split across units. Maintain data consistency during the reorganization process, and use a distributed lock mechanism to ensure the atomicity of critical data operations. Store the reorganized task units in a task queue according to the execution order. The queue uses a priority structure, where urgent tasks have a higher priority than ordinary tasks, and the priority is set according to the task latency sensitivity. The sensitivity index is dynamically evaluated by the system. Generate a decoupled and reorganized task data object, which contains task unit IDs, dependency relationships, and execution order. Group and aggregate the decoupled and reorganized task data according to dependencies to generate dependency-grouped and aggregated data. Group the task units according to the dependency relationship strength between them. Task units with strong dependency relationships are divided into the same group, and task units with medium and weak dependency relationships are assigned to different groups. Use a hierarchical clustering algorithm for grouping, with the distance metric being the reciprocal of the dependency relationship strength, and the clustering radius set to 0.5. Control the number of task units in each group between 5 and 20. Calculate the overall dependency characteristics of each group of task units, including the average dependency strength, the maximum dependency distance, and the data transfer volume. Establish an inter-group communication matrix, record the data interaction volume and latency between different groups, and implement the optimization of merging and splitting task units according to the group characteristics and the communication matrix. Merge adjacent groups with a large communication volume, and split groups with a high communication density and independent processing logics. Generate a dependency grouping aggregation data object, which records the group ID, the included task units, the dependency relationship graph, and the communication characteristics. Based on the dependency grouping aggregation data, perform parallel mapping and distribution to obtain task reorganization parallel processing data. Allocate an independent processing resource pool for each dependency group. The resource pool includes CPU cores, GPU accelerators, and memory blocks. The resource allocation ratio is determined according to the group task characteristics. Allocate more CPU and GPU resources for compute-intensive tasks, and allocate a larger memory capacity for data-intensive tasks. Use a round-robin scheduling algorithm to allocate the grouped tasks to available processing nodes. The node selection priority is determined by the resource utilization rate, network bandwidth, and processing capacity. The weight allocation is 30% for resource utilization rate, 40% for bandwidth, and 30% for processing capacity. Set the task parallelism parameter. The number of parallel tasks does not exceed 2 times the number of CPU cores and is not less than 2. Perform thread-level parallel optimization on the tasks allocated to the same node. The number of threads matches the number of tasks, and the work-stealing algorithm is used between threads to balance the load. Package the task execution plan after parallel mapping into a task reorganization parallel processing data object, which includes a task allocation table, parallel parameters, and a scheduling strategy. The data is stored in the shared memory area.

[0035] Of particular importance is the identification of bottleneck nodes based on the dependency path characteristics, including: Archive the path delay gradient of the dependency path characteristic data to generate a path delay distribution; Extract the path delay distribution of the path delay distribution to obtain the delay peak indexing data; Perform path overlap metric detection on the delay peak indexing data to obtain the path conflict density data; Based on the path conflict density data, perform node resource load mapping to generate node load occupancy data; According to the node load occupancy data, perform the maximum residence-waiting ratio screening to obtain the bottleneck node data.

[0036] In this embodiment, timestamps and node identification information in the dependent path feature data are collected to construct a dataset containing at least 10,000 records. Each record includes a path ID, a start node, an end node, and a transmission delay. The transmission delays of each path are sorted in chronological order, and the delay change amount between adjacent time points is calculated to form a delay gradient sequence. The sliding window algorithm is used to process the delay gradient sequence, with the window size set to 50 time points and the step size to 10. The average value and standard deviation of the delay gradient within the window are calculated, and the calculation results are archived and stored according to the path ID to form a path delay gradient archive. The archive data structure adopts the key-value pair form, where the key is the path ID and the value is a tuple containing the average gradient, standard deviation, and gradient sequence. The archived path delay gradient data is grouped and statistically analyzed according to the path ID, and the delay distribution histogram of each path is calculated. The number of bins in the histogram is set to 50, and the range covers the minimum to maximum delay values of all paths to generate a path delay distribution dataset. The path delay distribution is extracted to obtain the delay peak indexing data. For each path in the path delay distribution dataset, the bin interval with the largest count in the delay distribution histogram is located, and the midpoint value of this interval is marked as the delay peak of the path. The bimodal detection algorithm is used to identify the delay distribution with multimodal characteristics. When multiple peaks are detected and the height ratio of the sub-peak to the main peak exceeds 0.At 3 o'clock, mark this path as a multi-peak path. For a single-peak path, directly record the position of the main peak. For a multi-peak path, record all peak positions and their relative heights, and construct a delayed peak index table. The table structure includes the path ID, the main peak delay value, the secondary peak delay value (if any), and the kurtosis coefficient. The kurtosis coefficient is calculated using the fourth-order central moment formula, with the denominator being the fourth power of the standard deviation and the numerator being the sum of the fourth powers of the deviations. Perform path overlap measurement detection on the delayed peak indexing data to obtain path conflict density data, and construct a path overlap detection matrix. The rows and columns of the matrix represent different paths respectively, and the matrix element value represents the degree of overlap of the delayed peaks of the corresponding path pair. The overlap degree calculation formula is the proportion of the overlap time in the total time when the absolute value of the difference in peak delays between two paths is less than the set threshold τ. The threshold τ is initially set to 5 milliseconds and dynamically adjusted according to the network environment within the range of [3, 8] milliseconds. Traverse all path pairs to calculate the overlap degree, and form an N×N-dimensional overlap matrix, where N is the total number of paths. Sum the overlap matrix by row to obtain the total conflict time of each path, and divide it by the total monitoring time to obtain the path conflict density index. Construct a path conflict density vector, where the vector elements are the conflict density values of each path. After sorting from high to low density, store them in the distributed cache system Memcached. Based on the path conflict density data, perform node resource load mapping to generate node load occupancy data, establish a node resource monitoring system, collect the CPU usage rate, memory occupancy rate, and network bandwidth utilization data of each node, and set the sampling frequency to once per second. Use a sliding window to calculate the average and peak resource usage in the last 60 seconds. Map the paths in the path conflict density vector to the corresponding start and end nodes, and accumulate the conflict density values of all paths passing through the node to obtain the total conflict load of the node. Combine the node resource monitoring data to calculate the node comprehensive load index. The calculation formula is Comprehensive Load =. × Conflict Load + × CPU Usage Rate + × Memory Occupancy Rate + × Bandwidth Utilization Rate, Weight Coefficient = 0.4, = 0.3, = 0.2, Let \(= 0.1\). Normalize the calculated comprehensive load index and map it to the interval \([0, 1]\) to generate a node load occupancy data table. The table structure includes node ID, comprehensive load index, the usage rate of each resource, and the contribution rate of conflict load. The data is stored in the distributed time-series database InfluxDB. Set the data retention policy to 30 days. Screen the maximum residence-waiting ratio based on the node load occupancy data to obtain bottleneck node data. Define the node residence time as the average processing time of data packets within the node, and the waiting time as the average waiting time of data packets in the node queue. Collect the residence time and waiting time data of each node through the distributed tracing system. Calculate the residence-waiting ratio \(RWR\) of each node, where \(RWR=\frac{residence\ time}{waiting\ time}\). Set the \(RWR\) threshold range to \([0.5, 2.0]\). Nodes outside this range are marked as abnormal nodes. Sort the nodes within the normal range in descending order of the comprehensive load index, and select the node with the highest load index and an \(RWR\) value close to the lower limit of the threshold as the candidate bottleneck node. Further analyze the resource usage trend of the candidate node, and calculate the moving standard deviation of the CPU, memory, and bandwidth usage rates in the last 10 minutes. When the standard deviation exceeds 30% of its respective mean, it is confirmed that the resource fluctuation is too large. Finally, determine the node with the highest comprehensive load index and stable resource fluctuation as the bottleneck node, and generate a bottleneck node data table, which includes node ID, comprehensive load index, \(RWR\) value, resource fluctuation index, and confirmation timestamp.

[0037] Preferably, the pipelining scheduling of the task reorganization parallel processing data in step S5 and the execution of eliminating parallel conflicts include: Divide the task reorganization parallel processing data into pipeline segments to generate preliminary pipeline segment division data; Based on the preliminary pipeline segment division data, perform resource affinity matching to obtain resource affinity matching data; According to the resource affinity matching data, perform parallel timing optimization on the task reorganization parallel processing data to generate timing optimization scheduling data; Detect potential conflict clusters in the timing optimization scheduling data, and strip and reconstruct the conflict paths based on the potential conflict clusters to obtain conflict stripping and reconstruction data; Fuse the conflict stripping and reconstruction data into a conflict-free path to obtain parallel conflict decoupling data.

[0038] In this embodiment, the task dependency graph in the data of the parsing task reorganization parallel processing is parsed, the execution time and resource requirement parameters of each task node are extracted, the pipeline segments are divided by using the method based on critical path analysis. All paths from the start node to the end node in the task dependency graph are traversed, the total execution time of each task node on each path is calculated, the longest path is determined as the critical path, the task nodes on the critical path are evenly sliced into several pipeline segments according to the execution time, and the number of task nodes included in each segment is dynamically adjusted according to the number of system hardware resources. For example, when the system has 8 computing units, the number of task nodes included in each segment is set to 1 / 8 of the total number of tasks. The task nodes on the non-critical path are allocated to each pipeline segment by using the backward shift strategy to ensure that the difference in the total execution time of each pipeline segment does not exceed 15%, and a preliminary pipeline segment division data structure is generated, including the pipeline segment number, the list of task nodes included, the estimated execution time and the total resource requirement. Based on the preliminary pipeline segment division data, resource affinity matching is performed to obtain resource affinity matching data. The system hardware resource configuration information is collected, including the number of CPU cores, the number of GPU accelerators, the memory capacity and the cache size, and a resource feature matrix is established. The rows of the matrix represent different types of hardware resources, and the columns represent the resource requirement characteristics of each pipeline segment. The cosine similarity algorithm is used to calculate the matching degree between each pipeline segment and the hardware resources. The similarity calculation formula is the dot product of vectors divided by the product of the vector norms. The resource affinity threshold range is set to [0.7, 1.0]. When the similarity is lower than 0.7, it is marked as a low-affinity pipeline segment. For the low-affinity pipeline segments, a resource reallocation strategy is implemented, and the computing unit with more idle resources is preferentially selected for migration. The task dependency relationship is kept unchanged during the migration process. The greedy algorithm is used to select the migration target node, and the node that can maximize the affinity is selected each time. A resource affinity matching data table is generated, recording the pipeline segment number, the matching hardware resource number, the affinity score and the migration status flag. According to the resource affinity matching data, parallel timing optimization is performed on the data of the parsing task reorganization parallel processing to generate timing optimization scheduling data. A task execution timeline model is constructed, the pipeline segments are allocated to the corresponding hardware resources according to the resource affinity matching results, and the dynamic programming algorithm is used to optimize the task scheduling order on each resource. The state transition equation is defined as the total completion time of the task sequence on the current resource is equal to the completion time of the previous state plus the execution time of the current task. The constraint conditions are the task dependency relationship and the resource exclusivity requirements. For the tasks with data dependency relationships, a pipeline parallel strategy is implemented, and the pipeline depth parameter is set to 3, that is, each task is split into 3 stages and executed in parallel on the pipeline. The earliest start time and the latest end time of each pipeline segment are calculated to generate a scheduling plan in the form of a Gantt chart, which is converted into a timing optimization scheduling data structure, including the time slice number, the allocated pipeline segment number, the start time and the end time. Potential conflict cluster detection is performed on the timing optimization scheduling data, and conflict path stripping and reconstruction are performed based on the potential conflict clusters to obtain conflict stripping and reconstruction data.A conflict detection matrix is established. The rows and columns of the matrix respectively represent the pipeline segments on different time slices. The matrix element value of 1 indicates that there are resource competition or data dependence conflicts in the pipeline segments on the corresponding time slices. The scan line algorithm is used to traverse the time slice sequence in the scheduling data to detect whether there are conflicts in the pipeline segments on the same resource on adjacent time slices. The conflict threshold parameter is set that the number of resource contention times exceeds 2 times. When a conflict is detected, the relevant pipeline segments are marked as potential conflict clusters. Path analysis is performed on the potential conflict clusters to identify the resource types and data dependence paths where the conflicts occur. The path stripping strategy is used to migrate the conflicting pipeline segments to idle resources or adjust the execution time, and the task dependence relationship remains unchanged during the migration process. For the reconstructed scheduling scheme, the earliest start time and the latest end time of each pipeline segment need to be recalculated, and a conflict stripping reconstruction data table is generated, recording the original pipeline segment number, the newly allocated pipeline segment number, the time slice number after migration, and the conflict resolution mark. The conflict stripping reconstruction data is fused into a conflict-free path to obtain parallel conflict decoupled data, and a conflict-free scheduling graph model is constructed. The nodes represent the pipeline segments, and the edges represent the task dependence relationship or resource allocation constraints. The depth-first search algorithm is used to traverse the scheduling graph to detect whether there is a loop structure. When a loop is detected, a loop-breaking operation is performed, and the edge with the smallest weight in the loop is selected for deletion. The weight is defined as the product of the task execution time and the resource competition degree. The path fusion operation is performed on the reconstructed scheduling graph to merge the pipeline segments without dependence relationships to be executed on different resources in the same time slice to improve resource utilization rate. During the merging process, the task load balance on each resource is maintained, and the difference does not exceed 10%. A parallel conflict decoupled data structure is generated, including the fused time slice number, the set of allocated pipeline segments, the execution order of each pipeline segment, and the resource allocation scheme. The data is stored as an XML format file. The root node records the scheduling period and the total number of resources, and the child nodes record the allocation information of each pipeline segment in the order of time slices. The attributes include the execution order and the dependence relationship mark.,

[0039] The present invention also provides a memory-computation integrated parallel processing system for executing the memory-computation integrated parallel processing method as described above. The memory-computation integrated parallel processing system includes: A multi-scale feature fusion module for obtaining the original stored data; performing topological skeleton projection on the original stored data to obtain a projection skeleton structure; and performing feature hierarchical integration on the original stored data according to the projection skeleton structure to generate multi-scale spatial fusion data; A weight matching activation module for determining a weight relationship matrix based on the multi-scale spatial fusion data; performing gradient optimization adjustment on the multi-scale spatial fusion data through the weight relationship matrix, and performing memory-computation unit matching to obtain multiple memory-computation unit activation mapping data; An interactive chain self-calibration module is used to mine the chain interactive correlation data between the activation mapping data of the memory-computation units; recursively self-calibrate the original stored data according to the chain interactive correlation data to generate self-calibration updated data; A channel reconstruction flow control integration module is used to predict heterogeneous access channels based on the self-calibration updated data, and reconstruct each hierarchical data access link through the predicted heterogeneous access channels; perform multi-layer collaborative flow control cache integration according to the reconstructed hierarchical data access links to obtain access latency elimination optimized data; A task decoupling scheduling module is used to perform dependency path task recombination on the access latency elimination optimized data to generate task recombination parallel processing data; perform pipelining scheduling on the task recombination parallel processing data to execute and eliminate parallel conflicts, thereby obtaining parallel conflict decoupled data.

[0040] In the present invention, through the multi-scale feature fusion module, the acquisition of the original stored data is combined with the topological skeleton projection to form a clear projection skeleton structure. Feature hierarchical integration realizes multi-dimensional analysis of data on this basis. The generated multi-scale spatial fusion data improves the efficiency of data processing while retaining key features. The weight matching activation module provides a scientific basis for the gradient optimization of data based on the weight relationship matrix generated from the multi-scale spatial fusion data. The effective matching of the memory and computing units improves resource utilization. The activation mapping data of multiple memory-computation units realizes the basis of parallel processing through optimization and adjustment. The interactive chain self-calibration module enhances the correlation between data by mining the chain interactive correlation data between the activation mapping data of the memory-computation units. The updated data generated by recursive self-calibration ensures the consistency and accuracy of the original data. The channel reconstruction flow control integration module predicts heterogeneous access channels based on the self-calibration updated data, and the reconstructed hierarchical data access links optimize the data flow path. The multi-layer collaborative flow control cache integration formed after reconstruction effectively eliminates access latency, and the generated optimized data significantly improves the access efficiency. The task decoupling scheduling module creates conditions for parallel processing by performing dependency path task recombination on the access latency elimination optimized data. The generated task recombination parallel processing data improves the system response speed during execution. The pipelining scheduling mechanism effectively eliminates conflicts in parallel processing. The finally obtained parallel conflict decoupled data improves the overall system processing ability. The entire system realizes efficient and intelligent data processing within the framework of memory-computation integration, provides reliable support for the execution of complex computing tasks, promotes the development of new memory-computation architectures, enhances the flexibility and adaptability of the system, optimizes the utilization of storage and computing resources, provides an innovative technical path for future multi-task parallel processing. The high efficiency and accuracy of the overall process lay a solid foundation for the application in large-scale data processing scenarios, promote the improvement of data processing capabilities in high-performance computing environments, and drive the wide application and development of memory-computation integration technology.

[0041] Therefore, from any perspective, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the application documents are intended to be encompassed within the present invention.

[0042] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A storage and computing integrated parallel processing method, characterized in that: The following steps are involved: Step S1: Obtain original stored data; Perform topological skeleton projection on the original stored data to obtain the projection skeleton structure; perform feature hierarchical integration on the original stored data according to the projection skeleton structure to generate multi-scale spatial fusion data; Step S2: Determine a weight relationship matrix based on multi-scale spatial fusion data; The multi-scale spatial fusion data is gradient optimized and adjusted through the weight relationship matrix, and the memory-computing unit matching is performed to obtain activation mapping data of multiple storage and computing units; Step S3: mining the chain-like interactive association data between the storage and computing unit activation mapping data; recursively self-calibrating the original stored data according to the chain-like interactive association data to generate self-calibration update data; Step S4: predicting heterogeneous access channels based on self-calibration update data, and reconstructing each hierarchical data access link by predicting the heterogeneous access channels; performing multi-layer collaborative flow control cache integration according to the reconstructed hierarchical data access links, thereby obtaining access delay elimination optimization data; Step S5: reorganizing the dependent path tasks on the access delay elimination optimization data to generate task reorganization parallel processing data; The task reorganization parallel processing data is pipelined to eliminate parallel conflicts, thereby obtaining parallel conflict decoupling data.

2. The storage-computation integrated parallel processing method according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: performing channel domain division processing on the original stored data to obtain channel distribution fragment data, wherein the channel division granularity range is set to a pixel block dimension of 8×8 to 64×64; Step S12: performing local topological tensor mapping processing on the channel distribution fragment data to generate basic topological mapping data, wherein the mapping dimension is set to a 3rd-order tensor structure and the element dimension range is 16-512; Step S13: Perform skeletonized structural projection on the basic topological mapping data to obtain a projected skeleton structure, wherein the structural connectivity threshold is set to 0.35-0.65, and the skeleton sparsity rate is controlled to 15%-40%; Step S14: performing scale weight coupling according to the projected skeleton structure and nesting it into a skeleton scale label; Step S15: hierarchical feature compression and fusion are performed on the original stored data according to the skeleton scale label, thereby generating hierarchical feature integration data, wherein the fusion depth is set to 4 to 12 layers, and the compression ratio is controlled at 1:4-1:12; Step S16: Perform spatial redundancy regularization on the hierarchical feature integration data to obtain multi-scale spatial fusion data, wherein the redundant regularization window size is set to 5×5 to 11×11, and the residual filter threshold is set to 0.02-0.

12.

3. The storage-computation integrated parallel processing method according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: performing memory mapping expansion on the multi-scale spatial fusion data to obtain memory mapping structure expansion data; Step S22: identifying the pattern of the expanded data of the memory mapping structure, and classifying the identified pattern according to similarity to obtain a classification space feature pattern; Step S23: constructing a weight matrix based on the classification space feature pattern to generate a weight relationship matrix; performing gradient optimization adjustment on the weight relationship matrix and performing structured sparsity projection to obtain structured sparse mapping data; Step S24: Memory addressing optimization is performed according to the structured sparse mapping data, and a calculation unit activation mapping process is performed based on the addressing optimization data, thereby obtaining a plurality of storage and calculation unit activation mapping data.

4. The storage-computation integrated parallel processing method according to claim 1, characterized in that: The chain-type interactive association data between the mining storage and computing unit activation mapping data in step S3 includes: Perform interactive channel monitoring on the storage and computing unit activation mapping data to obtain interactive channel probe data; Extracting activation coupling from interactive channel probe data to generate activation coupling correlation data; Chain fusion mapping is performed according to the activation coupling degree correlation data to obtain chain interaction correlation data.

5. The storage-computation integrated parallel processing method according to claim 1, characterized in that: The recursive self-calibration of the original stored data according to the chained interactive correlation data in step S3 includes: Inferring interaction spike data from chained interaction correlation data; Perform time series reconstruction based on the interactive pulse data to generate time series interactive reconstruction data; Reconstruct the data verification correlation context through time series interaction, so as to obtain verification interaction correlation data; Perform feature residual decoupling on the verification interactive correlation data to obtain residual decoupling data; Perform autoregressive feature recursion on the residual decoupled data to generate recursive feature data; Determining a decay state of verification interactive correlation data according to the recursive characteristic data; The calibration cross-correlation data is synchronously gain adjusted based on the attenuation state to generate self-calibration update data.

6. The storage-computation integrated parallel processing method according to claim 1, characterized in that: Predicting heterogeneous access channels based on the self-calibration update data in step S4, and reconstructing each hierarchical data access link by predicting the heterogeneous access channels includes: Extracting multi-domain channel attribute spectra from calibration update data; Construct channel cluster grouping structure based on multi-domain channel attribute spectrum; According to the channel cluster grouping structure and storage-computing level, the correspondence between channel clusters and data blocks is sorted out; Decompose the channel cluster-data block correspondence into multiple dynamically reconfigurable links; Predict heterogeneous access channels based on dynamically reconfigurable links; Based on the preset storage capacity, the predicted heterogeneous access channel is adapted and reconstructed to generate a node adaptation link; Simulating a cache limit extension boundary through a node adaptation link, and determining an optimal cache layout based on the simulated cache limit extension boundary; The hierarchical links are reconstructed and integrated according to the preferred cache layout to obtain reconstructed hierarchical data access links.

7. The storage-computation integrated parallel processing method according to claim 1, characterized in that: The multi-layer collaborative flow control cache integration in step S4 according to the reconstruction of each hierarchical data access link includes: Reconstructing the node priority of each hierarchical data access link to generate a reconstructed data access link; Perform multi-source scheduling coupling processing on the reconstructed data access link to generate a collaborative scheduling link, wherein the multi-source scheduling coupling processing adopts a channel bandwidth balance coefficient range set to 0.4-0.8; Implement cache path cascade integration on the collaborative scheduling link data to obtain flow control cache integration data. During the cache path cascade integration process, the cache block granularity is set to 512KB to 2MB, and the hierarchical cascade does not exceed 4 layers. Based on the flow control cache, data is integrated to suppress node queue jitter, thereby generating a stable output cache, where the jitter suppression operation is performed based on the rule that the maximum transient queue delay does not exceed 5ms; The delay trace difference offset processing is performed by stabilizing the output cache data to obtain the access delay elimination optimization data.

8. The storage-computation integrated parallel processing method according to claim 1, characterized in that: The step S5 of reorganizing the dependent path tasks for the access delay elimination optimization data includes: Track the data flow path of the access delay elimination optimization data, and analyze the dependent path characteristics in the access delay elimination optimization data based on the data flow path; Identify bottleneck nodes based on dependency path characteristics; Determine the critical dependency path based on the bottleneck nodes; Decouple and reorganize the task units of the key dependency paths to obtain decoupled and reorganized task data; Dependent grouping and aggregation are performed on the decoupled and reorganized task data to generate dependent grouping and aggregation data; Based on the dependency grouping, the aggregated data is distributed in parallel, and the task reorganization and parallel processing data are obtained.

9. The storage-computation integrated parallel processing method according to claim 1, characterized in that: In step S5, the task reorganization parallel processing data is pipelined and executed to eliminate parallel conflicts, including: Divide the task reorganization parallel processing data into pipeline segments and generate preliminary pipeline segment division data; Perform resource affinity matching based on the preliminary division data of the pipeline segments, thereby obtaining resource affinity matching data; Perform parallel timing optimization on task reorganization parallel processing data according to resource affinity matching data to generate timing optimization scheduling data; Performing potential conflict cluster detection on the timing optimization scheduling data, and performing conflict path stripping and reconstruction based on the potential conflict clusters to obtain conflict stripping and reconstruction data; The conflict stripping and reconstructing data are fused into conflict-free paths to obtain parallel conflict decoupled data.

10. A storage and computing integrated parallel processing system, characterized in that: For executing the storage-computing integrated parallel processing method as claimed in claim 1, the storage-computing integrated parallel processing system comprises: The multi-scale feature fusion module is used to obtain the original stored data; perform topological skeleton projection on the original stored data to obtain the projection skeleton structure; perform feature hierarchical integration on the original stored data according to the projection skeleton structure, thereby generating multi-scale spatial fusion data; The weight matching activation module is used to determine the weight relationship matrix based on the multi-scale spatial fusion data; the multi-scale spatial fusion data is gradient optimized and adjusted through the weight relationship matrix, and the memory-computing unit matching is performed to obtain activation mapping data of multiple storage and computing units; The interactive chain self-calibration module is used to mine the chain-like interactive correlation data between the activation mapping data of the storage and computing units; recursively self-calibrate the original stored data according to the chain-like interactive correlation data to generate self-calibration update data; The channel reconstruction flow control integration module is used to predict heterogeneous access channels based on self-calibration update data, and reconstruct each hierarchical data access link by predicting the heterogeneous access channels; multi-layer collaborative flow control cache integration is performed according to the reconstruction of each hierarchical data access link, so as to obtain access delay elimination optimization data; The task decoupling scheduling module is used to reorganize the dependent path tasks for the access delay elimination optimization data and generate task reorganization parallel processing data; perform pipeline scheduling on the task reorganization parallel processing data and eliminate parallel conflicts to obtain parallel conflict decoupling data.

Images