Random subgraph sampling method and system for machine or deep learning training and reasoning

Abstract

The invention relates to a random subgraph sampling method and system for machine or deep learning training and reasoning. The method comprises the following steps: step 1, estimating a budget; 2, scale solving is carried out; 3, at least one sampling rule is selected from a preset sampling rule base to generate sub-image batches meeting constraints; step 4, obtaining sub-graph labels; 5, implementing a boundary retention mechanism and / or virtual neighbor convergence on the cut edges of the cross-sub-graph to compensate cross-sub-graph information; 6, in the training stage, importance weighting and / or reweighting loss are / is carried out on estimation deviation introduced by non-uniform sampling; step 7, training and / or reasoning the machine / deep learning model based on the subgraph batch, and performing budget adaptive adjustment on the subgraph scale and the sampling rule according to actual occupation and time delay during operation; and executing a corresponding task through the machine / deep learning model. According to the method, data which cannot be trained / inferred originally runs on limited equipment in a sub-graph batch form, and the problem of memory overflow is avoided.

Description

Technical Field

[0001] This invention belongs to the field of data processing and model training technology in machine learning and deep learning, specifically relating to a random subgraph sampling method and system for machine or deep learning training and inference. Background Technology

[0002] In recent years, the application of machine learning and deep learning in multimodal and large-scale scenarios has expanded rapidly. The scale of input (samples) and output (labels) data has continued to climb, significantly exceeding the computing power and memory capacity of single machines or edge devices. This mainly includes, but is not limited to, the following typical data forms: 1) Ultra-large format images: Natural images and remote sensing / satellite images exhibit ultra-high resolution and multispectral / multitemporal characteristics, commonly ranging from hundreds of millions to billions of pixels, and the corresponding pixel-level segmentation / change detection labels are also growing exponentially. 2) Graph structure data: Social networks, knowledge graphs, urban traffic networks and Internet of Things (IoT) node networks are massive in scale, with huge numbers of nodes / edges, obvious long tails in degree distribution, and significantly increased dimensionality in edge features and temporal attributes. 3) City-level and population-level data: Urban grid perception, population flow and macroeconomic indicators coexist with high temporal resolution and multiple spatial scales, which requires both large-scale input and label tensors that cover multi-dimensional targets; 4) Multi-task / large-scale prediction: In tasks such as multi-label classification, dense prediction, and long sequence multi-step prediction, the output dimension grows linearly or superlinearly with the number of tasks and prediction steps, causing the label side to become a bottleneck in memory and latency.

[0003] Faced with the above data characteristics, current engineering practices typically employ the following methods to make do with limited resources: 1) Hardware stacking (multi-card / multi-machine parallelism, model parallelism): high cost, complex deployment, and not universally applicable to edge / terminal devices; 2) Static downsampling and cropping (image slicing, fixed neighbor number sampling, offline graph partitioning): easily introduce semantic context loss and boundary effects, and have poor transferability between different datasets and models; 3) Memory tricks (mixed precision, gradient checkpointing, external memory mapping): can alleviate peak memory usage, but have no fundamental constraints on the size of the input / output itself, and may still trigger unpredictable memory overflow (Out-Of-Memory, OOM) or cause significant latency jitter. 4) General neighbor sampling / mini-batch training: Although it can reduce the size of a single batch, it often deviates from the label-side constraints. When the label dimension is extremely large (such as pixel level, grid level or multi-task multi-step), it still cannot guarantee overall trainability / inference capability. Moreover, the statistical bias, missing information across subgraphs, and inconsistent results caused by non-uniform sampling are difficult to control systematically.

[0004] Therefore, given the current reality of dual expansion on both the input and label sides, static segmentation or single-dimensional optimization alone is insufficient for stable training and inference on constrained devices (such as edge GPUs / CPUs, mobile terminals, or embedded platforms). There is an urgent need for an adaptive data organization mechanism that takes into account both the scale of input and labels while using device computing power and memory as hard constraints. This mechanism should, while ensuring statistical validity, divide the originally unprocessable massive datasets into manageable sub-problems online, and through bias control and cross-subgraph information compensation, approximate the full training / inference results to the greatest extent possible. This would enable universal support for scenarios such as massive natural / satellite images, large-scale graph structure data, and city-level population and sensor data. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a random subgraph sampling method and system for machine or deep learning training and inference; The technical problem to be solved by this invention is: when the computing power / memory of the device is insufficient to process the full input or output, how to divide the unprocessable original data into a series of processable subgraphs / subsamples through random subgraph sampling and budget adaptive control, and control the sampling bias and cross-subgraph information loss, so that the model can stably complete training and / or inference, and as close as possible to the effect of training / inference with full data.

[0006] This invention enables the model to complete training and inference on a limited device even when the input (sample) or output (label) scale exceeds the computing power or memory capacity of the user device.

[0007] Terminology Explanation: 1. Community detection: Techniques for identifying closely structured subsets (i.e., "communities") or those with specific relationships in graph-structured data; 2. Spectral Clustering: An algorithm that represents data in matrix form and performs data clustering based on matrix eigenvalue decomposition (spectral analysis); 3. Spatial grid division: This method divides continuous map / image data (here, multi-dimensional data is represented as space) into a grid with spatial structure, using elements as vertices and edges as connections, and then splits it into discrete grid units according to regular shapes (such as rectangles and hexagons).

[0008] 4. Space-filling curve partitioning: A partitioning technique that uses a continuous and non-intersecting curve to traverse all grid cells in space to achieve a spatial dimension to one-dimensional sequence mapping.

[0009] The technical solution of this invention is as follows: A random subgraph sampling method for machine or deep learning training and inference includes: Step 1: Budget Assessment: A budget assessment is performed by obtaining equipment capability parameters, including the available video memory / main memory budget. Computing power budget and / or delay threshold ; Step 2: Scale Calculation: Based on the model structure and optimizer cost, calculate the maximum number of nodes that can be processed under the budget in Step 1. and the maximum edge limit and the largest tile ; Step 3: Select at least one sampling rule from the preset sampling rule library to generate a rule that satisfies ( The sampling rule base includes any or a combination of random walk sampling based on the center point, random block partitioning sampling, random vertex sampling and random edge sampling. The above sampling rules are repeatedly used to form a subgraph batch with the same number of nodes and the same shape, which can be used for machine or deep learning training. Step 4: For tasks whose output size exceeds the budget, perform label-coupled sampling corresponding to the subgraph, and obtain the corresponding subgraph label based on the index of the sampled subgraph batch; Step 5: Implement boundary preservation mechanisms and / or virtual neighbor aggregation for edges that are cut across the subgraph to compensate for cross-subgraph information; Step 6: During the training phase, apply importance-weighted and / or reweighted loss to the estimation bias introduced by non-uniform sampling; Step 7: Train and / or infer the machine / deep learning model based on the subgraph batch. During runtime, the subgraph size and sampling rules are adaptively adjusted according to actual usage and latency. Perform the corresponding tasks using machine / deep learning models.

[0010] According to a preferred embodiment of the present invention, step 2 includes the following specific implementation process: Step 2.1: Given the model structure and optimizer configuration, estimate the memory footprint of a single batch of subgraphs, including the number of nodes n and the number of edges m. As shown in equation (1): (1); Where d is the feature dimension of a single node. The number of bytes for each feature dimension. The average byte cost for intermediate states of each node. The average byte cost per edge, For model parameter usage, Additional usage for the optimizer; Step 2.2: Under constraints The upper bound of the maximum number of nodes that can be processed under constraints of computation, memory, and latency is then obtained. and the maximum edge limit As shown in equation (2): (2); in, Indicates that in making At its maximum, and The value; Secondly, for image data, it is estimated using equation (1). Where p is the image tile size and C is the number of image channels, find the largest pixel tile. As shown in equation (3): (3); in, It means that Maximum The possible values ​​of ; Step 2.3: To satisfy With / or For each batch of actual iteration time If linear or empirical model approximation is performed, And if the delay requirement is met, then ( Qualified; otherwise, reduce the size without violating memory constraints. To meet the budget or .

[0011] According to a preferred embodiment of the present invention, random walk sampling based on a center point includes: Multiple vertices are randomly selected from the graph data to form a set of centroids. A multi-level truncated random walk with restart is then performed starting from this set of centroids. For the probability of restarting, For the number of floors, For each layer step size, The number of the center point, The average degree of the entire map, This is the deduplication factor. The edge reduction coefficient is the node selected during its random walk. With edge Expected size Approximately Equation (4): (4); Furthermore, the walking depth and the step size of each layer satisfy equation (5): (5); in, Selected nodes for random walk With edge Memory usage For nodes With edge The actual iteration time for each batch.

[0012] According to a preferred embodiment of the present invention, random block partitioning sampling includes: First, the original graph structure data is divided into several blocks using community detection, spectral clustering, spatial gridding, or spatial filling curves. Then, several blocks are randomly selected and... Within constraints, the blocks are joined to form subgraphs; specifically, this includes: dividing the blocks into... The number of blocks selected each time is , The priority within the block is Define block edge density ,in, The number of edges inside the block. The number of nodes within the block, and the cross-boundary ratio. Extract several blocks from high to low and splice them together; if it exceeds ( (, prioritize from the lowest) The block edges are trimmed to obtain the desired result ( () constrains one or more random blocks.

[0013] According to a preferred embodiment of the present invention, random vertex sampling includes: Distribution by node importance Extract the vertex set and its first- or multi-order neighborhood to form a subgraph; Determined by node degree, betweenness, task relevance, or learning score; specifically including: The initial sampling ratio at the vertex. It is a closure order. To determine the distribution of node importance, we can learn equation (6) based on node attributes: (6); in, Let r(i) be the degree of node i, and r(i) be the arrival rate estimate. For task-related categories, , where is the weighting coefficient. If the closure is of order one, then at the maximum node-edge ( Under constraints, nodes With edge The expected size is written as equation (7): (7); Where |V| is the total number of nodes in the graph. This is the deduplication coefficient for closures. The edge reduction factor; If the closure is second-order, then replace it with a decay form. And decrease accordingly Vertex sampling is achieved.

[0014] According to a preferred embodiment of the present invention, random side sampling includes: Distribution by edge importance Extract a set of edges and perform closure on the two endpoints of the extracted edges to ensure connectivity; sample random edges and let... The initial mining ratio is as follows: The importance distribution of edges is defined by equation (8): (8); in, For edge weights or similarity, For the sake of difficulty, For time decay kernel, As weight; according to Weighting Some adjacent edges and Some adjacent edges, ensure and It has more connections in the subgraph; if or If a component already belongs to a connected component during the sampling process, then it is accessed through the edge. The two components are merged to form a connected subgraph.

[0015] According to a preferred embodiment of the present invention, step 4 includes the following specific implementation process: Step 4.1: When the output data size of the machine or deep learning model corresponding to the original graph structure or image input data is too large, i.e., the output data size exceeds the memory and computing power of the current device, perform label coupling sampling on the current subgraph, retaining only the subset of labels associated with the subgraph nodes / edges / time steps; specifically, this includes defining a mapping operator from the subgraph to the label index. For a node task, let the set of vertices in the subgraph corresponding to the label be... ,in For the set of vertices of the subgraph, For the full label tensor, and the edge task, let the set of edges in the subgraph corresponding to the label be . ,in For the set of edges in the subgraph; for multi-step / multi-task, Select only with or Related task / time step index; Step 4.2: If it is a multi-task or multi-step prediction, select the corresponding label according to the task / step corresponding to the sampled sub-graph.

[0016] According to a preferred embodiment of the present invention, step 5 includes the following specific implementation process: Step 5.1: Cut the edges between the captured subgraph and its outer edge. Selectively including a small subset of cross-boundary adjacencies based on priority is called boundary preservation; specifically, it includes: assuming the probability of preserving a certain edge is... Then define the edge. Priority score for: (9); in, and Both are priority scores. The edge set after partitioning. It is a temperature constant; In video memory budget Constraints by Priority Score Sampling is performed from top to bottom; if the budget is tight, the number of edges to be retained is adaptively reduced based on the retention probability to meet memory resource constraints. Defined as: (10); in, For the actual use in this batch, This is a scaling factor; adding boundary edges will cause the number of edges to exceed the limit. Then according to the retention probability Eliminate from low to high until the budget is met; Step 5.2: For cross-edge effects that were not preserved, introduce a virtual neighbor aggregation approximation; specifically, this includes: re-aggregating the original nodes of node i. Decomposed into the visible part within the sub-image and the cut-off invisible part. Only for To approximate, see below: (11); (12); (13); in, As vertices All the neighbors, Visible neighbors within the subgraph; On behalf of the neighbors, For edge weights / attention; Is with vertex The boundary mean of the same block, To pre-maintain several global prototype embedding sets, For similarity function, For the sake of attention to the prototype; An adaptive trade-off factor; For scale compensation; ultimately used Alternative This compensates for cross-subgraph information without significantly increasing overhead and injects virtual messages into the message aggregation of the corresponding nodes.

[0017] According to a preferred embodiment of the present invention, step 6 includes the following specific implementation process: Step 6.1: Importance weighting is used to restore the deviation subplot sampling to an unbiased estimate of the full risk; Let the first The probability of a vertex being included in the current batch is: Define weights ; Determined by the sampling strategy: uniform vertex sampling without replacement indivual, , The total number of vertices; distributed by degree with probability. Perform replacement sampling Second-rate, ; When m edges are sampled uniformly, for edge e, we have E represents the total number of edges; when the label is coupled with the structure, the edge-level target is used. Node-level targets Normalization and clipping are employed. Keep the desired batch size constant, or scale it down. ,in Temperature coefficient; Step 6.2: Define the weighted loss As shown in equation (14): (14); in, For the current sub-chart, For mission losses, For parameters Machine / deep learning models, Features of the subgraph Labels for subgraph features; Step 6.3: For the same node or edge that appears repeatedly in different subgraph batches, let its representation in batch a and batch b be respectively... and Add consistency regular expressions As shown in equation (15): (15); in, , is the regularization coefficient. It is a set of repeating subgraphs across different batches.

[0018] According to a preferred embodiment of the present invention, step 7 is specifically implemented as follows: Step 7.1: Using sub-graph batches as the smallest scheduling unit, execute forward / reverse or forward only; Step 7.2: During inference, fuse the predictions for repeated subgraphs across different batches; assuming the predictions for the same node in different batches are... Define the fusion operator As shown in equation (16): (16); in, It can be a weighted average or a lightweight learning-gated aggregator to combine large-scale predicted labels to obtain the final result; Step 7.3: In multi-device scenarios, batch parallel scheduling of different subgraphs and synchronization under a parameter server or decentralized strategy; in single-device scenarios, sequential or threaded pipelined execution is adopted to ensure synchronization with... compatible.

[0019] A further preferred embodiment of step 7.3 includes the following specific implementation process: In multi-device scenarios, different subgraph batches are distributed to multiple computing devices for independent forward and backward computation in a data-parallel manner. A parameter server or decentralized synchronization strategy is used to aggregate gradients and update global parameters, ensuring consistency across devices after synchronization. Let the... The computing device at time The processed subgraph batch is ᵢ(t), and the local gradient is Then the global gradient Represented as: (17); in, For the collection of equipment participating in the training, For the number of devices; The model parameters are updated after aggregation. ,in, for Model parameters at time 10:00 A function that performs backpropagation on the model to update the parameters; The broadcast is then sent back to each computing device to continue the next round, thereby enabling parallel training on a large-scale graph while ensuring a semantically consistent global model. In a single-device scenario, subgraph batches are scheduled according to budget order, that is, 𝒢(1), 𝒢(2), ... are processed sequentially in a lightweight threaded pipeline manner, and the current computing load and latency are dynamically detected to ensure that they meet the computing power limit. and latency limit If the estimated computational cost of a batch meets the memory, computation, and latency limits, the batch is allowed to execute; otherwise, the subgraph size is automatically reduced and the cost is re-estimated.

[0020] In a further preferred embodiment, in step 7, the machine / deep learning model is a Transformer, recurrent, convolutional, or graph neural network, and the task refers to a dense prediction task, including image generation, segmentation, and editing; graph node classification and regression.

[0021] According to a preferred embodiment of the present invention, for long sequence or raster data, a generalized graph is constructed that treats time steps as nodes and connects edges according to temporal adjacency or similarity, or the variables × time blocks are used to form a two-dimensional grid for re-graphing; specifically including: sequence In, each time step Except for time adjacency In addition, connect edges based on similarity. , border rights Two-dimensional variable × time or space raster in units For each node, edges are formed using 4 / 8 neighborhoods or based on a similarity threshold; the maximum processing capacity is calculated under budget constraints. The maximum processing capacity will be increased. The mapping is a generalized graph structure. For edge-level tasks, random edge sampling is preferred; for node-level tasks, random walks or vertex sampling are preferred; for graph-level tasks, random block partitioning is preferred, and the block-level representations are pooled during the fusion stage. Execute steps 1 to 7 to sample random sub-maps of long sequences or raster data.

[0022] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement steps of a random subgraph sampling method for machine or deep learning training and inference.

[0023] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a random subgraph sampling method for machine or deep learning training and inference.

[0024] A random subgraph sampling system for machine or deep learning training and inference, comprising: The budget assessment module is configured to perform a budget assessment by acquiring device capability parameters, including available video memory / memory budget. Computing power budget and / or delay threshold ; The scaling module is configured to: based on the model structure and optimizer cost, solve for the maximum upper bound of nodes that can be processed under the budget in step 1. and the maximum edge limit and the largest tile ; The subgraph batch generation module is configured to: select at least one sampling rule from a preset sampling rule library to generate a subgraph that satisfies ( The sampling rule base includes any or a combination of random walk sampling based on the center point, random block partitioning sampling, random vertex sampling and random edge sampling. The above sampling rules are repeatedly used to form a subgraph batch with the same number of nodes and the same shape, which can be used for machine or deep learning training. The subgraph label acquisition module is configured to: perform label coupling sampling corresponding to the subgraph for tasks whose output size exceeds the budget, and obtain the corresponding subgraph label based on the index of the sampled subgraph batch; The cross-subgraph information compensation module is configured to: implement a boundary preservation mechanism and / or virtual neighbor aggregation for edges that are cut across the subgraph to compensate for cross-subgraph information; The training and execution module is configured to: during the training phase, apply importance-weighted and / or reweighted loss to the estimation bias introduced by non-uniform sampling; train and / or infer the machine / deep learning model based on the subgraph batch; adaptively adjust the subgraph size and sampling rules according to actual usage and latency during runtime; and execute corresponding tasks through the machine / deep learning model.

[0025] The beneficial effects of this invention are as follows: 1. Budget Adaptive: Without replacing the hardware, data that was originally untrainable / inference can be run on limited devices in the form of subgraph batches, avoiding memory overflow issues.

[0026] 2. Controllable accuracy: By preserving boundaries, weighting importance, and regularizing consistency, the bias and variance introduced by sampling are significantly reduced, achieving results close to those of full training.

[0027] 3. High versatility: Applicable to convolutional, graph neural networks, Transformer, and scenarios that map sequence / grid data to generalized graphs; supports node-level, edge-level, and graph-level tasks.

[0028] 4. Project Deployable: Runtime monitoring and dynamic scaling, interruptibility and streaming support, single / multi-device scheduling, facilitating deployment at the edge and in cloud-edge collaboration.

[0029] 5. Reproducibility: Reproducibility is ensured through stratified random seeds and sampling logs. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the overall process of a random subgraph sampling method for machine or deep learning training and inference according to the present invention. Figure 2 This is a schematic diagram illustrating the training and testing process of a random subgraph sampling method for machine or deep learning training and inference according to the present invention. Detailed Implementation

[0031] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0032] Example 1 A random subgraph sampling method for machine or deep learning training and inference, such as Figure 1 As shown, it includes: Step 1: Budget Assessment: A budget assessment is performed by obtaining equipment capability parameters, including the available video memory / main memory budget. (Unit: bytes) and computing power budget (e.g., maximum acceptable iteration time per batch, in milliseconds) and / or delay threshold (Unit: milliseconds); Step 2: Scale Calculation: Based on the model structure and optimizer cost, calculate the maximum number of nodes that can be processed under the budget in Step 1. and the maximum edge limit and the largest tile ; Step 3: Select at least one sampling rule from the preset sampling rule library to generate a rule that satisfies ( The sampling rule base includes any or a combination of random walk sampling based on the center point, random block partitioning sampling, random vertex sampling and random edge sampling. The above sampling rules are repeatedly used to form a subgraph batch with the same number of nodes and the same shape, which can be used for machine or deep learning training. Step 4: For tasks whose output (label) size exceeds the budget, perform label coupling sampling corresponding to the subgraph, and obtain the corresponding subgraph label based on the index of the sampled subgraph batch; Step 5: Implement boundary preservation mechanisms and / or virtual neighbor aggregation for edges that are cut across the subgraph to compensate for cross-subgraph information; Step 6: During the training phase, apply importance-weighted and / or reweighted loss to the estimation bias introduced by non-uniform sampling; Step 7: Train and / or infer the machine / deep learning model based on the subgraph batch. During runtime, the subgraph size and sampling rules are adaptively adjusted according to actual usage and latency. Perform the corresponding tasks using machine / deep learning models.

[0033] Example 2 The difference between the random subgraph sampling method for machine or deep learning training and inference described in Example 1 and the method described in Example 1 is as follows: The specific implementation process of step 2 includes: Step 2.1: Given the model structure and optimizer configuration, estimate the memory footprint of a single batch of subgraphs, including the number of nodes n and the number of edges m. As shown in equation (1): (1); Where d is the feature dimension of a single node. The number of bytes for each feature dimension (e.g., 4 bytes for 32-bit floating-point). The average byte cost for each node's intermediate states (such as activations, gradients, and caches). The average byte cost per edge (including index, edge features, etc.) For model parameter usage, Additional usage for the optimizer; Step 2.2: Under constraints The upper bound of the maximum number of nodes that can be processed under constraints of computation, memory, and latency is then obtained. and the maximum edge limit As shown in equation (2): (2); in, Indicates that in making At its maximum, and The value; Secondly, for image data, it is estimated using equation (1). Where p is the image tile size and C is the number of image channels, find the largest pixel tile. As shown in equation (3): (3); in, It means that The value of p when it is at its maximum; Step 2.3: To satisfy With / or For each batch of actual iteration time If linear or empirical model approximation is performed, And if the delay requirement is met, then ( Qualified; otherwise, reduce the size without violating memory constraints. To meet the budget or .

[0034] Center-based random walk sampling includes: Multiple vertices are randomly selected from the graph data to form a set of centroids. A multi-level truncated random walk with restart is then performed starting from this set of centroids. For the probability of restarting, For the number of floors, For each layer step size, The number of the center point, The average degree of the entire map, This is the deduplication factor (the average uniqueness rate when accessing the same level or across levels repeatedly). The edge reduction coefficient (considering deduplication and orientation) is the node selected during its random walk. With edge Expected size Approximately Equation (4): (4); Furthermore, the walking depth and the step size of each layer satisfy equation (5): (5); in, Selected nodes for random walk With edge Memory usage For nodes With edge The actual iteration time for each batch.

[0035] Random block partitioning sampling; including: First, the original graph structure data is divided into several blocks using community detection, spectral clustering, spatial gridding, or spatial filling curves. Then, several blocks are randomly selected and... Within constraints, the blocks are joined to form subgraphs; specifically, this includes: dividing the blocks into... The number of blocks selected each time is , The priority within the block is Define the edge density of block B. ,in, The number of edges inside the block. The number of nodes within the block, and the cross-boundary ratio. Extract several blocks from high to low and splice them together; if it exceeds ( (, prioritize from the lowest) The block edges are trimmed to obtain the desired result ( () constrains one or more random blocks.

[0036] Random vertex sampling; including: Distribution by node importance Extract the vertex set and its first- or multi-order neighborhood to form a subgraph; Determined by node degree, betweenness, task relevance, or learning score; specifically including: The initial sampling ratio at the vertex. It is a closure order. The distribution of node importance can be learned from node attributes, such as node degree, order, and task relevance, to obtain equation (6): (6); in, For nodes The degree, r(i) is the arrival rate estimate. For task-related categories, , where is the weighting coefficient. If the closure is of order one, then at the maximum node-edge ( Under constraints, nodes With edge The expected size is written as equation (7): (7); in, This represents the total number of nodes in the graph. This is the deduplication coefficient for closures. The edge reduction factor; If the closure is second-order, then replace it with a decay form. And decrease accordingly Vertex sampling is achieved.

[0037] Random side sampling; including: Distribution by edge importance Extract a set of edges and perform closure on the two endpoints of the extracted edges to ensure connectivity; sample (close) random edges and let The initial mining ratio is as follows: The importance distribution of edges is defined by equation (8): (8); in, For edge weights or similarity, For the sake of difficulty, For time decay kernel, As weight; according to Weighting Some adjacent edges (excluding) (outside) and Ensure that some adjacent edges (excluding e) are protected and It has more connections in the subgraph; if or If a component already belongs to a connected component during the sampling process, then it is accessed through the edge. The two components are merged to form a connected subgraph.

[0038] The specific implementation process of step 4 includes: Step 4.1: When the output data size of the machine or deep learning model corresponding to the original graph structure or image input data is too large, i.e., the output data size exceeds the memory and computing power of the current device, perform label coupling sampling on the current subgraph, retaining only the subset of labels associated with the subgraph nodes / edges / time steps; specifically, this includes defining a mapping operator from the subgraph to the label index. For a node task, let the set of vertices in the subgraph corresponding to the label be... ,in For the set of vertices of the subgraph, For the full label tensor, and the edge task, let the set of edges in the subgraph corresponding to the label be . ,in For the set of edges in the subgraph; for multi-step / multi-task, Select only with or Related task / time step index; Step 4.2: For multi-task or multi-step prediction, select the corresponding label based on the task / step size of the sampled subgraph. This reduces label resident memory and maintains input-output consistency.

[0039] The specific implementation process of step 5 includes: Step 5.1: To reduce information fragmentation caused by subgraph segmentation, cut the edges between the captured subgraph and its outer edges. Selectively including a small subset of cross-boundary adjacencies based on priority is called boundary preservation; specifically, it includes: assuming the probability of preserving a certain edge is... Then define the edge. Priority score for: (9); in, and Both are priority scores. The edge set after partitioning. It is a temperature constant; In video memory budget Constraints by Priority Score Sampling is performed from top to bottom; if the budget is tight, the number of edges to be retained is adaptively reduced based on the retention probability to meet memory resource constraints. Defined as: (10); in, For the actual use in this batch, This is a scaling factor; adding boundary edges will cause the number of edges to exceed the limit. Then according to the retention probability Eliminate from low to high until the budget is met; Step 5.2: For cross-edge effects that were not preserved, introduce a virtual neighbor convergence approximation; specifically, this includes: […]. The original aggregation ( (The set of neighbors of node i) is split into the visible part within the subgraph and the cut-off invisible part. Only for To approximate, see below: (11); (12); (13); in, As vertices All the neighbors, Visible neighbors within the subgraph; On behalf of the neighbors, For edge weights / attention; Is with vertex The boundary mean of the same block, To pre-maintain several global prototype embedding sets, For similarity function, For the sake of attention to the prototype; As an adaptive trade-off factor, it can be taken as follows: ; For scale compensation, the total impact of approximate missing edges is taken as follows: ( (where i is the average edge weight of vertex i); finally, use Alternative This mechanism recovers cross-subgraph information without significantly increasing overhead and injects virtual messages into the message aggregation of the corresponding nodes. It compensates for the insufficient representation caused by cross-subgraph fragmentation without introducing additional real node / edge storage.

[0040] like Figure 2 As shown, the specific implementation process of step 6 includes: Step 6.1: To address the estimation bias caused by non-uniform sampling, assign importance weights to training samples or structural units within the subgraph; importance weighting is used to restore the biased subgraph sampling to an unbiased estimate of the full risk. Let the first The probability of a vertex being included in the current batch is: Define weights ; Determined by the sampling strategy: uniform vertex sampling without replacement indivual, V is the total number of vertices; distributed by degree with probability Perform replacement sampling Second-rate, ; When sampling uniformly When there are edges, for the edges ,have E represents the total number of edges; when the label is coupled with the structure, the edge-level target is used. Node-level targets Furthermore, to suppress variance and numerical instability, normalization and pruning are employed: Keep the desired batch size constant, or scale it down. ,in Temperature coefficient; The above weighting also applies to non-uniform sampling caused by uneven positive and negative sample distribution and cross-block boundary retention, maintaining statistical consistency between training / inference and full learning under budget constraints.

[0041] Step 6.2: Define the weighted loss As shown in equation (14): (14); in, For the current sub-chart, For mission losses, For parameters Machine / deep learning models, Features of the subgraph Labels for subgraph features; Step 6.3: For the same node or edge that appears repeatedly in different subgraph batches (edges that are cut across subgraphs), let it be in the batch With criticism The representations in are respectively and Add consistency regular expressions As shown in equation (15): (15); in, , is the regularization coefficient. A set of repeated subplots across different batches to reduce variance and drift.

[0042] The specific implementation process of step 7 includes: Step 7.1: Using subgraph batches as the smallest scheduling unit, perform forward / backward (during training) or forward only (during inference); Step 7.2: During inference, fuse the predictions for repeated subgraphs across different batches; assuming the predictions for the same node in different batches are... Define the fusion operator As shown in equation (16): (16); in, It can be a weighted average or a lightweight learning-gated aggregator to combine large-scale predicted labels to obtain the final result; Step 7.3: In multi-device scenarios, batch parallel scheduling of different subgraphs and synchronization under a parameter server or decentralized strategy; in single-device scenarios, sequential or threaded pipelined execution is adopted to ensure synchronization with... compatible.

[0043] For multi-device scenarios, the original graph structure or image input is divided into multiple sub-graph batches based on the number of devices. Parallel scheduling allocates one sub-graph batch to each device for machine / deep learning model training. Simultaneously, model parameter updates employ a decentralized strategy, meaning devices directly communicate and exchange parameters. For single-device scenarios, model parameters are passed sequentially according to the dependency order of the subgraphs (without thread switching overhead), or threaded pipelined execution is used. This involves using a lightweight thread pool and lock-free queue to achieve pipelined parallelism of the subgraphs, allowing the execution parts of different subgraphs to overlap, improving single-device utilization while reducing latency.

[0044] The specific implementation process of step 7.3 includes: In multi-device scenarios, different subgraph batches are distributed to multiple computing devices (such as multiple GPUs or multiple edge nodes) for independent forward and backward computation in a data-parallel manner. A parameter server or decentralized synchronization strategy is used to aggregate gradients and update global parameters, ensuring consistency across devices after synchronization. Let the... The computing device at time The processed subgraph batch is ᵢ(t), and the local gradient is Then the global gradient Represented as: (17); in, For the collection of equipment participating in the training, For the number of devices; The model parameters are updated after aggregation. ,in, for Model parameters at time 10:00 A function that performs backpropagation on the model to update the parameters; The broadcast is then sent back to each computing device to continue the next round, thereby enabling parallel training on a large-scale graph while ensuring a semantically consistent global model. In a single-device scenario, subgraph batches are scheduled according to budget order, that is, 𝒢(1), 𝒢(2), ... are processed sequentially in a lightweight threaded pipeline manner, and the current computing load and latency are dynamically detected to ensure that they meet the computing power limit. and latency limit When the estimated computational cost of a batch meets memory, computation, and latency constraints, the batch is allowed to execute; otherwise, the subgraph size is automatically reduced and the cost is re-estimated. This avoids triggering overload. Through the above-described parallel / sequential scheduling strategies, this invention can achieve high throughput in a multi-device environment and run stably on a resource-constrained single device without sacrificing availability.

[0045] In step 7, the machine / deep learning model is Transformer, recurrent, convolutional and graph neural networks, etc., and the task refers to intensive (high-dimensional) prediction task, including image generation, segmentation and editing; graph node classification and regression, etc.

[0046] For long sequences or raster data, this method is applied to construct generalized graphs, treating time steps as nodes and connecting them by temporal adjacency or similarity, or by constructing a two-dimensional grid from variables and time blocks for re-graphification; specifically including: sequence In, each time step Except for time adjacency In addition, connect edges based on similarity. , border rights Two-dimensional variable × time or space raster in units For each node, edges are defined using 4 / 8 neighborhoods (pixels) or based on a similarity threshold; the maximum processable limit is calculated under budget constraints. The maximum processing capacity will be increased. The mapping is a generalized graph structure. For edge-level tasks (such as connection prediction), random edge sampling is preferred; for node-level tasks, random walks or vertex sampling are preferred; for graph-level tasks, random block partitioning is preferred, and the block-level representations are pooled during the fusion stage. Execute steps 1 to 7 to sample random sub-maps of long sequences or raster data.

[0047] Example 3 The difference between the random subgraph sampling method for machine or deep learning training and inference described in Example 2 and the method described in Example 2 is as follows: This embodiment focuses on adaptive subgraph training for super-resolution image scenes; Task: Remote sensing building segmentation; Data: A subset of urban areas from a publicly available remote sensing segmentation dataset (several ultra-large orthophotos, with the long side ≥ 10,000 pixels) was selected and divided into training / validation sections at an 8:2 ratio; Model: DeepLabv3+ (ResNet-50 backbone, output stride of 8); Loss = Cross-entropy + Boundary Weights; Metrics: mIoU (%), F1 (%); video memory usage (GB); single-step latency (ms, average of the first 100 warm-up steps after 1000 steps); Equipment: Restricted equipment: single RTX 3060 8GB, CUDA 12.x; Reference equipment (for comparison of the upper limit of "full training" only): single RTX 3090 24GB.

[0048] Optimizer: AdamW, learning rate 3e-4, batch size adaptively determined by budget; 40 training steps; mixed precision off (FP32) for clear contrast over memory.

[0049] Sampling rules: R2 (random blocks / tiles) + R3 (random tile importance sampling); Tile size and overlap: Corresponding to 512×512, overlapping ; Tag coupling: Only crop the tag clippings corresponding to the tiles; Importance weighting: ; Budget Adaptive: Online Monitoring Dynamically adjust batch size and number of candidate blocks.

[0050] Table 1 is a data comparison table between this embodiment and baseline A (full training, upper limit) and baseline B (full training, 8GB); Table 1

[0051] Example 4 The difference between the random subgraph sampling method for machine or deep learning training and inference described in Example 2 and the method described in Example 2 is as follows: This embodiment is a budget-adaptive subgraph training method based on random walks; Task: Node multi-classification (testing accuracy); Data: Publicly available product co-occurrence graph (scale similar to the OGB product graph benchmark); Model: 3-layer graph neural network (hidden=256, ReLU, Dropout 0.5); Optimizer: Adam (learning rate 1e-3, weight decay 5e-4), early stopping (tolerance for epochs 10); Sampling / Batch Size: Full training (Baseline-NS): 20000 nodes, batch size 1; This invention's random subgraph sampling (RSS): random walk + random blocks, automatically satisfying ( ); q=0.2; batch size 32, importance-weighted and consistency regularization ; Hardware and accuracy specifications: Limited device is a single RTX 3060 8GB card, FP32; full reference is A100 80GB (for reference only); Measurement methods: peak memory (GB), training latency (s / epoch), inference latency (s / 100,000 nodes).

[0052] Table 2 is a data comparison table between this embodiment and Full-batch GCN; Table 2

[0053] Example 5 The difference between the random subgraph sampling method for machine or deep learning training and inference described in Example 2 and the method described in Example 2 is as follows: This embodiment is a long sequence prediction for an industrial wind turbine network; Task: 96-step prediction (regression) of multi-unit, multivariate long series data; Indicator: MAPE (%) / RMSE; Data: Actual wind farm turbine-sensor network (nodes are turbines or turbine-sensor pairs, edges are electrical / physical neighbors and power correlation). Model: Graph augmentation time series model (hidden dimension 256, window length 288); Optimizer: Adam (learning rate 1e-3, weight decay 5e-4), cosine annealing, early stopping (tolerance for 10 rounds); Sampling / Batch Size: Full Training (Baseline-NS): Sequence blocks are 96, batch size is 1; This invention uses random subgraph sampling (RSS): random blocks + random edges, label-coupled sampling; Batch size 32, importance-weighted and consistency regularization ; Hardware and precision caliber: Limited device: RTX 3060 8GB; Full reference: A100 80GB (full sequence × full image); Measurement methods: peak memory (GB), training latency (s / epoch), online inference latency (ms / unit·96 steps).

[0054] Table 3 is a data comparison table between this embodiment and full training; Table 3

[0055] In Tables 1-3, 1) all results are the mean ± standard deviation of three independent reproducible experiments; 2) the backbone model, optimizer, and training rounds are consistent for each method on the limited device; a unified early stopping strategy is used; 3) inference adopts the "whole graph slicing + result fusion" process; 4) RSS budget adaptation is achieved by monitoring memory / latency. ) The sampling rules are dynamically adjusted; 5) The above values ​​are for example in the patent specification and can be finely adjusted according to the equipment and data scale during actual deployment.

[0056] On a device with limited 8GB of VRAM, this invention can stably complete training, and while significantly optimizing VRAM usage (down by approximately 67%) and latency (down by approximately 31%), it maintains accuracy close to the level of "full training (upper limit)" (mIoU difference <1pt). In the public commodity graph node classification task, on an RTX 2070S 8GB environment, the method of this invention, without changing the hardware, controls the peak VRAM to below 6GB, approaching 79.1% of full training (hardware: A100 80GB, used for processing the entire graph) (see Table 1), but with significantly reduced hardware requirements. In the industrial wind turbine 96-step sequence prediction task, this invention achieves almost equivalent MAPE (8.0% vs 7.9%) on a 12GB VRAM device through label-coupled sampling, while significantly reducing training and inference latency (see Table 2), demonstrating its effectiveness and engineering feasibility in computing / memory-constrained scenarios.

[0057] Example 6 A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of a random subgraph sampling method for machine or deep learning training and inference as described in any of Examples 1-5.

[0058] Example 7 A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a random subgraph sampling method for machine or deep learning training and inference as described in any of Examples 1-5.

[0059] Example 8 A random subgraph sampling system for machine or deep learning training and inference, comprising: The budget assessment module is configured to perform a budget assessment by acquiring device capability parameters, including available video memory / memory budget. (Unit: bytes) and computing power budget (e.g., maximum acceptable iteration time per batch, in milliseconds) and / or delay threshold (Unit: milliseconds); The scaling module is configured to: based on the model structure and optimizer cost, solve for the maximum upper bound of nodes that can be processed under the budget in step 1. and the maximum edge limit and the largest tile ; The subgraph batch generation module is configured to: select at least one sampling rule from a preset sampling rule library to generate a subgraph that satisfies ( The sampling rule base includes any or a combination of random walk sampling based on the center point, random block partitioning sampling, random vertex sampling and random edge sampling. The above sampling rules are repeatedly used to form a subgraph batch with the same number of nodes and the same shape, which can be used for machine or deep learning training. The subgraph label acquisition module is configured to: for tasks whose output (label) size exceeds the budget, perform label coupling sampling corresponding to the subgraph, and obtain the corresponding subgraph label based on the index of the sampled subgraph batch; The cross-subgraph information compensation module is configured to: implement a boundary preservation mechanism and / or virtual neighbor aggregation for edges that are cut across the subgraph to compensate for cross-subgraph information; The training and execution module is configured to: during the training phase, apply importance-weighted and / or reweighted loss to the estimation bias introduced by non-uniform sampling; train and / or infer the machine / deep learning model based on the subgraph batch; adaptively adjust the subgraph size and sampling rules according to actual usage and latency during runtime; and execute corresponding tasks through the machine / deep learning model.

Claims

1. A random subgraph sampling method for machine or deep learning training and inference, characterized in that, Comprising: Step 1: Budget evaluation: budget evaluation by fetching device capability parameters, including available GPU / memory budget With computational power budget And / or latency threshold ; Step 2: Scale solution: Solve for the maximum node limit that can be handled under the budget of Step 1, depending on the model structure and optimizer overhead with a maximum edge limit and a maximum tile ; Step 3: Select at least one sampling rule from the preset sampling rule library to generate a rule that satisfies ( The sampling rule base includes any or a combination of random walk sampling based on the center point, random block partitioning sampling, random vertex sampling and random edge sampling. The above sampling rules are repeatedly used to form a subgraph batch with the same number of nodes and the same shape, which can be used for machine or deep learning training. Step 4: For tasks whose output size exceeds the budget, perform label-coupled sampling corresponding to the subgraph, and according to the index of the sampled subgraph batch, obtain the corresponding subgraph label; Step 5: Implement a boundary preservation mechanism and / or virtual neighbor aggregation for edges that are cut across subgraphs to compensate for cross-subgraph information; Step 6: In the training phase, importance-weight and / or re-weight the loss to compensate for the estimation bias introduced by non-uniform sampling; Step 7: Train and / or infer the machine / deep learning model based on the subgraph batch, and adaptively adjust the subgraph size and sampling rules according to the actual occupancy and latency during runtime; Perform the corresponding task through the machine / deep learning model.

2. The random subgraph sampling method for machine or deep learning training and inference according to claim 1, wherein, The specific implementation process of step 2 includes: Step 2.1: Estimate the memory size of a single batch subgraph including the number of nodes n and the number of edges m under the given model structure and the optimizer configuration As shown in equation (1): （1）； where d is the dimension of single node feature, is the number of bytes for each feature dimension, is the average byte overhead for each node intermediate state, is the average byte overhead for each edge, is the model parameter footprint, is the optimizer extra footprint; Step 2.2: In the constraint Next, the maximum node upper bound that can be processed under the computation, memory, and latency overhead constraints is obtained With the maximum edge upper bound As shown in equation (2): （2）； wherein, represents at the time of making maximum, and the value; Second, for the image data, the maximum pixel tile is estimated by equation (1) where is the image tile size, is the number of image channels, as shown in equation (3): （3）； wherein means that the value of p at which the maximum occurs; Step 2.3: To satisfy and / or , the actual iteration time for each batch is approximated by a linear or empirical model, if and the delay requirement is satisfied, then is accepted; otherwise, decrease without violating the memory constraint, to get that satisfies the budget, or .

3. The random subgraph sampling method for machine or deep learning training and inference of claim 1, wherein, Random walk sampling based on the center point, including: Multiple vertices are randomly selected from the graph data to form a set of centroids. A multi-level truncated random walk with restart is then performed starting from this set of centroids. For the probability of restarting, For the number of floors, For each layer step size, The number of the center point, The average degree of the entire map, This is the deduplication factor. The edge reduction coefficient is the node selected during its random walk. With edge Expected size Approximately Equation (4): （4）； And the walk depth and the step length of each layer satisfy formula (5): （5）； wherein, selected nodes for random walk with edges memory footprint, nodes with edges actual iteration time per batch; Further preferably, random block partition sampling; including: First, the original graph structure data is subjected to community discovery, spectral clustering, spatial grid or space-filling curve division, and the entire large graph is divided into several blocks, then several blocks are randomly selected and spliced into a subgraph within the constraint ; specifically comprising: setting the block division as , the number of blocks selected each time is , , the priority within the block is ; defining the edge density of block B , wherein is the number of edges within the block, is the number of nodes within the block, and the cross-border ratio is ; extracting several blocks from high to low and splicing; if it exceeds , the edges of the block with the lowest are preferentially cropped to obtain one or more random blocks that meet the constraint ; Further preferably, random vertex sampling; including: Distribution of node importance Extracting a set of vertices and their first or multi-order neighborhood to form a subgraph; Determined by node degree, betweenness, task relevance or learning score; Specifically including: let The initial sampling ratio of the vertex is The closure order is The distribution of node importance can be obtained by learning formula (6) according to node attributes: （6）； in, Let r(i) be the degree of node i, and r(i) be the arrival rate estimate. For task-related categories, , where is the weighting coefficient. If the closure is of order one, then at the maximum node-edge ( Under constraints, nodes With edge The expected size is written as equation (7): （7）； Wherein, |V| is the number of nodes in the graph, is the closure deduplication coefficient, is the edge conversion coefficient; If the closure is second order, replace with a decaying form and correspondingly reduce , implement vertex sampling; Further preferably, random edge sampling; including: importance distribution extracting the edge set and closing the two endpoints of the extracted edge to ensure connectivity; sampling the random edge, let the initial sampling proportion of the edge is the edge importance distribution is defined as formula (8): （8）； wherein, is an edge weight or similarity, is a hard example degree, is a temporal decay kernel, is a weight; the weights are incorporated into the partial edges of and to ensure that and have more connections in the subgraph; if or has already belonged to some connected component during the sampling process, then the two components are merged by the edge to form a connected subgraph.

4. The random subgraph sampling method for machine or deep learning training and inference of claim 1, wherein, The specific implementation process of step 4 includes: Step 4.1: When the output data scale of the original graph structure or image input data corresponding to the machine or deep learning model is too large, that is, the output data scale exceeds the memory and computing power range of the current device, perform label coupling sampling on the current subgraph, and only retain the label subset associated with the subgraph node / edge / time step; Specifically, define a mapping operator from the subgraph to the label index For node tasks, let the label corresponding subgraph vertex set be , where is the subgraph vertex set, and Y is the full label tensor. For edge tasks, let the label corresponding subgraph edge set be , where is the subgraph edge set. For multi-step / multi-task, only select the task / time step index related to Step 4.2: If it is a multi-task or multi-step prediction, select the corresponding label according to the task / step corresponding to the sampled subgraph; Further preferably, the specific implementation process of step 5 includes: Step 5.1: Cut the edge between the pair of subgraphs and its outside A small fraction of the cross-boundary edges are selectively included according to their priority, a process called border preservation. Specifically, the probability of preserving an edge is defined as , where the priority score of the edge is defined as: ​ (9); wherein, with and are priority scores, is the set of edges after partitioning, is a temperature constant; Under memory budget Prior score under constraints Top-down sampling; if budget is tight, adaptively reduce the number of edges reserved according to the reserved probability to meet the memory resource limit, and adaptively reserve the probability is defined as: (10)； wherein, is the actual occupancy of the batch, is a scaling factor, if adding the boundary edge results in a number of edges exceeding then the edge is removed with a reservation probability from low to high until the budget is met; Step 5.2: Introduce virtual neighbor aggregation approximation for the effect of unreserved cross-edges; specifically, break the original aggregation of nodes into visible part within the subgraph and the cut-off invisible part and only approximate as follows: Step 5.2: Introduce virtual neighbor aggregation approximation for the effect of unreserved cross-edges; specifically, break the original aggregation of nodes into visible part within the subgraph and the cut-off invisible part and only approximate as follows: (11)； (12)； (13)； where, is the set of all neighbors of vertex i, is the set of visible neighbors within the subgraph; is the representation of the neighbors, is the edge weight / attention; is the average of the boundary values of the same block as vertex i, is a set of global prototype embeddings maintained beforehand, is a similarity function, is the attention to the prototype; is an adaptive weighting factor; is a scale compensation; finally, replaces compensates for cross-subgraph information without significantly increasing the overhead and injects virtual messages into the message aggregation of the corresponding nodes.

5. The random subgraph sampling method for machine or deep learning training and inference of claim 1, wherein, The specific implementation process of step 6 includes: Step 6.1: Importance weighting is used to restore the biased subgraph sampling to an unbiased estimate of the full risk; Let the i-th vertex be included in the current batch with probability , define the weight ; Decided by the sampling strategy: uniform without replacement vertex sampling k, , V is the total number of vertices; According to the degree distribution, sample k times with replacement with probability , ; When m edges are uniformly sampled, for edge e, there is , E is the total number of edges; when labels are coupled with structure, edge-level objectives are used , node-level objectives are used ; normalization and clipping are applied ; keeping the desired batch size constant, or scaling , where is the temperature coefficient; Step 6.2: Define Weighted Loss As shown in equation (14): （14）； wherein B is the current subgraph batch, is the task loss, is the parameter of the machine / deep learning model, is the subgraph feature, is the label of the subgraph feature; Step 6.3: For a same node or edge that appears repeatedly in different subgraph batches, let its representation in batch a and batch b be denoted as and , add a consistency regularizer as shown in equation (15): （15）； wherein, is a regular coefficient, is a set of repeated subgraphs across different batches.

6. The random subgraph sampling method for machine or deep learning training and inference of claim 1, wherein, The specific implementation process of step 7 includes: Step 7.1: Take the subgraph batch as the minimum scheduling unit to perform forward / backward or only forward; Step 7.2: At inference time, result fusion is performed on the predictions of the repeated subgraphs across different batches; let the prediction of the same node in different batches be , define the fusion operator as in equation (16): （16）； wherein, The weighted average or light learning type gated aggregator can be used to combine the large-scale predicted labels to obtain the final result. Step 7.3: In multi-device scenarios, different subgraphs are batched and scheduled in parallel and synchronized under parameter servers or decentralized strategies; in single-device scenarios, sequential or threaded pipelining is adopted to ensure compatibility with compatibility; Further preferably, the specific implementation process of step 7.3 includes: In a multi-device scenario, different sub-graphs are allocated to multiple computing devices for independent forward and backward calculation in a data parallel manner, and the parameter server or decentralized synchronization strategy is used to aggregate the gradient and update the global parameters, so that each device remains consistent after synchronization; let the sub-graph batch processed by the i-th computing device at time t be G i(t), and the local gradient be g i(t), then the global gradient is represented as: (17)； Wherein, P is a set of devices participating in training, and |P| is the number of devices; updating the model parameters obtained after the aggregation , wherein is the model parameter at time t, is a function that the model performs back propagation to realize parameter update; Broadcast back to each computing device to continue the next round, thereby realizing parallel training on large-scale graphs while ensuring a semantically consistent global model; In single-device scenarios, subgraph batches are scheduled in budget order, i.e., sequentially in a lightweight thread-per-task manner, and dynamic checks are performed to determine whether the current computation load and latency meet the upper limit on computing power and the upper limit on latency When the computation cost of a batch is expected to meet the memory, computation, and latency constraints, the batch is allowed to execute; if not, the subgraph size is automatically reduced and the cost is re-estimated. Further preferably, in step 7, the machine / deep learning model is a Transformer, a loop, a convolution, and a graph neural network, and the task refers to an intensive prediction task, including image generation, segmentation, and editing; graph node classification and regression.

7. The random subgraph sampling method for machine or deep learning training and inference of any of claims 1-6, wherein, For long sequences or grid data, construct a generalized graph by regarding time steps as nodes, connecting edges according to time adjacency or similarity, or constructing a two-dimensional grid from variable-time blocks and then graphing; Specifically including: Sequence In each time step , except for time adjacency , similarity is connected , and the edge weight ; two-dimensional variable-time or spatial grid with cells as nodes and 4 / 8 neighborhoods or similarity threshold-based edges; obtain the upper limit of processable under budget constraints , map the upper limit of processable to a generalized graph structure, for edge-level tasks, prefer to use random edge sampling; for node-level tasks, prefer to use random walk or vertex sampling; for graph-level tasks, prefer to use random block division, and in the fusion stage, pool the block-level representation; Perform steps 1 to 7 to achieve random subgraph sampling for long sequences or grid data.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the random subgraph sampling method for machine or deep learning training and inference according to any one of claims 1-7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the random subgraph sampling method for machine or deep learning training and inference according to any one of claims 1-7.

10. A random subgraph sampling system for machine or deep learning training and inference, comprising: Comprising: a budget evaluation module configured to perform a budget evaluation by obtaining a device capability parameter, the device capability parameter comprising an available GPU / memory budget with a computing power budget and / or a latency threshold ; a scale solving module configured to solve a maximum node upper limit processable under the budget of step 1 according to the model structure and the optimizer overhead with a maximum edge upper limit and a maximum tile ​ The subgraph batch generation module is configured to: select at least one sampling rule from a preset sampling rule library to generate a subgraph that satisfies ( The sampling rule base includes any or a combination of random walk sampling based on the center point, random block partitioning sampling, random vertex sampling and random edge sampling. The above sampling rules are repeatedly used to form a subgraph batch with the same number of nodes and the same shape, which can be used for machine or deep learning training. A subgraph label acquisition module configured to: for tasks whose output size exceeds the budget, perform label-coupled sampling corresponding to the subgraph, and according to the index of the sampled subgraph batch, obtain the corresponding subgraph label; A cross-subgraph information compensation module configured to: implement a boundary preservation mechanism and / or virtual neighbor aggregation for edges that are cut across subgraphs to compensate for cross-subgraph information; A cross-subgraph information compensation module configured to: implement a boundary preservation mechanism and / or virtual neighbor aggregation for edges that are cut across subgraphs to compensate for cross-subgraph information; The training and execution module is configured to: in a training stage, importance-weight and / or re-weight a loss introduced by the non-uniform sampling; train and / or infer a machine / deep learning model based on the subgraph batch, and perform budget self-adaptive adjustment on a subgraph size and a sampling rule according to actual occupancy and latency in runtime; and execute a corresponding task through the machine / deep learning model.

Images