A long-text computing power optimized layered brain-like large model basic framework

Abstract

The application discloses a long-text computing power optimized layered brain-like large model basic framework, which is compatible with offline pre-training and online inference. The existing scheme can only alleviate the computing power waste and logic deviation of training and inference scenes respectively, and the two problems are homologous to the full-amount interaction data flow of the traditional model without layered verification. The application adopts a global-local dual-isolation layered framework: a top global scheduling module (1) allocates computing power; a layered screening scheduling module (2) filters redundant representations through block compression (2-1), token selection (2-2) and sliding window (2-3); a plurality of local subnets (3) operate in parallel through local links (6), and the refined abstract is uploaded through a global link (5); and a bottom feedback verification module (4) backtracks and corrects logical conflicts. In the pre-training stage, redundant corpus is intercepted to reduce invalid gradient calculation, and in the inference stage, low-correlation representation is deleted to suppress logical illusion, and the same set of scheduling logic is used in the two scenes. The application is compatible with GPU and ASIC computing power hardware, and is suitable for million-word long-text business in the fields of law, finance, medical treatment and the like.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing, multimodal deep neural networks, and distributed computing power scheduling. Specifically, it relates to a hierarchical brain-like large model infrastructure optimized for long text computing power and its supporting general hierarchical scheduling method. This is used for the transformation of the underlying interaction topology of dense Transformer and MoE sparse large models to build a brain-like large model system. It is compatible with multimodal inputs such as text, image, audio, and video, and supports general computing power chips such as NVIDIA, Ascend, and Cambricon. It can be adapted to all business scenarios of offline batch pre-training and online long text inference for dense, MoE, and multimodal large models. Background Technology

[0002] Current mainstream large-scale models such as Dense Transformer, MoE, Mamba, and MLA / NSA all suffer from three common underlying shortcomings when handling massive training corpora, million-word long documents, and multi-image / text tasks, whether in offline batch pre-training or online long text inference scenarios:

[0003] First, there is no top-level predictive scheduling unit, resulting in a one-size-fits-all approach to computing power allocation. During offline training, all batches of data are processed uniformly, and during online inference, all contexts of varying lengths undergo pairwise interactions with all tokens. In both scenarios, the computational load increases exponentially with sequence length. Conventional MoE expert routes are fixed and cannot dynamically open or close computation subnets based on the size of the training set or the volume of inference text. MLA and NSA only optimize caching and lack pre-layered filtering capabilities. Even small training sets and short question-and-answer sessions will continuously occupy GPU memory without load, while extremely long samples are prone to GPU memory overflow. At the same time, global and local data streams share caches and computing cores, leading to severe resource contention in batch training and high-concurrency inference scenarios, resulting in computational latency and drastic training fluctuations.

[0004] Second, existing sparse solutions only offer single-layer coarse-grained text filtering, lacking layered storage and computation mechanisms. They can only merge large chunks of text, failing to distinguish core, effective information, temporally related content, and worthless redundant background information. Batch training corpora and all features of long inference contexts are stored in a mixed manner, requiring a complete recalculation of all content for each training iteration and every inference request. They cannot extract lightweight summaries for separate caching, and therefore cannot perform domain-specific parallel training or business-specific parallel inference, continuously generating a large amount of repetitive and ineffective computational overhead.

[0005] Third, the data flow is unidirectional and linear, lacking segmented verification and autonomous iterative closed-loop processing. Inconsistencies in numerical values ​​and terms during online inference cannot be intercepted in real time; in the offline pre-training stage, conflicting samples directly participate in gradient calculations, continuously causing model fitting bias. In both scenarios, logical errors necessitate a complete rerun of the entire computational chain, and training biases and inference illusions can only be addressed through manual screening and verification. High-quality and anomalous samples generated from training batches and inference sessions cannot be automatically re-processed; optimization can only be achieved through complete base retraining, resulting in extremely high iterative optimization costs.

[0006] The above three types of defects coexist in both training and inference scenarios, but manifest differently: in the pre-training stage, redundant corpora participate in gradient calculations, leading to wasted computing power and fitting bias; in the inference stage, invalid representations participate in full interaction, causing computing power inflation and logical illusions. Both types of defects stem from the same underlying architectural problem—traditional models use global full-scale token interaction, lacking pre-layered filtering and segmented verification mechanisms. In existing technologies, data cleaning schemes for training scenarios and prompt word optimization schemes for inference scenarios are independent, failing to form a unified computing power scheduling system.

[0007] The human brain naturally possesses a hierarchical logic in information processing, including task prediction, hierarchical filtering, partitioned parallel processing, and result review. Existing large-scale models, whether in offline pre-training or online inference processes, do not utilize this hierarchical mechanism. Addressing common technical shortcomings in both training and inference scenarios, this invention draws inspiration from this hierarchical logic to design a data flow scheduling architecture. This architecture allows redundant interception logic in the training phase and invalid culling logic in the inference phase to share the same filtering, verification, and caching mechanism. This reduces computational overhead and suppresses fitting bias in the training scenario, while reducing computational costs and suppressing logical illusions in the inference scenario. Summary of the Invention

[0008] To address the shortcomings of existing dense Transformer, MoE, Mamba, and time-series models, which only perform single-layer lightweight optimization, lack a standardized hierarchical scheduling system, have no isolation of hardware and software resources, have high computational overhead for long texts, are prone to logical distortion in long text inference, and have high model iteration costs, this invention provides a hierarchical brain-like large model infrastructure with optimized computational overhead for long texts and a supporting general hierarchical scheduling method.

[0009] The "brain-like" concept used in this invention is not a biomimetic model based on biological neuronal impulses, synaptic plasticity, or physiological cortical structures, but rather a mapping at the functional logic level. Specifically, it borrows the hierarchical logic of the human brain in information processing—task prediction, hierarchical filtering, partitioned parallel processing, and result review—and transforms it into a technical implementation scheme consisting of a hardware scheduling module, transmission links, and hierarchical caching. Implementing this scheme requires no biomimetic modifications to the general-purpose Transformer or MoE base. The human brain's physiological cortex and neurons are not essential technical features for realizing this invention. The table below is for functional analogy only and does not constitute any limitation on the scope of protection.

[0010] A table showing the correspondence between the four-layer architecture and the functions of physiological units in the human brain.

[0011] First layer Top-level global scheduling module (1) dorsolateral prefrontal cortex Predict the task complexity and issue global computing power and task scheduling instructions. Second floor Hierarchical filtering and scheduling module (2) thalamus Filtering redundant and low-association semantic representation modules Third layer Local domain scheduling module cluster (3) hippocampus / visual / language cortex, etc. Parallel execution of subdivided inference and training computations across different domains. Fourth floor Feedback verification scheduling module (4) Review and Reflect on Brain Regions Segmented verification logic, training gradient conflicts, and iteration of lightweight sample parameters. Basic carrier Semantic Representation Module Fragmented memory neurons Carrying original feature fragments of text, images, audio, and video Global Path Overall coordination and control of the entire process (5) Transbrain neural pathways Top-level instructions are issued, summaries of simplified data from each subnet are compiled, and transmission anomaly indicators are set. Local pathways Local autonomous interaction link (6) Internal synaptic pathways of the partition Only features are exchanged within a single subnet; raw features are not transmitted globally.

[0012] The “global-local dual-isolation hierarchical data flow topology” described in this invention refers to a hierarchical data flow architecture consisting of at least two hierarchical scheduling modules. It uses two mutually isolated transmission links, namely the global overall control link (5) and the local autonomous interaction link (6), to respectively undertake the transmission of global scheduling instructions and simplified summaries, as well as the local computational interaction of the original semantic representation within each subnet. The computational cache hardware corresponding to the two types of links is physically partitioned or logically resource-exclusively isolated. The original complete semantic representation only flows within the local autonomous interaction link (6) and is not transmitted across links to the global network to participate in full-scale interaction.

[0013] This invention protects a standardized multi-layered hardware and software skeleton, without limiting the base algorithm, modality, or computing chip. Any solution that only replaces the internal computing logic and hardware carrier but retains the core architecture of multi-layered, globally and locally isolated data flow is considered an equivalent technical solution of this invention. Relying on layered filtering, partitioned parallelism, conflict backtracking, and lightweight adaptation parameter iteration mechanisms, it can reduce the computing power overhead in offline pre-training and online inference scenarios, reduce the workload of manual review, and adapt to various commercial businesses such as text and multimodal applications.

[0014] The entire architecture consists of three core modules: multiple semantic representation modules, a multi-layer hierarchical scheduling module, and two sets of isolated transmission links. The semantic representation module is compatible with fragmented input features of text, images, and audio / video. The four-layer hierarchical scheduling module is only one implementation structure of this invention.

[0015] The "number of layers" mentioned in this invention specifically refers to the number of deployment layers of the hierarchical scheduling module. Multi-type semantic representation modules, as basic data carrying units, are not included in the layer count. Under the premise of keeping the core architecture of "isolation between the global overall control link (5) and the local autonomous interaction link (6), and the original semantic representation limited to local flow" unchanged, the number of scheduling layers can be merged (e.g., merging the local cluster and feedback verification into the same execution unit) or decomposed and refined (e.g., decomposing the screening layer into coarse screening and fine screening layers). The resulting two-layer, three-layer or more layer variants are all equivalent to this invention because they constitute substantially the same technical means, achieve substantially the same function, and achieve substantially the same effect. They fall within the protection scope of the claims of this invention.

[0016] The hierarchical filtering logic of the hierarchical screening scheduling module (2), the conflict backtracking logic of the feedback verification scheduling module (4), and the hierarchical storage logic of the three-level hierarchical cache module (7) described in this invention are implemented based on the same data flow scheduling architecture in both the pre-training and inference scenarios.

[0017] In the pre-training scenario, the hierarchical screening and scheduling module (2) removes redundant and contradictory representations before the corpus enters the gradient calculation, reducing the amount of invalid gradient calculation; the feedback verification and scheduling module (4) verifies sample conflicts in segments within the training batch, and abnormal data is returned for re-screening to avoid the continuous propagation of contradictory gradients; the three-level hierarchical caching module (7) distinguishes between storing valid and abnormal samples, supports lightweight adaptation parameter fine-tuning, and can iteratively optimize the screening rules without the need for complete base retraining.

[0018] In the reasoning scenario, the hierarchical filtering and scheduling module (2) removes low-association representations before long texts enter the global attention calculation, reducing the scale of global interaction; the feedback verification and scheduling module (4) verifies the consistency of reasoning logic in segments and intercepts contradictory outputs across paragraphs and documents; the three-level hierarchical caching module (7) stores KV features in a hierarchical manner according to access frequency, reducing memory usage and reasoning latency.

[0019] Each of the above filtering branches performs judgments based on preset repetition rate thresholds, semantic relevance thresholds, and time sequence window parameters, eliminating redundant, low-relevance, and invalid semantic representations.

[0020] The filtering rules, verification logic, and caching strategies in the two scenarios mentioned above are implemented based on the same set of scheduling logic. The abnormal sample features accumulated during the training phase can be fine-tuned by lightweight adaptation parameters to synchronously optimize the filtering threshold during the inference phase, thereby enabling the cross-scenario reuse of optimization experience.

[0021] The computing and caching hardware corresponding to the global overall control link (5) and the local autonomous interaction link (6) are completely partitioned and isolated, or the logical resources are isolated exclusively through software. This can be achieved through NVIDIA, Ascend chip firmware, scheduling interface or operating system memory management mechanism: the global overall control link (5) allocates independent computing cores and high-speed caches, and each subnet of the local autonomous interaction link (6) allocates independent sharded caches. There is no read / write preemption conflict between the two types of hardware or logical resources. The extended supporting modules include a three-layer filtering branch, a three-level hierarchical caching module (7), an external tool access interface (8), and a task complexity scoring module (9), all of which are described in the dependent claims.

[0022] From the perspective of architectural theory, the computing power consumption variation pattern is analyzed: In the traditional dense MoE model, all tokens interact in pairs, and the theoretical computational complexity is O(N²). This invention uses a hierarchical screening and scheduling module (2) to remove low-association redundant semantic representations in advance. The original features of each subnet only rely on local autonomous interaction links (6) for local interaction, and only simplified summaries participate in the top-level global operation. At the hardware level, global and local computing resources are partitioned and isolated, eliminating the additional computing power loss caused by resource contention. Here, k is the number of summary representations retained after hierarchical screening and participating in the top-level global interaction, and the value is much smaller than the total number of original text tokens. Under typical deployment conditions, the global interaction computation scale converges from the traditional O(N²) to O(kN). This computing power optimization effect can be achieved in offline pre-training batch gradient calculation and online long text inference.

[0023] The complete four-layer architecture described below is only one implementation structure of this invention; this four-layer architecture has four essential constraints that cannot be omitted:

[0024] First, there is no top-level global scheduling module (1): it is impossible to predict the complexity of tasks in advance and dynamically allocate computing power, and the effect of hierarchical optimization is completely lost;

[0025] Second, the non-hierarchical screening and scheduling module (2): all text representations participate in global O(N²) interaction, and the computing power overhead returns to the level of traditional models;

[0026] Third, there is no global overall control link (5) and local autonomous interaction link (6) isolation mechanism: global and local data streams preempt cache and computing resources, resulting in severe latency fluctuations in high-concurrency training and inference scenarios;

[0027] Fourth, the lack of a hierarchical semantic splitting mechanism makes it impossible to achieve parallel training and parallel computation of multiple subnet partitions, thus preventing the implementation of the entire hierarchical architecture.

[0028] The hierarchical scheduling method of this invention fully corresponds to all steps of claim 11: S1 Constructing multiple semantic representation modules to uniformly carry various multimodal original features; S2 Building a core hierarchical architecture of a top-level global scheduling module (1) and a hierarchical filtering scheduling module (2); S3 The top-level global scheduling module (1) issues global computing power and task scheduling instructions; S4 The hierarchical filtering scheduling module (2) filters low-association redundant semantic representations; S5 Dividing two sets of transmission links, namely a global overall control link (5) and a local autonomous interaction link (6), to realize hierarchical scheduling; S6 Deploying multiple groups of local domain scheduling module clusters (3) in parallel, with only the summary being aggregated upwards; S7 Relying on the feedback verification scheduling module (4), completing logical conflict backtracking and lightweight adaptation parameter iteration.

[0029] Compared with existing technologies, this invention has several technical advantages:

[0030] First, pre-training computational cost optimization and bias suppression. Redundant data is filtered out through pre-level hierarchical screening, reducing the number of invalid representations involved in gradient calculation; conflicting samples are intercepted in real-time through feedback verification, preventing the continuous propagation of contradictory gradients; and lightweight adaptation parameters are fine-tuned, allowing iterative optimization of the screening rules without requiring full base retraining. These mechanisms collectively reduce pre-training computational costs and suppress the accumulation of fitting bias.

[0031] Second, inference computational power optimization and illusion suppression. This involves reducing the scale of global attention interactions by pre-selecting and filtering low-relevance representations; intercepting contradictory outputs across paragraphs and documents by verifying logical consistency through feedback checks; and reducing memory usage and inference latency through a three-level hierarchical cache for tiered storage of key-value features. These mechanisms collectively reduce inference computational costs and suppress logical illusions.

[0032] Third, the training optimization results continue to compound in the inference scenario. The three core mechanisms of this invention—layered filtering, conflict backtracking, and hierarchical caching—are implemented based on the same data flow scheduling architecture in both training and inference scenarios. During the training phase, through iterative analysis of massive corpora, the optimal filtering threshold, verification rules, and caching strategy parameters have been converged. When the same scheduling logic is used in the inference phase, the above training results are directly inherited, achieving higher interception efficiency and lower computational consumption without re-tuning. Specifically, the optimal filtering threshold converged during the training phase is directly used for invalid representation reduction in the inference phase; the abnormal sample features accumulated during the training phase are used to optimize the filtering rules in the inference phase; and the optimal cache migration strategy determined during the training phase is directly used for memory management in the inference phase. These mechanisms enable the actual operating cost of the inference phase to receive additional pressure reduction with the support of the optimization results in the training phase, forming a continuous compounding effect of "training optimization, inference benefit." The more thorough the training and the more comprehensive the corpus coverage, the more accurate the scheduling strategy inherited in the inference phase, and the more significant the computational saving effect. The two are not simply superimposed; rather, the same architecture generates independent benefits in each of the two stages, and then the inference stage inherits all the optimization results from the training stage to maximize the total benefit.

[0033] Fourth, adaptive computing resource allocation. The top-level global scheduling module has a built-in task complexity scoring module that distinguishes between simple and deep derivation tasks, lightweighting idle redundant computing subnets in lightweight scenarios and reducing idle hardware power consumption.

[0034] Fifth, two-way software and hardware adaptation. ASIC and computing chip manufacturers can develop dedicated training and push hardware based on hardware partitioning and isolation architecture; cloud platforms can deploy only hierarchical scheduling software logic, and both implementation paths can be implemented independently.

[0035] Sixth, multi-modal and multi-industry adaptation. Text, images, audio, and video can all be encapsulated as semantic representation modules, and adding corresponding subnets can adapt to vertical businesses such as law, finance, healthcare, and code.

[0036] Those skilled in the art should understand that the task complexity scoring method of the top-level global scheduling module (1), the filtering branch type and number of the hierarchical filtering scheduling module (2), the cache hierarchy division and data migration strategy of the three-level hierarchical cache module (7) described in this invention are all illustrative examples. Any alternative means that can achieve essentially the same function and effect (including but not limited to different complexity quantification algorithms, different filtering branch combinations, and different cache hierarchy division methods) and can be conceived without creative effort are all equivalent technical solutions of this invention. Attached Figure Description

[0037] The attached diagram shows the complete correspondence between modules and steps: 1: Top-level global scheduling module; 2: Hierarchical filtering scheduling module; 2-1: Global block compression branch; 2-2: Fine-grained token selection branch; 2-3: Short-term sliding window branch; 3: Local domain scheduling module cluster, including three types of subdivided operation units: industry domain subnet, logic verification subnet, and general summarization subnet; 4: Feedback verification scheduling module; 5: Global overall management and control link; 6: Local autonomous interaction link; 7: Three-level hierarchical caching module; 7-1: Long-term global high-speed cache; 7-2: Mid-level active KV cache; 7-3: Low-speed redundant hibernation storage; 8: External tool access interface; 9: Task complexity scoring module; T1: Global summary temporary storage step; T2: Subnet private KV cache read and write step; T3: Subnet redundant text hibernation storage step.

[0038] Figure 1 The four-layer global-local dual-layer topology diagram shows the top-level global scheduling module (1), the hierarchical filtering scheduling module (2), the local domain scheduling module cluster (3), and the feedback verification scheduling module (4) hardware topology from top to bottom. The top-level global scheduling module (1) is configured with an external tool access interface (8) and a task complexity scoring module (9). The right side is the three-level hierarchical caching module (7). The thick connecting line represents the global overall control link (5), and the thin solid line represents the local autonomous interaction link (6). This is the attached figure in the abstract of this application.

[0039] Figure 2 Four-layer hierarchical unit linkage diagram: From top to bottom, the top-level global scheduling module (1) along the global overall control link (5) sends the degree instruction, the hierarchical screening scheduling module (2) filters redundant representations, the local domain scheduling module cluster (3) performs parallel operations, and the feedback verification scheduling module (4) summarizes the complete training and inference bidirectional data flow process.

[0040] Figure 3 Global and local transmission link hardware partition isolation architecture: hardware partitions or logical resource partitions are divided by global overall control link (5) and local autonomous interaction link (6). The long-term global high-speed cache (7-1) on the left executes step T1; the middle-level active KV cache (7-2) on the right executes step T2, and the low-speed redundant hibernation storage (7-3) executes step T3. The two types of storage hardware circuits or memory resources are physically / logically isolated, and there is no cross-partition direct read and write channel.

[0041] Figure 4Schematic diagram of lightweight iterative pipeline data flow for feedback verification: The top-level global scheduling module (1) diverts training and inference samples to the three-level hierarchical cache module (7) through the global overall management and control link (5). High-quality samples execute the T1 global summary temporary storage step, and abnormal samples execute the T2 subnet private KV cache read and write step. The samples are aggregated to the feedback verification scheduling module (4) to complete the lightweight adaptation parameter fine-tuning and form a general self-optimizing closed loop for training and inference. Detailed Implementation

[0042] General scheduling hardware and operating logic: As shown in Figure 1, the complete four-layer topology, the entire system consists of a top-level global scheduling module (1), a hierarchical filtering scheduling module (2), a local domain scheduling module cluster (3), a feedback verification scheduling module (4), a global overall control link (5), a local autonomous interaction link (6), and a three-level hierarchical cache module (7), forming a complete hierarchical data flow architecture; Figure 2 shows the hierarchical linkage data flow, Figure 3 shows the cache hardware partition isolation structure, and Figure 4 shows the lightweight iterative closed-loop pipeline; T1: temporary access step of global summary, T2: read and write step of subnet private KV cache, T3: hibernation storage step of subnet redundant text. The following embodiments distinguish the training and inference differentiated deployment logic, and the general hierarchical flow process will not be repeated.

[0043] Implementation 1: Integrated Hardware Deployment (Ascend 910 + 67B MoE for Million-Contract Inference)

[0044] As shown in Figures 1 and 3, this embodiment fully deploys a three-level hierarchical cache module (7), dividing it into a long-term global high-speed cache (7-1), a mid-level active KV cache (7-2), and a low-speed redundant hibernation storage (7-3); it also fully deploys a four-layer scheduling unit, with global overall control of the link (5) and local autonomous interaction link (6). Hardware resources are completely partitioned and isolated, and there is no conflict of computing power or video memory preemption.

[0045] The semantic representation module of the unified encapsulation of the million-unit project contract text is sent to the top-level global scheduling module (1). The task complexity scoring module (9) determines it to be an ultra-high complexity reasoning task and simultaneously activates all three filtering branches of the hierarchical screening scheduling module (2). In this embodiment, the task complexity scoring module (9) adopts the following quantification rules: the input text volume is graded according to the number of tokens (≤4K ​​is low complexity, 4K-128K is medium complexity, >128K is high complexity), the semantic association degree is calculated by cross-paragraph co-occurrence word density (threshold is set to 0.3), and the estimated reasoning steps are preset according to the task type (1-3 steps for single document summary, 5-10 steps for multi-document comparison). The activation thresholds for the three filtering branches are as follows: the global block compression branch (2-1) removes paragraphs with a repetition rate > 60%; the fine-grained token selection branch (2-2) retains the top 30% of tokens by TF-IDF value; and the short-term sliding window branch (2-3) sets the window size to 2048 tokens. The filtered features are then distributed to the legal domain subnet and the price inference subnet for parallel computation.

[0046] Each subnet's original representation relies solely on local autonomous interaction links (6) for local caching and interaction, and only a simplified summary is uploaded to the top-level global scheduling module (1) along the global overall management and control link (5). After a subnet detects numerical or clause contradictions, an anomaly identifier is generated and fed back to the top-level global scheduling module (1) along the global overall management and control link (5). The screening threshold of the hierarchical screening scheduling module (2) is then derived a second time. After inference is completed, the sample is sent to the three-level hierarchical caching module (7) to distinguish between high-quality and abnormal samples, and to complete the lightweight adaptation parameter fine-tuning (LoRA is used in this embodiment, with rank r=8 and α=16). The scheduling rules can be updated without complete base retraining. In this embodiment, the capacity of the three-level hierarchical caching module (7) is allocated in a ratio of 8:4:1.

[0047] Technical benefits: Relying on a four-layer hierarchical scheduling unit, the scale of global interactive data is reduced, cross-clause logical conflicts are automatically verified, and it is adapted to online review and reasoning scenarios of million-word commercial contracts.

[0048] Example 2: General-purpose GPU pure software hierarchical scheduling (H10 cluster online question answering inference)

[0049] As shown in Figures 1 and 4, this embodiment achieves global overall management and control link (5) and local autonomous interaction link (6) through software multi-process and independent memory pool, and realizes logical resource exclusive isolation (corresponding to "logical resource exclusive isolation" in claim 1); the top-level global scheduling module (1) configures the external tool access interface (8), and automatically calls the retrieval and numerical calculation module when information is insufficient; the hierarchical screening scheduling module (2) filters redundant text and sends it to the general inductive subnet for lightweight calculation; the inference sample is diverted to the three-level hierarchical cache module (7) to complete the lightweight adaptation parameter fine-tuning and update the scheduling rules of the hierarchical scheduling module (2).

[0050] Technical benefits: It can reduce the computing power overhead of online long text inference by relying solely on software layered logic, without the need for customized dedicated ASIC chips, and is compatible with lightweight online Q&A and code parsing services on public clouds.

[0051] Example 3: Multi-document review and reasoning of million-word investment research financial reports

[0052] As shown in Figures 1 and 4, multiple financial reports and research reports are uniformly converted into semantic representation modules and sent to the top-level global scheduling module (1). The task complexity scoring module (9) scores and determines that it is a multi-document deep reasoning task. The hierarchical screening scheduling module (2) fully activates all three filtering branches: global block compression branch (2-1), fine-grained token selection branch (2-2), short-term sliding window branch (2-3), and local domain scheduling module cluster (3). Duplicate announcements are stored in low-speed redundant hibernation storage (7-3), core revenue summaries are stored in long-term global high-speed cache (7-1), and time-series data are stored in mid-level active KV cache (7-2). Cross-document numerical contradictions flow back to the top-level global scheduling module (1) along the global overall control link (5) for re-deriving. The sample summary iteratively optimizes the scheduling rules of the hierarchical screening scheduling module (2).

[0053] Technical benefits: Layered differentiation between redundant and core text reduces the computational load of global interaction across multiple documents, automatically reviews and corrects logical inference deviations, and reduces the workload of manual review and verification.

[0054] Example 4: Multimodal Medical Image and Text Record Reasoning

[0055] As shown in Figures 1 and 3, the images and test images are uniformly encoded into a multimodal semantic representation module and sent to the top-level global scheduling module (1) to configure a medical-specific domain subnet; the hierarchical screening scheduling module (2) filters blank images and invalid medical record templates through the global block compression branch (2-1), the fine-grained token selection branch (2-2), and the short-term sliding window branch (2-3), while retaining the core features of lesions and test values; when the top-level global scheduling module (1) determines that the medical record information is insufficient, it calls the medical retrieval tool through the external tool access interface (8), and the image and text numerical conflicts are deduced for the second time along the global overall control link (5); online consultation reasoning and batch medical record pre-training samples are uniformly sent to the three-level hierarchical cache module (7) to complete the lightweight adaptation parameter iteration.

[0056] Technical effect: The original features of the multimodal system only flow within the local autonomous interaction link (6) and do not participate in the large-scale interaction of the global overall control link (5). This can reduce the computing power overhead of mixed text and image reasoning, automatically verify cross-modal numerical contradictions, and adapt to the medical online consultation medical record reasoning scenario.

[0057] Example 5: Offline Batch Layered Pre-training of Massive Long Documents

[0058] As shown in Figures 1, 3, and 4, this embodiment fully utilizes a four-layer architecture consisting of a top-level global scheduling module (1), a hierarchical filtering scheduling module (2), a local domain scheduling module cluster (3), and a feedback verification scheduling module (4). It is adapted to the batch offline pre-training process of legal and financial long text datasets. The operating logic of the hierarchical filtering scheduling module (2), the feedback verification scheduling module (4), and the three-level hierarchical caching module (7) is based on the same scheduling logic as the aforementioned inference embodiment.

[0059] The batch of training corpora of varying lengths are uniformly encoded into semantic representation modules and sent to the top-level global scheduling module (1). The task complexity scoring module (9) outputs the computing power scheduling weights based on the total character length of a single batch of corpora and the cross-document correlation. For ultra-long corpora batches, three filtering branches are enabled: global block compression branch (2-1), fine-grained token selection branch (2-2), and short-term sliding window branch (2-3). For short sample batches, the global block compression branch (2-1) is disabled to reduce preprocessing overhead. The hierarchical screening scheduling module (2) enables the matching screening auxiliary loss function during the pre-training stage to automatically filter templates and meaningless repetitive training corpora, retaining only representations with effective factual information and sending them to the local domain scheduling module cluster (3) for parallel gradient calculation.

[0060] The original training corpus within each subnet completes gradient calculations only within the local autonomous interaction link (6), and only lightweight gradient summaries and abnormal sample identifiers are transmitted upwards along the global overall management link (5). The global overall management link (5) and the local autonomous interaction link (6) are completely isolated in hardware partitions, and there is no memory resource preemption conflict during large-scale parallel training. After the subnet detects factual and numerical conflict training samples, it generates an abnormal identifier and flows back to the top-level global scheduling module (1) along the global overall management link (5). The conflict text association weights are reduced, the hierarchical screening scheduling module (2) is controlled, and the effective corpus is re-screened before gradient iteration is performed again.

[0061] After each training batch is completed, the samples are stored in three types of three-level hierarchical cache modules (7): long-term global high-speed cache (7-1), mid-level active KV cache (7-2), and low-speed redundant dormant storage (7-3). They are then sent to the feedback verification scheduling module (4) to complete the lightweight adaptation parameter fine-tuning. Only the scheduling rules of the hierarchical screening scheduling module (2) and the top-level global scheduling module (1) are fine-tuned, without the need for complete base retraining. The abnormal sample features and screening thresholds accumulated in the pre-training stage are synchronized to the inference stage through the lightweight adaptation parameters, directly optimizing the filtering efficiency of the hierarchical screening scheduling module (2) and realizing the continuous compounding of "training optimization and inference benefit".

[0062] Technical benefits: The offline pre-training stage automatically filters massive redundant training data, greatly reducing the scale of global interactive computation; it automatically removes conflicting samples to avoid fitting bias; the training and push reuse the same hierarchical scheduling module (1)(2)(3)(4) architecture, eliminating the need to develop two separate pipelines, resulting in lower iteration computing power costs.

[0063] The above embodiments and accompanying drawings are only used to help understand the technical solutions of the present invention and are not intended to exhaustively limit the scope of protection. Equivalent substitutions and modifications made based on the core concept of the present invention all fall within the protection scope of the claims of the present invention.

Claims

1. A hierarchical large model infrastructure optimized for long text computing power, used for offline pre-training of dense Transformer or MoE sparse large models and online inference of long text, including a semantic representation module carrying multimodal information, computing power hardware and cache storage unit; Its features are, A global-local dual-isolation layered data flow topology is adopted, and at least two layers of layered scheduling modules, two sets of independent transmission links, and a three-level layered buffer module are configured (7); The multi-layered scheduling module includes at least a top-level global scheduling module (1) and a layered filtering scheduling module (2); the top-level global scheduling module (1) has a built-in task complexity scoring module (9) and an external tool access interface (8) for issuing computing power allocation and scheduling instructions, and calling external computing modules when input information is insufficient; the layered filtering scheduling module (2) includes at least a global block compression branch (2-1) and a fine-grained token selection branch (2-2) for eliminating low-associative semantic representations; The two transmission links are divided into a global overall control link (5) and a local autonomous interaction link (6). The corresponding computing cache hardware is physically partitioned or logically resource-exclusive isolated, with no read / write preemption conflict. The three-level hierarchical caching module (7) is divided into a long-term global high-speed cache (7-1), a mid-level active KV cache (7-2), and a low-speed redundant hibernation storage (7-3), and the data migration is dynamically managed by the top-level global scheduling module (1).

2. The hierarchical large-scale model infrastructure according to claim 1, characterized in that, The multi-layer hierarchical scheduling module also includes multiple groups of parallel local domain scheduling module clusters (3); each cluster only performs local operations through local autonomous interaction links (6), and only uploads the simplified summary along the global overall management and control link (5), while the complete original representation does not participate in the global full-volume interaction.

3. The hierarchical large-scale model infrastructure according to claim 1, characterized in that, The multi-layer hierarchical scheduling module also includes a feedback verification scheduling module (4), which is used to verify logical conflicts in segments and trigger the hierarchical filtering scheduling module (2) to re-filter based on abnormal data return.

4. The hierarchical large-scale model infrastructure according to claim 3, characterized in that, The feedback verification scheduling module (4) is equipped with a virtual and real iteration pipeline. Based on the output samples, it uses a lightweight adaptation parameter fine-tuning method to update the scheduling rules. The fine-tuning method includes, but is not limited to, LoRA, Adapter or prefix fine-tuning, without the need for complete base model retraining.

5. The hierarchical large-scale model infrastructure according to claim 1, characterized in that, The execution logic of the three filtering branches of the hierarchical filtering scheduling module (2) is as follows: the global block compression branch (2-1) removes text templates and redundant paragraphs; the fine-grained token selection branch (2-2) retains semantic representations with factual relevance and core processing value; and the short-term sliding window branch (2-3) maintains the temporal contextual coherence of long texts.

6. The hierarchical large-scale model infrastructure according to claim 1, characterized in that, The data flow rules of the three-level hierarchical cache module (7) are as follows: the long-term global high-speed cache (7-1) stores the simplified summary uploaded by each local domain scheduling module cluster (3); the middle-level active KV cache (7-2) stores the original multimodal features of each subnet's local operation; Low-speed redundant hibernation storage (7-3) stores the filtered redundant text representations.

7. The hierarchical large-scale model infrastructure according to claim 1, characterized in that, The task complexity scoring module (9) quantifies and scores the input text volume, semantic relevance, and estimated processing steps to generate differentiated computing power scheduling weights.

8. The hierarchical large-scale model infrastructure according to claim 7, characterized in that, The estimated number of processing steps includes the estimated number of reasoning steps required for online reasoning of long texts. The task complexity scoring module (9) presets the range of reasoning steps according to the type of reasoning task.

9. The hierarchical large-scale model infrastructure according to claim 7, characterized in that, The estimated number of processing steps includes an estimate of the total number of training iterations required for offline pre-training. The task complexity scoring module (9) presets the range of iteration steps based on the ratio of the total number of training set samples to the effective batch.

10. The hierarchical large-scale model infrastructure according to claim 1, characterized in that, The low-association semantic representation is determined by the association weight threshold output by the task complexity scoring module (9), and representations below the preset weight threshold are eliminated by the hierarchical screening and scheduling module (2).

11. A general hierarchical scheduling method for large-scale hierarchical models, running on GPU or ASIC general-purpose computing chips, used for offline pre-training of large models and online inference of long texts, characterized in that... Perform the following steps: S1. Construct a multi-class semantic representation module to encapsulate text tokens, image / audio / video multimodal features, and segmented processing sub-conclusions into independent information carriers; S2. Build at least two layers of hierarchical scheduling modules, including at least a top-level global scheduling module (1) and a hierarchical filtering scheduling module (2); the top-level global scheduling module (1) has a built-in task complexity scoring module (9) and an external tool access interface (8); S3. The task complexity scoring module (9) of the top-level global scheduling module (1) quantifies and scores the input task, generates differentiated computing power scheduling weights, coordinates the entire inference / training process, and issues global computing power allocation and task scheduling instructions. When the input information is insufficient to complete the inference or training task, the external computing module is automatically called through the external tool access interface (8). S4. Identify and filter low-association invalid semantic representations through the global block compression branch (2-1), fine-grained token selection branch (2-2), and short-time sliding window branch (2-3) of the hierarchical screening and scheduling module (2); S5. Divide the transmission links into two sets: the global overall control link (5) and the local autonomous interaction link (6). The global and local operation cache partitions are read and written independently to eliminate the competition for computing power and storage resources. S6. Build multiple parallel local domain scheduling module clusters (3), each cluster independently processes subdivided modalities and subdivided inference / training tasks; the complete original representation only flows locally in the local autonomous interaction link (6), and only the summarized summary is transmitted upward along the global overall control link (5); S7. Set up the feedback verification scheduling module (4) and the matching conflict backtracking process: segmented verification of the conflict between the reasoning logic and the training gradient, and abnormal data is returned to the hierarchical screening scheduling module (2) for re-filtering along the global overall control link (5); based on the distinguished valid and abnormal samples, perform lightweight adaptation parameter fine-tuning and update the hierarchical scheduling rules.

12. A computing power processing device, characterized in that, It includes a processor and a memory; the memory stores a computer program, which, when executed by the processor, implements the hierarchical large model general hierarchical scheduling method as described in claim 11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the general hierarchical scheduling method for hierarchical large models as described in claim 11.

Images