A controlled submission method for large language model multi-thread reasoning candidate library

Abstract

The application belongs to the technical field of artificial intelligence and natural language processing, and particularly relates to a controlled submission method for a large language model multi-thread reasoning candidate library, wherein an input problem to be solved is collected; a unified standard candidate record structure is built, a logical interaction layer and a physical compression layer are split to complete lightweight coding, and storage segments are shared by homologous equivalent reasoning contents; a globally heterogeneous candidate library suitable for multi-thread operation is built, four types of evaluation features are extracted from each reasoning content in the library, a submission strategy parameter is solved and permanently locked, a frozen parameter is used for scoring, semantically equivalent contents are merged, a multi-layer index is built when the candidate exceeds a threshold, and the optimal reasoning content is selected according to the feature boundary by cutting off low-score branches; and finally, the final answer, full-link computing power consumption data, and multiple sets of audit diagnosis indexes are output, and it is distinguished whether a fault occurs in a candidate generation link or an answer submission link. Thus, the problems of large storage and transmission overhead and long reasoning delay in the prior art are solved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing technology, specifically relating to a controlled submission method for a multi-threaded inference candidate library for large language models. Background Technology

[0002] Currently, mainstream solutions for large language model reasoning include three types of technologies: single-threaded thought chain direct answering, self-consistent majority voting, and validator-ranked search-based reasoning. These constitute the basic implementation system for generating multi-threaded reasoning candidate libraries and submitting answers. Single-threaded thought chaining, by decomposing step-by-step reasoning logic, guides the model to output a complete derivation trajectory, directly improving the accuracy of basic reasoning for complex mathematical and common-sense problems. Self-consistent voting constructs a candidate answer set through multi-path sampling and aggregates results based on answer frequency statistics, achieving enhanced stability of reasoning results at low cost, and is widely used in scenarios such as enhanced question answering and offline batch reasoning. Validator ranking and search-based reasoning introduce external scorers to perform quality screening on multiple reasoning branches, further mining highly reliable reasoning paths through thought tree expansion and path pruning, making them suitable for professional business scenarios with extremely high requirements for reasoning accuracy. These three solutions can be used individually or in combination to build multi-threaded reasoning candidate libraries, enabling multi-path answer generation, preliminary screening, and result output. They support basic business implementations such as low-latency online reasoning and batch evaluation, and are the common basic implementation methods for large-scale model reasoning services in the industry.

[0003] Existing technologies lack a unified standard candidate record structure, heterogeneous reasoning content lacks a standardized storage format, and cannot fully retain standardized answers, reasoning links, and various metadata. They also lack a layered logical interaction layer and physical compression layer for lightweight encoding, preventing equivalent reasoning content from sharing storage fragments and resulting in high storage and transmission overhead. Furthermore, they cannot build a global heterogeneous candidate library adapted to multi-threaded operation, cannot extract four comprehensive evaluation features (quality, consensus, source prior, and cost), lack a mechanism for solving and permanently locking submission strategy parameters based on a development dataset, and suffer from dynamic and unstable reasoning scoring strategies. They also lack processing for merging semantically equivalent content, and when the number of candidates exceeds a threshold, they lack multi-layered indexing and low-score branch pruning based on feature boundaries, requiring complete scoring of all candidates and resulting in high reasoning latency. Simultaneously, they cannot synchronously output end-to-end computing power overhead data and multiple sets of audit diagnostic indicators, cannot distinguish whether the fault originates from the candidate generation stage or the answer submission stage, and lack quantifiable and traceable end-to-end audit diagnostic capabilities. Summary of the Invention

[0004] This application provides a controlled submission method for a multi-threaded inference candidate library for large language models, in order to solve the problems of high storage and transmission overhead and high inference latency in the prior art.

[0005] The first aspect of this application provides a controlled submission method for a multi-threaded inference candidate library for large language models, comprising the following steps: collecting input problems to be solved; building a unified standard candidate record structure based on the input problems to be solved, splitting the logical interaction layer and the physical compression layer to complete lightweight encoding, sharing storage fragments for equivalent inference content from the same source, and completely retaining standardized answers, inference links, and various metadata; building a global heterogeneous candidate library adapted to multi-threaded operation based on the unified standard candidate records, extracting four evaluation features—quality, consensus, source prior, and cost—from each inference content in the library, solving and permanently locking submission strategy parameters based on the development dataset, using frozen parameters for scoring, merging semantically equivalent content, building a multi-layer index when candidates exceed a threshold, and pruning low-scoring branches based on feature boundaries to select the optimal inference content; outputting the final answer and full-link computing power overhead data based on the optimal inference content, and simultaneously outputting multiple sets of audit diagnostic indicators to distinguish whether the fault occurs in the candidate generation stage or the answer submission stage.

[0006] Preferably, a unified standard candidate record structure is constructed, including: constructing a general logical interaction interface to fully retain all answer text, reasoning trajectory, and source metadata; dividing a physical compression layer based on the constructed general logical interaction interface and matching differentiated encoding methods, with answers using hash numbers, source identifiers and reasoning metadata stored using dictionary bitmaps, and computing power and scores using differential fixed-point compression; and using the divided physical compression layer to store long reasoning text using summaries and incremental fragments to obtain standard candidate records.

[0007] Preferably, four evaluation features—quality, consensus, source prior, and cost—are extracted from each inference content in the database. These include: using standard candidate records to extract verification scores, confidence levels, format compliance, and reasoning coherence to generate quality features; based on the quality features, calculating the percentage of candidates supporting the same answer and the consistency score across different source content to generate consensus features; and relying on the consensus features to extract inference source identifiers, inference checkpoints, and historical inference performance to obtain source prior features. Finally, cost features are generated by summarizing the total amount of text corresponding to the inference, the number of verification calls, and the estimated inference latency, thus forming a complete set of four evaluation features.

[0008] Preferably, scoring using frozen parameters and merging semantically equivalent content includes: inputting the complete four categories of evaluation features into the submitted strategy parameters after freezing, performing comprehensive scoring on all candidates, and generating scoring results; matching semantic similarity based on the scoring results, merging semantically equivalent candidates, and obtaining a candidate set; and outputting the candidate set for constructing a multi-level index.

[0009] Preferably, the optimal reasoning content is selected by pruning low-scoring branches based on feature boundaries, including: building a multi-level hierarchical index based on the merged candidate set, recording the feature value range corresponding to each index node, and obtaining an index structure with feature range markers; calculating the theoretical maximum score that each index branch can achieve based on the index structure with feature range markers and the frozen submission strategy parameters; comparing the theoretical maximum score corresponding to each index branch with the currently obtained candidate scores, pruning branches whose scores cannot enter the target range, and selecting the optimal reasoning content.

[0010] Preferably, distinguishing whether the fault occurs in the candidate generation stage or the answer submission stage includes: constructing a candidate recoverable space based on the optimal reasoning content, the actual number of correctly recovered samples, the correct answer recovery rate, and the newly added erroneous submission rate indicators to obtain a complete audit indicator dataset; statistically analyzing the data deviations in the candidate generation stage and the answer submission stage based on the complete audit indicator dataset; identifying the fault attribution stage by comparing the data deviations in each stage, and outputting a targeted optimization plan.

[0011] The second aspect of this application provides a controlled submission system for a multi-threaded inference candidate library for large language models, comprising: a problem acquisition module for acquiring input problems to be solved; a candidate encoding module for building a unified standard candidate record structure based on the input problems to be solved, splitting the logical interaction layer and the physical compression layer to complete lightweight encoding, sharing storage fragments for equivalent inference content from the same source, and completely retaining standardized answers, inference links, and various metadata; a candidate library construction and optimal inference module for building a global heterogeneous candidate library adapted to multi-threaded operation based on the unified standard candidate records, extracting four evaluation features (quality, consensus, source prior, and cost) from each inference content in the library, solving and permanently locking submission strategy parameters based on the development dataset, scoring using frozen parameters, merging semantically equivalent content, building a multi-layer index when candidates exceed a threshold, and pruning low-scoring branches based on feature boundaries to select the optimal inference content; and a result output and audit diagnosis module for outputting the final answer and full-link computing power cost data based on the optimal inference content, simultaneously outputting multiple sets of audit diagnosis indicators, and distinguishing whether the fault occurs in the candidate generation stage or the answer submission stage.

[0012] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement a controlled submission method for a multi-threaded inference candidate library for a large language model as described in the above embodiments.

[0013] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement a controlled submission method for a multi-threaded inference candidate library for a large language model as described in the above embodiments.

[0014] A fifth aspect of this application provides a computer program product, including a computer program or instructions, for implementing a controlled submission method for a multi-threaded inference candidate library for a large language model as described in the above embodiments.

[0015] Therefore, this application offers the following beneficial effects: By unifying standard candidate records and using layered lightweight encoding, and sharing common-source inference storage fragments, it significantly reduces the memory and transmission overhead of multi-threaded inference candidate libraries, and is compatible with various heterogeneous inference content such as thought chains, tool results, and checkpoints to achieve fair comparison; relying on four-dimensional features of quality, consensus, source prior, and cost, it jointly develops a solution set and permanently locks the submission strategy parameters, achieving a configurable trade-off between accuracy and computing power cost. The frozen strategy ensures stable online deployment and versionable management. Coupled with a layered index and scoring upper bound pruning mechanism, it can efficiently select the optimal inference content without full scoring in large-scale candidate scenarios. At the same time, it is equipped with multi-dimensional audit and diagnostic indicators to quantify candidate coverage and submission selection effects, clearly distinguishing whether the root cause of system failure is in the candidate generation stage or the answer submission stage, and taking into account the core requirements of low latency, low cost, high accuracy, and full-process auditability and iterative optimization of inference services. Thus, it solves the problems of high storage and transmission overhead and high inference latency in existing technologies.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0017] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating a controlled submission method for a multi-threaded inference candidate library for large language models, according to an embodiment of this application. Figure 2 This is a schematic diagram of the structure of a controlled submission system for a multi-threaded inference candidate library for a large language model, according to an embodiment of this application. Figure 3 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0019] The following describes a controlled submission method for a multi-threaded inference candidate library for large language models, based on embodiments of this application, with reference to the accompanying drawings. Addressing the high inference latency issue mentioned in the background section, this application provides a controlled submission method for a multi-threaded inference candidate library for large language models. This method significantly reduces the memory and transmission overhead of the multi-threaded inference candidate library by unifying standard candidate records and using layered lightweight encoding, and sharing common inference storage fragments. It is compatible with various heterogeneous inference content such as thought chains, tool results, and checkpoints, enabling fair comparison. Relying on four-dimensional features—quality, consensus, source prior, and cost—a joint development set is used to solve and permanently lock the submission strategy parameters, achieving a configurable trade-off between accuracy and computational cost. The freezing strategy ensures stable online deployment and versionable management. Combined with a layered index and scoring upper bound pruning mechanism, optimal inference content can be efficiently selected without full scoring in large-scale candidate scenarios. Simultaneously, multi-dimensional audit diagnostic indicators quantify candidate coverage and submission selection effects, clearly distinguishing whether the root cause of system failure lies in the candidate generation stage or the answer submission stage. This method balances the core requirements of low latency, low cost, high accuracy, and full-process auditability and iterative optimization for inference services. This solves the problems of high storage and transmission overhead and high inference latency in existing technologies.

[0020] Specifically, Figure 1 This application provides a controlled submission method for a candidate library of multi-threaded inference for large language models.

[0021] like Figure 1 As shown, this controlled submission method for a multi-threaded inference candidate library for large language models includes the following steps: In step S101, the input problem to be solved is collected.

[0022] It is understood that the embodiments of this application can provide the original input basis for subsequent full-process reasoning, candidate library construction and accurate screening, unify the input source of all reasoning tasks, avoid the problems of messy input data, inconsistent standards and semantic deviation of questions in the process of multi-threaded parallel reasoning, and ensure that each stage of subsequent standardized candidate record structure construction, lightweight coding and splitting, extraction of four types of evaluation features, candidate screening and fault auditing are carried out around the goal of solving problems in a real, complete and unified manner. It ensures that the equivalent reasoning content of the same source problem can accurately match the shared storage fragment, and at the same time provides accurate original data benchmarks for task scheduling, feature quantization scoring, branch pruning and screening and fault location of multi-threaded reasoning tasks, ensuring the standardization, consistency and accuracy of multi-threaded reasoning and subsequent processing results from the input source.

[0023] In step S102, a unified standard candidate record structure is built based on the input problem to be solved. The logical interaction layer and the physical compression layer are split to complete lightweight encoding. Equivalent reasoning content from the same source shares the same storage fragment, and standardized answers, reasoning links and various metadata are completely preserved.

[0024] Lightweight coding is a coding methodology that aims to minimize resource consumption by simplifying syntax, optimizing data structures, and reusing modular logic to improve system execution efficiency and maintainability.

[0025] It is understood that, by splitting the logical interaction layer and the physical compression layer, this application adopts differentiated compression storage strategies for different data types while fully preserving core effective information such as reasoning answers, reasoning links, and various metadata. It implements summary and incremental fragment storage for long reasoning texts, realizing storage resource sharing of equivalent reasoning content from the same source. This effectively solves the problems of large candidate data volume, high redundancy, large storage occupation, and low read / write efficiency in the multi-threaded reasoning process of large language models. It significantly reduces the storage space overhead and memory occupation of the global heterogeneous candidate library, reduces the data transmission pressure and latency during multi-threaded concurrent read / write, and improves the overall operating efficiency of candidate library data retrieval, feature extraction, semantic merging, and index construction. At the same time, it takes into account the requirements of data integrity and lightweight storage, providing efficient, concise, and standardized data storage support for efficient multi-threaded reasoning, accurate candidate selection, and stable and controlled submission.

[0026] In this embodiment, a unified standard candidate record structure is constructed, including: constructing a general logical interaction interface to fully retain all answer text, reasoning trajectory, and source metadata; dividing a physical compression layer based on the constructed general logical interaction interface and matching differentiated encoding methods, with answers using hash numbers, source identifiers and reasoning metadata stored using dictionary bitmaps, and computing power and scores using differential fixed-point compression; and using the divided physical compression layer to store long reasoning text using summaries and incremental fragments to obtain standard candidate records.

[0027] Among them, the general logic interaction interface is a standardized system boundary definition. It encapsulates reusable business rules and decision processing capabilities, and provides a unified request-response communication channel for upstream callers to achieve decoupling of logical calculation, state transition and data exchange.

[0028] It is understood that the embodiments of this application can completely retain the answer text, reasoning trajectory and various source metadata, realize the full-element and traceable information retention of the reasoning process of the large language model, and at the same time, as the connecting hub between the logic layer and the physical compression layer, it unifies the information interaction specifications in multi-threaded reasoning scenarios, provides a standardized data entry point for subsequent differentiated physical encoding and shared storage fragments of the same source reasoning content, avoids the problems of messy data format, information fragmentation and difficulty in tracing the source caused by multi-threaded parallel reasoning, ensures the standardization, integrity and consistency of candidate library data, and supports the orderly development of subsequent feature extraction, candidate screening, fault auditing and other processes.

[0029] In step S103, a global heterogeneous candidate library adapted to multi-threaded operation is built based on the unified standard candidate records. Four evaluation features, namely quality, consensus, source prior and cost, are extracted from each inference content in the library. The submission strategy parameters are solved and permanently locked based on the development dataset. The frozen parameters are used for scoring. Semantically equivalent content is merged. When the candidate exceeds the threshold, a multi-layer index is built. Low-scoring branches are pruned according to the feature boundary to select the optimal inference content.

[0030] Among them, the global heterogeneous candidate library is a unified storage collection built on a system-wide scale, integrating diverse candidate objects from various data sources, data structures, or recall strategies. It aims to break down data barriers and provide a comprehensive pool of alternatives for subsequent accurate ranking or comprehensive decision-making.

[0031] It is understood that the embodiments of this application are built on standardized candidate records, which are compatible with various heterogeneous inference content generated concurrently by multiple threads. They uniformly collect inference candidate samples from different sources, links, and with different qualities and costs, breaking down the data silos of multi-threaded inference. At the same time, the candidate library can support the unified extraction and calculation of four core evaluation features: quality, consensus, source prior, and cost. With the help of freezing strategy parameters, it can complete candidate scoring, semantically equivalent content merging, multi-level index construction, and low-scoring branch pruning and screening. It can efficiently complete the regularization, noise reduction, and optimal selection of massive inference candidates, providing comprehensive, regular, and schedulable underlying data support for the model to accurately output the optimal inference content, statistically analyze the computing power cost of the entire link, and accurately locate the faults in the candidate generation and answer submission stages.

[0032] In this embodiment, four evaluation features—quality, consensus, source prior, and cost—are extracted from each inference content in the database. These features include: using standard candidate records to extract verification scores, confidence levels, format compliance, and inference coherence to generate quality features; based on the quality features, the percentage of candidates supporting the same answer and the consistency score across different source content are statistically analyzed to generate consensus features; relying on the consensus features, inference source identifiers, inference checkpoints, and historical inference performance are extracted to obtain source prior features; and cost features are generated by summarizing the total amount of text corresponding to the inference, the number of verification calls, and the estimated inference latency information, thus forming a complete set of four evaluation features.

[0033] It is understood that the embodiments of this application evaluate the accuracy, standardization, and coherence of a single inference by using quality features, measure the group consistency and credibility of the inference results by using consensus features, distinguish the reliability of the inference source by using source prior features based on source information and historical performance, and quantify the resource and time-series overhead of inference by using cost features. The four types of features complement each other and fully cover the inference effect, credibility, source reliability, and operating cost, thus overcoming the one-sidedness of a single scoring dimension. This provides a standardized, refined, and quantifiable basis for subsequent candidate comprehensive scoring, semantic equivalence merging, branch pruning and selection, and optimal inference content selection, effectively improving the accuracy of multi-threaded inference candidate selection and overall scheduling efficiency, while taking into account both inference output quality and computing cost control.

[0034] In this embodiment of the application, scoring using frozen parameters and merging semantically equivalent content includes: inputting the complete four categories of evaluation features into the submission strategy parameters after freezing, performing comprehensive scoring on all candidates, and generating scoring results; matching semantic similarity based on the scoring results, merging semantically equivalent candidates, and obtaining a candidate set; and outputting the candidate set for constructing a multi-level index.

[0035] Among them, multi-level indexing is a data storage technology that organizes the index structure into a multi-level tree hierarchy. The upper-level index stores pointers to the lower-level index or data blocks. By filtering layer by layer, the search range is narrowed, thereby significantly improving query efficiency and reducing disk I / O overhead in the context of massive data.

[0036] It is understood that the embodiments of this application define the value ranges of four evaluation features—quality, consensus, source prior, and cost—for each index node, forming a hierarchical index structure with precise feature range markings. This enables structured and hierarchical management of massive multi-threaded inference candidates. It can predict the theoretical maximum score of each index branch by combining the frozen submission strategy parameters, quickly identify and remove low-scoring invalid branches, without having to traverse all candidate samples one by one. This significantly reduces the computational overhead and retrieval latency of large-scale candidate screening, while accurately retaining high-value inference content. This significantly improves the efficiency and accuracy of selecting the best inference content, providing efficient data retrieval support for the model to efficiently and accurately complete the selection of inference candidates and controlled submission.

[0037] In this embodiment, the optimal reasoning content is selected by pruning low-scoring branches based on feature boundaries. This includes: building a multi-level hierarchical index based on the merged candidate set, recording the feature value range corresponding to each index node, and obtaining an index structure with feature range labels; calculating the theoretical maximum score that each index branch can achieve based on the index structure with feature range labels and the frozen submission strategy parameters; comparing the theoretical maximum score corresponding to each index branch with the currently obtained candidate scores, pruning branches whose scores cannot enter the target range, and selecting the optimal reasoning content.

[0038] It is understood that the embodiments of this application achieve hierarchical and structured management of inference candidates by constructing a multi-level hierarchical index with feature interval markings. Combined with fixed submission strategy parameters, the theoretical maximum score of each index branch is accurately calculated. Interval boundary judgment replaces the full candidate traversal comparison, which can eliminate low-scoring invalid branches that do not meet the upper limit of the score and cannot enter the optimal interval in advance. This significantly reduces the amount of screening calculation and retrieval latency of massive multi-threaded candidates and avoids unnecessary consumption of computing power. At the same time, it relies on four-dimensional feature boundary constraints to complete accurate selection, avoiding the problems of strong randomness, high redundancy and insufficient selection accuracy of traditional screening methods. Under the premise of ensuring the optimality and reliability of the inference results, it significantly improves the efficiency of multi-threaded inference candidate screening and the overall submission stability of large language models.

[0039] In step S104, the final answer and full-link computing power overhead data are output based on the optimal reasoning content, and multiple sets of audit diagnostic indicators are output simultaneously to distinguish whether the fault occurs in the candidate generation stage or the answer submission stage.

[0040] It is understood that the embodiments of this application output complete end-to-end computing power consumption data and multi-dimensional audit diagnostic indicators while outputting the final reasoning answer. On the one hand, it can quantify and statistically analyze the resource consumption of the entire multi-threaded reasoning process, supporting computing power scheduling optimization and operating cost assessment. On the other hand, relying on differentiated audit indicators, it can independently calculate the data deviation between the candidate generation stage and the answer submission stage, quickly distinguish the stage to which the fault belongs, and break the bottleneck of difficulty in locating faults in the two stages due to mutual confusion. It not only realizes the integration of reasoning result delivery and resource consumption perception, but also provides quantitative basis for system anomaly investigation and targeted iterative optimization of modules, ensuring that the controlled submission process of the multi-threaded reasoning candidate library of the large language model is observable, auditable, and maintainable.

[0041] In this embodiment, distinguishing whether the fault occurs in the candidate generation stage or the answer submission stage includes: constructing a candidate recoverable space based on the optimal reasoning content, the actual number of correctly recovered samples, the correct answer recovery rate, and the newly added erroneous submission rate indicators to obtain a complete audit indicator dataset; statistically analyzing the data deviations in the candidate generation stage and the answer submission stage based on the complete audit indicator dataset; identifying the fault attribution stage by comparing the data deviations in each stage, and outputting a targeted optimization plan.

[0042] Among them, optimal reasoning content refers to the optimal combination of all premise propositions, intermediate assumptions and derivation rules that can maximize the truthfulness or posterior probability of the conclusion and minimize reasoning error and uncertainty under a given cognitive context, information foundation and resource constraints, by comprehensively weighing the evidence support, logical deductive validity and conciseness standards.

[0043] It is understood that the optimal inference content in this application embodiment, as a high-value inference sample selected after multi-dimensional feature scoring, semantic equivalence candidate merging, and multi-level index branch pruning, comprehensively considers inference quality, consensus credibility, source reliability, and computing power overhead. Using this as the output basis can ensure the accuracy and stability of the final answer. At the same time, the optimal inference content carries the complete inference chain and standardized metadata, which can be directly used to generate the final answer for users, and also provides a complete data source for full-chain computing power statistics and audit indicator calculation, supports fault segmentation and system diagnosis, and ensures that the controlled submission process of the multi-threaded inference candidate library of the large language model outputs credibility, information integrity, and is easy to trace and maintain.

[0044] The following will illustrate a controlled submission method for a multi-threaded inference candidate library for large language models through a specific embodiment, including: After initiating the inference process, the input problem to be solved is first collected. This input problem is a standard multi-step algebraic calculation problem. Eight inference threads are started in parallel to generate heterogeneous inference candidates from multiple sources. Simultaneously, a unified standard candidate record structure is built, and lightweight coding is completed by dividing the logical interaction layer and the physical compression layer. The unified candidate record adopts the triple formula. Complete standardized packaging, among which Represents a standardized answer text. Represents the complete reasoning chain text. The structured metadata includes source type, total number of generated tokens, validator call count, inference latency, sampling parameters, and model version. A shared storage mechanism is implemented for equivalent inference fragments generated by multi-threaded concurrency. The logical interaction layer fully preserves the original answer, inference steps, and all metadata without truncation. The physical compression layer executes differential encoding rules, mapping the normalized answer to a 64-bit hash equivalence class number. Candidate source, model version, checkpoint identifier, and validity Boolean flag are jointly encoded using a global dictionary and bitmap. Continuous numerical fields such as the number of tokens generated per sample, validation score, and inference latency are compressed using differential encoding combined with 16-bit fixed-point quantization. Fields exceeding 5 bytes in length... The 12-character inference link text is split into a basic summary and incremental fragments. Inference branches with the same origin share the basic summary storage block. Only fragment reference pointers are stored in the candidate records. When auditing the complete inference trajectory, the original text is traced back based on the pointers. The original storage of a single candidate is 21.6KB, which is reduced to only 3.2KB after layered compression encoding. The total memory usage of 48 candidate records generated in parallel by 8 threads is compressed from 1036.8KB to 153.6KB. When transmitting to the submission layer, only the candidate index, equivalence class number, basic values ​​of four-dimensional features and fragment pointers are transmitted, which greatly reduces the network transmission overhead in multi-threaded concurrent scenarios and takes into account both lightweight storage and the need for traceable auditing of the complete inference link.

[0045] After completing the standardized encapsulation of individual candidates, a global heterogeneous candidate library adapted for multi-threaded operation is built based on all unified candidate records, using set formulas. Complete the mathematical definition of the candidate library; the total number of candidates corresponding to this input problem. The candidate sources cover six heterogeneous channels: explicit thought chain sampling, internal reasoning checkpoints, multi-hint etymological variants, direct short answers, retrieval-enhanced external knowledge, and verifier-assisted generation. Reasoning entries generated from all sources are uniformly included in the same candidate database for unified management, eliminating issues of source isolation and incompatibility of comparison rules, thus addressing the shortcoming of unfair horizontal comparison of heterogeneous reasoning content. Subsequently, four types of evaluation features are extracted from each reasoning record in the database and combined into a four-dimensional feature vector formula. The four-dimensional features correspond to quality features, consensus features, source prior features, and cost features, respectively. The typical candidate with index 17 in the candidate pool is selected. Complete numerical decomposition calculations and quality characteristics. The consensus feature is calculated by normalizing and weighting the sum of four indicators: validator score (0.86), answer confidence (0.91), format compliance (0.98), and reasoning step coherence (0.83). The score is 0.685, weighted by the proportion of candidates with equivalent answers (0.72) and the cross-source consistency score (0.65). This score is derived from the prior features of the source. Combining the historical development set average accuracy of 0.79 for this candidate source type, the inference checkpoint validity label of 1, and the model version baseline score of 0.84, a cost feature is calculated to be 0.815. The total number of tokens generated from a single sample is 128, the number of validator calls is 1, and the estimated inference latency is 142 milliseconds. A normalized, inversely weighted average yields 0.230, resulting in the final candidate complete feature vector. The remaining 47 candidates in the database simultaneously completed the four-dimensional feature value calculation to form a complete candidate feature matrix. All feature value intervals were uniformly mapped to the standard interval of 0 to 1 to eliminate the interference of different units on the subsequent scoring strategy.

[0046] After all feature vectors are generated, the solution process proceeds to the fixed submission strategy stage, employing an objective function that includes both accuracy and computational power constraints. Complete the optimal parameter solution and develop the corresponding set average cost calculation formula. To standardize the computational cost, this experiment selected a development set-scale... Single sample cost The cost weight is set by weighting three metrics: total token generation, number of verification calls, and inference latency. To balance inference accuracy and computational cost, a grid search is used to traverse the entire feasible weight space. The fixed optimal policy parameters are then obtained by solving. The four-dimensional parameters correspond to the linear weights of quality features, consensus features, source prior features, and cost features, respectively. A negative weight for the cost feature indicates that the higher the inference computing power consumption, the lower the candidate's comprehensive score, thus realizing computing power-aware decision submission. After the strategy parameters are solved, they are permanently locked and no longer dynamically adjusted with online samples, ensuring deployment stability and reproducibility. Different business scenarios can be configured with exclusive weight combinations.

[0047] Use a linear scoring formula after locking the strategy parameters. A comprehensive score was given to all candidates, and the candidates were substituted into the candidate's score. The feature vector and fixed parameters are used to complete the calculation. The score calculation process is as follows: 0.42 multiplied by 0.895, plus 0.31 multiplied by 0.685, plus 0.19 multiplied by 0.815, minus 0.08 multiplied by 0.230. The step-by-step calculation yields 0.3759 plus 0.21235 plus 0.15485 minus 0.0184, resulting in the final comprehensive score. Simultaneously, all 47 remaining candidates were scored. After scoring, semantically equivalent answers were merged. Candidates with differences in string expression, unit writing, and step description but identical mathematical results were categorized by standardized hash equivalence class numbering. The 48 original candidates were reduced to 12 independent answer equivalence clusters after equivalence merging. The average consensus features, average quality features, and average computing power cost were summarized within each equivalence cluster to reduce the interference of expression differences on the final selection. After merging, the total number of candidate sets was reduced to 12 groups. It was determined that the number of current candidates exceeded the preset threshold of 8, and the hierarchical indexing and pruning filtering process was initiated to quickly filter the optimal reasoning content.

[0048] A four-layer hierarchical index structure is built based on 12 sets of equivalent candidate data, namely the answer equivalence class index, candidate source index, quality bucket index, and cost bucket index. Each index node independently maintains the lower bound of the values ​​corresponding to the four-dimensional features. and the upper bound of the value Taking the index branch node v as an example, the lower bound vector of this node's four-dimensional features... Four-dimensional feature upper bound vector Total number of feature dimensions The formula for calculating the upper bound of index node safety is adopted. To calculate the highest score for this branch of the theory, the calculation is performed dimension by dimension. The first dimension, quality feature, has a weight of 0.42, with a maximum value of 0.42 multiplied by 0.93, equaling 0.3906. The second dimension, consensus feature, has a weight of 0.31, with a maximum value of 0.31 multiplied by 0.78, equaling 0.2418. The third dimension, source prior, has a weight of 0.19, with a maximum value of 0.19 multiplied by 0.87, equaling 0.1653. The fourth dimension, cost feature, has a weight of -0.08, with a maximum value of -0.08 multiplied by 0.18, equaling -0.0144. The sum of these four values ​​yields the highest score for this branch of the index theory. The priority queue is used to retrieve the top-1 best candidate for the target, and the score of the current first candidate that has been accurately scored is obtained. Refer to the pruning judgment formula Branch selection is performed. The theoretical upper limit of this branch (0.7833) is greater than the current first-place score (0.7512), so this branch is retained for the precise scoring process. The theoretical maximum scores of the other three index branches are 0.6921, 0.6475, and 0.7106, respectively, all of which are less than... The pruning rules are directly triggered, skipping the precise scoring of all candidates within a branch and only performing full scoring and sorting on equivalent candidates within the retained branch. Finally, the optimal candidate index formula is used. Locate the candidate with the highest global score. According to the final answer formula Output the normalized optimal answer and simultaneously summarize the total generated tokens (128), validator calls (1), and total inference latency (142 milliseconds) of the candidate's total computing power overhead.

[0049] After the answer is output, a complete set of audit diagnostic indicators is generated. Based on the coverage of all correct answers in the candidate library, the core quantitative indicators OracleGap, Recovered, RecoveryRate, NetFlip, and FalseFlipRate are calculated. In this case, the original 48 candidates in the candidate library contain 8 equivalent records of correct answers. Without the introduction of controlled submission logic, the traditional majority voting scheme can only select 3 correct samples. The actual number of correct samples recovered by the controlled submission process is Recovered=5. OracleGap represents the recoverable space of the candidate library as 5. RecoveryRate is calculated as 5 divided by 5 equals 100%, which means that all correct answers in the library can be successfully filtered by the current weight strategy. The NetFlip indicator counts the difference between the error correction samples and the correct misjudged samples. In this execution, 4 samples were corrected from erroneous submissions to correct submissions, and 0 samples were reversed from correct submissions to erroneous submissions. NetFlip=4 and FalseFlipRate is 0, which means that the submission filtering logic does not introduce the risk of error reversal. By splitting the candidate generation stage and answer submission stage deviation data through a complete audit indicator dataset, the candidate generation stage deviation indicator shows that there are still 12 input samples whose candidate library does not cover the correct answer. The root cause is insufficient coverage of the multi-threaded sampling generation path. It is necessary to expand the number of inference checkpoint sampling to optimize the candidate generation stage. The answer submission stage deviation indicators are all close to 0, proving that there is no obvious selection defect under the fixed submission weight configuration. Based on this, a phased targeted optimization plan is output. The optimization direction of the candidate generation stage is to expand the number of internal inference checkpoint branches and increase the sampling channels for multiple prompt word variants. The submission screening stage does not need to adjust the fixed strategy parameters. It can run stably by simply maintaining the existing hierarchical index pruning rules.

[0050] In summary, this invention relies on triplet encapsulation and hierarchical compression to balance concurrent storage and transmission overhead with the traceability of the inference link. It achieves fair comparison of multi-source inference through a unified candidate library, solves computationally-aware decision weights offline, reduces computational load by combining equivalent answer clustering and four-layer index boundary pruning, and uses a series of audit indicators to distinguish the shortcomings in the candidate generation and screening stages. The verification scheme can balance accuracy and computational overhead, while providing a reproducible implementation process and quantitative evaluation paradigm.

[0051] Figure 2 This is a schematic diagram of the structure of a controlled submission system for a multi-threaded inference candidate library for a large language model, according to an embodiment of this application.

[0052] like Figure 2As shown, the controlled submission system 10 for a multi-threaded inference candidate library for large language models includes: a problem collection module 100, a candidate encoding module 200, a candidate library construction and optimal inference module 300, and a result output and audit diagnosis module 400.

[0053] The system comprises the following modules: Problem Acquisition Module 100, which collects the input problem to be solved; Candidate Encoding Module 200, which builds a unified standard candidate record structure based on the input problem, splits the logical interaction layer and physical compression layer to complete lightweight encoding, shares storage fragments for equivalent inference content, and fully preserves standardized answers, inference links, and various metadata; Candidate Library Construction and Optimal Inference Module 300, which builds a global heterogeneous candidate library adapted to multi-threaded operation based on the unified standard candidate records, extracts four evaluation features (quality, consensus, source prior, and cost) from each inference content in the library, solves and permanently locks the submission strategy parameters based on the development dataset, scores using frozen parameters, merges semantically equivalent content, builds multi-layer indexes when candidates exceed the threshold, and prunes low-scoring branches based on feature boundaries to select the optimal inference content; and Result Output and Audit Diagnosis Module 400, which outputs the final answer and full-link computing power cost data based on the optimal inference content, and simultaneously outputs multiple sets of audit diagnosis indicators to distinguish whether the fault occurs in the candidate generation stage or the answer submission stage.

[0054] It should be noted that the foregoing explanation of an embodiment of a controlled submission method for a multi-threaded inference candidate library for a large language model also applies to the controlled submission system for a multi-threaded inference candidate library for a large language model in this embodiment, and will not be repeated here.

[0055] This application proposes a controlled submission system for a multi-threaded inference candidate library for large language models. By standardizing candidate records and using layered lightweight encoding, and sharing common inference storage fragments, it significantly reduces the memory and transmission overhead of the multi-threaded inference candidate library. It is compatible with various heterogeneous inference content such as thought chains, tool results, and checkpoints, enabling fair comparison. Based on four dimensions of features—quality, consensus, source prior, and cost—it jointly develops a solution set and permanently locks submission strategy parameters, achieving a configurable trade-off between accuracy and computational cost. The freezing strategy ensures stable online deployment and versionable management. Combined with a layered index and scoring upper bound pruning mechanism, it can efficiently filter the optimal inference content without full scoring in large-scale candidate scenarios. Simultaneously, it provides multi-dimensional audit and diagnostic indicators to quantify candidate coverage and submission selection effectiveness, clearly distinguishing whether system failure originates in the candidate generation stage or the answer submission stage. It addresses the core requirements of low latency, low cost, high accuracy, and full-process auditability and iterative optimization for inference services. Therefore, it solves the problems of high storage and transmission overhead and high inference latency in existing technologies.

[0056] Figure 3A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0057] When the processor 302 executes the program, it implements the controlled submission method for a multi-threaded inference candidate library for large language models provided in the above embodiments.

[0058] Furthermore, electronic devices also include: Communication interface 303 is used for communication between memory 301 and processor 302.

[0059] The memory 301 is used to store computer programs that can run on the processor 302.

[0060] The memory 301 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0061] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0062] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0063] Processor 302 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0064] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the controlled submission method for a multi-threaded inference candidate library for a large language model as described above.

[0065] Furthermore, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed, implement the aforementioned controlled submission method for a multi-threaded inference candidate library for large language models.

[0066] In the description of this specification, the references to the terms "an embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0067] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0068] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0069] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by suitable instructions. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0070] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0071] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A controlled submission method for a candidate library of multi-threaded inference for large language models, characterized in that, include: Collect the input problem to be solved; Based on the input problem to be solved, a unified standard candidate record structure is constructed. A lightweight encoding is achieved by splitting the logical interaction layer and the physical compression layer. Equivalent inference content from the same source shares a storage segment, and standardized answers, inference chains, and various metadata are fully preserved. The unified standard candidate record uses a triplet formula. Complete standardized packaging, among which Represents standardized answer text. Represents the complete reasoning chain text. The system represents structured metadata including source type, total number of generated tokens, number of validator calls, inference latency, sampling parameters, and model version. It implements a shared storage mechanism for equivalent inference fragments generated by multiple threads concurrently. The logical interaction layer completely preserves the original answer, inference steps, and all metadata without truncation. The physical compression layer implements differential encoding rules, and the normalized answer is mapped to a 64-bit hash equivalence class number. Candidate source, model version, checkpoint identifier, and validity Boolean flag are jointly encoded using a global dictionary and bitmap. The number of tokens generated per sample, the verification score, and the continuous numerical fields of inference latency are compressed using differential encoding combined with 16-bit fixed-point quantization. Inference link texts with a length exceeding 512 characters are split into a basic summary and incremental fragments. Inference branches with the same source share the basic summary storage block. Only fragment reference pointers are retained in candidate records. When auditing the complete inference trajectory, the original text is traced back based on the pointers. A global heterogeneous candidate library adapted for multi-threaded operation is built based on the unified standard candidate records. Four evaluation features—quality, consensus, source prior, and cost—are extracted from each inference entry in the library. These features are solved using a development dataset, and the submission strategy parameters are permanently locked. Score is then calculated using frozen parameters. After all feature vectors are generated, a fixed submission strategy solution phase is initiated, employing an objective function that includes accuracy and computational power constraints. Complete the optimal parameter solution and develop the corresponding set average cost calculation formula. Unified quantification of computing power overhead and cost per sample The total number of generated tokens, the number of verification calls, and the inference latency are weighted and combined to balance inference accuracy and computing power consumption. The entire feasible weight space is traversed through a grid search. After solving, the fixed optimal strategy parameters are obtained. The four-dimensional parameters correspond to the linear weights of quality features, consensus features, source prior features, and cost features, respectively. A negative value for the cost feature weight indicates that the inference computing power consumption is higher and the candidate comprehensive score is lower. A computing power-aware submission decision is made. After the strategy parameters are solved, they are permanently locked and no longer dynamically adjusted with online samples. Semantically equivalent content is merged. When the candidate exceeds the threshold, a multi-layer index is built. Low-scoring branches are pruned according to the feature boundary to select the optimal inference content. Based on the optimal reasoning content, the final answer and full-link computing power overhead data are output, and multiple sets of audit diagnostic indicators are output simultaneously to distinguish whether the fault occurs in the candidate generation stage or the answer submission stage.

2. The controlled submission method for a multi-threaded inference candidate library for large language models according to claim 1, characterized in that, Establish a unified standard candidate record structure, including: Build a general logic interaction interface; Based on the established general logical interaction interface, physical compression layers are divided and matched with differentiated encoding methods.

3. The controlled submission method for a candidate library of multi-threaded inference for large language models according to claim 1, characterized in that, Extract four evaluation features from each inference entry in the database: quality, consensus, source prior, and cost. Validation scores, confidence levels, format compliance, and reasoning coherence are extracted from standard candidate records to generate quality features; Based on the aforementioned quality features, the percentage of candidates supporting the same answer and the consistency score across different sources of content are statistically analyzed to generate consensus features. Based on the consensus features, the source identifier, inference checkpoint, and historical inference performance are extracted to obtain the source prior features. The total amount of text corresponding to the inference, the number of verification calls, the inference latency prediction information generation cost features are summarized to form a complete set of four types of evaluation features.

4. The controlled submission method for a candidate library of multi-threaded inference for large language models according to claim 3, characterized in that, Scoring is performed using frozen parameters, and semantically equivalent content is merged, including: Input the complete four categories of evaluation features into the submitted strategy parameters after freezing, perform comprehensive scoring on all candidates, and generate scoring results; Based on the scoring results, semantic similarity is matched, and semantically equivalent candidates are merged to obtain a candidate set; The candidate set is output for constructing a multi-level index.

5. The controlled submission method for a multi-threaded inference candidate library for large language models according to claim 4, characterized in that, Based on feature boundaries, low-scoring branches are pruned to select the optimal inference content, including: A multi-level hierarchical index is built based on the merged candidate set, and the feature value range corresponding to each index node is recorded to obtain an index structure with feature range labels. Based on the index structure with feature interval markers, and combined with the frozen submission strategy parameters, the theoretical maximum score that each index branch can achieve is calculated. By comparing the theoretical highest score corresponding to each index branch with the currently obtained candidate scores, branches whose scores cannot enter the target interval are removed, and the optimal reasoning content is obtained.

6. The controlled submission method for a candidate library of multi-threaded inference for large language models according to claim 1, characterized in that, Distinguish between failures occurring during candidate generation and answer submission, including: Based on the optimal reasoning content, a complete audit indicator dataset is obtained by constructing a candidate recoverable space, the actual number of correctly recovered samples, the correct answer recovery rate, and the newly added erroneous submission rate. Based on the complete audit indicator dataset, the data deviations in the candidate generation stage and the answer submission stage were statistically analyzed respectively; By comparing the data deviations described in each stage, the fault attribution point is located, and a targeted optimization plan is output.

7. A controlled submission system for a candidate library of multi-threaded inference for large language models, characterized in that, include: The problem acquisition module is used to collect the input problem to be solved; The candidate encoding module is used to build a unified standard candidate record structure based on the input problem to be solved, split the logical interaction layer and the physical compression layer to complete lightweight encoding, share storage fragments for equivalent inference content, and completely retain standardized answers, inference links and various metadata. The unified standard candidate record adopts a triple formula. Complete standardized packaging, among which Represents standardized answer text. Represents the complete reasoning chain text. The system represents structured metadata including source type, total number of generated tokens, number of validator calls, inference latency, sampling parameters, and model version. It implements a shared storage mechanism for equivalent inference fragments generated by multiple threads concurrently. The logical interaction layer completely preserves the original answer, inference steps, and all metadata without truncation. The physical compression layer implements differential encoding rules, and the normalized answer is mapped to a 64-bit hash equivalence class number. Candidate source, model version, checkpoint identifier, and validity Boolean flag are jointly encoded using a global dictionary and bitmap. The number of tokens generated per sample, the verification score, and the continuous numerical fields of inference latency are compressed using differential encoding combined with 16-bit fixed-point quantization. Inference link texts with a length exceeding 512 characters are split into a basic summary and incremental fragments. Inference branches with the same source share the basic summary storage block. Only fragment reference pointers are retained in candidate records. When auditing the complete inference trajectory, the original text is traced back based on the pointers. The candidate library construction and optimal reasoning module is used to build a global heterogeneous candidate library adapted for multi-threaded operation based on the unified standard candidate records. It extracts four evaluation features—quality, consensus, source prior, and cost—from each inference entry in the library, solves for and permanently locks the submission strategy parameters using the development dataset, and scores using frozen parameters. After all feature vectors are generated, the module enters the fixed submission strategy solution stage, employing an objective function that includes accuracy and computational power constraints. Complete the optimal parameter solution and develop the corresponding set average cost calculation formula. Unified quantification of computing power overhead and cost per sample The total number of generated tokens, the number of verification calls, and the inference latency are weighted and combined to balance inference accuracy and computing power consumption. The entire feasible weight space is traversed through a grid search. After solving, the fixed optimal strategy parameters are obtained. The four-dimensional parameters correspond to the linear weights of quality features, consensus features, source prior features, and cost features, respectively. A negative value for the cost feature weight indicates that the inference computing power consumption is higher and the candidate comprehensive score is lower. A computing power-aware submission decision is made. After the strategy parameters are solved, they are permanently locked and no longer dynamically adjusted with online samples. Semantically equivalent content is merged. When the candidate exceeds the threshold, a multi-layer index is built. Low-scoring branches are pruned according to the feature boundary to select the optimal inference content. The results output and audit diagnosis module is used to output the final answer and full-link computing power overhead data based on the optimal reasoning content, and simultaneously output multiple sets of audit diagnosis indicators to distinguish whether the fault occurs in the candidate generation stage or the answer submission stage.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the controlled submission method for a multi-threaded inference candidate library for large language models as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When a computer program or instruction is executed, it implements a controlled submission method for a multi-threaded inference candidate library for a large language model as described in any one of claims 1-6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When a computer program or instruction is executed, it implements a controlled submission method for a multi-threaded inference candidate library for a large language model as described in any one of claims 1-6.

Images