Big language model reasoning method and device, equipment and storage medium

Abstract

The invention provides a big language model reasoning method and device, equipment and a storage medium, and is suitable for the fields of financial science and technology and artificial intelligence. The method comprises the following steps: acquiring an original large language model and a plurality of reasoning units of the original large language model; for each reasoning unit, constructing a compressed coding tree according to the plurality of weight parameters of the reasoning unit, and performing parameter compression on the weight parameters through the compressed coding tree to generate a compressed bit stream; dividing the compressed bit stream into a plurality of compressed bit segments, and for each compressed bit segment, calculating the initial bit offset of the first complete code to obtain an offset array; in response to the obtained target reasoning task, aiming at each reasoning unit, decoding the compressed bit stream according to the corresponding offset array, and restoring to obtain a plurality of linear weight matrixes; and executing a target reasoning task by using the plurality of linear weight matrixes. According to the method, the performance loss of the large language model can be reduced while efficient compression of the large language model is realized.

Description

Technical Field

[0001] This application applies to the fields of fintech and artificial intelligence, and particularly relates to a reasoning method, apparatus, device, and storage medium for a large language model. Background Technology

[0002] Large language models have been widely used in numerous industries, including fintech, due to their powerful natural language processing capabilities. However, the scale of these models is growing at an unprecedented rate, posing a significant challenge to hardware deployment. For example, a large model like Llama-3.1 has 405 billion parameters in 16-bit floating-point format, requiring approximately 810GB of memory for a single inference operation. This far exceeds the capacity of a typical high-end GPU server, making economical and efficient deployment of these models extremely difficult.

[0003] In related technologies, memory consumption is mainly reduced by decreasing the precision of weight values. However, this method leads to a decline in the quality and accuracy of model inference output. In scenarios such as financial risk control and precision medicine, where stringent requirements for model accuracy are necessary, a significant loss of inference performance is unacceptable. Therefore, how to effectively compress model size to reduce deployment costs without sacrificing inference accuracy is a pressing technical challenge in this field. Summary of the Invention

[0004] The main objective of this application is to propose a reasoning method, apparatus, device, and storage medium for large language models, aiming to achieve efficient compression of large language models while reducing the loss of reasoning performance.

[0005] To achieve the above objectives, a first aspect of this application proposes a reasoning method for a large language model, the method comprising:

[0006] Obtain the original large language model and multiple inference units of the original large language model; wherein each inference unit includes multiple linear weight matrices, and each linear weight matrix includes multiple weight parameters;

[0007] For each inference unit, a compressed coding tree is constructed based on multiple weight parameters of the inference unit, and the weight parameters are compressed using the compressed coding tree to generate a compressed bitstream;

[0008] The compressed bitstream is divided into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, the offset of the first fully encoded starting bit is calculated to obtain an offset array.

[0009] In response to obtaining the target inference task, for each inference unit, the compressed bitstream is decoded according to the corresponding offset array to restore multiple linear weight matrices;

[0010] The target inference task is performed using multiple linear weight matrices.

[0011] In some embodiments, the weight parameters include a weight exponent, a weight mantissa, and a weight sign. The step of constructing a compressed coding tree for each inference unit based on the multiple weight parameters of the inference unit includes:

[0012] Calculate the frequency of occurrence of each index value in the weighted index;

[0013] Based on the frequency of occurrence of each index value, a corresponding compressed coding value is assigned to each index value to obtain the compressed coding tree.

[0014] In some embodiments, the step of compressing the weight parameters using the compressed coding tree to generate a compressed bitstream includes:

[0015] The weight mantissas and weight signs of multiple weight parameters are concatenated to generate an uncompressed data block for the inference unit;

[0016] For each of the weight parameters, the compressed code value corresponding to the weight index is determined as the compressed index according to the compressed coding tree;

[0017] The compressed bitstream is generated by concatenating the compressed exponents of multiple weight parameters.

[0018] In some embodiments, the compressed bitstream is divided into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, the starting bit offset of the first fully encoded segment is calculated to obtain an offset array, including:

[0019] The compressed bit stream is divided into multiple compressed bit segments based on the segment length;

[0020] For each of the compressed bit segments, a scan is performed to determine the starting bit position of the first complete compressed code value within it, and the starting bit offset of the compressed bit segment is calculated based on the starting bit position.

[0021] The offset array is constructed based on the multiple starting bit offsets calculated from the multiple compressed bit segments.

[0022] In some embodiments, for each inference unit, decoding the compressed bitstream according to the corresponding offset array to reconstruct multiple linear weight matrices includes:

[0023] Based on the offset array, multiple thread blocks are scheduled to pre-decode multiple compressed bit segments of the inference unit to determine the number of elements after decoding each compressed bit segment; wherein, each thread block includes multiple parallel processing threads, and each parallel processing thread is used to process one compressed bit segment;

[0024] The precise write position of each parallel processing thread is calculated based on the number of elements after decoding each compressed bit segment.

[0025] The corresponding compressed bit segment is re-decoded by the parallel processing thread, and the decoding result is written to the corresponding precise writing position to obtain multiple linear weight matrices.

[0026] In some embodiments, calculating the precise write position of each parallel processing thread based on the number of elements after decoding each compressed bit segment includes:

[0027] Within the same thread block, the prefix sum of the number of elements of all parallel processing threads is calculated to determine the prefix sum value of each parallel processing thread;

[0028] The write offset of each parallel processing thread within the thread block is determined based on the prefix sum value of each parallel processing thread.

[0029] In some embodiments, the parallel processing thread re-decodes the corresponding compressed bit segment and writes the decoding result to the corresponding precise write position to obtain multiple linear weight matrices, including:

[0030] Based on the write offset within the thread block and the preset block-level output start point of the thread block, the precise write position of each parallel processing thread in global memory is determined.

[0031] The parallel processing thread re-decodes the corresponding compressed bit segment and writes the decoding result to the corresponding precise write position to form a decompressed aggregated data block in global memory.

[0032] Based on the boundary information of each of the linear weight matrices recorded before compression within the inference unit, a plurality of the linear weight matrices are recovered from the decompressed aggregated data block.

[0033] To achieve the above objectives, a second aspect of this application provides a reasoning apparatus for a large language model, the apparatus comprising:

[0034] An acquisition module is used to acquire the original large language model and multiple inference units of the original large language model; wherein each inference unit includes multiple linear weight matrices, and each linear weight matrix includes multiple weight parameters;

[0035] A compression module is used to construct a compressed coding tree for each inference unit based on multiple weight parameters of the inference unit, and to compress the weight parameters using the compressed coding tree to generate a compressed bitstream;

[0036] The calculation module is used to divide the compressed bit stream into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, calculate the starting bit offset of the first fully encoded segment to obtain an offset array;

[0037] The decoding module is used to decode the compressed bitstream according to the corresponding offset array for each inference unit in response to the acquisition of the target inference task, and restore multiple linear weight matrices.

[0038] An execution module is used to perform the target inference task using multiple linear weight matrices.

[0039] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0040] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0041] The inference method, apparatus, device, and storage medium for large language models proposed in this application first obtain the original large language model and multiple inference units of the original large language model; wherein, each inference unit includes multiple linear weight matrices, and each linear weight matrix includes multiple weight parameters; for each inference unit, a compressed coding tree is constructed based on the multiple weight parameters of the inference unit, and the weight parameters are compressed through the compressed coding tree to generate a compressed bitstream; the compressed bitstream is divided into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, the starting bit offset of the first complete encoding is calculated to obtain an offset array; in response to obtaining the target inference task, for each inference unit, the compressed bitstream is decoded according to the corresponding offset array to restore multiple linear weight matrices; the target inference task is executed using the multiple linear weight matrices.

[0042] According to the inference method of the large language model provided by the present invention, firstly, the original large language model and its multiple inference units are obtained, providing processing objects for subsequent compression and inference operations; then, a compression coding tree is specifically constructed for the weight parameters within each inference unit. This step, through statistical analysis of the actual distribution of the weight data, ensures that the compression scheme can best fit the distribution characteristics of the data itself, thus laying the algorithmic foundation for achieving a high compression ratio; subsequently, the weight parameters are compressed using this compression coding tree. This step employs lossless encoding based on data redundancy, avoiding a decrease in model output quality and accuracy due to compression; then, while generating the compressed bitstream, it is divided into multiple compressed bit segments and... The offset array is calculated, and by calculating and storing the decoding start point of each processing segment, the variable-length encoding and decoding problem, which was originally difficult to parallelize, is transformed into a highly parallel task, providing the necessary prerequisite for subsequent high-speed decompression. Then, when responding to the inference task, the compressed bitstream is efficiently decoded in parallel according to the offset array, realizing real-time and fast restoration of model weights during inference, overcoming the shortcomings of traditional lossless compression schemes, such as slow decompression speed and unsuitability for inference scenarios. Finally, the target inference task is executed using the restored linear weight matrix, ultimately achieving effective compression and deployment of large language models without sacrificing any inference accuracy, solving the technical problem of difficulty in achieving both compression and accuracy in existing technologies. Attached Figure Description

[0043] Figure 1 This is a flowchart of the reasoning method for the large language model provided in the embodiments of this application;

[0044] Figure 2 This is another flowchart of the reasoning method for the large language model provided in the embodiments of this application;

[0045] Figure 3This is another flowchart of the reasoning method for the large language model provided in the embodiments of this application;

[0046] Figure 4 This is another flowchart of the reasoning method for the large language model provided in the embodiments of this application;

[0047] Figure 5 This is another flowchart of the reasoning method for the large language model provided in the embodiments of this application;

[0048] Figure 6 This is another flowchart of the reasoning method for the large language model provided in the embodiments of this application;

[0049] Figure 7 This is another flowchart of the reasoning method for the large language model provided in the embodiments of this application;

[0050] Figure 8 This is a schematic diagram of the structure of the reasoning device for the large language model provided in the embodiments of this application;

[0051] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0055] Large language models have been widely used in numerous industries, including fintech, due to their powerful natural language processing capabilities. However, the scale of these models is growing at an unprecedented rate, posing a significant challenge to hardware deployment. For example, a large model like Llama-3.1 has 405 billion parameters in 16-bit floating-point format, requiring approximately 810GB of memory for a single inference operation. This far exceeds the capacity of a typical high-end GPU server, making economical and efficient deployment of these models extremely difficult.

[0056] In related technologies, memory consumption is mainly reduced by decreasing the precision of weight values. However, this method leads to a decline in the quality and accuracy of model inference output. In scenarios such as financial risk control and precision medicine, where stringent requirements for model accuracy are necessary, a significant loss of inference performance is unacceptable. Therefore, how to effectively compress model size to reduce deployment costs without sacrificing inference accuracy is a pressing technical challenge in this field.

[0057] Based on this, embodiments of this application provide a reasoning method, apparatus, device, and storage medium for large language models, aiming to achieve efficient compression of large language models while reducing the loss of reasoning performance of large language models.

[0058] The reasoning method, apparatus, device, and storage medium of the large language model provided in this application are specifically described through the following embodiments. First, the reasoning method of the large language model in this application is described.

[0059] The large language model inference method provided in this application is applicable to the fields of fintech and medical services. The large language model inference method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the large language model inference method, but is not limited to the above forms.

[0060] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0061] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0062] Figure 1 This is an optional flowchart of the reasoning method for the large language model provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S105.

[0063] Step S101: Obtain the original large language model and multiple inference units of the original large language model.

[0064] Step S102: For each inference unit, a compressed coding tree is constructed based on multiple weight parameters of the inference unit, and the weight parameters are compressed using the compressed coding tree to generate a compressed bit stream.

[0065] Step S103: Divide the compressed bit stream into multiple compressed bit segments according to the preset segment length, and for each compressed bit segment, calculate the starting bit offset of the first fully encoded segment to obtain the offset array.

[0066] In step S104, in response to obtaining the target inference task, for each inference unit, the compressed bitstream is decoded according to the corresponding offset array to restore multiple linear weight matrices.

[0067] Step S105: Perform the target inference task using multiple linear weight matrices.

[0068] Steps S101 to S105, as illustrated in this embodiment, firstly, obtain the original large language model and its multiple inference units to provide processing objects for subsequent compression and inference operations. Next, a compressed encoding tree is specifically constructed for the weight parameters within each inference unit. This step, through statistical analysis of the actual distribution of the weight data, ensures that the compression scheme best matches the distribution characteristics of the data itself, thus laying the algorithmic foundation for achieving a high compression ratio. Subsequently, the weight parameters are compressed using this compressed encoding tree. This step employs lossless encoding based on data redundancy, avoiding a decrease in model output quality and accuracy due to compression. Then, while generating the compressed bitstream, it is divided into multiple compressed bits. The algorithm calculates the offset array for each processed segment and stores the decoding start point of each segment, transforming the variable-length encoding and decoding problem, which was originally difficult to parallelize, into a highly parallel task, providing the necessary prerequisite for subsequent high-speed decompression. Then, when responding to the inference task, the compressed bitstream is efficiently decoded in parallel according to the offset array, realizing real-time and fast restoration of model weights during inference, overcoming the shortcomings of traditional lossless compression schemes, such as slow decompression speed and unsuitability for inference scenarios. Finally, the target inference task is executed using the restored linear weight matrix, ultimately achieving effective compression and deployment of large language models without sacrificing any inference accuracy, solving the technical problem of difficulty in achieving both compression and accuracy in existing technologies.

[0069] In step S101 of some embodiments, the method first needs to acquire a pre-trained original large language model as the processing object, such as a model with billions of parameters specifically designed for the financial field to analyze financial reports and market announcements. Subsequently, the method logically divides this financial large language model into multiple inference units according to preset organizational rules. An inference unit can be a transformer block, which refers to a set of functionally related weights in the model and is the basic structural unit constituting the large language model. "Weight parameters" refer to the most basic numerical units constituting a neural network, such as a floating-point number in BFloat16 format; a "linear weight matrix" is a two-dimensional array composed of a large number of weight parameters, representing a complete computational layer in the model; and an "inference unit" is a collection of multiple linear weight matrices combined to achieve a specific inference function. For example, in the fintech field, the inference unit of a large language model used for credit risk assessment can be a transformer block responsible for processing customer historical transaction sequences. In the medical service field, the inference unit of a model used for medical image analysis can be a transformer block responsible for extracting high-level features from image data. Each inference unit contains multiple linear weight matrices for different computational tasks, such as query, key, and value weight matrices for the attention mechanism, and weight matrices for the feedforward network layers.

[0070] In step S102 of some embodiments, a parameter compression process is performed independently for each inference unit divided in the previous step. For example, when processing the aforementioned credit risk assessment inference unit, the method traverses all weight parameters of all linear weight matrices and constructs an optimal "compressed coding tree" specifically for the unit based on the statistical distribution characteristics of their values. This "compressed coding tree" is a data structure that implements the entropy coding concept, typically represented as a Huffman tree. Its core principle is to assign shorter binary codes to weight parameter values ​​that appear more frequently in the inference unit, and vice versa. After constructing this tree that highly matches the unit's data characteristics, the method performs a "parameter compression" operation, replacing the original, fixed-length BFloat16 representation with the variable-length binary code corresponding to each weight parameter in the tree. All the replaced codes are then concatenated to form a compressed bitstream that carries all the weight information of the entire inference unit in a lossless manner.

[0071] Please see Figure 2In some embodiments, the weight parameters include weight exponent, weight mantissa, and weight sign. Step S102, which constructs a compressed coding tree for each inference unit based on multiple weight parameters of the inference unit, may include, but is not limited to, steps S201 to S202.

[0072] Step S201: Calculate the frequency of occurrence of each index value in the weighted index.

[0073] Step S202: Assign a corresponding compression code value to each index value according to the frequency of occurrence of each index value to obtain a compression code tree.

[0074] Before implementation, this embodiment is based on an in-depth analysis of the components of the weight parameter. The weight parameter, as a floating-point number (e.g., BFloat16 format), comprises three components: the weight sign, indicating whether the parameter is positive or negative; the weight mantissa, defining the precision or significant digits of the value; and the weight exponent, determining the overall size range or magnitude of the value. This embodiment does not blindly compress the entire floating-point number, but rather performs targeted compression based on Shannon entropy analysis. Analysis revealed that the measured entropies of the 1-bit weight sign and the 7-bit weight mantissa are close to 1 and 7 respectively, meaning their data distribution is nearly random and has almost no potential for lossless compression. In stark contrast, the weight exponent, allocated 8 bits, has a measured entropy of only about 2.6 bits, revealing a huge data redundancy of about 5.4 bits, making it an ideal compression target.

[0075] In some embodiments, the core task of step S201 is to perform precise data distribution statistics. However, the statistical object is not the complete weight parameters, but rather the "weight index" portion identified as having the greatest compression potential. This step iterates through all weight parameters within an inference unit, extracting the corresponding weight index value for each parameter. This step maintains a frequency table or histogram to record how many times each unique index value appears in all weight parameters of the inference unit, i.e., the "occurrence frequency." For example, in an inference unit, the index value 127 may appear tens of thousands of times, while the index value 98 may only appear a few dozen times. The final output of this step is a statistical data set that accurately describes the distribution pattern of the index within the inference unit, providing a quantitative basis for subsequent optimal encoding allocation.

[0076] In some embodiments, the exponent values ​​and their frequencies obtained in step S201 are used as input, and an entropy coding algorithm (typically Huffman coding) is applied to generate the optimal variable-length code. The core idea of ​​the algorithm is to assign a shorter binary "compressed code value" to exponent values ​​with higher "occurrence frequency," and conversely, to exponent values ​​with lower occurrence frequency with a relatively longer code. This allocation process is ultimately materialized into a "compressed code tree" data structure. This tree not only embodies the code allocation logic but also serves as an efficient lookup dictionary: starting from the root node of the tree, traversing left or right according to each bit (0 or 1) of the binary code, the final leaf node corresponds to an original exponent value. Thus, a compressed code tree customized for this inference unit and specifically designed for compressing the weighted exponent is constructed.

[0077] Please see Figure 3 In some embodiments, step S102, which compresses the weight parameters by means of a compressed coding tree to generate a compressed bitstream, may include, but is not limited to, steps S301 to S303.

[0078] Step S301: The weight mantissas and weight signs of multiple weight parameters are concatenated to generate an uncompressed data block for the inference unit.

[0079] Step S302: For each weight parameter, determine the compressed code value corresponding to the weight index as the compressed index based on the compressed coding tree.

[0080] Step S303: Concatenate the compressed exponents of multiple weight parameters to generate a compressed bit stream.

[0081] In step S301 of some embodiments, the portions of the weight parameters that lack compression potential are first processed. Based on the analysis in the previous steps, the information entropy contained in the "weight mantissa" and "weight sign" portions of the weight parameters is close to their bit width, making further lossless compression difficult. Therefore, this embodiment first traverses all weight parameters within the current inference unit, sequentially extracting the bits corresponding to the weight mantissa and weight sign of each parameter, and then directly concatenating these extracted bits in their original order to obtain a continuous, binary "uncompressed data block" that completely and without any modification preserves the sign and precision information of all weight parameters.

[0082] In step S302 of some embodiments, each weight parameter within the inference unit is also traversed, and its "weight index" value is extracted. Then, using this index value as an index, the "compressed coding tree" specifically constructed for this inference unit in a previous step (such as S202) is consulted. By searching in the tree, the unique, optimized variable-length binary "compressed coding value" corresponding to the index value can be determined. This coding value found in the tree is defined here as the "compressed index," which is the new form of the original weight index after lossless compression. This step converts the fixed-length index value into a variable-length, shorter coding representation parameter by parameter.

[0083] In step S303 of some embodiments, the scattered compressed exponents generated in the previous step are aggregated into a complete data stream. First, the compressed exponents corresponding to each parameter are concatenated sequentially according to the original order of the weight parameters in the inference unit. Since the bit length of each compressed exponent is different, this concatenation process requires connecting them end-to-end to form a single, continuous bit sequence. This final sequence, composed of all the compressed exponents, is the complete "compressed bit stream" corresponding to the inference unit.

[0084] Through steps S301 to S303, this embodiment first separates the mantissa and sign portions, which have high information density and lack compression value, in step S301 to form "uncompressed data blocks," thus avoiding wasting computational power on invalid data. Subsequently, steps S302 and S303 concentrate all compression capabilities on the truly redundant weight index portion. By using a customized coding tree for "lookup-style" replacement and splicing, maximum spatial compression is achieved. The originally homogeneous weight parameter stream is deconstructed and recombined into two data blocks: one is the "uncompressed data block" that is preserved as is to ensure 100% accuracy, and the other is the "compressed bitstream" compressed using data statistical patterns. This structured data separation not only achieves a very high overall compression rate but also creates conditions for the subsequent decoder to adopt efficient restoration strategies for different data blocks.

[0085] In step S103 of some embodiments, to address the difficulties posed by variable-length encoding generated by the compressed coding tree to parallel decoding, the entire compressed bitstream can be divided into multiple fixed-size "compressed bit segments" based on a preset segment length (e.g., 64 bytes or 256 bytes) for subsequent allocation to parallel processing threads. However, since the encoding is variable-length, this division is highly likely to truncate the encoding in the middle. Therefore, a "starting bit offset" needs to be calculated for each segment. The "starting bit offset" indicates how many bits a processing thread needs to skip backward from the start of its assigned segment to reach the boundary of the first "complete encoding." A "complete encoding" refers to an undivided binary encoding that can be decoded independently. The offsets of all segments are collected to form an independent, compact "offset array," providing a decoding starting point for all parallel threads.

[0086] Please see Figure 4 In some embodiments, step S103 may include, but is not limited to, steps S401 to S403.

[0087] Step S401: Divide the compressed bit stream into multiple compressed bit segments according to the segment length.

[0088] Step S402: Scan each compressed bit segment to determine the starting bit position of the first complete compressed code value within it, and calculate the starting bit offset of the compressed bit segment based on the starting bit position.

[0089] Step S403: Construct an offset array based on multiple starting bit offsets calculated from multiple compressed bit segments.

[0090] In step S401 of some embodiments, the single, continuous compressed bitstream representing the exponential information of the entire inference unit, generated in the preceding steps, is first structurally segmented. Based on a pre-defined, fixed-size "segment length," such as 64 bytes or 256 bytes (this length is typically optimized based on the parallel processing capabilities of the target hardware, such as the thread bundles or thread blocks of a GPU), the long bitstream is divided into a series of equal-length, continuous, and non-overlapping "compressed bit segments." Essentially, this operation transforms a large, unstructured data stream into a series of standardized work units that are easy to distribute and manage in parallel.

[0091] In step S402 of some embodiments, since the compressed code is variable-length while the segmentation in S401 is fixed-length, the starting position of a segment is likely to be located in the middle of a certain code. The "scanning" process in this step uses the previously constructed compressed code tree to parse the bit sequence starting from the first bit of the segment to identify the first uncrunted, "complete compressed code value" contained within the segment. After determining this complete code, its "starting bit position" within the segment is recorded. Based on this position, the "starting bit offset" of the segment can be calculated, that is, the number of bits that need to be skipped from the beginning of the segment to this effective decoding position.

[0092] In step S403 of some embodiments, a new array, the "offset array," is created. The length of this array is equal to the total number of compressed bit segments. Then, each "starting bit offset" value calculated in step S402 is filled into the corresponding position of this offset array in a one-to-one correspondence. For example, the Nth element of the array stores the starting bit offset of the Nth compressed bit segment. This constructed offset array serves as crucial metadata, stored along with the compressed bitstream, providing essential and efficient lookup functionality for the parallel decoding stage during actual inference.

[0093] Through steps S401 to S403, this embodiment transforms a compressed data stream, inherently dependent on serial decoding due to variable-length encoding, into a data organization form supporting large-scale parallel decoding via a preprocessing flow. Step S401 creates basic units for parallel task distribution through standardized chunking. Step S402 solves the core problem of uncertain decoding start positions for each unit introduced by chunking through content scanning and calculation, calculating a clear decoding start point for each unit. Finally, step S403 materializes these calculation results into a structured, directly queryable "offset array," equipping each independent compressed data segment with the start position information required for its decoding. The existence of this information allows the decoding operation of each segment to be a completely independent task, eliminating decoding dependencies between different segments. This enables hundreds or thousands of decoding threads to start their work directly and simultaneously without waiting or communication, forming the technical foundation for achieving the ultimate goal of lossless decompression matching real-time inference speed.

[0094] In step S104 of some embodiments, when a target inference task is received, such as a loan application requiring immediate approval received by a bank's risk control system, the decoding process is triggered in real time. The method first determines that a credit risk assessment inference unit is needed to execute this task. Then, the decoder retrieves the offset array generated in step S103 corresponding to the inference unit, and distributes multiple compressed bit segments of the inference unit, along with the corresponding starting bit offsets in the offset array, to a large number of parallel processing threads on the GPU. Each thread, upon receiving the task, can directly jump to the correct decoding position of its assigned segment without waiting for other threads and independently begin decoding. This transforms the previously sequential decoding process into a large-scale parallel operation, significantly reducing decoding latency and ultimately quickly reconstructing the multiple linear weight matrices required by the inference unit in memory.

[0095] Please see Figure 5 In some embodiments, step S104 may include, but is not limited to, steps S501 to S503.

[0096] Step S501: Based on the offset array, schedule multiple thread blocks to pre-decode multiple compressed bit segments of the inference unit and determine the number of elements after decoding each compressed bit segment.

[0097] Step S502: Calculate the precise write position of each parallel processing thread based on the number of elements after decoding each compressed bit segment.

[0098] Step S503: The corresponding compressed bit segment is re-decoded by a parallel processing thread, and the decoding result is written to the corresponding precise writing position to obtain multiple linear weight matrices.

[0099] In step S501 of some embodiments, multiple "thread blocks" are scheduled to collaboratively process the decoding task of an inference unit based on the previously generated offset array. Here, a "thread block" is a basic unit in a parallel computing architecture, containing multiple "parallel processing threads" capable of executing in parallel. Each parallel processing thread is assigned a compressed bit segment for processing. This stage of operation is called "pre-decoding," and its key feature is that when a thread executes the decoding logic, its purpose is not to write the decoded data, but only to count the decoding results. That is, each thread completely decodes its assigned segment to accurately determine how many "elements" the segment will ultimately be restored to, for example, how many weight indices.

[0100] In step S502 of some embodiments, the exact number of elements to be produced by each thread is known in the previous step, but it is unclear from which position in the output buffer each thread should start writing to avoid data overwriting or corruption. This step calculates the "precise write position" for each thread based on the "number of elements" information for all threads. This calculation process determines a starting address or index in the global output memory for each thread. The determination of this address takes into account the total amount of data to be written by all other threads preceding this thread, thereby ensuring that the output areas of all threads can be seamlessly and non-overlapping.

[0101] Please see Figure 6 In some embodiments, step S502 may include, but is not limited to, steps S601 to S602.

[0102] Step S601: Within the same thread block, calculate the prefix sum of the number of elements for all parallel processing threads to determine the prefix sum value for each parallel processing thread.

[0103] Step S602: Determine the write offset of each parallel processing thread within the thread block based on the prefix sum value of each parallel processing thread.

[0104] In step S601 of some embodiments, within a "thread block," all "parallel processing threads" work together to perform a "prefix sum calculation" on the sequence composed of the "number of elements" they each counted in step S501. Specifically, the prefix sum calculation includes: receiving a sequence of numbers as input (e.g., the number of elements for each thread [c0, c1, c2, ...]), and generating a new sequence where the new value at each position is the sum of all elements preceding it in the original sequence, i.e., [0, c0, c0+c1, ...], thus allowing each parallel processing thread to obtain its own "prefix sum value."

[0105] In step S602 of some embodiments, for each parallel processing thread, the "prefix sum value" obtained in step S601 is directly determined as its relative write start position within the overall output data block that the current thread block is responsible for, which is the "write offset within the thread block". For example, if a thread calculates a prefix sum value of 256, it means that when finally writing data, it should start writing from the 257th element position (i.e., the position with index 256) of the output buffer of its own thread block.

[0106] In step S503 of some embodiments, each parallel processing thread performs a "re-decoding" operation once. "Re-decoding" means that the thread needs to repeatedly execute the exact same decoding logic as in step S501 to parse its corresponding compressed bit segment. However, the difference is that the purpose of this decoding is to generate the actual decoded data. As each element is decoded, the thread immediately writes it into a contiguous memory space starting from the "precise write position" calculated in step S502. After all parallel processing threads have completed this write operation, the data they have written are correctly concatenated in global memory, together forming multiple linear weight matrices that have been completely restored.

[0107] Please see Figure 7 In some embodiments, step S503 may include, but is not limited to, steps S701 to S703.

[0108] Step S701: Determine the precise write position of each parallel processing thread in global memory based on the write offset within the thread block and the preset block-level output start point of the thread block.

[0109] Step S702: The corresponding compressed bit segment is re-decoded by a parallel processing thread, and the decoding result is written to the corresponding precise write position to form a decompressed aggregated data block in global memory.

[0110] Step S703: Based on the boundary information of each linear weight matrix recorded before compression within the inference unit, recover multiple linear weight matrices from the decompressed aggregated data block.

[0111] In step S701 of some embodiments, the calculation is based on two inputs: first, the local "write offset within the thread block" calculated for each thread in the preceding step (such as S602) within its respective thread block; and second, a "block-level output start point" allocated to the entire thread block. Here, the "block-level output start point" refers to the starting address of the contiguous memory region allocated to the entire thread block for writing all its decoding results. By adding these two values ​​(i.e., global address = block-level start point + block offset), a unique "precise write position" in global memory is determined for each thread, thereby ensuring that the write regions of different thread blocks and different threads within the same thread block do not overlap.

[0112] In step S702 of some embodiments, each "parallel processing thread" performs a "re-decoding" operation, that is, it completely decodes the compressed bit segment it is responsible for again to produce the actual element data. Unlike the pre-decoding stage, this time, whenever an element is decoded, the thread immediately writes it to the "precise write location" in global memory calculated for it in step S701. Since all threads in all thread blocks write data to a pre-planned, contiguous global memory region, when all threads have finished their work, their outputs are seamlessly stitched together, forming a single, contiguous "decompressed aggregate data block" in global memory. At this point, this data block is only a one-dimensional raw data array and has not yet recovered its logical structure as multiple different matrices.

[0113] In step S703 of some embodiments, before initially compressing the inference unit, the size information of each "linear weight matrix" (such as Query, Key, Value matrix, etc.) contained in the unit or its start and end positions in the logical concatenation order have been pre-recorded. This metadata is called "boundary information." This step uses this pre-stored boundary information to logically segment the physically continuous "decompressed aggregated data block" generated in the previous step. It does not move data, but creates pointers or views pointing to different segments inside the aggregated data block based on the boundary information, thereby logically re-establishing each independent linear weight matrix. After this step, the originally single data block is successfully interpreted and restored into multiple structurally correct linear weight matrices that can be directly used by the model.

[0114] Through steps S501 to S503 described above, this embodiment of the application separates the tasks of "determining the output data volume and location" and "actually writing the output data" in time. Step S501 collects the output size information of all parallel tasks without incurring actual memory write overhead. Step S502 uses the size information collected in the first stage to pre-plan the precise position of each thread in the final output target, fundamentally eliminating the address conflict problem that may occur during parallel writing. Finally, step S503 allows each thread to perform decoding and writing operations with a clear target address. Through this "probe planning first, then execute writing" strategy, a lock-free, conflict-free, and highly deterministic parallel data restoration process is achieved, making parallel decoding of variable-length encoding in a shared memory environment both efficient and reliable. This allows full utilization of the computing power of massively parallel processors to meet the stringent performance requirements of real-time inference scenarios.

[0115] In step S105 of some embodiments, after the multiple linear weight matrices (such as Query, Key, Value, and feedforward network matrices) of the credit risk assessment inference unit are completely restored to GPU memory through parallel decoding, the method enters the final inference execution stage. Using these newly restored linear weight matrices, which are numerically identical to the weight parameters in the original large language model, the various information contained in the loan application (such as the applicant's credit history, income verification, etc.) are processed. The model performs a series of calculations, including matrix multiplication, and finally outputs an accurate credit score and corresponding risk probability. Since the entire compression and decompression process is completely lossless, it ensures that the weight parameters used are completely consistent with the uncompressed original model. Therefore, the final output evaluation result also maintains 100% consistency with the original model, meeting the highest fidelity requirements for model output results in fields such as financial risk control.

[0116] Please see Figure 8 This application also provides a reasoning apparatus for a large language model, which can implement the above-mentioned reasoning method for a large language model. The apparatus includes:

[0117] The acquisition module is used to acquire the original large language model and multiple inference units of the original large language model; wherein, each inference unit includes multiple linear weight matrices, and each linear weight matrix includes multiple weight parameters;

[0118] The compression module is used to construct a compressed coding tree for each inference unit based on multiple weight parameters of the inference unit, and to compress the weight parameters through the compressed coding tree to generate a compressed bit stream.

[0119] The calculation module is used to divide the compressed bit stream into multiple compressed bit segments according to the preset segment length, and for each compressed bit segment, calculate the starting bit offset of the first fully encoded bit to obtain the offset array.

[0120] The decoding module is used to decode the compressed bitstream according to the corresponding offset array for each inference unit in response to the acquisition of the target inference task, and restore multiple linear weight matrices.

[0121] The execution module is used to perform the target inference task using multiple linear weight matrices.

[0122] The specific implementation of the reasoning device of this large language model is basically the same as the specific implementation of the reasoning method of the large language model described above, and will not be repeated here.

[0123] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned reasoning method for the large language model. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0124] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0125] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0126] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the large language model inference method of the embodiments of this application.

[0127] The input / output interface 903 is used to implement information input and output;

[0128] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0129] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0130] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0131] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described reasoning method for a large language model.

[0132] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0133] The inference method, apparatus, device, and storage medium for large language models proposed in this application first obtain the original large language model and multiple inference units of the original large language model; wherein, each inference unit includes multiple linear weight matrices, and each linear weight matrix includes multiple weight parameters; for each inference unit, a compressed coding tree is constructed based on the multiple weight parameters of the inference unit, and the weight parameters are compressed through the compressed coding tree to generate a compressed bitstream; the compressed bitstream is divided into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, the starting bit offset of the first complete encoding is calculated to obtain an offset array; in response to obtaining the target inference task, for each inference unit, the compressed bitstream is decoded according to the corresponding offset array to restore multiple linear weight matrices; the target inference task is executed using the multiple linear weight matrices.

[0134] According to the inference method of the large language model provided by the present invention, firstly, the original large language model and its multiple inference units are obtained, providing processing objects for subsequent compression and inference operations; then, a compression coding tree is specifically constructed for the weight parameters within each inference unit. This step, through statistical analysis of the actual distribution of the weight data, ensures that the compression scheme can best fit the distribution characteristics of the data itself, thus laying the algorithmic foundation for achieving a high compression ratio; subsequently, the weight parameters are compressed using this compression coding tree. This step employs lossless encoding based on data redundancy, avoiding a decrease in model output quality and accuracy due to compression; then, while generating the compressed bitstream, it is divided into multiple compressed bit segments and... The offset array is calculated, and by calculating and storing the decoding start point of each processing segment, the variable-length encoding and decoding problem, which was originally difficult to parallelize, is transformed into a highly parallel task, providing the necessary prerequisite for subsequent high-speed decompression. Then, when responding to the inference task, the compressed bitstream is efficiently decoded in parallel according to the offset array, realizing real-time and fast restoration of model weights during inference, overcoming the shortcomings of traditional lossless compression schemes, such as slow decompression speed and unsuitability for inference scenarios. Finally, the target inference task is executed using the restored linear weight matrix, ultimately achieving effective compression and deployment of large language models without sacrificing any inference accuracy, solving the technical problem of difficulty in achieving both compression and accuracy in existing technologies.

[0135] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0136] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0137] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0139] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0140] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0142] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0144] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A reasoning method for a large language model, characterized in that, The method includes: Obtain the original large language model and multiple inference units of the original large language model; wherein each inference unit includes multiple linear weight matrices, and each linear weight matrix includes multiple weight parameters; For each inference unit, a compressed coding tree is constructed based on multiple weight parameters of the inference unit, and the weight parameters are compressed using the compressed coding tree to generate a compressed bitstream; The compressed bitstream is divided into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, the offset of the first fully encoded starting bit is calculated to obtain an offset array. In response to obtaining the target inference task, for each inference unit, the compressed bitstream is decoded according to the corresponding offset array to restore multiple linear weight matrices; The target inference task is performed using multiple linear weight matrices.

2. The reasoning method for a large language model according to claim 1, characterized in that, The weight parameters include a weight exponent, a weight mantissa, and a weight sign. The step of constructing a compressed coding tree for each inference unit based on the multiple weight parameters of the inference unit includes: Calculate the frequency of occurrence of each index value in the weighted index; Based on the frequency of occurrence of each index value, a corresponding compressed coding value is assigned to each index value to obtain the compressed coding tree.

3. The reasoning method for a large language model according to claim 2, characterized in that, The step of compressing the weight parameters using the compressed coding tree to generate a compressed bitstream includes: The weight mantissas and weight signs of multiple weight parameters are concatenated to generate an uncompressed data block for the inference unit; For each of the weight parameters, the compressed code value corresponding to the weight index is determined as the compressed index according to the compressed coding tree; The compressed bitstream is generated by concatenating the compressed exponents of multiple weight parameters.

4. The reasoning method for a large language model according to claim 3, characterized in that, The compressed bitstream is divided into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, the starting bit offset of the first fully encoded segment is calculated to obtain an offset array, including: The compressed bit stream is divided into multiple compressed bit segments based on the segment length; For each of the compressed bit segments, a scan is performed to determine the starting bit position of the first complete compressed code value within it, and the starting bit offset of the compressed bit segment is calculated based on the starting bit position. The offset array is constructed based on the multiple starting bit offsets calculated from the multiple compressed bit segments.

5. The reasoning method for a large language model according to claim 4, characterized in that, For each of the inference units, the compressed bitstream is decoded according to the corresponding offset array to reconstruct multiple linear weight matrices, including: Based on the offset array, multiple thread blocks are scheduled to pre-decode multiple compressed bit segments of the inference unit to determine the number of elements after decoding each compressed bit segment; wherein, each thread block includes multiple parallel processing threads, and each parallel processing thread is used to process one compressed bit segment; The precise write position of each parallel processing thread is calculated based on the number of elements after decoding each compressed bit segment. The corresponding compressed bit segment is re-decoded by the parallel processing thread, and the decoding result is written to the corresponding precise writing position to obtain multiple linear weight matrices.

6. The reasoning method for a large language model according to claim 5, characterized in that, The step of calculating the precise write position of each parallel processing thread based on the number of elements after decoding each compressed bit segment includes: Within the same thread block, the prefix sum of the number of elements of all parallel processing threads is calculated to determine the prefix sum value of each parallel processing thread; The write offset of each parallel processing thread within the thread block is determined based on the prefix sum value of each parallel processing thread.

7. The reasoning method for a large language model according to claim 6, characterized in that, The parallel processing thread re-decodes the corresponding compressed bit segment and writes the decoding result to the corresponding precise writing position to obtain multiple linear weight matrices, including: Based on the write offset within the thread block and the preset block-level output start point of the thread block, the precise write position of each parallel processing thread in global memory is determined. The parallel processing thread re-decodes the corresponding compressed bit segment and writes the decoding result to the corresponding precise write position to form a decompressed aggregated data block in global memory. Based on the boundary information of each of the linear weight matrices recorded before compression within the inference unit, a plurality of the linear weight matrices are recovered from the decompressed aggregated data block.

8. A reasoning device for a large language model, characterized in that, The device includes: An acquisition module is used to acquire the original large language model and multiple inference units of the original large language model; wherein each inference unit includes multiple linear weight matrices, and each linear weight matrix includes multiple weight parameters; A compression module is used to construct a compressed coding tree for each inference unit based on multiple weight parameters of the inference unit, and to compress the weight parameters using the compressed coding tree to generate a compressed bitstream; The calculation module is used to divide the compressed bit stream into multiple compressed bit segments according to a preset segment length, and for each compressed bit segment, calculate the starting bit offset of the first fully encoded segment to obtain an offset array; The decoding module is used to decode the compressed bitstream according to the corresponding offset array for each inference unit in response to the acquisition of the target inference task, and restore multiple linear weight matrices. An execution module is used to perform the target inference task using multiple linear weight matrices.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the reasoning method of the large language model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, characterized in that, when the computer program is executed by a processor, it implements the reasoning method of the large language model according to any one of claims 1 to 7.

Images