Text processing method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

This application provides a text processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. The method includes: dividing the reasoning text generated by a large language model for a target problem into multiple semantic units; converting each semantic unit into a semantic vector, and combining multiple semantic vectors into a one-dimensional semantic vector sequence; extracting local features from the one-dimensional semantic vector sequence to obtain multiple local correlation features; combining the multiple local correlation features into a local feature sequence according to a first direction, and integrating the local feature sequence into a global feature sequence to obtain a global feature sequence; segmenting and fusing the global feature sequence to obtain multiple dimensionality-reduced features, and combining the multiple dimensionality-reduced features into a dimensionality-reduced feature sequence according to a second direction; and inputting the dimensionality-reduced feature sequence into a large language model for text generation to obtain a reasoning result. This application can improve the reasoning efficiency of a large language model.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a text processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In recent years, large language models have made significant progress in the field of natural language processing and have been widely applied to various complex tasks. When processing a target problem, large language models typically perform multi-step logical deductions, generating inference text containing numerous intermediate steps. This inference text encapsulates the complete process of model analysis and problem-solving, serving as a crucial foundation for subsequent generation.

[0003] In related technologies, when large language models perform inferences on complex target problems, the generated inference texts often contain a massive amount of information, resulting in extremely long sequences. When large language models rely on these lengthy inference texts for subsequent text generation, it incurs a significant computational burden and memory consumption, leading to low processing efficiency. Furthermore, because the extremely long sequences contain a large amount of redundant information, large language models struggle to maintain focus on the core logic during processing, easily losing or ignoring crucial inference steps while handling massive sequences. This, in turn, leads to decreased accuracy and slow generation speed of the final inference results used to respond to the target problem. Summary of the Invention

[0004] This application provides a text processing method, apparatus, electronic device, computer-readable storage medium, and computer program product that can improve the reasoning efficiency of large language models.

[0005] The technical solution of this application embodiment is implemented as follows: This application provides a text processing method, the method comprising: Obtain the reasoning text generated by a pre-trained large language model for the target problem, and divide the reasoning text into multiple semantic units; Each of the semantic units is converted into a semantic vector, and based on the order of each of the semantic units in the reasoning text, multiple semantic vectors are combined into a one-dimensional semantic vector sequence; Based on a preset one-dimensional sliding window, the one-dimensional semantic vector sequence is slid along a first direction. Local features are extracted from the semantic vectors covered by the one-dimensional sliding window at each sliding position to obtain multiple local association features. The first direction is the direction from the first position to the last position of the one-dimensional semantic vector sequence. The multiple local associated features are combined according to the first direction to obtain a local feature sequence, and the local feature sequence is then integrated with global features to obtain a global feature sequence. The global feature sequence is segmented and fused along the second direction with a preset step size to obtain multiple dimensionality-reduced features. The multiple dimensionality-reduced features are then combined along the second direction to obtain a dimensionality-reduced feature sequence. The second direction is the direction from the first position to the last position of the global feature sequence. The sequence length of the dimensionality-reduced feature sequence is less than the sequence length of the global feature sequence. The reduced feature sequence is input into the large language model to generate text, thereby obtaining the reasoning result used to respond to the target question.

[0006] This application provides a text processing apparatus, the apparatus comprising: The text segmentation module is used to obtain the reasoning text generated by the pre-trained large language model for the target problem, and to divide the reasoning text into multiple semantic units. A vector combination module is used to convert each of the semantic units into a semantic vector, and to combine multiple semantic vectors into a one-dimensional semantic vector sequence based on the order of each of the semantic units in the reasoning text; The feature extraction module is used to slide a preset one-dimensional sliding window along a first direction on the one-dimensional semantic vector sequence, and to perform local feature extraction on the semantic vector covered by the one-dimensional sliding window at each sliding position to obtain multiple local associated features, wherein the first direction is the direction from the first position to the last position of the one-dimensional semantic vector sequence. The feature combination module is used to combine the multiple local associated features according to the first direction to obtain a local feature sequence, and to perform global feature integration on the local feature sequence to obtain a global feature sequence. The feature dimensionality reduction module is used to segment and fuse the global feature sequence along a second direction with a preset step size to obtain multiple dimensionality reduction features, and combine the multiple dimensionality reduction features according to the second direction to obtain a dimensionality reduction feature sequence, wherein the second direction is the direction from the first position to the last position of the global feature sequence, and the sequence length of the dimensionality reduction feature sequence is less than the sequence length of the global feature sequence. The text generation module is used to input the dimensionality-reduced feature sequence into the large language model to generate text and obtain the reasoning result used to respond to the target question.

[0007] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the text processing method provided in the embodiments of this application.

[0008] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the text processing method provided in this application when executed by a processor.

[0009] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the text processing method provided in this application.

[0010] The embodiments of this application have the following beneficial effects: The reasoning text generated by the large language model for the target problem is divided into multiple semantic units, and each semantic unit is converted into a semantic vector. This transforms unstructured natural language text into continuous vector features that are easy for the model to process. Furthermore, based on the order of each semantic unit in the reasoning text, multiple semantic vectors are combined into a one-dimensional semantic vector sequence, preserving the temporal logical relationship of the original reasoning text. This ensures that the generated one-dimensional semantic vector sequence accurately maps the structural information of the original text, providing a high-quality input foundation for subsequent feature extraction. Based on a preset one-dimensional sliding window, the system slides along the first direction on the one-dimensional semantic vector sequence. Local feature extraction is performed on the semantic vectors covered by the one-dimensional sliding window at each sliding position. The one-dimensional sliding window limits the scope of each feature extraction, allowing the extraction process to focus on adjacent semantic vectors. This accurately extracts the local logical dependencies between adjacent reasoning steps in the one-dimensional semantic vector sequence, and the generated local association features effectively preserve the contextual reasoning details within a short distance. Multiple local related features are combined in the first direction to obtain a local feature sequence. The local feature sequence is then integrated with global features to obtain a global feature sequence. While preserving the order of the first direction, global feature integration can overcome the local limitations of the one-dimensional sliding window and establish long-distance logical connections between different local related features. This allows the extraction of a global feature sequence that represents the main logic of the entire reasoning text, ensuring that core global reasoning information is not lost in the subsequent dimensionality reduction process.

[0011] The global feature sequence is segmented and fused along the second direction with a preset step size to obtain multiple dimensionality-reduced features. By introducing a preset step size for segmented fusion, multiple global features are merged into dimensionality-reduced features, achieving efficient information integration. Subsequently, multiple dimensionality-reduced features are combined along the second direction to obtain a dimensionality-reduced feature sequence. Through the segmented fusion mechanism, the sequence length of the dimensionality-reduced feature sequence is made shorter than the sequence length of the global feature sequence, effectively eliminating redundant information in the reasoning text, achieving a physical reduction in the dimension of the sequence representation, and reducing the amount of data for subsequent processing. The dimensionality-reduced feature sequence is input into a large language model for text generation. Since the input dimensionality-reduced feature sequence retains the logic of local correlation features and global feature sequences while reducing its sequence length, it directly reduces the computational resource consumption and memory consumption of the large language model when performing text generation tasks, improving generation efficiency. At the same time, the concise and shorter dimensionality-reduced feature sequence reduces the interference of irrelevant redundant information, enabling the large language model to utilize key features more stably and focusedly, thereby improving the accuracy of the final reasoning results used to respond to the target question. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the architecture of the text processing system 100 provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the electronic device 500 provided in the embodiments of this application; Figure 3A This is a first flowchart illustrating the text processing method provided in this application embodiment; Figure 3B This is a second flowchart illustrating the text processing method provided in the embodiments of this application; Figure 3C This is a schematic diagram of the third process of the text processing method provided in the embodiments of this application; Figure 3D This is a schematic diagram of the fourth process of the text processing method provided in the embodiments of this application; Figure 3E This is a schematic diagram of the fifth process of the text processing method provided in the embodiments of this application; Figure 3F This is a sixth flowchart illustrating the text processing method provided in the embodiments of this application; Figure 3G This is a schematic diagram of the seventh process of the text processing method provided in the embodiments of this application; Figure 4A This is a schematic diagram illustrating the generation of a one-dimensional semantic vector sequence provided in an embodiment of this application; Figure 4B This is a schematic diagram of local association feature extraction provided in an embodiment of this application; Figure 4C This is a schematic diagram of cross-modal feature processing provided in an embodiment of this application; Figure 4D This is a schematic diagram of the generation of dimensionality reduction feature sequences provided in the embodiments of this application; Figure 5 This is a schematic diagram of the process of obtaining reasoning features after compressing reasoning text, as provided in the embodiments of this application.

[0013] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0016] In the following description, the terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0017] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0018] Unless otherwise specified, "at least one" as used below refers to one or more cases, and "multiple" can refer to two or more cases.

[0019] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0020] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0021] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0022] 1) Human-computer interaction interface, which is an interface used to provide human-computer interaction functions / display reasoning results.

[0023] For example, graphical user interfaces (GUIs) include augmented reality (AR) interfaces, virtual reality (VR) interfaces, voice user interfaces (VUIs), interactive projection interfaces (using projection technology to display information on a flat surface), eye-tracking interfaces (interfaces controlled by detecting the user's gaze), holographic interfaces (three-dimensional holograms formed by projecting images using holographic projection technology, allowing users to see stereoscopic images without wearing special glasses), multimodal interfaces (interfaces that combine multiple interaction methods, such as tactile, visual, and auditory interaction), and brain-machine interfaces (BMIs).

[0024] 2) Inference text refers to the text sequence data representing the logical deduction steps generated by the large language model based on the intermediate hidden layer states before outputting the final result for the target problem. The inference text records in detail the complete multi-step deduction process, including model decomposition conditions, derivation sub-conclusions, and logical chain propagation. As the initial input basis for processing, the inference text is divided into multiple semantic units and transformed into structured continuous features to address the problems of high computational resource consumption and low efficiency caused by excessively long contexts. For example, the intermediate computation text records output by the model when processing multi-step logical deduction and state update tasks constitute this type of intermediate output data.

[0025] 3) The sliding position refers to the sequence index or physical relative coordinate location corresponding to the translation of the preset sliding window within the one-dimensional sequence features. As the one-dimensional sliding window moves forward from the first position to the last position of the sequence, this coordinate point will be continuously updated and changed. The algorithm pauses briefly at each such current coverage state to extract the corresponding local context feature set.

[0026] 4) Segmented fusion refers to the process of dividing continuous high-dimensional sequence data into multiple fixed-length feature subsequences based on a preset truncation length, and then merging each subsequence into a dimensionality-reduced feature with a lower feature dimension than the original sequence using a specific aggregation algorithm. Segmented fusion is performed by alternating between span-based skip-style truncation and dynamic feature aggregation, which can eliminate redundant transition information while preserving the core main logic and key node features of long sequences.

[0027] 5) The inference result refers to the conclusive character sequence generated and output by the large language model after receiving the target input sequence constructed for the target problem, through autoregressive decoding calculations of its internal deep neural network. This sequence directly responds to the task at hand. Unlike the intermediate deduction steps in the generation process, the inference result directly presents the final solution, decision, or judgment for the target problem. By reading high-information-density, dimensionality-reduced feature sequences containing highly condensed context, the large language model can accurately and efficiently output this final conclusion.

[0028] 6) Gated state mapping refers to the process of using small evaluation network units containing nonlinear activation functions to perform algebraic operations and smooth compression on each global feature input, thereby outputting proportional weights reflecting the degree to which the feature information is retained or discarded. Gated state mapping gives the algorithm the ability to adaptively identify feature value, dynamically determining which features belong to the core logic and which belong to redundant noise that should be suppressed. The calculated probability scalar between 0 and 1 is directly used as multiplication coefficients to numerically scale the features. For example, based on the contribution of the global feature to the loss function gradient in the current context, non-negative activation weight values ​​corresponding to the feature scaling or suppression ratio are output.

[0029] 7) Inference accuracy metrics refer to pre-defined standard constraint thresholds for a specific target problem, used to measure the accuracy, logical integrity, or fault tolerance of the final response conclusion of a large language model. Inference accuracy metrics objectively reflect the specific business requirements of the current processing task regarding the rigor of the underlying logic and the degree of detail preservation. In actual dynamic adaptive adjustment mechanisms, inference accuracy metrics are used to deduce the lower limit of the context length required to calculate the large language model, ensuring that the feature compression process does not compromise the accurate expression of the core logic.

[0030] In related technologies, when large language models perform inferences on complex target problems, the generated inference texts often contain a massive amount of information, resulting in extremely long sequences. When large language models rely on these lengthy inference texts for subsequent text generation, it incurs a significant computational burden and memory consumption, leading to low processing efficiency. Furthermore, because the extremely long sequences contain a large amount of redundant information, large language models struggle to maintain focus on the core logic during processing, easily losing or ignoring crucial inference steps while handling massive sequences. This, in turn, leads to decreased accuracy and slow generation speed of the final inference results used to respond to the target problem.

[0031] Based on the above analysis, the applicant found that the text processing methods of related technologies cannot properly reduce the reasoning text, resulting in slow reasoning speed. In order to address the above problem, this application provides a text processing method that can improve the reasoning efficiency of large language models.

[0032] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. The electronic devices provided in the embodiments of this application can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and vehicle terminals, or they can be implemented as servers.

[0033] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the text processing system 100 provided in the embodiments of this application. In order to support a text processing application, the terminal 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0034] Terminal 400 is used to respond to the input operation on the target question in the human-computer interaction interface 410 and send the target question to server 200. Server 200 is used to generate inference text for the target question, and perform dimensionality reduction and compression on the features formed by the transformation of the inference text to obtain a dimensionality reduction feature sequence. Based on the dimensionality reduction feature sequence, it generates an inference result for responding to the target question and returns the inference result to terminal 400 for display on human-computer interaction interface 410.

[0035] Taking a teaching scenario as an example, the terminal responds to input operations for complex physics application problems (target problems) involving multiple kinematic formulas and sends the problem to the server. Upon receiving the physics application problem, the server performs in-depth problem-solving deduction using a large language model, generating extremely lengthy problem-solving steps (reasoning text) including force analysis, multi-step formula derivation, intermediate variable substitution, and calculation. The features generated from these problem-solving steps are then segmented and weighted to reduce their dimensionality, condensing them into a dimensionality-reduced feature sequence containing the core physical logic. Based on this dimensionality-reduced feature sequence, the final correct numerical answer and a brief physical conclusion (reasoning result) are generated and returned to the terminal for display.

[0036] Taking a code development scenario as an example, the terminal responds to input operations based on a business requirement description (target problem) and sends the description to the server. Upon receiving the business requirement description, the server parses it using a large language model, generating a detailed record of the internal thought process (reasoning text) that includes analyzing data structures, designing algorithmic logic flow, considering time and space complexity, and evaluating boundary conditions. The server then performs dimensionality reduction and compression operations, such as segmentation, truncation, and weighted fusion, on the features generated from this thought process record, condensing it into a dimensionality-reduced feature sequence containing the core algorithm architecture. Based on this dimensionality-reduced feature sequence, the server generates a syntactically correct, directly compileable and runnable target code fragment (reasoning result), and returns this target code fragment to the terminal for display.

[0037] Taking a smart after-sales scenario as an example, the terminal responds to an input operation describing a comprehensive fault phenomenon (target problem) in an industrial device at location a, where multiple indicator lights flash alternately and the motor fails to start. This comprehensive fault phenomenon description is then sent to the server. Upon receiving the comprehensive fault phenomenon description, the server uses a large language model to logically investigate the comprehensive fault, generating a long-chain troubleshooting logic process (reasoning text) that involves checking hardware power supply step by step, querying fault tree nodes in conjunction with alarm light combinations, and inferring the possibility of abnormalities in underlying sensors. The features generated from this long-chain troubleshooting logic process are then segmented, truncated, and weighted to reduce the dimensionality, condensing them into a dimensionality-reduced feature sequence pointing to the core fault source. Based on this dimensionality-reduced feature sequence, a definitive conclusion about the fault cause and clear repair operation instructions in three steps are generated (reasoning result), which are then returned to the terminal for display.

[0038] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0039] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 500 provided in the embodiments of this application. Figure 2 The illustrated electronic device 500 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 540.

[0040] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0041] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0042] The memory 550 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 550 may optionally include one or more storage devices physically located away from the processor 510.

[0043] The memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0044] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0045] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 552 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. Presentation module 553 is used to enable the presentation of information (e.g., user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., display screen, speaker, etc.) associated with user interface 530. The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0046] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A text processing device 555 stored in memory 550 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a text segmentation module 5551, a vector combination module 5552, a feature extraction module 5553, a feature combination module 5554, a feature dimensionality reduction module 5555, and a text generation module 5556. These modules are logically connected and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0047] In some embodiments, the terminal or server can implement the text processing method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as a question-and-answer APP; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0048] The text processing method provided in this application will be described in conjunction with exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0049] The text processing method provided in the embodiments of this application will be described below. As mentioned above, the electronic device implementing the text processing method of the embodiments of this application can be a terminal or a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0050] See Figure 3A , Figure 3A This is a first flowchart illustrating the text processing method provided in this application embodiment, with an electronic device as the main body, and combining... Figure 3A The steps shown are explained.

[0051] In step 101, the reasoning text generated by the pre-trained large language model for the target problem is obtained, and the reasoning text is divided into multiple semantic units.

[0052] In some embodiments, the system receives a target question for a task to be solved. The target question refers to text data such as initial input instructions, questions, mathematical problems, or logical task descriptions that require the model to analyze, deduce logically, and provide a final solution. The target question is transmitted as input data through a data interface and fed into a pre-trained large language model. The pre-trained large language model is a deep learning network model trained on a massive text dataset, possessing a large number of network parameters and deep natural language understanding, context analysis, and text generation capabilities. Upon receiving the target question, the pre-trained large language model activates the corresponding network layers internally, employing a multi-head attention mechanism to perform lexical analysis and feature mapping on the words in the target question, constructing an internal hidden layer representation corresponding to the target question. Subsequently, based on this hidden layer representation, the pre-trained large language model uses an autoregressive generation method, predicting and generating the next lexical unit one by one according to the existing contextual conditional probability distribution. In this text generation stage, to solve target questions containing complex logic, the pre-trained large language model is configured not to directly output the final short answer, but instead to output a series of intermediate logical deduction steps. These detailed intermediate text sequences, recording the pre-trained large language model's analysis of problem conditions, breakdown of intermediate steps, derivation of sub-conclusions, and logical chain transmission process, constitute the inference text. Inference text refers to the intermediate output data of the pre-trained large language model, showcasing its internal thought process and logical reasoning before reaching the final conclusion. By establishing a data receiving stream at the output end of the pre-trained large language model, and completely collecting and caching all logical deduction character sequences output by the pre-trained large language model before generating the final answer identifier, the complete inference text generated by the pre-trained large language model for the target problem can be obtained, serving as the input basis for subsequent processing.

[0053] For example, suppose user A inputs a target problem involving complex engineering calculations into a pre-trained large language model: "A project department has three groups: Group A, Group B, and Group C. Group A can complete the project alone in 10 days, Group B can complete it alone in 15 days, and Group C can complete it alone in 20 days. If Group A works for 2 days first, and then Group B and Group C work together, how many more days will it take to complete the project?" After inputting the target problem into a pre-trained large language model, the model performs calculations in the background and outputs a detailed logical derivation process before giving the final number of days required: "Step 1, calculate the daily work efficiency of each engineering team: Team A's work efficiency is 1 / 10, Team B's work efficiency is 1 / 15, and Team C's work efficiency is 1 / 20. Step 2, calculate the workload completed by Team A in 2 days: 2 multiplied by 1 / 10 equals 1 / 5. Step 3, calculate the remaining workload: 1 minus 1 / 5 equals 4 / 5. Step 4, calculate the combined work efficiency of Team B and Team C: 1 / 15 plus 1 / 20 equals 7 / 60. Step 5, calculate the number of days required to complete the remaining workload: 4 / 5 divided by 7 / 60 equals 48 / 7 days." This text output by the pre-trained large language model, from Step 1 to Step 5, completely includes the calculation decomposition, intermediate numerical deduction, and logical chain transmission. This is the reasoning text generated by the pre-trained large language model for the target problem.

[0054] In some embodiments, step 101, "dividing the inference text into multiple semantic units," can be achieved by performing the following processing: scanning the characters of the inference text generated by the large language model for the target problem to obtain multiple delimiters; truncating the inference text based on the multiple delimiters to obtain multiple candidate text segments; performing semantic integrity verification on each candidate text segment to obtain a verification result; and using the verification result to represent multiple semantically complete candidate text segments as multiple semantic units.

[0055] For example, a character-by-character sliding reading method or a regular expression matching engine can be used to scan the character sequence of the reasoning text. Character scanning refers to the process of reading and analyzing each character or character combination in the text sequentially according to its arrangement. During the scanning process, symbols used to divide the text into logical levels are extracted, which are called separators. Separators typically include punctuation marks (such as periods, semicolons, and newlines) defined in a predefined rule base, as well as structured character entities that represent logical order, such as character combinations with sequential enumeration attributes or specific causal / parallel logical transition conjunctions. By matching the internal predefined rule base, these special characters or words are identified, and their position indices in the reasoning text are recorded, resulting in multiple separators. After obtaining multiple separators and their position information in the reasoning text, these positions are used as cutting points to segment the sequence. Fragment truncation refers to the operation of truncating and recombining a continuous text sequence into several discrete subsequence data blocks according to predefined boundary marker features or a maximum data length threshold. Starting from the end position of the previous delimiter and ending at the beginning position of the current delimiter, the character sequence between these two positions is extracted to form an independent text string. This process is repeated for all delimiters, dividing the lengthy, continuous reasoning text into shorter segments, resulting in multiple candidate text fragments. These candidate text fragments are preliminary segments, not yet confirmed to possess independent and complete logic. After obtaining multiple candidate text fragments, their internal logic and language structure are checked one by one. Semantic integrity verification refers to using algorithms to evaluate whether a text fragment contains a complete subject-verb-object structure and can independently express a clear semantic or logical deduction process. Each candidate text fragment is input into a lightweight language model, its internal features are extracted, and the presence of core predicate actions and missing related argument data are checked. Based on preset integrity scoring rules, the fragment is judged to be self-contained. The binary classification judgment value or confidence score output by the judgment mechanism provides a test result reflecting the logical completeness of the fragment. All test results are summarized, and candidate text fragments are then filtered and reorganized based on these results. The process iterates through all candidate text fragments. If a candidate text fragment's verification result indicates semantic completeness, it is directly retained and confirmed as a qualified item. If a candidate text fragment with incomplete semantic representation is encountered, it is concatenated with adjacent fragments to form a new combined text fragment, triggering semantic completeness verification again, until the concatenated text meets the completeness standard. Finally, after filtering and concatenation, the text blocks that can independently carry a complete logical node are identified as multiple semantic units.

[0056] As an example, for reasoning text involving engineering calculations, the following is a character scan: "Step 1, calculate the daily work efficiency of each engineering team: ... Step 2, calculate the workload completed by team A in 2 days: ...". The scanning engine identifies explicit logical connectors such as "Step 1" and "Step 2," as well as punctuation marks "." indicating the end of statements. These words and punctuation marks, along with their corresponding positions, are extracted to obtain multiple delimiters. Based on these extracted delimiters, the entire text is segmented at these points. The originally continuous long string of reasoning is broken into several independent parts. For example, "Step 1, calculate the daily work efficiency of each engineering team: Team A's work efficiency is 1 / 10, Team B's work efficiency is 1 / 15, and Team C's work efficiency is 1 / 20." is extracted as one candidate text segment, and "Step 2, calculate the workload completed by team A in 2 days: 2 multiplied by 1 / 10 equals 1 / 5." is extracted as another candidate text segment, and so on, resulting in multiple candidate text segments. For each candidate text fragment, such as "Step 1, calculate the daily work efficiency of each engineering team: A's work efficiency is 1 / 10, B's work efficiency is 1 / 15, and C's work efficiency is 1 / 20," the process examines the sentence. It identifies that the sentence possesses a complete subject action and its corresponding explanation, with clear and consistent logic, thus outputting a test result representing the semantic completeness of the fragment. Suppose that due to some non-standard output, an abnormal fragment "A's work efficiency is" is segmented. This fragment lacks a numerical entity and has a semantically incomplete loop, thus outputting a test result representing its semantic incompleteness. Based on the test results, the semantically complete fragment "Step 1, calculate the daily work efficiency of each engineering team: A's work efficiency is 1 / 10, B's work efficiency is 1 / 15, and C's work efficiency is 1 / 20" is directly treated as a semantic unit. For the semantically incomplete abnormal fragment "A's work efficiency is," the abnormal fragment is concatenated with the subsequent numerical part to form a complete statement and included in the semantic unit. Finally, the original reasoning text is successfully decomposed into 5 independent semantic units, corresponding to the complete steps from Step 1 to Step 5, completing the semantic unit division.

[0057] This application addresses the technical problem of inaccurate local feature extraction and easy truncation of core context in long text inference sequences due to deep internal logical structures and mixed information, thus affecting the quality of feature transformation. By scanning the inference text to obtain multiple delimiters, and then performing segmentation based on these delimiters to obtain multiple candidate text segments, the physical isolation and preliminary segmentation of lengthy, continuous, and unstructured long text sequences are achieved, providing fine-grained discrete data support for structured modeling. Semantic integrity verification is performed on each candidate text segment to obtain the verification result, and the verification result represents the semantically complete candidate text segments as multiple semantic units, ensuring that each segmented unit maintains independent coherence and self-consistency in linguistics and logical chains. This dual segmentation mechanism based on recognition truncation combined with semantic verification effectively avoids logical breaks and omissions caused by mechanical segmentation, laying a solid and reliable data layer foundation for the subsequent high-quality and accurate transformation of each structural segment into representational features.

[0058] In step 102, each semantic unit is converted into a semantic vector, and multiple semantic vectors are combined into a one-dimensional semantic vector sequence based on the order of each semantic unit in the reasoning text.

[0059] In some embodiments, step 102, "converting each semantic unit into a semantic vector," can be achieved by performing the following processes for each semantic unit: dividing the semantic unit into multiple initial lexical units; performing feature mapping processing on the multiple initial lexical units to obtain multiple initial lexical vectors; generating local position vectors corresponding to each initial lexical vector based on the arrangement order of each initial lexical unit in the semantic unit; performing feature fusion on the initial lexical vector and the local position vector corresponding to the initial lexical vector for each initial lexical vector to obtain a first position fusion vector; and performing feature aggregation on multiple first position fusion vectors to obtain a semantic vector.

[0060] For example, a word segmentation algorithm is used to scan each semantic unit. Lexical segmentation refers to the process of decomposing a continuous natural language string into smaller, discrete sequences of particles with basic linguistic meaning. Based on a pre-constructed sub-word vocabulary (e.g., a vocabulary generated by a byte-pair encoding algorithm), pattern matching is performed on semantic units from left to right. When an entity word, root word, or symbol combination in the vocabulary is matched, it is cut out from the original string to form a series of discrete basic data units, which are the initial lexical units. The initial lexical units represent the lowest-level text symbols that constitute the semantic units. After obtaining a series of discrete initial lexical units, they need to be converted into a numerical form that can be processed by a computer. The cut initial lexical units are traversed, and the unique integer index number corresponding to each initial lexical unit is found in the embedding matrix. Based on the index number, the corresponding row of weight parameters is extracted from the matrix. This fixed-dimensional weight array is the initial lexical unit vector. The initial lexical unit vector encodes the basic semantic attributes of the lexical unit in a multi-dimensional abstract space. Since feature mapping is performed independently, the original sequential information of the text is lost, and positional encoding is required. The order of arrangement refers to the relative position index of the initial word element from left to right within the current semantic unit (e.g., the 1st, 2nd). This index value for each initial word element is obtained and substituted into the trigonometric formulas of sine and cosine, or input into a learnable relative position embedding network. The output is a numerical array with the same dimension as the initial word element vector, which is the local position vector. In mathematical space, the local position vector is specifically used to represent the specific order information of a particular word element within a short text segment. After obtaining the vector representing word meaning and the vector representing position, their information needs to be combined. According to the sequence correspondence, an element-wise addition operation is performed on a given initial word element vector and its dedicated local position vector, or the two are concatenated and then passed through a linear mapping layer to restore the dimension. The new feature array generated by this algebraic operation retains the meaning of the word itself and adds positional information parameters to the semantic unit; this is the first position fusion vector. In the sequence dimension, average pooling is performed on all first-position fusion vectors belonging to the same semantic unit, i.e., the average value across all dimensions is calculated; or a specific global classification marker is artificially added to the beginning of the sequence, and its corresponding output after multi-layer network computation is used as the representative of the entire sentence. The condensed output, which contains the overall basic semantic information, is the semantic vector.

[0061] As an example, process the previously determined first semantic unit: "Step 1, calculate the daily work efficiency of each engineering team: the work efficiency of Team A is 1 / 10, the work efficiency of Team B is 1 / 15, and the work efficiency of Team C is 1 / 20." The tokenization algorithm segments it according to the vocabulary, breaking it down into discrete characters or phrase particles such as "Step", "1", ",", "calculate", "each", "engineering", "team", etc., thus obtaining multiple initial tokens. For the obtained initial token "calculate", the lookup table engine retrieves its corresponding index number and extracts a set of dense floating-point arrays with, for example, 256 dimensions from the embedding matrix. The same lookup and extraction operations are also performed on all the remaining initial tokens such as "1 / 10" and "engineering", thus obtaining multiple initial token vectors. For the "calculate" token arranged in the 5th position, obtain its arrangement order value "5". Using the positional encoding algorithm, a set of 256-dimensional floating-point arrays is calculated based on the value 5. This array specifically identifies the feature of "being in the 5th position", which is the local position vector. For tokens in other positions, the corresponding local position vectors are also generated respectively according to their order values. The 256-dimensional initial token vector representing "calculate" and the 256-dimensional local position vector specifically representing "the 5th position" are added dimension by dimension one by one to merge and generate a comprehensive 256-dimensional array. This new array not only represents the meaning of "calculate", but also accurately represents "calculate at the 5th position at the beginning of the sentence", that is, the first position fusion vector. All the first position fusion vectors transformed from the first semantic unit (including all the fusion features with position information such as "calculate", "engineering", "team", "1 / 10", etc.) are respectively calculated for the mathematical average in the 256 dimensions of the sequence, condensing these more than a dozen variable-length token features into a unified and single 256-dimensional comprehensive feature. This feature is the unique semantic vector representing the overall semantic unit of "calculate the efficiency of each group in Step 1".

[0062] This application addresses the technical problem that discrete text sequences generated by large language models cannot be directly extracted for high-dimensional features, and that conventional basic feature mapping easily loses the logical order of local contexts within the text. By dividing each semantic unit into multiple initial words and performing feature mapping on each initial word to obtain multiple initial word vectors, discrete natural language symbols are transformed into high-dimensional continuous features that the model can directly compute, providing a foundation for subsequent deep feature extraction. Based on the arrangement order of each initial word in the semantic unit, a local position vector corresponding to each initial word vector is generated, and the initial word vector and the corresponding local position vector are fused to obtain a first position fusion vector. This accurately injects temporal position information within the text into the semantic features, enabling the model to precisely perceive the exact order of each initial word within the semantic unit, effectively avoiding the loss of logical order. Feature aggregation is performed on multiple first position fusion vectors to obtain a semantic vector, condensing discrete word-level sequences of varying lengths into continuous feature vectors containing basic semantics, providing a high-quality input foundation for subsequently constructing structured one-dimensional semantic vector sequences.

[0063] In some embodiments, step 102, "combining multiple semantic vectors into a one-dimensional semantic vector sequence based on the order of each semantic unit in the inference text," can be achieved by performing the following processing: concatenating multiple semantic vectors into an initial vector sequence according to the arrangement order of each semantic unit in the inference text; generating a global position vector corresponding to each semantic vector based on the third order of each semantic vector in the initial vector sequence; performing feature fusion on the semantic vector and the global position vector corresponding to the semantic vector for each semantic vector to obtain a second position fusion vector; combining multiple second position fusion vectors into a fusion vector sequence according to the arrangement order; and performing self-attention encoding on the fusion vector sequence to obtain a one-dimensional semantic vector sequence.

[0064] For example, the logical order of each semantic unit within the complete original text is clearly defined, i.e., the arrangement order. Based on this arrangement order, the semantic vectors representing each independent step are spatially rearranged by concatenating them end-to-end or stacking them row-by-row. Through this splicing operation based on the chronological and logical order of occurrence, multiple previously isolated semantic vectors are integrated into a continuous data block, i.e., the initial vector sequence. The initial vector sequence structurally reconstructs the long chain form of the original text. After obtaining the initial vector sequence, the global absolute index of each feature is determined. The third order refers to the absolute rank of each semantic vector within the fully spliced ​​initial vector sequence (e.g., the first, second, etc. in the overall sequence). This third order is used as an input parameter and passed to a preset positional encoding algorithm (such as a mathematical formula containing sine and cosine functions of different frequencies), or it is input into a pre-trained relative position embedding table for lookup operations. Through the above operations, a dense numerical array with the same dimension as the semantic vectors is output, which is the global position vector. The global position vector is specifically used in mathematical space to accurately mark the absolute coordinates of a step within the entire macroscopic reasoning chain. Having obtained both the representation of the step content and the representation of the global coordinates, information combination is required. All semantic vectors are traversed, and the current semantic vector is added element-wise with its corresponding global position vector along the same numerical dimension. Alternatively, these two vectors are concatenated along the feature dimension and then fused and reduced in dimensionality through a linear mapping layer. The resulting new feature array retains the basic meaning of the current inference step while adding global order coordinate information; this is the second position fusion vector. After completing the dual information injection of all features, the macroscopic sequence structure is reconstructed. Based on the initially determined arrangement, the generated second position fusion vectors are placed back into their corresponding sequence slots, thus forming a continuous high-dimensional data matrix with strong positional attributes—the fusion vector sequence. To establish deep dependencies between the steps within the sequence, high-order encoding operations are performed, linearly transforming the fusion vector sequence to generate the query matrix, key matrix, and value matrix. The relevance score between any two features in the sequence is calculated by performing a dot product operation on the query matrix and the key matrix. This score is then converted into attention weights using a normalized exponential function. Finally, these weights are used to perform a weighted summation of the value matrix. Through this encoding method, the originally isolated features are aggregated with contextually relevant information from the entire sequence, resulting in a one-dimensional semantic vector sequence as the output sequence.

[0065] As an example, consider the five semantic vectors previously obtained, representing the calculation processes from step 1 to step 5. Their order is defined as 1, 2, 3, 4, 5. The semantic vector representing step 1 is placed first, followed by the semantic vector representing step 2, and so on. These five independent 256-dimensional feature arrays are concatenated or stacked to form a complete initial vector sequence. For the semantic vector representing step 4 (calculating joint working efficiency), its third order in the entire initial vector sequence is determined to be 4. The value 4 is input into the position encoder, which calculates and generates a set of 256-dimensional feature arrays specifically representing "located in the fourth core step globally" using trigonometric function formulas. This array is the global position vector corresponding to that semantic vector. The 256-dimensional semantic vector representing "calculating joint working efficiency in step 4" and the 256-dimensional global position vector representing "the fourth global position" are added and merged one by one on their corresponding 256 feature channels. The merged result is a new 256-dimensional comprehensive feature array, which is the second position fusion vector containing dual information. The second position fusion vector carrying the position information from step 1 is placed in the first position, the one carrying the information from step 2 in the second position, and so on up to step 5. These five fused feature arrays are arranged sequentially to form a matrix structure with a unified sequence dimension, which is the fusion vector sequence. The fusion vector sequence containing features from the five steps is input into a self-attention network. During computation, the feature representing the "joint efficiency of step 4" automatically acquires a high correlation with the "individual efficiency of step 1" features and absorbs their numerical information to update its own feature representation. After this round of computation where all features interact and update each other, the output high-dimensional feature sequence rich in contextual information is the one-dimensional semantic vector sequence.

[0066] This application addresses the technical problem that isolated local text features lack a global arrangement order and long-distance contextual relationships, failing to accurately reflect the overall macro-level logical deduction chain. Multiple semantic vectors are concatenated into an initial vector sequence based on the arrangement order of each semantic unit in the reasoning text, physically reconstructing the original linear logical framework of the long text at the data structure level. A global position vector is generated based on the third order in the initial vector sequence, and the semantic vector is fused with it to obtain a second position fusion vector. This operation not only injects basic semantics into the features but also accurately anchors the absolute position coordinates of each local step in the long macro-level deduction chain. Multiple second position fusion vectors are combined into a fusion vector sequence according to the arrangement order, reshaping the continuous feature arrangement with spatiotemporal attributes. Self-attention encoding is applied to the fusion vector sequence to obtain a one-dimensional semantic vector sequence, prompting dynamic interaction between any two step features within the sequence, thereby transforming isolated semantic representations into a high-dimensional continuous feature sequence implicitly containing global contextual logical dependencies. Through the above rigorous encoding and fusion mechanism, a structural foundation is laid for subsequent accurate extraction of fine-grained logical relationships between reasoning steps and global compression.

[0067] As an example, see Figure 4A , Figure 4A This is a schematic diagram illustrating the generation of a one-dimensional semantic vector sequence provided in this application embodiment. The input semantic units are segmented into initial words; feature mapping is performed on the initial words to generate initial word vectors, and local position vectors are generated based on their arrangement order; the initial word vectors and local position vectors are fused to obtain a first position fusion vector, which is then aggregated to output a semantic vector. Further, multiple semantic vectors are concatenated into an initial vector sequence, and combined with a third sequence to generate a corresponding global position vector; the semantic vectors and global position vectors are fused again to obtain a second position fusion vector; multiple second position fusion vectors are combined to obtain a fusion vector sequence, and finally, self-attention encoding is performed on the fusion vector sequence to output a one-dimensional semantic vector sequence.

[0068] In step 103, based on a preset one-dimensional sliding window, the window slides along the first direction on the one-dimensional semantic vector sequence. Local features are extracted from the semantic vectors covered by the one-dimensional sliding window at each sliding position to obtain multiple local association features. The first direction is the direction from the first position to the last position of the one-dimensional semantic vector sequence.

[0069] In some embodiments, a one-dimensional sliding window refers to a mathematical data view model with a fixed numerical length that can extract local continuous segments in the sequence dimension. The first direction is defined as a unidirectional vector direction from the first position of the one-dimensional semantic vector sequence (corresponding to the data node with the earliest sequence data generation timestamp or the starting anchor point of the causal logical chain) to the last position (corresponding to the data node when the deductive logical link terminates or the conclusion is finalized). A preset window length for the one-dimensional sliding window is obtained and aligned to the starting point of the one-dimensional semantic vector sequence. At this point, the one-dimensional sliding window will cover a specific number of adjacent semantic vectors at the beginning of the sequence; this coverage state forms the current sliding position. Subsequently, the algorithm drives the one-dimensional sliding window to successively translate along the first direction in the one-dimensional semantic vector sequence according to a fixed unit offset (usually element-by-element translation). Each physical slide causes the semantic vector combination covered by the one-dimensional sliding window to shift and update. Through this continuous sliding operation, a global, ultra-long, continuous one-dimensional semantic vector sequence is dynamically exposed as multiple local continuous subsequence views that change with the sliding position, thus providing a clear scope for accurately focusing and extracting local contextual information at each sliding position.

[0070] For example, the preset length of the one-dimensional sliding window is set to cover three semantic vectors. The first direction is from the feature in step 1 to the feature in step 5. First, the one-dimensional sliding window is placed at the initial sliding position (i.e., sliding position 1), at which point the window precisely covers the semantic vectors representing steps 1, 2, and 3. Next, the one-dimensional sliding window slides backward one unit along the first direction to reach sliding position 2, at which point the target covered by the window is updated to the semantic vectors representing steps 2, 3, and 4. The one-dimensional sliding window slides backward again to sliding position 3, at which point the window covers the semantic vectors representing steps 3, 4, and 5. Through this sliding action, multiple local feature combinations are gradually selected from the global sequence.

[0071] In some embodiments, see Figure 3B , Figure 3B This is a schematic diagram of the second process of the text processing method provided in the embodiments of this application. Figure 3A Step 103, "Extracting local features from the semantic vectors covered by the one-dimensional sliding window at each sliding position to obtain multiple local related features," can be achieved through... Figure 3B Steps 1031 to 1036 are implemented, and the details are explained below.

[0072] In step 1031, based on the sliding window length of the one-dimensional sliding window, a local subsequence corresponding to the sliding position is determined in the one-dimensional semantic vector sequence.

[0073] In some embodiments, a pre-defined window length value for a one-dimensional sliding window is obtained. At the currently reached sliding position, using the semantic vector being processed as a reference point, a fixed number of continuous feature representations are truncated from the surrounding area (e.g., including itself and adjacent features before and after it) according to the sliding window length. A local subsequence refers to a short data block composed of multiple high-dimensional features arranged consecutively and physically adjacent within a one-dimensional long sequence structure. After the truncation operation is completed, the set of continuous features successfully separated from the one-dimensional semantic vector sequence is determined as the local subsequence corresponding to the current sliding position.

[0074] For example, suppose the sliding window length is set to include 3 features, and the current sliding position is at the semantic vector representing "step 2". In the one-dimensional semantic vector sequence, with this position as the center, the features containing steps 1, 2 and 3 are extracted, and the combination of these 3 consecutive features is determined as the local subsequence corresponding to the sliding position.

[0075] In step 1032, query feature mapping is performed on the semantic vectors corresponding to the sliding positions in the one-dimensional semantic vector sequence to obtain the first local query features.

[0076] In some embodiments, the main target feature at the current sliding position is clearly defined, which is the core semantic vector directly aligned with the current sliding position. A linear transformation operation is performed on this target feature using a first learnable weight matrix. Query feature mapping refers to the process of inputting the original high-dimensional features into a linear projection layer and transforming them into a specific attention representation space through matrix multiplication, specifically used to characterize the feature's "actively seeking relevant information in the surrounding environment" attribute. The semantic vector corresponding to the sliding position is multiplied by the first learnable weight matrix, and the resulting new mapping vector is the first local query feature.

[0077] For example, the core objective corresponding to the sliding position is extracted, namely, a 256-dimensional semantic vector representing "Step 2 (calculating the workload completed by Group A in 2 days)". This vector is then input into a query weight matrix of dimension 256×256 for algebraic multiplication. The product results in a new 256-dimensional vector, which is specifically used to express the query requirement of finding supporting evidence in Step 2. This is the first local query feature.

[0078] In step 1033, key feature mapping is performed on the local subsequence to obtain the local key feature sequence, and value feature mapping is performed on the local subsequence to obtain the local value feature sequence.

[0079] In some embodiments, for the previously extracted local subsequences, it is necessary to construct the queried index labels and the actual underlying content. Key feature mapping refers to the process of using a second learnable weight matrix to convert each feature in the sequence into an index attribute representing "what information it contains"; value feature mapping refers to the process of using a third learnable weight matrix to convert features into a representation of their "actual logical numerical expression". Each semantic vector in the local subsequence is traversed and multiplied by both the second and third learnable weight matrices. The resulting set of index vectors forms the local key feature sequence, and the resulting set of content vectors forms the local value feature sequence.

[0080] For example, the local subsequences containing the features from steps 1, 2, and 3 are input in parallel into the key weight matrix network and the value weight matrix network. After matrix transformation, the original local subsequences are separated into two new sets of feature sequences: one set is the local key feature sequence for accepting matching (consisting of 3 new 256-dimensional key vectors), and the other set is the local value feature sequence containing the actual logical details (consisting of 3 new 256-dimensional value vectors).

[0081] In step 1034, the first correlation between the first local query feature and each local key feature in the local key feature sequence is determined, and the multiple first correlations are normalized to obtain multiple local attention weights.

[0082] In some embodiments, a first relevance is determined between a first local query feature and each local key feature in the local key feature sequence. Normalization refers to using a mathematical function (such as the Softmax function) to convert a set of real numbers in an arbitrary range into a probability distribution whose sum is always 1. These first relevance values ​​are then substituted into the normalization formula. First, the relative differences between the values ​​are amplified by exponential operations using the natural constant. Then, each exponential result is divided by the sum of all exponential results, resulting in a series of decimal values ​​between 0 and 1, which represent the multiple local attention weights.

[0083] For example, suppose the first relevance values ​​of the query feature in "Step 2" and the key features of the local subsequences (Steps 1, 2, and 3) are 2.5, 8.0, and 1.2, respectively. Substituting these three unconstrained real numbers into the Softmax function, after exponential amplification and summation / division operations, we obtain a set of probability distributions that sum to 1, such as 0.01, 0.98, and 0.01. This set of values ​​scientifically reflects the proportion of attention received, i.e., multiple local attention weights.

[0084] In some embodiments, step 1034, "determining the first relevance between the first local query feature and each local key feature in the local key feature sequence," can be achieved by performing the following processing: determining the first order of the semantic vector corresponding to the first local query feature in the one-dimensional semantic vector sequence; for each local key feature in the local key feature sequence, determining the second order of the semantic vector corresponding to the local key feature in the one-dimensional semantic vector sequence, and determining the difference between the first order and the second order as the relative positional distance between the first local query feature and the local key feature; determining the feature inner product value between the first local query feature and the local key feature, and determining the first relevance based on the relative positional distance and the feature inner product value.

[0085] For example, the process locates the original base vector upon which the current first local query feature depends, and searches for the absolute rank index of this base vector in the one-dimensional global sequence structure. The first order refers to the absolute integer coordinates of a specific query node arranged from left to right in the complete one-dimensional semantic vector sequence. By searching the position record table of the one-dimensional semantic vector sequence, the absolute index value of the semantic vector corresponding to the current first local query feature is directly extracted, and this integer value is determined as the first order. All queried index tags within the local scope are traversed. For each local key feature contained in the local key feature sequence, its original source node is traced back. The second order refers to the absolute integer coordinates of a specific key node arranged from left to right in the complete one-dimensional semantic vector sequence. For each local key feature within the window, the absolute rank index value of its corresponding base semantic vector is searched in the global position record table, and each of these integer values ​​is determined as the corresponding second order. After obtaining the absolute coordinates of the query node and each key node, mathematical subtraction is performed. Relative positional distance refers to a numerical indicator used to quantify the physical span and relative direction between any two feature nodes in a sequence. The extracted first-order value is subtracted from the corresponding second-order values ​​(i.e., first order minus second order, or vice versa, depending on the coordinate system). Through this simple algebraic subtraction, the absolute coordinates are transformed into offset values ​​centered on the query node, with positive or negative signs or absolute values. This offset value is directly considered as the relative positional distance between the first local query feature and the local key feature. The feature inner product value is a pure algebraic scalar result obtained by multiplying and summing the elements of two high-dimensional vectors in their corresponding dimensions. It is specifically used to measure the pure content similarity between features in the abstract semantic space. Perform a matrix dot product operation on the vector array of the first local query features and the vector array of the local key features to calculate the feature inner product value; construct a learnable relative position bias matrix or lookup table function, take the relative position distance as the index input, and map it to obtain a scalar bias value; add the scalar bias value to the feature inner product value, and through this numerical combination calculation that combines semantic relevance and physical spatial distance, output the first relevance that finally reflects the comprehensive matching degree.

[0086] As an example, the first local query feature is known to be mapped from the core semantic vector representing "Step 2 (calculate the workload completed by Group A in 2 days)". This semantic vector is searched within the one-dimensional semantic vector sequence (containing steps 1 to 5), and its absolute rank index in the sequence is found to be 2. Therefore, the first order corresponding to this semantic vector is determined to be the value 2. For the three local key features contained in the local key feature sequence (derived from the semantic vectors of steps 1, 2, and 3 respectively), their absolute ranks in the complete one-dimensional semantic vector sequence are traced back. The rank index of the key feature from step 1 is found to be 1, the rank index of the key feature from step 2 is 2, and the rank index of the key feature from step 3 is 3. Thus, the second orders corresponding to these three local key features are determined to be 1, 2, and 3 respectively. The first order of the query node is found to be 2, and the second orders of the three key nodes are 1, 2, and 3 respectively. Subtraction operations are performed separately: subtracting the second order 1 from the first order 2 yields a difference of 1; subtracting the second order 2 from the first order 2 yields a difference of 0; and subtracting the second order 3 from the first order 2 yields a difference of -1. The calculated values ​​1, 0, and -1 are used as the relative distances between the first local query feature and these three local key features, respectively. A vector dot product is calculated between the 256-dimensional first local query feature (representing step 2) and the 256-dimensional local key features (representing step 1), resulting in a feature inner product value of 3.5, representing the pure semantic correlation between the two. Next, based on the relative distance of 1 between them, a numerical bias of 0.8 is obtained through table lookup mapping. The feature inner product value of 3.5 is added to the bias value of 0.8, yielding the final result of 4.3. The same dot product is performed on the local key features of steps 2 and 3, and the result is calculated using the bias corresponding to their relative distances (0 and -1). The resulting comprehensive values, such as 4.3, represent the first relevance.

[0087] This application addresses the technical problem that conventional attention mechanisms rely solely on the pure semantic inner product of the feature space for matching when calculating local relevance, lacking explicit awareness of the relative physical distances between elements within the sliding window, resulting in insufficient extraction of local logical temporal sequences. By accurately extracting the first order of the first local query features and the second order of the local key features and calculating their difference, the absolute ranking coordinates of the sequence features are transformed into relative positional distances within the local space. In determining the first relevance, the traditional calculation framework relying solely on feature inner product values ​​is broken. The derived relative positional distance is mapped as a numerical bias depth and integrated into the relevance calculation, enabling the model to explicitly perceive the relative proximity of semantic vectors within the sliding window in terms of physical sequence when extracting local features for similarity weighting. The forced attention mechanism, while focusing on semantic relevance, must also consider the logical order of elements, thereby further improving the ability of the one-dimensional sliding window to extract and model the underlying local logical order.

[0088] In step 1035, based on multiple local attention weights, multiple local value features in the local value feature sequence are fused into an initial local aggregated feature.

[0089] In some embodiments, the calculated local attention weights are used as multiplier parameters to perform a scalar-vector multiplication with the corresponding local value features in the local value feature sequence. This scaling is achieved by proportionally scaling the numerical dimension of each value feature according to its weight. After scaling, all processed feature vectors are summed term-by-term across the same numerical channel. This weighted summation method condenses multiple scattered value features within a local window into a single comprehensive feature representation containing core information of the local context; this is the initial local aggregated feature.

[0090] For example, the resulting weights (0.01, 0.98, 0.01) are multiplied by the corresponding value vectors from steps 1, 2, and 3 in the local value feature sequence, respectively. Since step 2 has the highest weight, its features are largely preserved. The three multiplied vectors are then summed. This results in a new 256-dimensional feature array that incorporates the most relevant information for the current step within the local context; this is the initial local aggregated feature.

[0091] In step 1036, the initial local aggregated features are subjected to one-dimensional convolution to obtain the local associated features corresponding to the sliding position.

[0092] Here, a feature extraction strategy based on multi-scale receptive field parallelism can be adopted. Instead of using a single fixed-size convolutional kernel, a parallel processing unit is constructed containing multiple one-dimensional convolutional kernels of different sizes (e.g., kernel sizes set to 2, 3, and 5 respectively). The initial local aggregated features are simultaneously input into these convolutional computation branches of different scales for nonlinear mapping. After extraction, the convolutional features of different scales output from each branch are concatenated and fused along the channel dimension. This allows the model to simultaneously extract extremely short-distance adjacent lexical transitions and slightly longer-distance phrase-level smooth transitions in a single processing step, enhancing the ability of one-dimensional convolution to characterize complex logical patterns at multiple scales.

[0093] As an example, see Figure 4B , Figure 4BThis is a schematic diagram of local association feature extraction provided in an embodiment of this application. Based on the sliding window length of a one-dimensional sliding window, a local subsequence corresponding to the sliding position is determined in the one-dimensional semantic vector sequence; query feature mapping is performed on the semantic vector corresponding to the sliding position in the one-dimensional semantic vector sequence to obtain the first local query feature; key feature mapping and value feature mapping are performed on the local subsequences respectively to obtain the local key feature sequence and the local value feature sequence in sequence; the first correlation between the first local query feature and the local key feature sequence is determined, where the first correlation can be calculated by the relative position distance and the feature inner product value; the first correlation is normalized to obtain the local attention weight; based on the local attention weight, the local value feature sequence is fused to obtain the initial local aggregated feature; finally, the initial local aggregated feature is subjected to one-dimensional convolution processing to output the local association feature.

[0094] This application addresses the technical problem that long text features derived from large language models, in a one-dimensional sequence arrangement, only contain their own isolated semantics and lack awareness of their adjacent logical context, making it difficult to accurately extract fine-grained logical connections between consecutive reasoning steps. Based on the length of a one-dimensional sliding window to determine local subsequences, without disrupting the global temporal order, the effective boundary for extracting local contextual information is physically defined and limited, avoiding interference from distant, irrelevant noise. Query feature mapping is performed on the semantic vector corresponding to the sliding position, and key-value feature mapping is performed on the local subsequences, reshaping the matching and content representation of target nodes and their surrounding context within a unified high-dimensional attention mathematical space. By normalizing the first relevance, multiple local attention weights are obtained, transforming unbounded similarity values ​​into a strict probability distribution, scientifically measuring the supporting importance of each feature node within the local neighborhood to the current core step. By weighting local value features based on local attention weights and fusing them into initial local aggregated features, the originally isolated features located at the sliding position are successfully integrated with the most critical logical clues in their local context. This achieves efficient evolution of single semantic features into comprehensive features containing neighborhood logical dependencies, and improves the model's ability to extract and model fine-grained relationships between consecutive reasoning steps.

[0095] In some embodiments, step 1036, "performing one-dimensional convolution on the initial local aggregated features to obtain local associated features corresponding to the sliding position," can be achieved by performing the following processing: obtaining initial local aggregated features corresponding to multiple candidate sliding positions adjacent to the sliding position, as historical local aggregated features; performing interval sampling from multiple historical local aggregated features based on a preset interval step size to obtain historical feature subsequences; concatenating the initial local aggregated features with the historical feature subsequences to obtain a fused feature sequence; and performing convolution kernel mapping processing on the fused feature sequence to obtain local associated features corresponding to the sliding position.

[0096] For example, as the algorithm progresses step-by-step along a one-dimensional sequence, a feature cache queue is set up. A candidate sliding position refers to the sequence ranking coordinates of other nodes that are spatially or logically adjacent to the currently processed core node in the one-dimensional semantic vector sequence. The feature cache queue is queried to extract feature arrays that have been calculated for these adjacent coordinates in previous processing steps. This set of feature arrays corresponding to adjacent coordinates extracted from the cache queue is directly identified as historical local aggregation features, used to provide a broader neighborhood structure reference for the processing of the current node. A pre-set interval step size is obtained; the interval step size is an integer parameter used to control the data extraction frequency, representing how many elements in the continuous sequence data are sampled at a time. Based on this interval step size, multiple arranged historical local aggregation feature sets are traversed. Interval sampling refers to breaking the continuous, no-miss reading method and selecting features from the data pool according to a fixed jump distance, for example, skipping several features in the middle and only retaining feature nodes that conform to the interval period. The feature vectors retained after this skip-sampling process are rearranged into a new short sequence structure according to their original order; this structure is the historical feature subsequence. The core feature nodes currently being processed are extracted, and the initial local aggregated features independently calculated at the current sliding position are joined end-to-end or side-by-side with the historical feature subsequences extracted through skip sampling in the sequence arrangement dimension. This generates a macroscopic data matrix that is continuous in physical memory and contains both current focus and historical span information; this is the fused feature sequence. The fused feature sequence is then fed into a one-dimensional neural network filtering layer. Convolutional kernel mapping refers to the mathematical process of using a set of sliding filters with learnable parameters to perform dot product operations on specific local regions of the input matrix and achieving nonlinear transformation through network activation. A one-dimensional convolutional kernel with a size covering the entire length of the concatenated sequence is constructed. The weight matrix of the one-dimensional convolutional kernel is multiplied and added with each numerical channel in the fused feature sequence to uncover hidden high-order combination patterns and nonlinear relationships between the current features and those skip-sampled historical features. After this filtering, condensation, and dimensional transformation, a brand-new high-dimensional feature array with fixed dimensions is output. This array is the local correlation feature corresponding to the current sliding position that finally completes the deep logic modeling.

[0097] As an example, the current sliding position is at the semantic feature of step 4. At this point, the feature cache queue already contains the features of the previously calculated adjacent coordinates. The algorithm extracts the 256-dimensional initial local aggregated features of steps 1, 2, and 3 adjacent to step 4, and stores these three feature vectors in the current working area as multiple historical local aggregated features. Assume the preset interval step size is 2. For the cached historical local aggregated features of steps 1, 2, and 3, extraction jumps backward from the preceding node immediately adjacent to step 4 (i.e., step 3). Features from step 3 are extracted and retained, then the features from step 2 are skipped across the interval step size of 2, and features from step 1 are directly extracted and retained. Combining the extracted features from step 3 and step 1 yields a historical feature subsequence of length 2. The 256-dimensional initial local aggregated features corresponding to the current step 4 are then tightly concatenated with the historical feature subsequence containing the features from steps 3 and 1 along the sequence sorting dimension. This results in a novel feature matrix with a shape of 3 x 256. This continuous matrix, containing the aggregated information from steps 4, 3, and 1, is the fused feature sequence. This 3 x 256 fused feature sequence is then input into a one-dimensional convolutional layer with a size parameter of 3. The filter matrix within the convolutional layer performs dot product summation and nonlinear activation calculations on the three feature vectors from different time steps (steps 4, 3, and 1). Finally, the information from these three vectors is completely integrated, outputting a single, novel 256-dimensional comprehensive feature array, which represents the local correlation feature corresponding to the sliding position in step 4.

[0098] This application addresses the technical problem that one-dimensional convolutional or local self-attention mechanisms have limited receptive fields. Extracting long-distance, step-wise logical patterns requires increasing the number of network layers or kernel size, leading to increased computational parameters and significant memory overhead. By acquiring initial local aggregated features from adjacent candidate sliding positions as historical local aggregated features, and performing interval sampling based on a preset interval step size to obtain historical feature subsequences, a discrete data extraction mechanism with expanded intervals is introduced in the sequence dimension. The current initial local aggregated features are concatenated with these historical feature subsequences to form a fused feature sequence. Finally, convolutional kernel mapping is applied to obtain local association features. Without increasing the size of the underlying convolutional kernels or the number of computational parameters, the model is allowed to skip adjacent ordinary text features and extract longer-distance core logical combination patterns in a leapfrog manner. This effectively expands the receptive field for extracting local logic, resulting in locally associated features with a broader logical perspective and stronger contextual expressive power.

[0099] In some embodiments, before step 103 "sliding along a first direction on a one-dimensional semantic vector sequence based on a preset one-dimensional sliding window", the following processing may be performed: obtaining the original inference data generated by the large language model for the target problem, wherein the original inference data includes inference text; if the original inference data also includes image data, inputting the image data into a pre-trained visual encoding network for feature extraction to obtain a visual feature sequence; combining the visual feature sequence with the one-dimensional semantic vector sequence to form a multimodal feature sequence; using the multimodal feature sequence as a new one-dimensional semantic vector sequence, and proceeding to the step of sliding along a first direction on a one-dimensional semantic vector sequence based on a preset one-dimensional sliding window.

[0100] For example, raw inference data refers to all the initial mixed-format content generated by the large language model when parsing complex instructions, without modality physical stripping. It includes inference text composed of natural language characters, and may also contain non-text elements depending on the question's attributes. The raw binary data stream output by the model is structured and decoded, extracting the text portions and saving them as inference text. Simultaneously, accompanying data in other formats is packaged and collected to obtain the complete raw inference data generated by the large language model for the target question. Internal format detection is performed on the obtained raw inference data. If an image data format header is detected in the data packet, the image data is converted into a standard-resolution three-channel tensor and input into the visual encoding network. Internally, the visual encoding network extracts spatial texture and shape logic between pixels through multi-layer two-dimensional sliding convolution or image block self-attention mechanisms, ultimately flattening and mapping the two-dimensional visual information into a series of fixed-dimensional high-dimensional continuous feature arrays, which is the visual feature sequence obtained after feature extraction. A one-dimensional semantic vector sequence representing the text logic, already generated by the text processing branch, is extracted. To merge information from two different sources, modality embedding vectors are introduced. A visual modality identifier vector is added and fused to each feature of the visual feature sequence, and a text modality identifier vector is added and fused to each feature of the one-dimensional semantic vector sequence. Then, based on the relative time or physical location markers inserted into the original logic of the image data, the visual feature sequence is directly concatenated and inserted into the corresponding positions of the one-dimensional semantic vector sequence along the sequence length dimension, resulting in a multimodal feature sequence. After the multimodal feature fusion and arrangement are completed, it needs to be connected to the subsequent general compression pipeline. The multimodal feature sequence matrix containing visual and textual features is redirected or its variables are overwritten in memory. The variable addresses of the old one-dimensional semantic vector sequence containing only plain text information are directly replaced with this multimodal feature sequence. After the assignment and replacement are completed, this multimodal feature sequence becomes the nominally new one-dimensional semantic vector sequence in the process flow, sliding along the first direction based on a preset one-dimensional sliding window.

[0101] As an example, user A inputs an engineering calculation problem accompanied by a Gantt chart of project progress. During inference, the large language model not only outputs the aforementioned textual processes such as "Step 1...Step 5...", but also internally generates and outputs a heatmap of the engineering allocation as an auxiliary explanation. By intercepting and extracting all output content containing these textual steps and this auxiliary image, the original inference data containing the inference text is obtained. The heatmap image data is detected as an attachment to the original inference data. After resizing, it is input into a pre-trained visual attention network. This network segments the image into multiple small patches and calculates and extracts a 256-dimensional high-dimensional feature array set representing the meaning of the color and line distribution within the chart. This discrete set is the transformed visual feature sequence. A one-dimensional semantic vector sequence containing features of the five steps is obtained, along with a visual feature sequence representing the heatmap generated by the image branch. Based on the dependency of the heatmap appearing in the text after "Step 4", the visual feature sequence is directly inserted between the features of Step 4 and Step 5. The resulting long sequence matrix structure containing textual and image features is the multimodal feature sequence. The matrix concatenated with textual logic and heatmap features is then identified as a new one-dimensional semantic vector sequence. Subsequently, based on a pre-defined one-dimensional sliding window, the window slides backward along the first direction, starting from the first textual feature of the new one-dimensional semantic vector sequence. During this sliding process, the window not only covers the features extracted from the pure text but also slides over the inserted heatmap visual features, smoothly incorporating the entire cross-modal reasoning chain into the subsequent uniform compression processing framework.

[0102] As an example, see Figure 4C , Figure 4C This is a schematic diagram of cross-modal feature processing provided in an embodiment of this application. The original inference data is divided into two branches at the bottom layer: inference text and image data. The inference text undergoes inference text preprocessing and is converted into a one-dimensional semantic vector sequence; concurrently, the image data is input into a pre-trained visual encoding network, which outputs a visual feature sequence. The one-dimensional semantic vector sequence and the visual feature sequence are input into a combination module to generate a unified multimodal feature sequence. Finally, the multimodal feature sequence is converted into a new one-dimensional semantic vector sequence, and under the action of a one-dimensional sliding window, the one-dimensional sliding window is driven to slide along a first direction.

[0103] This application addresses the technical problem that, in multimodal inference scenarios, the mixed-modal data generated by pre-trained large language models cannot be processed by dimensionality reduction algorithms designed only for pure text structures. This leads to the complete discarding or isolation of multimedia logic branches, such as visual ones, during sequence compression, thus disrupting the overall coherence of the multimodal inference chain. By acquiring the original inference data before triggering the one-dimensional sliding extraction process and performing feature extraction using a pre-trained visual encoding network while including image data, the original two-dimensional pixel image data, which could not be uniformly matched, is successfully transformed into a continuous visual feature sequence aligned with the text feature dimension. Next, the visual feature sequence and the one-dimensional semantic vector sequence are concatenated to form a multimodal feature sequence. This hybrid sequence is then used as a new one-dimensional semantic vector sequence to input into subsequent algorithms. This achieves seamless integration of cross-modal features at the underlying data structure level. As a result, the subsequent one-dimensional sliding local extraction and dimensionality reduction integration mechanisms do not require any decoupling and reconstruction of the core network architecture. They can naturally and uniformly handle complex multimodal reasoning scenarios that include non-textual modal data, breaking the limitations of pure text compression. This preserves the cross-modal associations and logical chain support in the image-text hybrid inference, and improves the model's sequence compression capability and environmental generalization under complex multimodal tasks.

[0104] In step 104, multiple local associated features are combined in the first direction to obtain a local feature sequence, and the local feature sequence is then integrated with global features to obtain a global feature sequence.

[0105] In some embodiments, the corresponding feature data are extracted sequentially according to the sliding time sequence of the one-dimensional sliding window when extracting each local associated feature. In the specified tensor concatenation dimension, the first generated local associated feature is placed at the beginning, followed by other locally associated features generated sequentially along the first direction, stacked accordingly. Through this strictly sequential assembly operation, the locally extracted discrete high-dimensional features are restored to a macroscopic linear sequence, thereby generating a continuous new data block that completely preserves the chronological order of the deduction, which is the local feature sequence.

[0106] For example, a one-dimensional sliding window slides across a sequence containing features from five steps, generating three 256-dimensional local correlation features corresponding to sliding positions 1, 2, and 3, respectively. The first direction is defined as the direction from step 1 to subsequent steps. First, the local correlation feature corresponding to sliding position 1 is placed in the first slot of the matrix arrangement. Then, the local correlation feature corresponding to sliding position 2 is placed in the second slot, and finally, the local correlation feature corresponding to sliding position 3 is placed in the third slot. These three independent 256-dimensional feature arrays are arranged and combined strictly according to the above order, merging into a long strip feature matrix of size 3 x 256 dimensions, which is the assembled local feature sequence.

[0107] In some embodiments, see Figure 3C , Figure 3C This is a schematic diagram of the third process of the text processing method provided in the embodiments of this application. Figure 3A Step 104, "Integrating local feature sequences into global features to obtain a global feature sequence," can be achieved through... Figure 3C Steps 1041A to 1045A are implemented, and the details are explained below.

[0108] In step 1041A, query feature mapping is performed on the local feature sequence to obtain the global query feature sequence.

[0109] In some embodiments, a linear transformation operation is performed on the local feature sequence using a global projection weight matrix containing learnable parameters. The local feature sequence is input as a whole into this linear layer, and matrix multiplication traverses each feature node in the sequence, transforming it into an attention query space specifically designed to extract long-range macroscopic dependencies. The global query feature sequence is a continuous tensor composed of multiple query vectors, specifically designed to initiate information matching at the macroscopic level. After this global-dimensional feature space transformation operation, a new set of feature vectors with active addressing properties is output, thus obtaining the global query feature sequence.

[0110] For example, a 3*256 dimensional local feature sequence is extracted based on the sliding window local extraction assembly. This sequence is then input into a 256*256 dimensional global query weight matrix for matrix multiplication. After the operation, the original short sequence matrix is ​​transformed into a new 3*256 dimensional matrix. The features in this new matrix are specifically assigned query requirements attributes for finding global supporting information, thus obtaining the global query feature sequence.

[0111] In step 1042A, key feature mapping is performed on the local feature sequence to obtain the global key feature sequence, and value feature mapping is performed on the local feature sequence to obtain the global value feature sequence.

[0112] In some embodiments, for the same local feature sequence, two sets of independent linear projection operations are performed in parallel to construct the searchable database and content base for subsequent global similarity calculation. Key feature mapping refers to the matrix operation process of converting features into key vector representations for passive matching and retrieval using a second weight matrix; value feature mapping refers to the matrix operation process of converting features into numerical representations carrying actual underlying logical derivations using a third weight matrix. Matrix multiplication of the local feature sequence using the second globally learnable weight matrix generates a set of index attributes representing the global logical information contained within the feature, thus obtaining the global key feature sequence. Simultaneously, matrix multiplication of the local feature sequence using the third globally learnable weight matrix generates a set of content representing the actual macroscopic derivation details contained within the feature, thus obtaining the global value feature sequence.

[0113] For example, the 3*256 dimensional local feature sequence is multiplied and transformed by two other independent weight matrices. The result is two independent sets of 3*256 dimensional matrices: one is a global key feature sequence containing three key vectors used for passively receiving similarity query matching, and the other is a global value feature sequence containing three value vectors with the underlying content of the actual global computation and inference.

[0114] In step 1043A, for each global query feature in the global query feature sequence, the third correlation between the global query feature and each global key feature in the global key feature sequence is determined, and the multiple third correlations are normalized to obtain multiple global attention weights.

[0115] In some embodiments, the global query feature sequence is traversed, and for each global query feature, a vector dot product operation is performed with all global key features contained in the global key feature sequence. Multiple third-order relevance values ​​are substituted into a normalized exponential function for feature scaling transformation, generating a set of probability values ​​between 0 and 1 whose sum equals 1; these are the multiple global attention weights. The global attention weights are used to accurately quantify the effective proportion of the contribution of all other nodes in the sequence to the macroscopic logical information of the current query node.

[0116] For example, the top-ranked global query feature is extracted, and it is multiplied by a pure algebraic dot product with the three global key features in the global key feature sequence. This yields three unrestricted scalar values, such as 5.0, 1.2, and -0.5. These three values ​​represent the third relevance. Then, these three values ​​are substituted into the Softmax normalization formula for a non-linear scaling and smoothing transformation, resulting in a set of probability distributions that sum to 1, such as 0.96, 0.03, and 0.01. These are the multiple global attention weights used for subsequent proportional feature merging.

[0117] In step 1044A, based on multiple global attention weights, each global value feature in the global value feature sequence is fused to obtain a global feature corresponding to the global query feature.

[0118] In some embodiments, the acquired global attention weights are used as scalar multipliers and multiplied and scaled with the corresponding global value features in the global value feature sequence. After weight scaling, all vector features are summed on the same feature dimension channel. Through this weighted summation based on global relevance, the supporting information of the most relevant parts of the macroscopic main logic of the scattered long-chain reasoning process in the sequence is collected, generating a new single high-order feature array containing long-distance macroscopic correlation dependencies. This array is the global feature that precisely corresponds to the current global query feature.

[0119] For example, the three global attention weights of 0.96, 0.03, and 0.01 are multiplied by the corresponding three global feature values. These three proportionally scaled feature vectors are then added together across their respective 256 channels. The summation operation generates a novel and dense 256-dimensional high-order feature array, i.e., the global features. This not only represents the original content of the first local dimension but also incorporates the most important macroscopic logical deductions for this step from a global perspective.

[0120] In step 1045A, multiple global features are combined in the first direction to obtain a global feature sequence.

[0121] In some embodiments, the algorithm traces back to the first direction set by the algorithm environment, which is the physical extension direction from the starting point of the long sequence inference to the ending point of the inference. In the tensor splicing space, the global features calculated corresponding to the earliest query coordinates are preferentially placed in the first arrangement slot. Then, multiple global features subsequently calculated along the first direction are sequentially arranged side by side behind it and continuously spliced ​​in physical memory to assemble a macroscopic continuous data block containing a complete global temporal dimension, which is the final extracted and integrated global feature sequence.

[0122] For example, the 256-dimensional global feature generated for query position 1, including the macroscopic context, is placed at the very beginning of the matrix sequence. Next, the global feature generated for query position 2 is placed in the second position immediately following it. Finally, the global feature for position 3 is placed in the third position. Along the first direction, these three high-order feature arrays, including the deductive logic, are sequentially connected and assembled side-by-side to form a complete macroscopic long-sequence comprehensive matrix framework of size 3 by 256 dimensions, which is the global feature sequence.

[0123] This application addresses the technical problem that local feature extraction strategies can only perceive short-range semantic details within a sliding window, failing to establish direct logical dependencies between two key nodes spanning vast physical distances within a sequence. This leads to the model easily losing macroscopic main logic when processing and compressing long-chain deductive texts containing multi-level nested causal relationships. By performing query feature mapping, key feature mapping, and value feature mapping on local feature sequences respectively, the local feature representations are uniformly projected onto a global attention high-dimensional computational space suitable for macroscopic comparison. The third relevance between global query features and each global key feature is determined and normalized to obtain global attention weights. Under conditions unrestricted by physical spatial distance, the long-distance macroscopic logical dependency strength between any two local features within a sequence is accurately extracted and rigorously quantified. The global features are weighted and fused based on the weight distribution to obtain global features, and then combined into a global feature sequence according to the inherent first direction of the deduction. This breaks the physical short-sightedness limitation caused by local sliding, so that each node, after being finally integrated into the sequence, fully and adaptively extracts the core main line logic dependency that spans the entire ultra-long text deduction process, enhancing the model's ability to characterize the overall global information and converge on extremely long sequence data.

[0124] In some embodiments, see Figure 3D , Figure 3D This is a schematic diagram of the fourth process of the text processing method provided in the embodiments of this application. Figure 3A Step 104, "Integrating local feature sequences into global features to obtain a global feature sequence," can be achieved through... Figure 3D Steps 1041B to 1047B are implemented, and the details are explained below.

[0125] In step 1041B, query feature mapping is performed on the local feature sequence to obtain the global query feature sequence.

[0126] In some embodiments, a global query weight matrix is ​​extracted, and the local feature sequence is input as a whole into a linear operation layer. By performing matrix multiplication, each local feature node in the sequence is traversed, and its feature space is projected and transformed into the attention query space used for macro-level matching. The global query feature sequence is a continuous tensor matrix consisting of multiple projected query vectors arranged consecutively, specifically designed for initiating similarity searching along a macro-length dimension. After this linear space transformation operation, a new set of feature vectors with active addressing retrieval attributes is output, thus obtaining the global query feature sequence.

[0127] For example, a 5*256 dimensional local feature sequence (corresponding to 5 inference steps) generated by extracting and assembling local features is input into a 256*256 dimensional global query weight matrix for algebraic matrix multiplication. After the operation, the original local feature matrix is ​​transformed into a new 5*256 dimensional matrix. All features in this new matrix are assigned the query requirement attribute of seeking global supporting information, thus obtaining the global query feature sequence.

[0128] In step 1042B, key feature mapping is performed on the local feature sequence to obtain the global key feature sequence, and value feature mapping is performed on the local feature sequence to obtain the global value feature sequence.

[0129] In some embodiments, for the same input local feature sequence, two sets of independent linear feature projection operations are initiated in parallel to construct the target retrieval library and underlying content base required for subsequent global relevance calculation. A second globally learnable weight matrix is ​​invoked to perform matrix multiplication on the local feature sequence, transforming the original features into a set of index attributes representing the global logical information they contain, thus obtaining the global key feature sequence. Simultaneously and in parallel, a third globally learnable weight matrix is ​​invoked to perform matrix multiplication on the local feature sequence, transforming the original features into a set of content representing the actual macroscopic derivation details they contain, thus obtaining the global value feature sequence.

[0130] For example, a 5*256 dimensional local feature sequence is multiplied in parallel with two other independent 256*256 dimensional weight matrices. The result is two completely independent sets of 5*256 dimensional high-dimensional matrices: one is a global key feature sequence consisting of five key vectors used for passively receiving similarity query matching scores; the other is a global value feature sequence consisting of five value vectors containing actual global computational deduction logic.

[0131] In step 1043B, the global query feature sequence is divided into multiple query feature blocks.

[0132] In some embodiments, a pre-defined fixed block length parameter is obtained to control the granularity of the segmentation. A sequential traversal is performed along the feature arrangement direction of the global query feature sequence. Whenever a cumulative number of consecutive feature vectors specified by the block length are encountered, a forced physical truncation operation of the tensor data is performed at that position. Through repeated truncation traversal, the original long matrix is ​​decomposed and stripped into multiple non-overlapping, independent sets of small, short matrices, thus successfully obtaining multiple query feature blocks.

[0133] For example, consider a global query feature sequence containing feature vectors corresponding to 5 steps. Set the block length parameter to 2. Perform a segmentation operation from the beginning of the sequence, combining the features representing the query attributes of steps 1 and 2 into the first block, combining the features representing steps 3 and 4 into the second block, and the remaining features representing step 5 into a separate third block. This physical segmentation decomposes the original high-dimensional global query feature sequence of length 5 into 3 independent query feature blocks.

[0134] In step 1044B, the global key feature sequence is divided into multiple key feature blocks, and the global value feature sequence is divided into multiple value feature blocks corresponding to the multiple key feature blocks.

[0135] In some embodiments, for the global key feature sequence and global value feature sequence used for matching the retrieval reference and the underlying logical data base, the same truncation position and boundary segmentation rules as those used to segment the query-side sequence are applied. The same physical segmentation operation is performed at coordinate points corresponding to the length of the global key feature sequence, dividing it into multiple independent subsets of the index retrieval library, thus obtaining multiple key feature blocks. Simultaneously, parallel truncation and stripping are performed on the global value feature sequence at perfectly consistent segmentation boundaries, ensuring that each segment of logical content and its corresponding key feature block achieves a strict parallel match at sequence coordinate points. After this parallel segmentation operation, multiple value feature blocks are finally obtained, each physically aligned with one of the multiple key feature blocks in terms of quantity and arrangement.

[0136] For example, a global key feature sequence and a global value feature sequence, both containing feature vectors from five steps, are extracted and divided using a parallel block segmentation rule of length 2. These two sequences are then segmented separately, successfully obtaining three independent key feature blocks (including index features from steps 1 to 2, 3 to 4, and 5, respectively), and simultaneously, three value feature blocks that are aligned one-to-one with them in sequence order are obtained.

[0137] In step 1045B, for each query feature block, the local relevance matrix between the query feature block and each key feature block is determined, and the local relevance matrix is ​​normalized to obtain the block attention weight matrix.

[0138] In some embodiments, a specific query feature block is extracted, and matrix multiplication is performed one by one between this query feature block and all key feature blocks in the database. At each block collision, all query vectors within the specific query feature block and all key vectors within the specific key feature block undergo dot product and inner product operations, producing a two-dimensional score grid matrix constrained by the block size, thus determining the local relevance matrix. Subsequently, this local relevance matrix is ​​fed into a non-linear exponential scaling and normalization activation function, which smoothly transforms the inner product scores along the extension axis of the corresponding key vector sequence into a non-negative probability distribution whose sum is strictly equal to 1. The two-dimensional grid output after smoothing and scaling is the block attention weight matrix.

[0139] For example, the first query feature block (2x256 dimensions) containing the features from steps 1 and 2 is extracted. A matrix-dot product matching calculation is then performed directly between this first query feature block and the second key feature block (also 2x256 dimensions) containing the features from steps 3 and 4. This produces a 2x2 local relevance matrix recording the similarity scores between the four pairs of features within these two blocks. This 2x2 matrix containing absolute real numbers is then substituted into the Softmax function for probabilistic scaling, with the vertical column sum constrained to 1. The output 2x2 full probability 2D table representing a specific similarity distribution ratio is the block attention weight matrix.

[0140] In step 1046B, based on the block attention weight matrix, the value feature blocks corresponding to multiple key feature blocks are fused to obtain local fused feature blocks, and multiple local fused feature blocks are fused into global features corresponding to the query feature block.

[0141] In some embodiments, the block attention weight matrix is ​​directly multiplied at the block level with specific value feature blocks that maintain a parallel binding mapping relationship with the target key feature block in the preceding matching. The underlying values ​​within the value feature blocks are amplified, reduced, and multi-channel accumulated and mixed according to a set probability distribution ratio. The high-information-density intermediate output result is the local fusion feature block. For the fixed query feature block currently executing the outer loop, after it has completely completed the matching and fusion with all key blocks, it will be collected and converged into a series of independent local fusion feature blocks. On a unified data channel space dimension, the most direct numerical matrix accumulation and summation operation is performed on these multiple converged local fusion feature blocks. By converging and merging all the macroscopic supporting clues extracted from the various block operation branches, a high-order comprehensive tensor that fully integrates the cross-block global correlation support logic is finally formed, which is the global feature precisely corresponding to the current query feature block.

[0142] For example, the 2x2 block attention weight matrix previously calculated for the second block is called, and it is used to dot-multiply the second value feature block containing steps 3 and 4. This operation scales and extracts the value features equally according to the weight probabilities, producing a new 2x256-dimensional local fusion feature block rich in the underlying data of the second block. Through the same block-level dot-multiplication operation, the corresponding local fusion feature blocks for the other 1st and 3rd value blocks are calculated. Finally, the three resulting matrix blocks are directly added and merged across all corresponding feature channels, ultimately aggregating and generating a new, unified 2x256-dimensional comprehensive tensor block that embodies the global macroscopic inference logic of the entire sequence. This block is then established as the global feature exclusive to the first query feature block.

[0143] In step 1047B, the global features corresponding to the multiple query feature blocks are combined in the first direction to obtain a global feature sequence.

[0144] In some embodiments, global features calculated for each of the multiple query feature blocks are obtained. A global feature sequence is a continuous sequence of features formed by concatenating multiple global features containing long-distance logical dependencies in a specific order. The first direction determined in the preceding steps is obtained, i.e., the direction from the first position to the last position of the sequence. Based on the original arrangement order of each query feature block before partitioning, the global features corresponding to each of the multiple query feature blocks are sequentially concatenated along the first direction along the length dimension of the feature sequence. Through the combination operation according to the first direction, the global features calculated for each block are reconnected into a complete continuous matrix, thereby obtaining a global feature sequence with the same length as before partitioning and internally incorporating global contextual relationship logic.

[0145] For example, the 2x256-dimensional global feature output first, belonging to the first query feature block (including steps 1 and 2), is placed at the very front of the long matrix assembly queue. Then, the corresponding global features generated independently from the second query feature block (including steps 3 and 4) and the third query feature block (including step 5) are taken and sequentially concatenated along the extension direction specified in the first direction to form a 5x256-dimensional global feature sequence.

[0146] This application addresses the technical problem that when performing global interactive integration of feature sequences from extremely long reasoning texts, directly calculating the complete global relevance matrix leads to a quadratic increase in computational and memory requirements with increasing sequence length, resulting in exhaustion of computing resources or low processing efficiency. The method involves performing query feature mapping, key feature mapping, and value feature mapping on local feature sequences to obtain global query feature sequences, global key feature sequences, and global value feature sequences, respectively. These long sequences are then divided into multiple query feature blocks, multiple key feature blocks, and corresponding multiple value feature blocks. This physical segmentation reduces the dimensionality of the massive global sequence into multiple smaller feature blocks. For each query feature block, the local relevance matrix between it and each key feature block is determined and normalized to obtain a block attention weight matrix. Based on this block attention weight matrix, the corresponding value feature blocks are fused to obtain local fused feature blocks. Finally, multiple local fused feature blocks are merged into the global feature corresponding to the query feature block. The global features corresponding to multiple query feature blocks are combined in the first direction to obtain a global feature sequence. By calculating the relevance of each block independently and performing local fusion, the massive global relevance matrix is ​​not instantiated completely during the calculation process, thereby reducing the memory activation and computational resource consumption when processing ultra-long inference texts. At the same time, through cross-matching between blocks and final combination assembly, global logical dependencies that span long distances in the sequence are effectively extracted while improving computational efficiency.

[0147] In step 105, the global feature sequence is segmented and fused along the second direction with a preset step size to obtain multiple dimensionality-reduced features. The multiple dimensionality-reduced features are then combined along the second direction to obtain a dimensionality-reduced feature sequence. The second direction is the direction from the first position to the last position of the global feature sequence. The sequence length of the dimensionality-reduced feature sequence is less than the sequence length of the global feature sequence.

[0148] In some embodiments, see Figure 3E , Figure 3E This is a schematic diagram of the fifth process of the text processing method provided in the embodiments of this application. Figure 3A Step 105, "Segmenting and fusing the global feature sequence along the second direction with a preset step size to obtain multiple dimensionality-reduced features," can be achieved through... Figure 3E Steps 1051A to 1053A are implemented, and the details are explained below.

[0149] In step 1051A, along the second direction, the global feature sequence is segmented based on a preset truncation length and a preset step size to obtain multiple first feature subsequences.

[0150] In some embodiments, a pre-set truncation length parameter for controlling the truncation range each time and a preset step size parameter for controlling the sliding span of the truncation window are obtained. A second direction is obtained, i.e., the temporal evolution direction from the first position to the last position of the global feature sequence. Starting from the beginning position of the global feature sequence, a first continuous set of feature vectors is selected according to the truncation length; the truncation starting point is then shifted forward along the second direction according to the preset step size, and a feature set of the same length is selected again. Since the preset step size is usually greater than 1, this jump-like truncation window shifting operation can achieve data span compression. This truncation and shifting operation is repeated until the end of the sequence. The resulting multiple shorter continuous feature sequence segments physically separated from the global sequence are the first feature subsequences.

[0151] For example, consider a global feature sequence containing five combined features (in order, features 1 to 5). Assume the truncation length is set to 2, and the preset step size is 2. The second direction is from feature 1 to feature 5. Starting from the beginning, features 1 and 2 are extracted and combined to form the first first feature subsequence. Moving backward by the preset step size 2 from the starting point, features 3 and 4 are extracted and combined to form the second first feature subsequence. Moving again, the remaining feature 5 forms the third first feature subsequence. Through this step-skipping truncation operation, three independent first feature subsequences are segmented from a long sequence.

[0152] In step 1052A, a one-dimensional convolution process is performed on each first feature subsequence to obtain the convolutional feature sequence corresponding to each first feature subsequence.

[0153] In some embodiments, one-dimensional convolution processing refers to the process of using filters (convolution kernels) with learnable weights to perform sliding dot products and nonlinear activation mappings on the data channels of the input sequence, thereby specifically extracting local dependencies and structural patterns. For each extracted first feature subsequence, it is independently input into a pre-defined one-dimensional convolutional layer. The filter matrix of the convolutional layer slides locally across each feature node within the first feature subsequence, multiplying and summing the low-level numerical features of adjacent nodes using weights. After this nonlinear filtering and mapping transformation, the node features in the original first feature subsequence are transformed into a deep feature matrix that integrates the information of its internal adjacent context, which is the convolutional feature sequence generated for that segment.

[0154] For example, the first feature subsequence containing features 1 and 2 (its shape is a 2x256 matrix) is input into a one-dimensional convolutional network layer. The core parameters of the convolutional layer perform dot product summation and non-linear mapping on features 1 and 2, identifying and solidifying the logical combination pattern between them, and outputting a new 2x256-dimensional comprehensive feature matrix. This newly generated higher-order mapped feature matrix is ​​the convolutional feature sequence specifically generated for the first feature subsequence. Similarly, parallel operations are performed on the other two first feature subsequences, and the same one-dimensional filtering process is applied to obtain their respective dedicated convolutional feature sequences.

[0155] In step 1053A, for each convolutional feature sequence, the convolutional fusion weights corresponding to each convolutional feature in the convolutional feature sequence are determined, and based on the multiple convolutional fusion weights, the multiple convolutional features are fused into a dimensionality-reduced feature corresponding to the first feature subsequence.

[0156] In some embodiments, for each generated convolutional feature sequence, the sequence is input into a lightweight weight evaluation network (e.g., an architecture combining global average pooling and a multilayer perceptron). This evaluation network analyzes the data activation distribution within each convolutional feature, calculates and outputs a set of normalized allocation probability weights, thereby determining the convolutional fusion weights for each convolutional feature in the sequence. Then, using these calculated convolutional fusion weights as multiplication coefficients, numerical multiplication scaling operations are performed with the convolutional features at corresponding coordinates in the convolutional feature sequence. After weight scaling, multiple convolutional features within the sequence are physically superimposed and summed along the same data channel dimension. Through this weighted summation operation, a short sequence containing multiple vectors is completely merged into a single high-order comprehensive feature representation, i.e., a dimensionality-reduced feature.

[0157] For example, consider the first convolutional feature sequence (containing convolutionally transformed features 1' and 2'). The weight evaluation network, after feature parsing, determines that feature 2' contains the core inference and transition result of this segment. Therefore, the convolutional fusion weight assigned to feature 1' is 0.2, and the convolutional fusion weight assigned to feature 2' is 0.8. Based on this set of weights, each numerical dimension of feature 1' is multiplied by 0.2, and each numerical dimension of feature 2' is multiplied by 0.8. Then, these two scaled 256-dimensional feature vectors are summed. The resulting single 256-dimensional high-density feature array replaces the first two features of the original sequence. This process is repeated for all segments to complete the dimensionality reduction extraction. Ultimately, the original long sequence of length 5 is efficiently compressed and condensed into dimensionality-reduced features containing only 3 high-information-density vectors.

[0158] As an example, see Figure 4D , Figure 4D This is a schematic diagram of the generation of dimensionality reduction feature sequences provided in this application embodiment. Along the second direction, the global feature sequence is segmented based on a preset truncation length and a preset stride to obtain multiple parallel first feature subsequences; each first feature subsequence is subjected to one-dimensional convolution processing to obtain a corresponding convolutional feature sequence; for each convolutional feature sequence, it is fused with the input convolutional fusion weights to obtain dimensionality reduction features; finally, the multiple dimensionality reduction features are physically combined along the second direction to output the final dimensionality reduction feature.

[0159] This application addresses the technical problem that directly pooling and reducing the dimensionality of a full-length feature sequence can easily lead to a significant loss of core reasoning logic details, while completely preserving the long sequence cannot effectively reduce the sequence representation length, resulting in extremely high computational overhead and memory overload for subsequent input processing of large language models. By segmenting the global feature sequence along the second direction based on a preset truncation length and a preset step size, multiple first feature subsequences are obtained. By introducing a physical truncation mechanism with a preset step size, a large-span discrete sampling partitioning in the temporal dimension is achieved, establishing a basic physical segmentation structure for subsequent significant sequence length reduction. One-dimensional convolution processing is performed on each first feature subsequence to obtain a convolutional feature sequence. The convolution operation is used to precisely enhance and refine the logical combination patterns of adjacent features within each truncated segment. The convolution fusion weights corresponding to each convolutional feature are determined, and based on these weights, multiple convolutional features are fused into a dimensionality-reduced feature corresponding to the first feature subsequence. By dynamically evaluating the importance of each feature within the truncated segment and performing adaptive weighted summation fusion, the conventional direct truncation or mean discarding is abandoned. By combining span truncation with a preset step size with a dynamic weighted fusion and interleaved execution mechanism based on feature weights, the system achieves accurate filtering and deep aggregation of locally redundant text information while preserving the global core reasoning logic and key node features. This efficiently transforms complex and massive long sequences into dimensionality-reduced features with a small number of features and extremely high information density, reducing the length of the input sequence for subsequent large language models and alleviating the computational load and memory pressure during subsequent text generation.

[0160] In some embodiments, the global feature sequence includes multiple global features, see [link to relevant documentation]. Figure 3F , Figure 3F This is a schematic diagram of the sixth process of the text processing method provided in the embodiments of this application. Figure 3A Step 105, "Segmenting and fusing the global feature sequence along the second direction with a preset step size to obtain multiple dimensionality-reduced features," can be achieved through... Figure 3F Steps 1051B to 1054B are implemented, and the details are explained below.

[0161] In step 1051B, along the second direction, the global feature sequence is segmented based on a preset truncation length and a preset step size to obtain multiple second feature subsequences.

[0162] In some embodiments, a preset truncation length for controlling the range of the truncation window and a preset step size for controlling the sliding span of the truncation window on the sequence are obtained. A second direction is defined, that is, the direction in which the global feature sequence physically extends from the first position to the last position. Starting from the starting coordinates of the global feature sequence, the first continuous set of global features is selected according to the truncation length; then, the starting coordinates of the truncation are translated along the second direction according to the preset step size, and the feature set of the corresponding length is selected again. The translation and selection operations are repeated until the entire global feature sequence is traversed, thereby physically dividing the originally continuous and huge sequence into multiple independent and shorter sets of segments, i.e., obtaining multiple second feature subsequences.

[0163] For example, for a global feature sequence containing 5 comprehensive features (labeled sequentially as features A to E), the truncation length is set to 2, the preset step size is 2, and the second direction is from feature A to feature E. Features A and B are truncated from the first position of the sequence to form the first second feature subsequence; after shifting by the preset step size, features C and D are truncated to form the next second feature subsequence; finally, the remaining feature E forms the third second feature subsequence. Through the truncation operation, three second feature subsequences are extracted from the long sequence of 5 features.

[0164] In step 1052B, for each second feature subsequence, a gating state mapping is performed on each global feature in the second feature subsequence to obtain the gating activation weight corresponding to each global feature.

[0165] In some embodiments, gated state mapping refers to a mapping process that uses a small evaluation unit containing an activation network to perform a nonlinear transformation on the input features, outputting probability weights that reflect the degree to which the feature information is retained or discarded. For each extracted second feature subsequence, the global features contained within it are extracted. These global features are then input one by one into a pre-defined gated mapping network. This network performs algebraic operations on the input features through its internal fully connected layer parameters, and then inputs the results into activation functions such as the Sigmoid function for nonlinear smooth compression mapping. The activation function maps undefined real values ​​to probability distribution scalars strictly between 0 and 1, where the scalar value determines the proportion of information of the corresponding feature that is allowed to "pass" and retained. The set of these probability scalars output after mapping calculation is the gated activation weight corresponding to each global feature.

[0166] For example, consider the first second feature subsequence (containing global feature A and global feature B). The gating network receives both features, performs network parameter calculations and compression / smoothing mapping using the activation function, and evaluates feature B to include the final result of this step, while feature A is merely a transitional condition. Therefore, it outputs a probability scalar of 0.1 for feature A and a probability scalar of 0.9 for feature B. This set of probability scalars corresponding to features A and B respectively represents the calculated gating activation weights.

[0167] In step 1053B, based on the gating activation weights corresponding to each global feature, multiple global features in the second feature subsequence are numerically scaled to obtain multiple gating filtering features.

[0168] In some embodiments, each gated activation weight is used as a product coefficient and multiplied by a vector with the corresponding global feature in the second feature subsequence. Under the control of the gated activation weights, the numerical amplitude of global features judged as redundant information (whose weights approach 0) is significantly weakened and suppressed, while the numerical amplitude of global features judged as core logic (whose weights approach 1) is fully preserved or even relatively amplified. After this round of proportional product control of physical amplitudes, the original set of global features is transformed into a new set of features that suppress noise and highlight key points, thus obtaining multiple gated filtering features.

[0169] For example, extract the previously calculated gating activation weights (0.1 for feature A and 0.9 for feature B). Multiply all 256 numerical dimensions of feature A by 0.1 to significantly suppress and filter its feature strength; multiply all numerical dimensions of feature B by 0.9 to preserve its core features more completely. The resulting array of new controlled features after these two scaling operations replaces the original two gating features.

[0170] In step 1054B, multiple gated screening features are fused into a dimensionality-reduced feature corresponding to the second feature subsequence.

[0171] In some embodiments, for the multiple gated filtering features generated for the current second feature subsequence, matrix or vector addition and summation operations are performed on the corresponding feature value channels. By thoroughly merging and unifying the feature segments within the subsequence that have been filtered to remove redundancy and retain the core main line, a single comprehensive feature array with a sequence length of only 1 is finally output, which covers all high-quality information of the extracted segment. This array is the dimensionality-reduced feature corresponding to the current second feature subsequence after information purification and dimensionality reduction.

[0172] For example, for the first subsequence with the second feature, the two gated features representing features A and B, after scaling and modulation, are added and accumulated across their 256 corresponding numerical channels. This results in a single, entirely new 256-dimensional comprehensive feature array, which refines and aggregates the core deductive logic from the original two features. This single high-density feature is the corresponding dimensionality-reduced feature. After processing all subsequences, the original sequence of length 5 is successfully reduced and compressed into a dense sequence containing only 3 core vectors.

[0173] This application addresses the technical problems of directly truncating and pooling the full-length feature sequence for dimensionality reduction, which can easily lead to the irreversible loss of core reasoning logic details, resulting in the interruption of the coherence of subsequent large language model reasoning text. Furthermore, simple pooling cannot effectively identify and remove low-value redundant information from the sequence. By segmenting the global feature sequence along a second direction based on a preset truncation length and preset step size, multiple second feature subsequences are obtained. This regular segmentation constructs the basic physical intervals for subsequent information extraction and dimensionality reduction, ensuring the local focus of long sequence feature processing. For each second feature subsequence, a gating state mapping is performed on each global feature to obtain a gating activation weight, and multiple gating filtering features are obtained based on the numerical scaling of this weight. By introducing a non-linear gating mechanism, the importance of each global feature within the segment can be accurately mapped numerically. During the numerical scaling stage, low-weight redundant pseudo-features are forcibly suppressed and weakened, while high-weight core logical features are fully highlighted and preserved, successfully achieving accurate filtering of redundant information. Finally, multiple gated filtering features are directly fused to obtain dimensionality-reduced features. This physically reduces the length of the feature sequence while precisely ensuring that core logical features are selectively and highly fused and preserved. Through the above-mentioned gated filtering and segmented fusion mechanism, high compression ratio sequence dimensionality reduction is achieved while improving the information purity and logical density of the output dimensionality-reduced features.

[0174] In some embodiments, the global feature sequence includes multiple global features, see [link to relevant documentation]. Figure 3G , Figure 3G This is a schematic diagram of the seventh process of the text processing method provided in the embodiments of this application. Figure 3A Step 105, "Segmenting and fusing the global feature sequence along the second direction with a preset step size to obtain multiple dimensionality-reduced features," can be achieved through... Figure 3G Steps 1051C to 1054C are implemented, and the details are explained below.

[0175] In step 1051C, along the second direction, the global feature sequence is segmented based on a preset truncation length and a preset step size to obtain multiple third feature subsequences.

[0176] In some embodiments, a preset truncation length for defining local feature boundaries and a preset step size for determining the translation span of the truncation window are obtained. Starting from the beginning coordinates of the global feature sequence, a continuous set of feature tensors is extracted according to the truncation length. Then, the starting coordinates of the truncation are pushed along the second direction according to the preset step size, and a set of feature tensors of the corresponding length is extracted again. This translation and extraction operation is repeated until the end of the sequence, thereby cutting the originally continuous macroscopic long sequence into multiple independent local short sequence matrices, which are the multiple third feature subsequences.

[0177] For example, consider a global feature sequence containing 5 global features (labeled as global features A to global features E). Assume a preset truncation length of 2, a preset step size of 2, and a second direction from global feature A to global feature E. Starting from the first point of the global feature sequence, truncate global features A and B to form the first third feature subsequence; then, shift the starting point backward by the preset step size to truncate global features C and D to form the next third feature subsequence; finally, truncate the remaining global feature E separately to form the third third feature subsequence. Through this step-skip segmentation operation, three independent third feature subsequences are extracted from the global feature sequence.

[0178] Here, for each third feature subsequence, steps 1052C to 1054C are performed.

[0179] In step 1052C, feature mapping is performed on the third feature subsequence to obtain the second local query feature corresponding to the third feature subsequence.

[0180] In some embodiments, for each acquired third feature subsequence, the third feature subsequence is input into a preset mapping network. The mapping network first performs an averaging operation on each global feature within the third feature subsequence to achieve feature dimensionality reduction, obtaining a feature representation that characterizes the macroscopic contour of the third feature subsequence; then, the feature representation is transformed by matrix projection through a fully connected linear transformation network to obtain the second local query feature corresponding to the third feature subsequence.

[0181] For example, the first third feature subsequence is extracted (containing global feature A and global feature B). These two global features are input into a mapping network. The network first calculates the mean of global features A and B to condense them into a single feature, and then uses matrix multiplication to linearly project it to obtain a single feature vector. This feature vector is specifically responsible for initiating a retrieval and matching request within the current third feature subsequence, which is the second local query feature.

[0182] In step 1053C, the second relevance between the second local query feature and each global feature in the third feature subsequence is determined, and the multiple second relevances are normalized to obtain multiple aggregation and fusion weights.

[0183] In some embodiments, for the currently processed third feature subsequence, the generated second local query feature is subjected to a vector dot product operation with each global feature in the third feature subsequence. The series of numerical scores reflecting the strength of the match, obtained by summing the dot products, is the second relevance. The calculated multiple second relevances are substituted into an exponential normalization function. Normalization refers to using an exponential function to amplify the numerical differences and performing summation division to scale, transforming a set of numerical values ​​of any range into a probability distribution with a constant sum of 1. After normalization, a set of probability values ​​is output where the sum of each value is strictly equal to 1. This set of probability values, indicating the fusion ratio of each global feature in the third feature subsequence, is the multiple aggregation fusion weights.

[0184] For example, the second local query feature representing the query attribute of the third feature subsequence is multiplied by the vector inner product of the original global feature A and global feature B within the third feature subsequence. The absolute relevance scores between them are calculated (e.g., 0.5 for feature A and 2.0 for feature B), and these two scores are the second relevance scores. Next, these two second relevance scores are normalized and probabilistically transformed. The transformed output shows the probability percentage for global feature A, for example, 0.1, and the probability percentage for global feature B, for example, 0.9. This set of probability distributions constitutes the multiple aggregation and fusion weights.

[0185] In step 1054C, based on multiple aggregation and fusion weights, multiple global features in the third feature subsequence are fused into a dimensionality-reduced feature corresponding to the third feature subsequence.

[0186] In some embodiments, multiple calculated aggregation and fusion weights are obtained, and these weights are used as multiplication coefficients for numerical scaling. A matrix multiplication scaling operation is then performed with the corresponding global features in the third feature subsequence. Depending on the magnitude of the aggregation and fusion weights, the numerical amplitude of each global feature within the third feature subsequence is proportionally amplified or reduced. After numerical scaling and modulation, the multiple global features within the third feature subsequence are summed and added together along the same feature channel's data dimension. Through this weighted fusion operation, which dynamically scales and sums using aggregation and fusion weights, the original third feature subsequence containing multiple global features is highly compressed into a single feature vector, which is the dimensionality-reduced feature corresponding to the third feature subsequence.

[0187] For example, the extracted aggregation and fusion weights (0.1 for global feature A and 0.9 for global feature B) are used as multipliers to dot-multiply the global features A and B within the third feature subsequence. After product scaling, direct addition and merging are performed at the corresponding numerical channel positions to obtain the dimensionality-reduced features. This process is repeated for all three third feature subsequences, transforming the original sequence containing five global features into a new sequence with only three dimensionality-reduced features.

[0188] This application addresses the technical problem that when using simple truncation pooling and other methods to segment and reduce the dimensionality of long-chain feature sequences, there is a lack of adaptive judgment capability for feature extraction. This leads to indiscriminate merging and discarding based on fixed mean or maximum values, easily resulting in the omission of crucial logical features that play a decisive role in the sequence. By segmenting the global feature sequence along a second direction based on a preset truncation length and preset step size, multiple third feature subsequences are obtained, establishing a basic segmentation structure for physically reducing the dimensionality of long sequences. For each third feature subsequence, feature mapping is performed to obtain a second local query feature. The second relevance between this second local query feature and each global feature within the third feature subsequence is determined, and multiple second relevances are normalized to obtain multiple aggregation and fusion weights. A higher-order feature selection logic based on feature mapping and query matching is introduced, endowing the feature fusion process with adaptive recognition and dynamic perception judgment capabilities. Finally, based on multiple aggregation and fusion weights, multiple global features in the third feature subsequence are fused into dimensionality-reduced features corresponding to the third feature subsequence. Within each divided third feature subsequence, the hidden core features can be adaptively found and purified with a large-scale weighting. While preserving the macro-logical clues, efficient dimensionality reduction and compression of global feature sequence data is achieved.

[0189] In some embodiments, before step 105 "segmenting and fusing the global feature sequence along the second direction with a preset step size to obtain multiple dimensionality-reduced features", the following processing may be performed: determining the upper limit of the context length for the target problem based on the preset memory capacity and preset inference latency upper limit of the large language model in the current operating environment; determining the lower limit of the context length for inference of the large language model based on the preset inference accuracy index for the target problem; determining the target sequence length based on the upper and lower length limits; determining the ratio of the original sequence length to the target sequence length of the global feature sequence as the sequence compression ratio; and determining the preset step size for segmenting and fusing the global feature sequence based on the sequence compression ratio.

[0190] For example, obtain the relevant resource parameters of the hardware environment where the large language model is running. The preset memory capacity refers to the maximum physical threshold of available video memory or memory space allocated when running the large language model. The preset inference latency limit refers to the maximum allowed time delay threshold from the input target question to the generated result. The context length refers to the total number of sequence feature nodes that the large language model can process in a single inference. The length limit refers to the maximum number of feature nodes that can be accommodated without causing memory overflow or timeout. Obtain the preset memory capacity value currently allocated to the large language model, and the preset inference latency limit in the task configuration for this response. Using a pre-established memory evaluation model, calculate the maximum number of feature nodes that can be accommodated under the preset memory capacity constraint; using a time cost evaluation formula, calculate the maximum number of feature nodes that can be processed within the preset inference latency limit constraint. Compare these two calculated maximum feature node numbers and take the minimum value; this minimum value is the determined context length limit for the target question. Analyze the task type and difficulty of the currently input target question. The preset inference accuracy index refers to the standard threshold that requires the large language model output results to achieve a specific accuracy, losslessness, or fault tolerance rate. The lower bound of length refers to the minimum number of feature nodes that must be retained to ensure logical coherence and meet accuracy requirements. A pre-defined inference accuracy index for the target problem is obtained. This index is then substituted into a pre-configured accuracy loss evaluation function for backpropagation calculation. This calculates the minimum number of basic feature nodes required to maintain and achieve the pre-defined inference accuracy index after sequence feature dimensionality reduction compression. This calculated number of basic feature nodes is determined as the lower bound of the context length for the large language model's inference. Within the defined upper and lower bound length range, interpolation is performed based on a set bias weight coefficient, which determines whether the result prioritizes computational efficiency or retains logical redundancy. An integer value within this range is obtained through mathematical linear interpolation and is determined as the target sequence length to guide the overall compression intensity. The global feature sequence before segmentation and fusion is read. The original sequence length refers to the total number of feature nodes in the global feature sequence before feature dimensionality reduction. The total number of feature nodes contained within the global feature sequence is calculated and determined as the original sequence length. Using the determined original sequence length as the dividend and the target sequence length as the divisor, an algebraic division operation is performed. The resulting quotient, reflecting the ratio between the two, is determined as the sequence compression ratio. The preset step size refers to the number of feature nodes that slide along the feature sequence each time during the segmentation and fusion truncation operation on the global feature sequence.Using a numerical mapping function or a direct algebraic rounding algorithm, the floating-point sequence compression ratio is converted into a discrete integer step size. The sequence compression ratio is then rounded down or up. The integer value obtained from the final rounding conversion, which guides the truncation sliding span, is determined as the preset step size for segmenting and fusing the global feature sequence.

[0191] As an example, the current environment for running a large language model is evaluated. The preset memory capacity allows for a maximum storage of 2000 feature nodes, while the preset inference latency limit, after conversion, allows for a maximum processing of 1500 feature nodes. Taking the minimum of these two values, the upper limit of the context length for the target problem in this project is determined to be 1500. For the target problem in this project, complete numerical accuracy is required; therefore, the preset inference accuracy metric is set to high-precision computation level. Substituting high-precision computation level into the evaluation function, it is derived that the input sequence needs to retain at least 500 core feature nodes. Therefore, the lower limit of the context length for large language model inference is determined to be 500. Currently, bias weights requiring high inference efficiency are interpolated. Values ​​with bias efficiency within this range are selected, determining the target sequence length to be 1000. The generated global feature sequence containing multiple global features is read, and it is found to contain 3000 feature nodes. The original sequence length of the global feature sequence is determined to be 3000. Dividing the original sequence length of 3000 by the determined target sequence length of 1000, the quotient is calculated to be 3. This value of 3 is then used as the sequence compression ratio. Using an algebraic rounding algorithm, it is determined that the sliding window needs to slide every 3 feature nodes during subsequent segmentation and fusion. Therefore, the value of 3 is set as the preset step size for segmentation and fusion of the global feature sequence.

[0192] This application addresses the technical problem that when using segmented fusion to physically reduce the dimensionality of feature sequences, the sliding truncation span is often fixed and cannot be adaptively adjusted according to the dynamic physical resource constraints of different hardware and software operating environments and the actual needs of different task accuracy. This can easily lead to over-compression causing loss of logical features, or under-compression causing memory overflow. Based on the preset memory capacity and preset inference latency upper limit of the large language model in the current operating environment, the upper limit of the context length for the target problem to ensure the stable operation of the model is strictly calculated and defined; at the same time, based on the preset inference accuracy index for the target problem, the lower limit of the context length to ensure the accuracy of the inference logic is derived. The target sequence length is determined based on the upper and lower length limits, and the sequence compression ratio is calculated by the ratio of the original sequence length to the target sequence length of the global feature sequence. Finally, the preset step size for segmented fusion of the global feature sequence is accurately determined based on the sequence compression ratio. It can intelligently derive the optimal sequence compression ratio and preset step size according to hardware bottlenecks and actual inference task requirements. This ensures that the output sequence after segmented fusion does not exceed the limits of hardware and time, while maximizing the preservation of the accuracy baseline of the underlying inference logic features. Thus, it realizes the intelligent adaptive elastic adjustment of the sequence feature compression mechanism and the optimal and safe configuration of large language model running resources.

[0193] In step 106, the dimensionality-reduced feature sequence is input into the large language model to generate text, and the reasoning result used to respond to the target question is obtained.

[0194] In some embodiments, step 106, "inputting the dimensionality-reduced feature sequence into a large language model for text generation to obtain an inference result for responding to the target question," can be achieved by performing the following processing: obtaining the historical context sequence of the large language model for the target question, wherein the historical context sequence includes inference text; deleting the inference text from the historical context sequence to obtain an updated historical context sequence; concatenating the updated historical context sequence with the dimensionality-reduced feature sequence to obtain the target input sequence; and inputting the target input sequence into the large language model for text generation to obtain an inference result for responding to the target question.

[0195] For example, the historical context sequence refers to the collection of all relevant sequence data accumulated before the model arrives at its final response conclusion, including initial input instructions, multi-turn interactive dialogue records, and intermediate steps generated by the model itself. Access the memory cache, extract the entire array of interactive data related to the current target question, and define it as the historical context sequence. Subsequently, boundary checks are performed on the extracted historical context sequence data content to ensure that the sequence's data structure completely contains the reasoning text generated by the large language model for logical deduction in the preceding steps. The precise start and end index positions of the reasoning text within the historical context sequence are determined. Data culling is performed using array slicing or tensor truncation algorithms. For example, data segments before the start index position of the reasoning text and data segments after the end index position of the reasoning text are extracted from the historical context sequence. These extracted and retained segments are reconnected and integrated, thereby removing the presence of lengthy reasoning text at the physical storage level. The shortened feature sequence generated after this slicing, erasing, and reconnection operation is the updated historical context sequence. In memory, the data matrix of the dimensionality-reduced feature sequence is directly appended to the end boundary of the updated historical context sequence. By aligning and combining two different feature sequences in memory in a continuous physical arrangement, a completely new target input sequence is reconstructed, containing both the original background problem and highly condensed inference logic. The reconstructed target input sequence is then fed back into the deep processing network of the large language model through the input interface. The attention unit of the large language model performs feature reading and cross-association calculations on the target input sequence. Because the target input sequence embeds the purified and compressed dimensionality-reduced feature sequence, the model can directly and quickly read and understand the core inference logic. Based on this highly condensed understanding, the decoder of the large language model performs autoregressive prediction and sequentially outputs the final conclusion text characters. This final generated text data, which directly provides the task answer attributes, is the inference result used to respond to the target problem.

[0196] As an example, the memory cache of the large language model is read to obtain interactive data for the engineering calculation target problem. The extracted data array contains the initial question text ("The project department has three groups: Group A, Group B, and Group C...how many more days are needed to complete it?") and the detailed derivation text subsequently generated by the large language model ("Step 1, calculate each engineering group...equals 48 / 7 days."). This complete data block containing the problem premise and derivation steps is the obtained historical context sequence, and it is clear that this sequence includes the previously generated derivation text. The position of the derivation text "Step 1, calculate each engineering group...equals 48 / 7 days" is located in the obtained historical context sequence. This extremely long character vector fragment is directly truncated and removed using array slicing operations. After slicing, the original sequence only retains the initial question "The project department has Group A...how many more days are needed to complete it?" This simplified sequence, after removing the lengthy derivation steps, is the updated historical context sequence. The updated historical context sequence containing only the question is extracted. Simultaneously, the dimensionality-reduced feature sequence generated by the previous processing, consisting of only three high-density vectors, is extracted. These three high-density feature vectors, which contain the core logic of the entire operation, are directly concatenated and arranged after the question sequence for combination and splicing, forming a new target input sequence composed of the question features and three condensed features. The target input sequence containing the original question and the three condensed logic vectors is then fed back into the decoding network of the large language model. After reading the question, the large language model directly parses the three condensed features, without needing to re-deduce the complex steps, and quickly generates the text at the output end: "Based on the above calculations, it will take another 48 / 7 days to complete." This final conclusion text, which directly answers the number of days, is the reasoning result output by the large language model in response to the target question.

[0197] This application addresses the technical problem that when large language models perform complex task reasoning, lengthy original reasoning text occupies a large amount of context window, leading to huge computational resource consumption in the subsequent generation stage. Furthermore, the model is prone to forgetting early key contextual clues when processing massive amounts of lengthy information, resulting in the final reasoning result deviating from the target or degrading in quality. The solution involves obtaining a historical context sequence containing the reasoning text and deleting the reasoning text to obtain an updated historical context sequence. The updated historical context sequence is then concatenated with a dimensionality-reduced feature sequence to obtain the target input sequence, which is then input into the large language model for text generation. By physically or logically eliminating lengthy original reasoning text from the large language model's context cache and directly replacing it with a volume-reduced, high-information-density dimensionality-reduced feature sequence, the length of the final input sequence used for the large language model is shortened, reducing the computational complexity and memory consumption of attention in the text generation and decoding stages. Simultaneously, because the dimensionality-reduced feature sequence is highly condensed and retains the original core reasoning logic, it effectively ensures that the model does not have to deal with massive amounts of useless redundant information again during the final text generation. This high-density feature replacement fundamentally alleviates the phenomenon of forgetting key information in ultra-long contextual reasoning, ensuring that the model can continuously focus on the core logic, thereby improving the efficiency of text generation and the accuracy of reasoning results used to respond to the target question.

[0198] In some embodiments, before step 106 "inputting the dimensionality-reduced feature sequence into the large language model for text generation to obtain the reasoning result for responding to the target question", the following processing can be performed: determining the number of times the large language model sequentially performs local feature extraction, global feature integration, and segmented fusion on the one-dimensional semantic vector sequence; if the number of processing is less than a preset threshold, then the dimensionality-reduced feature sequence is used as a new one-dimensional semantic vector sequence, and the process proceeds to the step of sliding along the first direction of the one-dimensional semantic vector sequence based on a preset one-dimensional sliding window until the number of processing equals the threshold; the last obtained dimensionality-reduced feature sequence is used as the input dimensionality-reduced feature sequence for text generation in the large language model.

[0199] For example, a global state iteration counter is constructed in the underlying algorithm architecture of text processing. During the process initialization phase, the value of this global state iteration counter is reset to zero. Each time the computation graph scheduling engine of the large language model completes the entire feature dimensionality reduction pipeline process described above, from local feature extraction to global feature integration and finally segmented fusion, an increment instruction is sent to the global state iteration counter to increase its internal integer value by one. By reading the current integer value in the memory address where the global state iteration counter is located, the actual number of times the large language model performs this set of core feature processing processes on the one-dimensional semantic vector sequence can be accurately determined. The logical judgment unit compares the number of processing operations with a preset threshold. If the judgment logic returns a result indicating that the number of processing operations is less than the preset threshold, it is determined that the current feature sequence has not yet reached the desired deep condensation level. At this time, a variable redirection operation is performed in the underlying tensor management space, directly assigning the data memory pointer of the shortened dimensionality reduction feature sequence output in the previous round to the one-dimensional semantic vector sequence variable of the input data stream, so that the current dimensionality reduction feature sequence takes over in physical memory and becomes the new one-dimensional semantic vector sequence. After the tensor renaming handover is completed, a program control flow jump instruction is triggered, forcing the algorithm execution flow to return to the frontmost data input node, restarting the subsequent extraction pipeline, and then transitioning to the step of executing a step based on a preset one-dimensional sliding window, which cyclically slides along the first direction on this new, shortened high-order one-dimensional semantic vector sequence. This multi-round iterative operation is executed repeatedly, in each round the original sequence is deeply reduced to higher-density features until the number of processing steps equals a preset threshold, at which point the jump logic terminates, completely ending the iterative architecture. When the loop control condition meets the termination requirement and successfully jumps out of the multi-layered nested compressed iterative network architecture, the physical buffer at the end of the computation pipeline is located, and the data tensor structure located in the physical buffer is extracted. After going through all the predetermined rounds of iteration, the finalized data matrix is ​​confirmed to no longer undergo any form of dimensionality splitting or numerical extraction calculations. Instead, it is standardized for dimensionality alignment and batch data encapsulation. Then, the final dimensionality reduction feature sequence is directly sent to the generation interaction pipeline interface of the large language model as the actual dimensionality reduction feature sequence used to splice the context and input into the large language model for subsequent substantive natural language text generation, thus completing the entire closed loop of the inference data dimensionality reduction architecture.

[0200] As an example, the counter is initially set to 0 at algorithm startup. After the algorithm completes the first round of local extraction, global aggregation, and segmentation fusion operations on the engineering computation inference text, and outputs a dimensionality-reduced feature sequence containing 3 feature vectors, the counter increments by 1. At this point, the counter variable is read to determine that the large language model has processed these steps sequentially on the one-dimensional semantic vector sequence 1 time. A preset threshold of 2 is set. The current number of processed steps is 1, and a comparison shows that 1 is less than 2. At the underlying level, the algorithm directly renames and marks the dimensionality-reduced feature sequence containing 3 vectors, which was just compressed in the first round, as a new one-dimensional semantic vector sequence. The preset one-dimensional sliding window restarts sliding extraction along the first direction on this new sequence containing only 3 higher-order vectors. After the second round of in-depth local and global processing and further fusion and truncation (e.g., merging and reducing the 3 features to 1 feature), the processing count is updated to 2. A comparison shows that the current processing count equals the preset threshold of 2, and the algorithm directly truncates the feedback jump instruction, stopping all cyclic extraction behavior. After two rounds of processing and the complete termination of the loop structure, a result containing only one condensed high-order feature vector is obtained (this feature is further compressed and refined from the three features in the previous round). This final output, containing only one feature vector, is directly extracted as a dimensionality-reduced feature sequence. At the end of the entire processing pipeline, this single condensed feature vector, containing the essence density of the entire deduction, is concatenated with the initial question history context, making it the direct input to the large language model to calculate the answer and ultimately generate the dimensionality-reduced feature sequence of the result text.

[0201] This application addresses the technical problem that when processing complex, long reasoning texts, single-step feature extraction and segmentation fusion are often limited by fixed extraction ranges and truncation steps, making it impossible to deeply reduce lengthy sequences while fully preserving core logical features, resulting in limited dimensionality reduction compression of feature sequences. The solution involves determining the number of times the large language model sequentially performs local feature extraction, global feature integration, and segmentation fusion on a one-dimensional semantic vector sequence. If this number of processing steps is less than a preset threshold, the resulting dimensionality-reduced feature sequence is used as a new one-dimensional semantic vector sequence, and the process proceeds to slide a preset one-dimensional sliding window along a first direction on the one-dimensional semantic vector sequence, triggering multiple iterative operations. This continues until the number of processing steps equals the threshold, at which point the final dimensionality-reduced feature sequence is used as the input to the large language model for text generation. A multi-level, interleaved sequence compression architecture is constructed, using the feature sequence output from the previous round of dimensionality reduction as the underlying input for the next round of extraction, enabling the model to steadily refine local context and fuse global information in each independent round. This iterative approach effectively avoids the risk of missing core features caused by large-scale segmentation in a single execution. While ensuring the logical coherence of the reasoning text, it achieves a deep reduction of lengthy one-dimensional semantic vector sequences into high-density features, thus improving the dimensionality reduction effect for complex reasoning texts.

[0202] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0203] Large Language Models (LLMs) have developed rapidly in recent years, demonstrating strong capabilities in understanding and generation. In particular, by introducing techniques such as Chain-of-Thought (CoT), LLMs can complete complex reasoning tasks step-by-step, improving problem-solving accuracy. However, this powerful reasoning ability brings a significant problem: LLMs generate long strings of text processes, or thought steps (i.e., the aforementioned semantic units), during thinking and reasoning. These lengthy reasoning processes consume substantial computational and memory resources, resulting in slow and inefficient reasoning. In scenarios requiring the processing of long historical information or repeated deliberation, LLMs struggle to maintain focus on early key information, easily forgetting previously reasoned content and affecting accuracy. While related technologies optimize the LLM model itself or change the questioning method, none have been able to effectively and universally compress the long reasoning processes of LLMs without compromising reasoning quality. Therefore, current reasoning models tend to slow down when encountering long thought processes.

[0204] When LLMs perform complex or multi-step reasoning, they generate very long thought process texts. Using these long texts directly as input or intermediate states for the LLM consumes significant computational resources and memory, slowing down the model's inference speed, especially in large-scale applications. In particularly long inference chains, LLMs struggle to consistently and effectively focus on and utilize key information generated earlier, leading to decreased reasoning coherence or errors. While various methods exist to optimize LLMs, they generally lack a mechanism for universally and efficiently compressing the LLM inference process itself, failing to reduce the length of the inference representation while preserving inference quality.

[0205] This application addresses the efficiency issues encountered by LLMs in complex reasoning. LLMs perform multi-step reasoning, generating a series of intermediate steps or thought chains, often resulting in very long text sequences. Processing these long sequences requires significant computational and memory resources, leading to slow reasoning speed and low efficiency. This application first transforms the complex logical reasoning text (i.e., the aforementioned reasoning text) generated by the large language model for the target problem into a structured, high-dimensional continuous representation. Specifically, each key step or semantic unit in the reasoning is encoded into a continuous vector carrying semantics and potential structural relationships (i.e., the aforementioned semantic vector). These vectors are laid out and associated in the concept space, forming a data structure similar to a multi-dimensional feature map, implicitly extracting the logical flow, dependencies, and importance hierarchy of the reasoning. Next, a reasoning compression encoder is used. This encoder does not process traditional two-dimensional images but directly processes this structured one-dimensional continuous vector representation. Through multi-level feature extraction and aggregation mechanisms, it intelligently identifies and extracts the most important logical paths and core arguments in the reasoning process, while filtering out redundant and secondary information. Ultimately, the encoder efficiently compresses the high-dimensional original reasoning representation into a small number of high-information-density reasoning feature tokens (i.e., the aforementioned dimensionality-reduced feature sequence). These tokens highly summarize and represent the essence of the entire reasoning process. Thus, when the LLM needs to review, reuse, or continue thinking based on previous reasoning, it no longer needs to input and process the entire lengthy original text reasoning process. Instead, it only needs to utilize these compressed dimensionality-reduced feature sequences, reducing the length of the LLM's context input, lowering computational resource consumption, and improving the efficiency and speed of reasoning. Simultaneously, this mechanism also helps the LLM more effectively remember key information and avoid forgetting when dealing with extremely long reasoning chains.

[0206] This application's embodiments transform the lengthy text reasoning process into a small number of high-density, dimensionality-reduced feature sequences. The LLM only needs to process these concise tokens, rather than the entire original text, shortening the input context length, thereby reducing computational resources and memory consumption, and improving reasoning speed and overall efficiency. Through compression mechanisms, the core logic and key information in the reasoning process can be extracted and preserved. Even with a long reasoning chain, the LLM can efficiently review and focus on the most important steps through these highly summarized features, effectively alleviating the problem of forgetting early information during long reasoning processes. By using a reasoning process compression method decoupled from the specific LLM model, complex reasoning can be effectively compressed without sacrificing reasoning quality, providing an efficient way of thinking and remembering.

[0207] This application embodiment transforms the complex reasoning process of a large language model into a structured, high-dimensional continuous representation, and then uses a customized compression encoder to efficiently extract core information and generate a small number of high-density reasoning feature tokens (i.e., the aforementioned dimensionality-reduced feature sequence). This can improve the reasoning speed of LLM, reduce the consumption of computing resources, and effectively alleviate the forgetting problem of LLM in long reasoning chains.

[0208] See Figure 5 , Figure 5 This is a flowchart illustrating the process of obtaining inference features after compressing inference text, as provided in an embodiment of this application. First, the original LLM inference text R is obtained; then, semantic unit segmentation and initial encoding are performed; subsequently, structural relationship modeling and high-dimensional layout are performed to generate a structured continuous representation X; after adapting the sequence input to the structure, inference-aware local feature extraction, global inference context integration, and multi-level inference token compression and fusion are performed sequentially; finally, a high-information-density inference feature token T is output. This feature token can be widely used in subsequent LLM context substitution, efficient inference, and forgetting mitigation, and possesses high flexibility.

[0209] To adapt to the reasoning scenarios of multimodal large language models, before generating structured continuous representations, non-textual modal data (such as images and charts) included in the reasoning process are uniformly encoded into continuous features to achieve unified compression processing of cross-modal reasoning chains.

[0210] Specifically, the original inference data (including inference text) generated by the large language model for the target problem is obtained; if the original inference data also includes image data, the image data is input into a pre-trained visual encoding network for feature extraction to obtain a visual feature sequence; the visual feature sequence is combined with the subsequent one-dimensional semantic vector sequence to form a multimodal feature sequence, and the multimodal feature sequence is used as a new one-dimensional semantic vector sequence to be input into the subsequent compression process.

[0211] The raw inference text generated by LLM is transformed into a non-image, but structural and locally / globally correlated, high-dimensional continuous vector representation, which serves as the input to the subsequent compression encoder. The raw inference process R is a sequence of text steps, denoted as... ,in, This represents a semantically complete reasoning step (such as a sentence, phrase, or sub-argument). For each semantic unit Using a pre-trained text encoder, such as a Transformer-based encoder like Bidirectional Encoder Representations from Transformers (BERT) or its variants, semantic units are... Transform into an initial continuous vector embedding This embedding extracts The semantic content. The text encoding process is shown in formula (1): (1) in, For the initial continuous vector embedding, It is a text encoding algorithm. It is a semantic unit.

[0212] In order to extract the logical relationships between reasoning steps (e.g., sequence, causality, support or refutation), these independent steps are not considered. Instead of simple concatenation, we go further with context awareness and relationship modeling. A Transformer-based self-attention mechanism can be used to process the initial embedded sequence. This process generates a series of context-aware vectors. .These In an abstract high-dimensional space, the relative positions and eigenvalues ​​of vectors implicitly encode the logical structure and semantic relationships between inference steps, forming a continuous data structure similar to a multi-dimensional feature map. The internal elements of this structure have local and global relationships that can be perceived by a similar visual processing mechanism. The structural encoding function is shown in formula (2): (2) in, It's the Transformer layer. It is a continuous data structure. It is the initial embedding sequence.

[0213] For the generated structured continuous representation Perform efficient feature extraction and information compression to generate a small number of high-information-density inference feature tokens. ,in, DeepEncoder is designed for 2D image pixel data. Its core advantage lies in the processing and compression of visual information. However, directly applying DeepEncoder to the high-dimensional structured continuous vector representation of LLMs encounters the following fundamental difficulties: DeepEncoder's input is a fixed 2D pixel grid, while the embodiments in this application use a 1D sequence of high-dimensional continuous vectors. Even if it possesses structure in the abstract space, it is not a traditional 2D image. The 2D convolution and 2D window attention (such as the Swing Transformer) used in DeepEncoder to handle spatial locality cannot be directly applied to 1D vector sequences. Furthermore, the locality of DeepEncoder is physical proximity in pixel space, which is problematic for LLM inference. The locality between them is semantic and logical proximity. Directly applying spatial window attention will result in the loss or misunderstanding of the true logical relationship between reasoning steps. DeepEncoder uses downsampling of 2D convolution (such as 16 times) to efficiently compress visual tokens. This grid-based 2D compression mechanism cannot be directly transferred to 1D vector sequences.

[0214] To overcome the aforementioned difficulties, this application's embodiments innovatively adapt the core idea of ​​DeepEncoder at the semantic-sequence level, constructing a "reasoning compression encoder" specifically for compressing inference sequences. It retains its core advantages of combining local and global attention, multi-level compression, and low activation memory, but has been redesigned to adapt to non-image structured continuous representations. Considering the input... It is a one-dimensional vector sequence, but it has an inherent logical structure. First, consider the input... Adaptation is performed to make the input... It can be processed by the subsequent DeepEncoder class. As the initial feature sequence input, pixel-level transformation is no longer performed; instead, it is directly... It is considered as a sequence with an implicit context.

[0215] The reasoning-aware local feature extraction layer innovatively adapts the window attention mechanism, which handles 2D spatial locality in DeepEncoder, to a mechanism for handling logical locality in reasoning sequences, extracting the fine logic between adjacent or short-distance related steps in the reasoning chain. It employs a one-dimensional sliding window self-attention mechanism for sequences... Each Token Only with a fixed-size window Attention is calculated for tokens within the one-dimensional sliding window (as described above). This window slides along the sequence, ensuring that each token can extract its semantically adjacent context. One-dimensional window self-attention function. As shown in formula (3): (3) in, , , , exist window Inside, , , This is the weight matrix. , It is a window All , The set, For feature dimension coefficients, The function is a normalized exponential function. Multiple layers can be stacked, and 1D convolutional layers (such as dilated convolution) can be introduced to further extract local patterns and expand the receptive field. Through a one-dimensional sliding window self-attention mechanism, fine-grained semantics of each step in the inference chain and short-range logical dependencies between steps (i.e., the aforementioned local association features) are effectively extracted.

[0216] The global reasoning context integration layer inherits the global attention concept from DeepEncoder, but applies it to the overall logical structure and key arguments of the reasoning sequence, rather than the global relationships of visual objects. After local feature extraction, or at a deeper level, a global self-attention mechanism is introduced, allowing each token in the encoder to interact with all other tokens in the sequence, thereby extracting the long-distance dependencies and overall logical flow of the entire reasoning process (i.e., the aforementioned global feature sequence). Global attention function. As shown in formula (4): (4) in, , , , , , This is the weight matrix. For feature dimension coefficients, is a normalized exponential function. Through a global self-attention mechanism, it can be ensured that the encoder understands the global context of reasoning, identifying core arguments and main logical threads, even if they are far apart in the sequence.

[0217] For 1D sequence data, an innovative 1D sequence token compression mechanism is designed to replace the 2D convolutional downsampling in DeepEncoder, achieving efficient token reduction with low information loss. A multi-level token aggregation strategy is adopted, for example, using 1D pooling layers (such as max pooling or average pooling along the sequence dimension), or a token fusion unit based on an attention mechanism, for example, aggregating tokens in a local region using a small query vector (i.e., the second local query feature mentioned above), or learning a gating mechanism to selectively fuse information. The 1D pooling function is shown in formula (5): (5) in, This indicates that after the pooling operation is completed, the first element in the output dimensionality-reduced feature sequence is... Each feature element. This indicates the specific aggregate function to be executed. This represents max pooling (preserving the most salient feature / maximum value within the window). This represents average pooling (calculating the average of all features within a window to preserve overall information). This represents the original one-dimensional feature sequence before input to the pooling layer. This represents the index (position subscript) of the output sequence, indicating which position in the output sequence is currently being calculated. It is the pooling step size. Indicates the first A pooling window in the input sequence The starting slicing index on, Indicates the first A pooling window in the input sequence The end of the truncation index.

[0218] Alternatively, a one-dimensional convolutional layer with a stride (stride>1) (i.e., the preset stride mentioned above) can be used to naturally reduce the sequence length while extracting features, thereby reducing the number of tokens. The 1D stride convolution function is shown in formula (6): (6) in, This indicates the first element in the output dimensionality-reduced feature sequence after the convolution operation. Each feature element. This represents a one-dimensional convolution operation, which performs linear weighted summation and non-linear mapping on the truncated local sequence through a set of learnable weights (convolution kernels). This represents a one-dimensional feature sequence input to the convolutional layer (usually represented as a matrix or a high-dimensional vector sequence). This represents the index (position subscript) of the output sequence, indicating which convolutional feature is currently being computed after dimensionality reduction. This indicates the kernel size, which is the number of consecutive input features that a convolution operation can cover / process in one operation (the size of the local receptive field). The stride represents the convolutional step size, which determines the span by which the convolutional kernel slides along the sequence each time. At the same time, it can naturally reduce the sequence length and achieve feature compression. Indicates the first During the convolution calculation, the convolution kernel is in the input sequence The sliding start index. This indicates that during this convolution calculation, the convolution kernel is in the input sequence The sliding end index is used. These compression operations are performed alternately across different levels of the encoder, progressively compressing N input vectors into M high-density inference feature tokens.

[0219] The low-activation memory design draws on the engineering optimization principles of DeepEncoder to ensure the entire encoder maintains high efficiency even when handling extremely long inference sequences. Gradient checkpointing and other techniques are employed to reduce memory usage during training, and efficient attention implementations are used to reduce the computational complexity and memory consumption of attention. Optimized layer design avoids unnecessary intermediate activation storage. This allows the compressed encoder to handle extremely long inference sequences generated by LLMs in real-world production environments without causing memory overflows due to memory bottlenecks. After processing by the multi-layered inference compressed encoder, the final output is a series of inference feature tokens with a reduced number of tokens. These tokens are concise and highly information-dense representations of the original, lengthy reasoning process, and are used for subsequent LLM tasks.

[0220] The compressed inference feature tokens are used in subsequent tasks of the LLM to improve efficiency and mitigate forgetting. The final output is a sequence of compressed inference feature tokens. When the LLM needs to continue generating text based on previous inference or review previous thought processes, the original lengthy text inference chain is replaced by a simplified one, shortening the input context length of the LLM. When processing subsequent tasks, the LLM only needs to focus on and utilize these few information-intensive tokens, thereby reducing computation and memory usage and improving inference speed. Because the core logic of the inference is extracted, even if the inference chain is very long, the LLM can efficiently recall and remember key information through these highly generalized tokens, thus effectively mitigating the forgetting problem that easily occurs when processing long contexts. The encoder can support different compression ratios, and the number of tokens generated at the end can be adjusted to adapt to different efficiency and accuracy requirements.

[0221] This application's embodiments compress the lengthy inference process into a small number of high-information-density tokens, reducing the input context length of LLM during inference, decreasing model computation and memory usage, and enabling LLM to complete complex inference tasks faster, especially suitable for scenarios requiring long-chain thinking. The simplified token representation means less computation, thereby reducing the GPU memory and computation time required for model operation, making LLM inference more efficient and facilitating the deployment and operation of more complex inference models in resource-constrained environments. The compression mechanism accurately extracts the core logic and key information of the inference. Through these refined tokens, LLM can efficiently review and focus on the most important thinking steps, effectively avoiding the forgetting of early key information in extremely long inference chains, thus maintaining the coherence and accuracy of inference. Because the core information is strengthened and simplified, the possibility of LLM being interfered with by noise and irrelevant information during inference is reduced, thereby improving the stability and reliability of its inference results. By decoupling from specific LLM models, it can be flexibly applied to various LLM architectures and supports different compression ratios to adapt to different application needs and performance requirements. The compressed inference feature token or its decoding result can present the LLM thinking path in a more concise form, which helps users quickly understand, review and debug the model's inference process.

[0222] The following description continues to illustrate the exemplary structure of the text processing device 555 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the text processing device 555 of the memory 550 may include: The text segmentation module 5551 is used to obtain the reasoning text generated by the pre-trained large language model for the target problem and to divide the reasoning text into multiple semantic units.

[0223] The vector combination module 5552 is used to convert each semantic unit into a semantic vector and combine multiple semantic vectors into a one-dimensional semantic vector sequence based on the order of each semantic unit in the reasoning text.

[0224] The feature extraction module 5553 is used to slide along a first direction on a one-dimensional semantic vector sequence based on a preset one-dimensional sliding window, and to extract local features from the semantic vectors covered by the one-dimensional sliding window at each sliding position to obtain multiple local associated features. The first direction is the direction from the first position to the last position of the one-dimensional semantic vector sequence.

[0225] The feature combination module 5554 is used to combine multiple local related features in a first direction to obtain a local feature sequence, and to perform global feature integration on the local feature sequence to obtain a global feature sequence.

[0226] The feature dimensionality reduction module 5555 is used to segment and fuse the global feature sequence along the second direction with a preset step size to obtain multiple dimensionality reduction features, and combine the multiple dimensionality reduction features according to the second direction to obtain a dimensionality reduction feature sequence. The second direction is the direction from the first position to the last position of the global feature sequence, and the sequence length of the dimensionality reduction feature sequence is less than the sequence length of the global feature sequence.

[0227] The text generation module 5556 is used to input the dimensionality-reduced feature sequence into the large language model to generate text and obtain the reasoning results used to respond to the target question.

[0228] In some embodiments, the text segmentation module 5551 is further configured to perform character scanning on the inference text generated by the large language model for the target problem to obtain multiple delimiters; based on the multiple delimiters, perform segmentation processing on the inference text to obtain multiple candidate text segments; for each candidate text segment, perform semantic integrity verification on the candidate text segment to obtain a verification result; and use the verification result to represent multiple semantically complete candidate text segments as multiple semantic units.

[0229] In some embodiments, the vector combination module 5552 is further configured to perform the following processing for each semantic unit: dividing the semantic unit into multiple initial lexical units; performing feature mapping processing on the multiple initial lexical units respectively to obtain multiple initial lexical unit vectors; generating local position vectors corresponding to each initial lexical unit vector based on the arrangement order of each initial lexical unit in the semantic unit; performing feature fusion on the initial lexical unit vector and the local position vector corresponding to the initial lexical unit vector for each initial lexical unit vector to obtain a first position fusion vector; and performing feature aggregation on multiple first position fusion vectors to obtain a semantic vector.

[0230] In some embodiments, the vector combination module 5552 is further configured to: concatenate multiple semantic vectors into an initial vector sequence according to the arrangement order of each semantic unit in the inference text; generate a global position vector corresponding to each semantic vector based on the third order of each semantic vector in the initial vector sequence; perform feature fusion on the semantic vector and the global position vector corresponding to the semantic vector for each semantic vector to obtain a second position fusion vector; combine multiple second position fusion vectors into a fusion vector sequence according to the arrangement order; and perform self-attention encoding on the fusion vector sequence to obtain a one-dimensional semantic vector sequence.

[0231] In some embodiments, the feature extraction module 5553 is further configured to: determine a local subsequence corresponding to the sliding position in a one-dimensional semantic vector sequence based on the sliding window length of the one-dimensional sliding window; perform query feature mapping on the semantic vector corresponding to the sliding position in the one-dimensional semantic vector sequence to obtain a first local query feature; perform key feature mapping on the local subsequence to obtain a local key feature sequence, and perform value feature mapping on the local subsequence to obtain a local value feature sequence; determine a first correlation between the first local query feature and each local key feature in the local key feature sequence, and normalize multiple first correlations to obtain multiple local attention weights; based on the multiple local attention weights, fuse multiple local value features in the local value feature sequence into an initial local aggregated feature; and perform one-dimensional convolution processing on the initial local aggregated feature to obtain a local association feature corresponding to the sliding position.

[0232] In some embodiments, the feature extraction module 5553 is further configured to determine the first order of the semantic vector corresponding to the first local query feature in the one-dimensional semantic vector sequence; for each local key feature in the local key feature sequence, determine the second order of the semantic vector corresponding to the local key feature in the one-dimensional semantic vector sequence, and determine the difference between the first order and the second order as the relative positional distance between the first local query feature and the local key feature; determine the feature inner product value between the first local query feature and the local key feature, and determine the first relevance based on the relative positional distance and the feature inner product value.

[0233] In some embodiments, the feature extraction module 5553 is further configured to obtain initial local aggregation features corresponding to multiple candidate sliding positions adjacent to the sliding position, respectively, as historical local aggregation features; perform interval sampling from multiple historical local aggregation features based on a preset interval step size to obtain historical feature subsequences; concatenate the initial local aggregation features with the historical feature subsequences to obtain a fused feature sequence; and perform convolution kernel mapping processing on the fused feature sequence to obtain local association features corresponding to the sliding position.

[0234] In some embodiments, the feature extraction module 5553 is further configured to acquire the original inference data generated by the large language model for the target problem, wherein the original inference data includes inference text; if the original inference data also includes image data, the image data is input into a pre-trained visual encoding network for feature extraction to obtain a visual feature sequence; the visual feature sequence is combined with a one-dimensional semantic vector sequence to form a multimodal feature sequence; the multimodal feature sequence is used as a new one-dimensional semantic vector sequence, and the process proceeds to the step of sliding along a first direction on the one-dimensional semantic vector sequence based on a preset one-dimensional sliding window.

[0235] In some embodiments, the feature combination module 5554 is further configured to perform query feature mapping on the local feature sequence to obtain a global query feature sequence; perform key feature mapping on the local feature sequence to obtain a global key feature sequence, and perform value feature mapping on the local feature sequence to obtain a global value feature sequence; for each global query feature in the global query feature sequence, determine a third correlation between the global query feature and each global key feature in the global key feature sequence, and normalize multiple third correlations to obtain multiple global attention weights; based on the multiple global attention weights, fuse each global value feature in the global value feature sequence to obtain a global feature corresponding to the global query feature; and combine multiple global features in a first direction to obtain a global feature sequence.

[0236] In some embodiments, the feature combination module 5554 is further configured to perform query feature mapping on the local feature sequence to obtain a global query feature sequence; perform key feature mapping on the local feature sequence to obtain a global key feature sequence, and perform value feature mapping on the local feature sequence to obtain a global value feature sequence; divide the global query feature sequence into multiple query feature blocks; divide the global key feature sequence into multiple key feature blocks, and divide the global value feature sequence into multiple value feature blocks corresponding to the multiple key feature blocks respectively; for each query feature block, determine the local relevance matrix between the query feature block and each key feature block, and normalize the local relevance matrix to obtain a block attention weight matrix; based on the block attention weight matrix, fuse the value feature blocks corresponding to the multiple key feature blocks respectively to obtain a local fused feature block, and fuse the multiple local fused feature blocks into a global feature corresponding to the query feature block; combine the global features corresponding to the multiple query feature blocks according to a first direction to obtain a global feature sequence.

[0237] In some embodiments, the feature dimensionality reduction module 5555 is further configured to perform segmentation on the global feature sequence along the second direction based on a preset truncation length and a preset step size to obtain multiple first feature subsequences; perform one-dimensional convolution processing on each first feature subsequence to obtain convolutional feature sequences corresponding to each first feature subsequence; for each convolutional feature sequence, determine the convolutional fusion weights corresponding to each convolutional feature in the convolutional feature sequence, and fuse the multiple convolutional features into a dimensionality reduction feature corresponding to the first feature subsequence based on the multiple convolutional fusion weights.

[0238] In some embodiments, the global feature sequence includes multiple global features. The feature dimensionality reduction module 5555 is further configured to perform segmentation on the global feature sequence along a second direction based on a preset truncation length and a preset step size to obtain multiple second feature subsequences; for each second feature subsequence, a gating state mapping is performed on each global feature in the second feature subsequence to obtain the gating activation weight corresponding to each global feature; based on the gating activation weight corresponding to each global feature, the multiple global features in the second feature subsequence are numerically scaled to obtain multiple gating filtering features; and the multiple gating filtering features are fused into a dimensionality reduction feature corresponding to the second feature subsequence.

[0239] In some embodiments, the global feature sequence includes multiple global features. The feature dimensionality reduction module 5555 is further configured to segment the global feature sequence along a second direction based on a preset truncation length and a preset step size to obtain multiple third feature subsequences. For each third feature subsequence, the following processing is performed: feature mapping is performed on the third feature subsequence to obtain a second local query feature corresponding to the third feature subsequence; a second correlation degree is determined between the second local query feature and each global feature in the third feature subsequence, and the multiple second correlation degrees are normalized to obtain multiple aggregation and fusion weights; based on the multiple aggregation and fusion weights, the multiple global features in the third feature subsequence are fused into a dimensionality reduction feature corresponding to the third feature subsequence.

[0240] In some embodiments, the feature dimensionality reduction module 5555 is further configured to: determine an upper limit for the context length of the target problem based on a preset memory capacity and a preset upper limit for inference latency of the large language model in the current operating environment; determine a lower limit for the context length of the large language model inference based on a preset inference accuracy index for the target problem; determine the target sequence length based on the upper and lower limits; determine the ratio of the original sequence length of the global feature sequence to the target sequence length as the sequence compression ratio; and determine a preset step size for segmenting and fusing the global feature sequence based on the sequence compression ratio.

[0241] In some embodiments, the text generation module 5556 is further configured to obtain a historical context sequence of a large language model for a target problem, wherein the historical context sequence includes inference text; delete the inference text from the historical context sequence to obtain an updated historical context sequence; concatenate the updated historical context sequence with the dimensionality-reduced feature sequence to obtain a target input sequence; and input the target input sequence into the large language model for text generation to obtain an inference result for responding to the target problem.

[0242] In some embodiments, the text generation module 5556 is further configured to determine the number of times the large language model sequentially performs local feature extraction, global feature integration, and segment fusion on the one-dimensional semantic vector sequence; if the number of processing steps is less than a preset threshold, the dimensionality-reduced feature sequence is used as a new one-dimensional semantic vector sequence, and the process proceeds to the step of sliding along the first direction of the one-dimensional semantic vector sequence based on a preset one-dimensional sliding window until the number of processing steps equals the threshold; the last obtained dimensionality-reduced feature sequence is used as the input dimensionality-reduced feature sequence for text generation in the large language model.

[0243] This application provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the text processing method described above in this application.

[0244] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the text processing method provided in this application, for example, such as... Figure 3A The text processing method shown.

[0245] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0246] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0247] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0248] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0249] In summary, through the embodiments of this application, the reasoning text generated by the large language model for the target problem is divided into multiple semantic units, and each semantic unit is converted into a semantic vector, thereby converting unstructured natural language text into continuous vector features that are easy for the model to process. Furthermore, based on the order of each semantic unit in the reasoning text, multiple semantic vectors are combined into a one-dimensional semantic vector sequence, preserving the temporal logical relationship of the original reasoning text. This allows the generated one-dimensional semantic vector sequence to accurately map the structural information of the original text, providing a high-quality input foundation for subsequent feature extraction. Based on a preset one-dimensional sliding window, the system slides along a first direction on the one-dimensional semantic vector sequence, and local feature extraction is performed on the semantic vectors covered by the one-dimensional sliding window at each sliding position. The one-dimensional sliding window limits the scope of each feature extraction, allowing the extraction process to focus on adjacent semantic vectors, thereby accurately extracting the local logical dependencies between adjacent reasoning steps in the one-dimensional semantic vector sequence. The generated local association features effectively preserve the contextual reasoning details within a short distance. Multiple locally related features are combined along a first direction to obtain a local feature sequence. This local feature sequence is then integrated with global features to obtain a global feature sequence. While preserving the sequential order of features along the first direction, global feature integration overcomes the local limitations of a one-dimensional sliding window, establishing long-distance logical connections between different locally related features. This extracts a global feature sequence representing the main logic of the entire reasoning text, ensuring that core global reasoning information is not lost during subsequent dimensionality reduction. The global feature sequence is then segmented and fused along a second direction with a preset step size to obtain multiple dimensionality-reduced features. Introducing a preset step size for segmented fusion merges multiple global features into a single dimensionality-reduced feature, achieving efficient information integration. Subsequently, these dimensionality-reduced features are combined along the second direction to obtain a dimensionality-reduced feature sequence. Through a segmented fusion mechanism, the sequence length of the dimensionality-reduced feature sequence is less than the sequence length of the global feature sequence, effectively eliminating redundant information in the reasoning text and achieving a physical reduction in the sequence representation dimension, thus reducing the amount of data processed subsequently. Inputting the dimensionality-reduced feature sequence into a large language model for text generation has several advantages. Since the input dimensionality-reduced feature sequence retains the logic of local and global feature sequences while reducing its length, it directly reduces the computational resource consumption and memory usage of the large language model when performing text generation tasks, thus improving generation efficiency. Simultaneously, the concise and shorter dimensionality-reduced feature sequence reduces interference from irrelevant and redundant information, enabling the large language model to utilize key features more stably and with greater focus, thereby improving the accuracy of the final inference results used to respond to the target problem.

[0250] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A text processing method, characterized in that, The method includes: Obtain the reasoning text generated by a pre-trained large language model for the target problem, and divide the reasoning text into multiple semantic units; Each of the semantic units is converted into a semantic vector, and based on the order of each of the semantic units in the reasoning text, multiple semantic vectors are combined into a one-dimensional semantic vector sequence; Based on a preset one-dimensional sliding window, the one-dimensional semantic vector sequence is slid along a first direction. Local features are extracted from the semantic vectors covered by the one-dimensional sliding window at each sliding position to obtain multiple local association features. The first direction is the direction from the first position to the last position of the one-dimensional semantic vector sequence. The multiple local associated features are combined according to the first direction to obtain a local feature sequence, and the local feature sequence is then integrated with global features to obtain a global feature sequence. The global feature sequence is segmented and fused along the second direction with a preset step size to obtain multiple dimensionality-reduced features. The multiple dimensionality-reduced features are then combined along the second direction to obtain a dimensionality-reduced feature sequence. The second direction is the direction from the first position to the last position of the global feature sequence. The sequence length of the dimensionality-reduced feature sequence is less than the sequence length of the global feature sequence. The reduced feature sequence is input into the large language model to generate text, thereby obtaining the reasoning result used to respond to the target question.

2. The method according to claim 1, characterized in that, The semantic vector covered by the one-dimensional sliding window at each sliding position is subjected to local feature extraction to obtain multiple local association features, including: For each of the aforementioned sliding positions, the following processing is performed: Based on the sliding window length of the one-dimensional sliding window, a local subsequence corresponding to the sliding position is determined in the one-dimensional semantic vector sequence; Perform query feature mapping on the semantic vectors corresponding to the sliding positions in the one-dimensional semantic vector sequence to obtain the first local query features; The local subsequence is subjected to key feature mapping to obtain a local key feature sequence, and the local subsequence is subjected to value feature mapping to obtain a local value feature sequence; Determine the first correlation between the first local query feature and each local key feature in the local key feature sequence, and normalize the multiple first correlations to obtain multiple local attention weights; Based on the multiple local attention weights, the multiple local value features in the local value feature sequence are fused into an initial local aggregated feature; The initial local aggregated features are subjected to one-dimensional convolution to obtain the local associated features corresponding to the sliding position.

3. The method according to claim 2, characterized in that, The step of performing one-dimensional convolution processing on the initial local aggregated features to obtain local association features corresponding to the sliding position includes: Obtain the initial local aggregation features corresponding to multiple candidate sliding positions adjacent to the sliding position, and use them as historical local aggregation features; Based on a preset interval step size, interval sampling is performed from multiple historical local aggregated features to obtain historical feature subsequences; The initial local aggregated features are concatenated with the historical feature subsequences to obtain a fused feature sequence; The fused feature sequence is subjected to convolution kernel mapping to obtain the local associated features corresponding to the sliding position.

4. The method according to claim 2, characterized in that, Determining the first relevance between the first local query feature and each local key feature in the local key feature sequence includes: Determine the first order of the semantic vector corresponding to the first local query feature in the one-dimensional semantic vector sequence; For each local key feature in the local key feature sequence, determine the second order of the semantic vector corresponding to the local key feature in the one-dimensional semantic vector sequence, and The difference between the first order and the second order is determined as the relative positional distance between the first local query feature and the local key feature; Determine the inner product value of the features between the first local query feature and the local key feature, and determine the first relevance based on the relative position distance and the inner product value.

5. The method according to claim 1, characterized in that, The global feature sequence is segmented and fused along the second direction with a preset step size to obtain multiple dimensionality-reduced features, including: Along the second direction, based on the preset truncation length and the preset step size, the global feature sequence is segmented to obtain multiple first feature subsequences; Perform one-dimensional convolution on each of the first feature subsequences to obtain the convolutional feature sequences corresponding to each of the first feature subsequences. For each of the convolutional feature sequences, the convolutional fusion weights corresponding to each convolutional feature in the convolutional feature sequence are determined, and based on the multiple convolutional fusion weights, the multiple convolutional features are fused into a dimensionality-reduced feature corresponding to the first feature subsequence.

6. The method according to claim 1, characterized in that, The global feature sequence includes multiple global features. The global feature sequence is segmented and fused along the second direction with a preset step size to obtain multiple dimensionality-reduced features, including: Along the second direction, based on the preset truncation length and the preset step size, the global feature sequence is segmented to obtain multiple second feature subsequences; For each second feature subsequence, a gating state mapping is performed on each global feature in the second feature subsequence to obtain the gating activation weight corresponding to each global feature. Based on the gating activation weights corresponding to each global feature, the multiple global features in the second feature subsequence are numerically scaled to obtain multiple gating filtering features. Multiple gated screening features are fused into a dimensionality-reduced feature corresponding to the second feature subsequence.

7. The method according to claim 1, characterized in that, The global feature sequence includes multiple global features. The global feature sequence is segmented and fused along the second direction with a preset step size to obtain multiple dimensionality-reduced features, including: Along the second direction, based on the preset truncation length and the preset step size, the global feature sequence is segmented to obtain multiple third feature subsequences; For each of the third feature subsequences, the following processing is performed: The third feature subsequence is subjected to feature mapping to obtain the second local query feature corresponding to the third feature subsequence; Determine the second relevance between the second local query feature and each of the global features in the third feature subsequence, and normalize the multiple second relevances to obtain multiple aggregation and fusion weights; Based on the multiple aggregation and fusion weights, multiple global features in the third feature subsequence are fused into a dimensionality-reduced feature corresponding to the third feature subsequence.

8. The method according to any one of claims 1 to 7, characterized in that, The step of converting each semantic unit into a semantic vector includes: For each of the semantic units, the following processing is performed: The semantic unit is divided into multiple initial lexical units; Each of the initial lexical units is subjected to feature mapping processing to obtain multiple initial lexical unit vectors; Based on the arrangement order of each initial word element in the semantic unit, a local position vector corresponding to each initial word element vector is generated; For each initial word vector, the initial word vector and the local position vector corresponding to the initial word vector are fused to obtain a first position fusion vector; Feature aggregation is performed on multiple first position fusion vectors to obtain semantic vectors.

9. The method according to any one of claims 1 to 7, characterized in that, The step of combining multiple semantic vectors into a one-dimensional semantic vector sequence based on the order of each semantic unit in the reasoning text includes: Based on the arrangement order of each semantic unit in the reasoning text, multiple semantic vectors are concatenated into an initial vector sequence; Based on the third order of each semantic vector in the initial vector sequence, a global position vector corresponding to each semantic vector is generated; For each semantic vector, the semantic vector is fused with the global position vector corresponding to the semantic vector to obtain a second position fusion vector; The multiple second-position fusion vectors are combined into a fusion vector sequence according to the aforementioned arrangement order; The fused vector sequence is self-attention encoded to obtain a one-dimensional semantic vector sequence.

10. The method according to any one of claims 1 to 7, characterized in that, The step of integrating the local feature sequences into global features to obtain a global feature sequence includes: Perform query feature mapping on the local feature sequence to obtain the global query feature sequence; The local feature sequence is subjected to key feature mapping to obtain a global key feature sequence, and the local feature sequence is subjected to value feature mapping to obtain a global value feature sequence; For each global query feature in the global query feature sequence, a third relevance is determined between the global query feature and each global key feature in the global key feature sequence. Multiple third relevances are then normalized to obtain multiple global attention weights. Based on the multiple global attention weights, each global value feature in the global value feature sequence is fused to obtain a global feature corresponding to the global query feature; The multiple global features are combined in the first direction to obtain a global feature sequence.

11. The method according to any one of claims 1 to 7, characterized in that, The step of integrating the local feature sequences into global features to obtain a global feature sequence includes: Perform query feature mapping on the local feature sequence to obtain the global query feature sequence; The local feature sequence is subjected to key feature mapping to obtain a global key feature sequence, and the local feature sequence is subjected to value feature mapping to obtain a global value feature sequence; The global query feature sequence is divided into multiple query feature blocks; The global key feature sequence is divided into multiple key feature blocks, and the global value feature sequence is divided into multiple value feature blocks corresponding to the multiple key feature blocks respectively; For each query feature block, a local relevance matrix is ​​determined between the query feature block and each key feature block, and the local relevance matrix is ​​normalized to obtain a block attention weight matrix. Based on the block attention weight matrix, the value feature blocks corresponding to the multiple key feature blocks are fused to obtain local fused feature blocks, and the multiple local fused feature blocks are fused into the global feature corresponding to the query feature block; The global features corresponding to the multiple query feature blocks are combined according to the first direction to obtain a global feature sequence.

12. The method according to any one of claims 1 to 11, characterized in that, The step of inputting the dimensionality-reduced feature sequence into the large language model for text generation to obtain inference results for responding to the target question includes: Obtain the historical context sequence of the large language model for the target question, wherein the historical context sequence includes the inference text; The inference text is deleted from the historical context sequence to obtain the updated historical context sequence; The updated historical context sequence is concatenated with the dimensionality-reduced feature sequence to obtain the target input sequence. The target input sequence is input into the large language model to generate text, thereby obtaining the reasoning result used to respond to the target question.

13. The method according to any one of claims 1 to 12, characterized in that, Before segmenting and fusing the global feature sequence along the second direction with a preset step size to obtain multiple dimensionality-reduced features, the method further includes: Based on the preset memory capacity and preset inference latency limit of the large language model in the current operating environment, the upper limit of the context length for the target problem is determined. Based on the pre-defined reasoning accuracy index for the target problem, the lower limit of the context length for reasoning in the large language model is determined. The length of the target sequence is determined based on the upper and lower length limits. The ratio of the original sequence length of the global feature sequence to the target sequence length is determined as the sequence compression ratio. Based on the sequence compression ratio, the preset step size for segmenting and fusing the global feature sequence is determined.

14. The method according to any one of claims 1 to 12, characterized in that, Before inputting the dimensionality-reduced feature sequence into the large language model for text generation to obtain the inference result for responding to the target question, the method further includes: The number of times the large language model sequentially performs the local feature extraction, the global feature integration, and the segmented fusion on the one-dimensional semantic vector sequence is determined. If the number of processing steps is less than a preset threshold, the dimensionality-reduced feature sequence is used as a new one-dimensional semantic vector sequence, and the process proceeds to the step of sliding the one-dimensional semantic vector sequence along the first direction based on a preset one-dimensional sliding window until the number of processing steps equals the threshold. The last obtained dimensionality-reduced feature sequence is used as the input to the large language model for text generation.

15. The method according to any one of claims 1 to 12, characterized in that, The process of dividing the reasoning text into multiple semantic units includes: The inference text generated by the large language model for the target problem is scanned to obtain multiple delimiters; Based on the multiple delimiters, the inference text is truncated to obtain multiple candidate text fragments; For each candidate text segment, a semantic integrity check is performed on the candidate text segment to obtain the check result; The test results represent multiple semantically complete candidate text segments as multiple semantic units.

16. The method according to any one of claims 1 to 12, characterized in that, Before sliding along the first direction on the one-dimensional semantic vector sequence based on the preset one-dimensional sliding window, the method further includes: Obtain the raw inference data generated by the large language model for the target problem, wherein the raw inference data includes the inference text; If the original inference data also includes image data, the image data is input into a pre-trained visual coding network for feature extraction to obtain a visual feature sequence. The visual feature sequence is combined with the one-dimensional semantic vector sequence to form a multimodal feature sequence; The multimodal feature sequence is used as a new one-dimensional semantic vector sequence, and the process proceeds to the step of sliding along the one-dimensional semantic vector sequence in a first direction based on a preset one-dimensional sliding window.

17. A text processing device, characterized in that, The device includes: The text segmentation module is used to obtain the reasoning text generated by the pre-trained large language model for the target problem, and to divide the reasoning text into multiple semantic units. A vector combination module is used to convert each of the semantic units into a semantic vector, and to combine multiple semantic vectors into a one-dimensional semantic vector sequence based on the order of each of the semantic units in the reasoning text; The feature extraction module is used to slide a preset one-dimensional sliding window along a first direction on the one-dimensional semantic vector sequence, and to perform local feature extraction on the semantic vector covered by the one-dimensional sliding window at each sliding position to obtain multiple local associated features, wherein the first direction is the direction from the first position to the last position of the one-dimensional semantic vector sequence. The feature combination module is used to combine the multiple local associated features according to the first direction to obtain a local feature sequence, and to perform global feature integration on the local feature sequence to obtain a global feature sequence. The feature dimensionality reduction module is used to segment and fuse the global feature sequence along a second direction with a preset step size to obtain multiple dimensionality reduction features, and combine the multiple dimensionality reduction features according to the second direction to obtain a dimensionality reduction feature sequence, wherein the second direction is the direction from the first position to the last position of the global feature sequence, and the sequence length of the dimensionality reduction feature sequence is less than the sequence length of the global feature sequence. The text generation module is used to input the dimensionality-reduced feature sequence into the large language model to generate text and obtain the reasoning result used to respond to the target question.

18. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the text processing method according to any one of claims 1 to 16.

19. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the text processing method according to any one of claims 1 to 16.

20. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the text processing method according to any one of claims 1 to 16.

Images