Task response method and device based on large language model, electronic device, and medium

Abstract

The present application relates to the field of artificial intelligence technology, and in particular to a task response method and device, electronic device, and medium based on a large language model. The present application needs to first obtain target sequence data; select current sequence sub-data from multiple sequence sub-data in the target sequence data; convert the current sequence sub-data into the corresponding current feature vector through the input layer; update the sequence memory parameter according to the current feature vector through the core layer, and reselect the current sequence sub-data from multiple sequence sub-data in the target sequence data according to the sorting of time steps; return to execute and input the current sequence sub-data into the large linear recurrent neural network model until all sequence sub-data of the target sequence data are selected, and generate a task response result according to the current sequence sub-data and sequence memory parameters through the output layer. In this way, when the large language model faces the processing requirements of longer sequence data, it can bring less computing power burden to the computing power terminal.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a task response method and device, electronic device, and medium based on a large language model. Background Art

[0002] Large language models (LLMs) are a type of deep learning-based AI model. They are trained on large amounts of text data to learn the complex patterns and structures of language, enabling them to understand and generate natural language. Many LLMs have been developed based on the Transformer architecture for a variety of natural language processing tasks.

[0003] It's worth noting that large language models based on the Transformer architecture perform exceptionally well in natural language processing tasks. However, when processing long sequences of data, the memory and energy consumption they incur on computing power terminals limits their efficiency and applicability. Therefore, how to minimize the computing burden on large language models while processing long sequences of data has become a pressing issue in the industry. Summary of the Invention

[0004] This application aims to solve at least one of the technical problems existing in the prior art. To this end, this application proposes a task response method and device based on a large language model, an electronic device, and a medium, which can reduce the computing power burden on the computing terminal when the large language model faces the processing requirements of long sequence data.

[0005] According to the first aspect of the present application, a task response method based on a large language model includes:

[0006] Acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, each of the sequence sub-data corresponds to one time step;

[0007] Selecting current sequence sub-data from a plurality of sequence sub-data in the target sequence data;

[0008] Inputting the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters;

[0009] Converting the current sequence sub-data into a corresponding current feature vector through the input layer;

[0010] updating the sequence memory parameter according to the current feature vector through the core layer, and reselecting the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the order of the time steps;

[0011] Return to execute inputting the current sequence sub-data into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and generate a task response result according to the current sequence sub-data and the sequence memory parameters through the output layer.

[0012] According to some embodiments of the present application, the core layer includes a plurality of module layers stacked by residual connections, the module layers include a time mixing layer, and the updating of the sequence memory parameters according to the current feature vector by the core layer and the reselection of the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the order of the time steps include:

[0013] A time dependency relationship between the current feature vector and the sequence memory parameter is determined in the time mixing layer, and the sequence memory parameter is incrementally adjusted according to the time dependency relationship and the current feature vector to update the sequence memory parameter.

[0014] According to some embodiments of the present application, the time mixing layer includes a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer, and a gating layer. The incremental adjustment of the sequence memory parameter based on the time dependency and the current feature vector to update the sequence memory parameter includes:

[0015] In the first data-dependent interpolation layer, a first interpolation conversion process is performed on the sequence memory parameter according to the time-dependent relationship to obtain a first memory parameter;

[0016] In the first linear layer, performing a first linear transformation on the current feature vector to obtain a first current linear vector;

[0017] In the incremental rule operation layer, the first memory parameter is incrementally adjusted according to the first current linear vector to obtain a transitional memory parameter;

[0018] In the gating layer, gate adjustment processing is performed on the first current linear vector and the transition memory parameter to obtain the updated sequence memory parameter.

[0019] According to some embodiments of the present application, the incrementally adjusting the first memory parameter according to the first current linear vector to obtain a transitional memory parameter includes:

[0020] Determining a memory writing parameter and a memory erasing parameter according to the first current linear vector;

[0021] The first memory parameter is incrementally adjusted according to the memory writing parameter and the memory erasing parameter to obtain the transition memory parameter.

[0022] According to some embodiments of the present application, the incrementally adjusting the first memory parameter according to the memory writing parameter and the memory erasing parameter to obtain the transitional memory parameter includes:

[0023] Based on the time step corresponding to the first current linear vector, configuring a decay weight for the first memory parameter, configuring a write weight for the memory write parameter, and configuring an erase weight for the memory erase parameter;

[0024] The transition memory parameter is obtained by performing calculation based on the first memory parameter and the attenuation weight, the memory write parameter and the write weight, and the memory erase parameter and the erase weight.

[0025] According to some embodiments of the present application, the module layer includes the time mixing layer and the channel mixing layer of the residual connection, different current feature vectors and the sequence memory parameters have respective corresponding feature channels, and after the sequence memory parameters are incrementally adjusted according to the time dependency and the current feature vector, the method further includes:

[0026] The channel dependency relationship between the current feature vector and the corresponding sequence memory parameter is determined in the channel mixing layer, and the sequence memory parameter after incremental adjustment is subjected to channel fusion adjustment according to the channel dependency relationship and the current feature vector to update the sequence memory parameter.

[0027] According to some embodiments of the present application, the channel mixing layer includes a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer, and the performing channel fusion adjustment on the incrementally adjusted sequence memory parameters according to the channel dependency and the current feature vector to update the sequence memory parameters includes:

[0028] In the second data-dependent interpolation layer, a second interpolation conversion process is performed on the sequence memory parameters after incremental adjustment in different feature channels according to the time-dependent relationship to obtain a plurality of second memory parameters;

[0029] In the second linear layer, a second linear transformation is performed on the current feature vectors in different feature channels to obtain multiple second current linear vectors;

[0030] In the element-wise multiplication fusion layer, the second memory parameter and the second current linear vector between different feature channels are element-wise multiplied to update the sequence memory parameter.

[0031] According to the second aspect of the present application, a task response device based on a large language model includes:

[0032] A data acquisition module, configured to acquire target sequence data; wherein the target sequence data comprises a plurality of sequence sub-data sorted according to time steps, each of the sequence sub-data corresponding to one time step;

[0033] a data selection module, configured to select current sequence sub-data from a plurality of sequence sub-data in the target sequence data;

[0034] A task processing module is used to input the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters; the current sequence sub-data is converted into a corresponding current feature vector through the input layer; the sequence memory parameter is updated according to the current feature vector through the core layer, and the current sequence sub-data is reselected from the multiple sequence sub-data in the target sequence data according to the sorting of the time steps; the current sequence sub-data is input into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and a task response result is generated according to the current sequence sub-data and the sequence memory parameters through the output layer.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the task response method based on a large language model as described in any one of the embodiments of the first aspect of the present application.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program, and the program is executed by a processor to implement the task response method based on a large language model as described in any one of the embodiments of the first aspect of the present application.

[0037] The task response method and apparatus based on a large language model, electronic device, and medium according to the embodiments of the present application have at least the following beneficial effects:

[0038] According to the task response method based on the large language model of the present application, it is necessary to first obtain the target sequence data; wherein, the target sequence data includes multiple sequence sub-data sorted according to the time step, and each sequence sub-data corresponds to a time step; select the current sequence sub-data from the multiple sequence sub-data in the target sequence data; input the current sequence sub-data into the linear recurrent neural network large model; wherein, the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with a sequence memory parameter; the current sequence sub-data is converted into the corresponding current feature vector through the input layer; the sequence memory parameter is updated according to the current feature vector through the core layer, and the current sequence sub-data is reselected from the multiple sequence sub-data in the target sequence data according to the sorting of the time step; return to execute and input the current sequence sub-data into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and generate a task response result according to the current sequence sub-data and the sequence memory parameter through the output layer. In this way, when the large language model faces the processing requirements of longer sequence data, it can bring less computing power burden to the computing power terminal.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0041] Figure 1 This is a flowchart of a task response method based on a large language model according to an embodiment of the present application;

[0042] Figure 2 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0043] Figure 3 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0044] Figure 4 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0045] Figure 5 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0046] Figure 6 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0047] Figure 7This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0048] Figure 8 A schematic diagram of the model architecture of a large linear recurrent neural network model provided in an embodiment of the present application;

[0049] Figure 9 Schematic diagram of the structure of a task response device based on a large language model provided in an embodiment of the present application;

[0050] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.

[0052] In the description of this application, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly specifying the number or order of the technical features indicated.

[0053] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, left, right, front, and back, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0054] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0055] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "set," "install," and "connect" should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution. In addition, the identification of specific steps below does not represent a limitation on the order of steps and execution logic. The execution order and execution logic between each step should be understood and inferred with reference to the content described in the embodiments.

[0056] Large Language Models (LLMs) are a type of AI model based on deep learning. They are trained on large amounts of text data to learn the complex patterns and structures of language, enabling them to understand and generate natural language. These large language models have revolutionized the field of natural language processing (NLP) and are widely used in fields such as text generation, translation, summarization, and question-answering systems.

[0057] The Transformer architecture is a core technology in natural language processing. Based on this architecture, many large language models have been developed for various natural language processing tasks, including BERT and GPT. While large language models based on the Transformer architecture excel at natural language processing tasks, they face a number of challenges when processing long sequences of data. These challenges stem primarily from their fundamental design and computational requirements.

[0058] First, the Transformer architecture relies on a self-attention mechanism, which allows it to compute the relationship between each element in a sequence and all other elements. This mechanism can theoretically capture long-range dependencies in a sequence, but in practice, the amount of computation required grows exponentially with sequence length. This means that for a long text containing thousands of words, the Transformer architecture needs to calculate millions of attention scores, which requires not only a large amount of computing resources but also a large amount of memory to store these calculations.

[0059] Secondly, because the Transformer architecture processes the entire sequence at each time step, it consumes a significant amount of memory. When processing long sequences, the Transformer architecture must simultaneously retain information about all elements in the sequence, which requires a significant amount of memory. As the sequence length increases, so does the memory requirement, which can be a problem in resource-constrained environments.

[0060] Furthermore, as computational workloads and memory usage increase, energy consumption also increases. During training and inference, a large number of computational operations consume energy, especially when using high-performance computing devices such as GPUs or TPUs. This high energy consumption not only increases operating costs but also has a certain impact on the environment.

[0061] Finally, the Transformer architecture's low efficiency when processing long sequences also limits its use in certain application scenarios. For example, in applications requiring fast response times, such as real-time translation or speech recognition, the inference speed of large models is crucial. However, due to the aforementioned computational and memory limitations, the Transformer architecture may not be able to meet real-time processing requirements when processing long sequences.

[0062] In summary, although the Transformer architecture is very effective in many aspects, its memory consumption and energy consumption issues on computing terminals limit its efficiency and applicability when processing very long sequences.

[0063] This application aims to solve at least one of the technical problems existing in the prior art. To this end, this application proposes a task response method and device based on a large language model, an electronic device, and a medium, which can reduce the computing power burden on the computing terminal when the large language model faces the processing requirements of long sequence data.

[0064] The following is a further explanation based on the accompanying drawings.

[0065] Reference Figure 1 According to the first aspect of the present application, the task response method based on a large language model may include:

[0066] Step S101, obtaining target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each sequence sub-data corresponds to a time step;

[0067] Step S102, selecting current sequence sub-data from multiple sequence sub-data in the target sequence data;

[0068] Step S103: input the current sequence sub-data into a linear recurrent neural network model; wherein the linear recurrent neural network model includes an input layer, a core layer, and an output layer, and the core layer is configured with sequence memory parameters;

[0069] Step S104, converting the current sequence sub-data into the corresponding current feature vector through the input layer;

[0070] Step S105 , updating the sequence memory parameters according to the current feature vector through the core layer, and reselecting the current sequence sub-data from the multiple sequence sub-data in the target sequence data according to the order of the time steps;

[0071] Step S106 , returning to the execution to input the current sequence sub-data into the linear recurrent neural network large model until all sequence sub-data of the target sequence data are selected, and generating a task response result according to the current sequence sub-data and sequence memory parameters through the output layer.

[0072] In the face of the memory consumption and energy consumption problems faced by large language models when processing long sequence data, a task response method based on a large language model is proposed through steps S101 to S106 of this application, aiming to reduce the computing power burden caused by task response and enable the large linear recurrent neural network model to have the ability to process long sequence data. The method first obtains target sequence data, which includes multiple sequence sub-data, each sequence sub-data corresponding to a time step. This method allows the large linear recurrent neural network model to process only one sequence sub-data at a time, instead of processing the entire long sequence at the same time, thereby reducing the amount of data loaded into the memory at one time.

[0073] When processing each sub-sequence data point, the linear recurrent neural network model first selects a current sub-sequence data point from multiple sub-sequence data points and then inputs it into the large linear recurrent neural network model. This large linear recurrent neural network model consists of an input layer, a core layer, and an output layer. The core layer is configured with sequence memory parameters to store and track important information in the sequence. Through the input layer, the current sub-sequence data point is converted into a corresponding feature vector, which captures the key information of the sequence data point.

[0074] Next, the core layer uses this feature vector to update the sequence memory parameters. This step is crucial because it allows the large linear recurrent neural network model to retain and update knowledge of the entire sequence state as it processes each sub-sequence data point. After the update is complete, the large linear recurrent neural network model reselects the next current sub-sequence data point based on the order of the time steps, and this process repeats until all sub-sequence data points have been processed.

[0075] Finally, after all the sequence sub-data have been processed, the large linear recurrent neural network model generates a task response result through the output layer. This result is based on the current sequence sub-data and the updated sequence memory parameters, and it reflects the information of the entire sequence.

[0076] Overall, this task-response-based approach effectively reduces the memory and computational requirements of large linear recurrent neural network models at any given point in time by processing sequence data in steps. This approach not only reduces the computational burden but also maintains the ability of large linear recurrent neural network models to process long sequences of data, enabling them to operate effectively even in resource-constrained environments. In this way, large language models can process long sequences of data in a more efficient and energy-efficient manner while still providing accurate task response results.

[0077] In some embodiments, step S101 is to obtain target sequence data, wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each sequence sub-data corresponds to a time step;

[0078] In some embodiments, the target sequence data contains all the information that needs to be processed and analyzed by the large linear recurrent neural network model. The target sequence data includes multiple sequence sub-data, wherein each sequence sub-data contains information associated with a specific time step, so that the large linear recurrent neural network model can focus on the context of that specific time point without having to process the entire sequence at once. The benefits of doing so are multifaceted: first, it reduces the memory consumption of the large linear recurrent neural network model during processing, because only one sequence sub-data needs to be processed at a time instead of the entire sequence; second, it helps to capture the temporal dependencies in the sequence, because each sequence sub-data can be analyzed independently while maintaining the coherence of the sequence through references across sequence sub-data; finally, this method can improve the flexibility and efficiency of the large linear recurrent neural network model in processing long sequence data, because it allows the large linear recurrent neural network model to selectively focus on specific parts of the sequence as needed.

[0079] In addition, decomposing the target sequence data into sequence sub-data ordered by time steps also helps the linear recurrent neural network large model better understand and predict patterns and trends in the sequence. For example, when processing a piece of text, each sequence sub-data can represent a word or phrase, while the entire sequence represents the entire paragraph or document. By analyzing each sequence sub-data separately, the linear recurrent neural network large model can learn the basic language structure and semantic information that constitutes the entire sequence. Then, by integrating the information of all sequence sub-data, the linear recurrent neural network large model can generate a comprehensive understanding of the entire sequence, thereby making accurate responses and predictions in various natural language processing tasks. Therefore, obtaining and segmenting the target sequence data is a key step in realizing an efficient and effective linear recurrent neural network large model for language processing.

[0080] In some more specific embodiments, the target sequence data and the sequence sub-data may vary depending on the actual application scenario. In the application scenario indicated below, the target sequence data and the sequence sub-data may specifically refer to the following:

[0081] This approach targets long document processing applications in natural language processing. The target sequence data consists of the entire long document, which could be a complete research paper, a technical report, or the full text of an e-book. Sequence sub-data consists of chapters, paragraphs, or individual sentences within the document. During processing, the large linear recurrent neural network model can analyze each paragraph or sentence individually, with each paragraph or sentence representing a sequence sub-data.

[0082] Applications for time series forecasting. Target sequence data: Continuous data points collected over a period of time, such as a single day's stock price fluctuations, a month's temperature changes, or a year's rainfall data. Sequence sub-data: Smaller data segments within the target sequence divided chronologically, such as a single day's stock price fluctuations, a week's temperature changes, or a month's rainfall data.

[0083] For applications in audio processing. Target sequence data: A complete audio recording, such as an entire music piece, a speech, or a conversation recording. Sequence sub-data: Smaller segments of audio, such as a single measure in a music piece, a single sentence in a speech, or the utterances of a single speaker in a conversation.

[0084] Applications in bioinformatics for gene sequence analysis. Target sequence data: Complete genome or protein sequences, potentially from a species' genome or a protein family. Sequence sub-data: Specific regions within a genome, such as individual genes or regulatory regions, or specific domains within a protein sequence.

[0085] Targeted at resource-constrained embedded systems. Target sequence data: Sensor data sequences collected over long periods of time, such as a month's worth of temperature and humidity records, a month's worth of factory machine performance metrics, or weeks of health monitoring data. Sequence subdata: Segments of sensor data, such as a single day's worth of temperature and humidity records, a single week's worth of machine performance metrics, or a single health monitoring cycle.

[0086] In these scenarios, the target sequence data represents the complete dataset to be analyzed, while the subsequence data are smaller, manageable pieces of data that make up the entire sequence. By processing the subsequence data in steps, large linear recurrent neural network models can effectively manage memory consumption and computational load while still capturing key information and long-term dependencies in the sequence data. This implementation is particularly suitable for processing long sequence data that cannot be loaded into memory all at once, or for efficient data processing in environments with limited computing resources.

[0087] In step S102 of some embodiments, current sequence sub-data is selected from a plurality of sequence sub-data in the target sequence data;

[0088] In some embodiments, a key step in processing target sequence data is selecting the subset of sequence data that needs to be processed from the entire sequence. This step is fundamental to efficient sequence processing, especially when dealing with long sequences, which are difficult to process in their entirety. Therefore, it is necessary to break the sequence into smaller, manageable components, known as subsets of sequence data.

[0089] The process of selecting the current sequence sub-data can be done by selecting each sub-data point one by one in chronological order. This method is suitable for situations where the sequence data has a clear chronological order and the information at each time step is relatively independent. After selecting the current sequence sub-data point, the linear recurrent neural network model will conduct an in-depth analysis of it, extract useful features, and update the state of the linear recurrent neural network model. For example, in natural language processing, this may involve semantic analysis of a paragraph or sentence; in time series analysis, it may involve identifying trends and patterns in the data within a certain time period. In this way, the linear recurrent neural network model can gradually build an understanding of the entire sequence.

[0090] It should be understood that by processing only one subsequence at a time, the amount of data that needs to be loaded into memory simultaneously can be reduced, thereby reducing the demand on computing resources. This is particularly important in resource-constrained environments or scenarios where extremely long sequence data needs to be processed.

[0091] In general, selecting the current sequence subdata from the target sequence data is a key step in achieving efficient and scalable sequence processing. This step not only determines how the large linear recurrent neural network model processes and analyzes sequence data, but also affects the model's computational efficiency and resource usage. This allows the large linear recurrent neural network model to adapt to different application scenarios and resource constraints while maintaining high performance.

[0092] In some embodiments, step S103 includes inputting the current sequence sub-data into a linear recurrent neural network model; wherein the linear recurrent neural network model includes an input layer, a core layer, and an output layer, and the core layer is configured with sequence memory parameters;

[0093] In some embodiments, step S103 involves inputting the selected current sequence sub-data into a large linear recurrent neural network model. This large linear recurrent neural network model is specifically designed to process sequence data. It is not only capable of processing a single data point, but also capable of understanding and memorizing previous data points in the sequence, thereby capturing long-term dependencies in the sequence.

[0094] The linear recurrent neural network model consists of three main parts: input layer, core layer and output layer.

[0095] It's important to note that the input layer is responsible for converting sequence data into a form that can be processed by the linear recurrent neural network model. For example, this involves converting text into word vectors or audio signals into feature representations. Since the current sequence data typically needs to be converted into a specific form (such as a feature vector), this step allows the linear recurrent neural network model to extract useful information from the current sequence data.

[0096] It should be noted that the core layer is the main body of the linear recurrent neural network model. It is equipped with sequence memory parameters, which act as the model's "memory," storing information from previous time steps. These sequence memory parameters enable the model to consider past context when processing the current sub-sequence data, which is crucial for understanding patterns and dependencies within sequence data. The core layer integrates new input information by updating these memory parameters, thereby maintaining an understanding of the sequence history at each step. It should be understood that the sequence memory parameters, as the model's "memory," refer to the information accumulated about previous sub-sequence data while processing the current sub-sequence data. Specifically, when processing sequence data, the sequence memory parameters are used to store and represent the internal state of information from previous time steps. This internal state encodes the model's past input information, enabling it to consider previous contextual information at the current time step.

[0097] It should be noted that the output layer generates a response based on the current sequence sub-data and the updated sequence memory parameters. In natural language processing tasks, the specific responses generated by the output layer can include various types of task responses such as classification, summarization, or translation.

[0098] In step S104 of some embodiments, the current sequence sub-data is converted into a corresponding current feature vector through the input layer;

[0099] It should be noted that the input layer can be used to convert the current sequence sub-data into a feature vector. This step involves extracting key information that represents the current sequence sub-data from the raw data and encoding this information into a vector in a high-dimensional space. For example, when processing text data, this may involve converting words or phrases into their embedded representations; when processing audio data, it may involve extracting the features of the sound clip. This feature vector provides a compact and information-rich representation for subsequent processing, allowing the large linear recurrent neural network model to capture the important features of the current sequence sub-data.

[0100] In step S105 of some embodiments, the core layer updates the sequence memory parameter according to the current feature vector, and reselects the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the order of the time steps;

[0101] It should be noted that the core layer uses the current feature vector obtained from the input layer to update the sequence memory parameters. The sequence memory parameters are a state within the linear recurrent neural network model that stores the linear recurrent neural network model's memory of previously processed data. By updating these sequence memory parameters, the linear recurrent neural network model can combine newly processed sequence sub-data with previously processed contextual information, thereby capturing long-term dependencies in the sequence. After the sequence memory parameters are updated, the linear recurrent neural network model reselects the next sequence sub-data to be processed based on the order of the time steps. This dynamic selection mechanism allows the linear recurrent neural network model to determine the next processing target based on the updated memory state and time sequence after processing the current sub-sequence, further optimizing the processing flow and enabling the linear recurrent neural network model to more flexibly handle long sequence data.

[0102] Reference Figure 2 According to some embodiments of the present application, the core layer includes multiple module layers stacked by residual connections, and the module layers include a time mixing layer. The core layer updates the sequence memory parameters according to the current feature vector and reselects the current sequence sub-data from multiple sequence sub-data in the target sequence data according to the order of time steps, which may include:

[0103] Step S201, determining the time dependency between the current feature vector and the sequence memory parameter in the time mixing layer;

[0104] Step S202 : incrementally adjusting the sequence memory parameters according to the time dependency and the current feature vector to update the sequence memory parameters.

[0105] It should be noted that the core layer of the embodiment of the present application is composed of multiple module layers, which are stacked through residual connections to promote effective information flow and gradient propagation. Each module layer carries a specific function, among which the time mixing layer plays a crucial role in processing sequence data.

[0106] In the temporal mixing layer, the temporal dependency between the current feature vector and the sequence memory parameters must first be determined. This step is key to capturing long-term dependencies in the sequence. Temporal dependencies reflect the correlation between feature vectors corresponding to different time steps, allowing the large linear recurrent neural network model to understand the influence of individual sequence sub-data in the target sequence data at the time step order. By analyzing the interaction between the current feature vector and the sequence memory parameters, the large linear recurrent neural network model can identify important patterns and structures in the sequence.

[0107] After determining the temporal dependencies, the next step is to incrementally adjust the sequence memory parameters based on the temporal dependencies and the current feature vector. Incremental adjustment is a sophisticated update method that modifies the sequence memory parameters in small, gradual steps based on the currently processed sequence sub-data. This update method not only reflects the impact of the newly processed sequence sub-data but also preserves the knowledge gained in previous processing steps. Incremental adjustment enables the large linear recurrent neural network model to dynamically adapt to new inputs while retaining historical memory of previous sequence sub-data within the large linear recurrent neural network model.

[0108] Furthermore, updating the sequence memory parameters involves reselecting the current sub-sequence data from multiple sub-sequence data in the target sequence data based on the order of the time steps. This dynamic selection mechanism enables the large linear recurrent neural network model to flexibly determine the next sub-sequence data to process based on the updated sequence memory parameters and time dependencies, thereby more efficiently processing the entire target sequence data.

[0109] In summary, the process described in the embodiments of this application enables large linear recurrent neural network models to effectively process long sequence data while maintaining a global understanding of the entire sequence by determining temporal dependencies in the temporal mixing layer and incrementally adjusting the sequence memory parameters accordingly. This approach not only improves the ability of large linear recurrent neural network models to handle more complex target sequence data, but also reduces the demand for computing resources, enabling them to excel in a variety of sequence processing tasks while adapting to different application scenarios and resource constraints.

[0110] Reference Figure 3According to some embodiments of the present application, the time mixing layer includes a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer, and a gating layer. Incrementally adjusting the sequence memory parameters based on the time dependency and the current feature vector to update the sequence memory parameters may include:

[0111] Step S301, in a first data-dependent interpolation layer, performing a first interpolation conversion process on a sequence memory parameter according to a time-dependent relationship to obtain a first memory parameter;

[0112] Step S302: performing a first linear transformation on the current feature vector in a first linear layer to obtain a first current linear vector;

[0113] Step S303: In the incremental rule operation layer, incrementally adjust the first memory parameter according to the first current linear vector to obtain a transitional memory parameter;

[0114] Step S304: In the gating layer, gate adjustment processing is performed on the first current linear vector and the transition memory parameter to obtain updated sequence memory parameters.

[0115] In some embodiments of the present application, the temporal mixing layer is a key component of the core layer of a large linear recurrent neural network model, responsible for handling temporal dependencies in time series data. The temporal mixing layer consists of multiple sublayers, including a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer, and a gating layer. These sublayers work together to incrementally adjust the sequence memory parameters, thereby updating these parameters and capturing the temporal dynamics in the sequence data.

[0116] In step S301 of some embodiments, in a first data-dependent interpolation layer, the time dependency is used to perform a first interpolation transformation on the sequence memory parameters. This step aims to generate a smooth representation of the sequence memory parameters that reflects the time-dependent structure of the sequence. Through this interpolation transformation, the large linear recurrent neural network model can fill in gaps or missing data points in the sequence while maintaining the continuity of the time series data.

[0117] In step S302 of some embodiments, in a first linear layer, the current feature vector is subjected to a first linear transformation. This transformation maps the feature vector to a new space to highlight the features that are most important for subsequent processing steps. This linear transformation typically involves matrix multiplication and bias addition, providing a way for the linear recurrent neural network model to extract and emphasize key information in the current sequence sub-data.

[0118] In step S303 of some embodiments, in the incremental rule operation layer, the first current linear vector is used to incrementally adjust the first memory parameter. This step is key to updating the sequence memory parameters based on new feature information. Incremental adjustment ensures that the large linear recurrent neural network model can gradually integrate new data while avoiding the instability that may be caused by a large one-time update. Incremental rules can take different forms, for example, updating memory parameters by adding, replacing, or accumulating new information.

[0119] In step S304 of some embodiments, in the gating layer, the first current linear vector and the transition memory parameter are used to perform a gated adjustment process. The gating layer controls the flow of information, determining which information should be retained and which should be forgotten. This adjustment process is implemented through a gating mechanism, which can be a learned parameter or a dynamically calculated result based on the current context. The output of the gating layer is the updated sequence memory parameter, which reflects the linear recurrent neural network model's understanding of the current sequence sub-data while incorporating information from previous time steps.

[0120] It should be understood that the processing flow described in the embodiments of the present application achieves incremental updates to the sequence memory parameters by performing a series of sophisticated operations in the time mixing layer. This approach not only improves the ability of the large linear recurrent neural network model to process time series data, but also enables it to flexibly adapt to new inputs while maintaining continuous memory of the sequence history. In this way, the large linear recurrent neural network model can more efficiently process long sequence data, capture temporal dependencies, and provide accurate responses in various sequence processing tasks.

[0121] Reference Figure 4 According to some embodiments of the present application, incrementally adjusting the first memory parameter according to the first current linear vector to obtain a transitional memory parameter may include:

[0122] Step S401, determining a memory writing parameter and a memory erasing parameter according to the first current linear vector;

[0123] Step S402 , incrementally adjusting the first memory parameter according to the memory writing parameter and the memory erasing parameter to obtain a transitional memory parameter.

[0124] In some embodiments of the present application, the process of incrementally adjusting the first memory parameter is achieved by carefully manipulating parameters of memory writing and memory erasing.

[0125] In step S401 of some embodiments, the linear recurrent neural network large model determines a memory write parameter and a memory erase parameter based on the first current linear vector. The memory write parameter determines the extent to which new information is updated to the memory parameter, reflecting the content of the current sequence sub-data that is worth remembering. Conversely, the memory erase parameter determines which old information is removed from memory, identifying information that is no longer relevant or outdated based on the current context and previous memory state.

[0126] In step S402 of some embodiments, after determining the memory write and erase parameters, the large linear recurrent neural network model then performs incremental adjustments, specifically adjusting the first memory parameters based on the memory write and erase parameters to calculate transitional memory parameters. Incremental adjustments are a dynamic update mechanism that allows the large linear recurrent neural network model to gradually integrate new information and forget old information. This adjustment does not simply replace memory content, but rather selectively retains and updates information in memory based on the importance and relevance of the current sequence sub-data.

[0127] In this way, the large linear recurrent neural network model is able to continuously update its internal state to reflect the latest understanding of the entire target sequence data. The sophisticated operation of memory writing and erasing enables the large linear recurrent neural network model to not only capture the characteristics of the current sequence sub-data, but also maintain its understanding of the target sequence data over a longer timeframe. This incremental adjustment mechanism is very effective when processing long sequence data because it allows the large linear recurrent neural network model to adaptively update its memory state when processing each sequence sub-data, thereby better capturing the temporal dependencies and complex patterns in the sequence data.

[0128] It should be understood that the incremental adjustment process described in the embodiments of this application provides a flexible and efficient method for updating sequence memory parameters by combining memory write parameters and memory erase parameters. This method enables large linear recurrent neural network models to process long sequence data in a controlled and step-by-step manner while maintaining a comprehensive understanding of the sequence history and current context, laying the foundation for generating accurate task response results.

[0129] Reference Figure 5 According to some embodiments of the present application, incrementally adjusting the first memory parameter according to the memory write parameter and the memory erase parameter to obtain the transition memory parameter may include:

[0130] Step S501, based on the time step corresponding to the first current linear vector, configuring an attenuation weight for the first memory parameter, configuring a write weight for the memory write parameter, and configuring an erase weight for the memory erase parameter;

[0131] Step S502 , calculating based on the first memory parameter and the attenuation weight, the memory write parameter and the write weight, and the memory erase parameter and the erase weight to obtain a transition memory parameter.

[0132] In some embodiments of the present application, the method for incrementally adjusting the first memory parameter controls the memory update process by finely configuring weights. This method involves configuring specific weights for different parameters to reflect the importance and role of each parameter in the update process.

[0133] In step S501 of some embodiments, based on the time step corresponding to the first current linear vector, a decay weight is configured for the first memory parameter, a write weight is configured for the memory write parameter, and an erase weight is configured for the memory erase parameter. The decay weight is used to control the influence of old memory information on the update of new memory. By configuring the decay weight for the first memory parameter, the linear recurrent neural network large model can determine to what extent the memory information in the previous time step is retained. This weight reflects the natural decay of memory over time, helping the linear recurrent neural network large model adapt to new input data while preventing important information from being forgotten too quickly. The write weight and erase weight, combined with the memory write parameter and memory erase parameter, determine the extent to which new information is written into the memory and the extent to which old information is forgotten. The write weight enhances the memory information associated with the current feature vector, while the erase weight reduces the memory information that is no longer relevant or outdated. This mechanism enables the linear recurrent neural network large model to process sequence data more flexibly and dynamically adjust its memory content according to the current context.

[0134] In step S502 of some embodiments, a calculation is performed based on these weighted parameters to obtain a transition memory parameter. This calculation process combines the effects of the decay weight, the write weight, and the erase weight to generate a new memory state that contains both updated memory information and removes outdated information. The transition memory parameter becomes the basis for the linear recurrent neural network model to make predictions or classifications at the next time step, providing the linear recurrent neural network model with an updated memory representation that is relevant to the current context.

[0135] It should be understood that the incremental adjustment process described in these embodiments provides a mechanism for controlling memory updates by assigning weights to memory parameters and performing calculations. This approach enables large linear recurrent neural network models to more accurately process time series data, capturing key information and forgetting content that is no longer relevant. In this way, large linear recurrent neural network models can maintain a dynamically updated memory state, providing strong support for processing complex sequence tasks.

[0136] In some more specific embodiments, if the time step is represented as t, the first memory parameter is represented as The memory write parameters are expressed as The memory erasure parameter is expressed as The transition memory parameter Memory is calculated based on the first memory parameter and attenuation weight, memory write parameter and write weight, and memory erase parameter and erase weight. t+1 , which can be expressed as:

[0137]

[0138] in, is the attenuation weight, α t To erase the weight, β t is the write weight. It should be noted that λ t In order to control the speed of memory decay over time, we can choose a suitable t The large linear recurrent neural network model can maintain sensitivity to key information over a longer time span, thereby effectively capturing long-distance dependencies. In addition, λ t , α t and β t All depend on the time step t. It should be noted that the memory write parameter and memory erasure parameters It is also dynamically calculated based on the current time step t.

[0139] Reference Figure 6 According to some embodiments of the present application, the module layer includes a residual-connected time mixing layer and a channel mixing layer, different current feature vectors and sequence memory parameters have corresponding feature channels, and after incrementally adjusting the sequence memory parameters based on the time dependency and the current feature vector, the following may also be included:

[0140] Step S601, determining the channel dependency between the current feature vector and the corresponding sequence memory parameters in the channel mixing layer;

[0141] Step S602 : performing channel fusion adjustment on the incrementally adjusted sequence memory parameters according to the channel dependency and the current feature vector to update the sequence memory parameters.

[0142] In some embodiments of the present application, the design of the module layer includes a time mixing layer and a channel mixing layer of residual connections. Such an architecture is intended to enhance the understanding and processing capabilities of the linear recurrent neural network large model for target sequence data through the fusion of feature channels. Under this framework, different current feature vectors and sequence memory parameters are assigned to their respective feature channels, so that the linear recurrent neural network large model can process information from different sources in parallel. After completing the incremental adjustment of the sequence memory parameters based on the time dependency and the current feature vector, the linear recurrent neural network large model enters the processing stage of the channel mixing layer.

[0143] In step S601 of some embodiments, the primary task of the channel mixing layer is to determine the channel dependencies between the current feature vector and the corresponding sequence memory parameters. This channel dependency reveals the interactions and influences between different feature channels, which may involve correlations or complementarities between different types of features. By identifying these channel dependencies, the large linear recurrent neural network model can understand how different features jointly influence the final task output.

[0144] In step S602 of some embodiments, the large linear recurrent neural network model performs channel fusion adjustment on the incrementally adjusted sequence memory parameters based on the channel dependencies and the current feature vector. This process involves integrating information from different channels to generate an updated sequence memory parameter that integrates multi-channel information. Channel fusion adjustment enables the large linear recurrent neural network model to not only capture dynamic changes in time series but also fuse multi-dimensional information at the feature level, thereby obtaining a more comprehensive and rich sequence representation.

[0145] It should be understood that the channel fusion adjustment process in the embodiments of the present application can be implemented using a variety of methods, such as weighted summation, feature interpolation, or attention mechanisms to integrate the contributions of different channels. In this way, the large linear recurrent neural network model can adaptively adjust the influence of different feature channels, strengthening those features that are most helpful for the current task while suppressing those that are less relevant or redundant.

[0146] The processing flow described in the embodiments of the present application further enhances the linear recurrent neural network large model's ability to understand and process sequence data by determining channel dependencies and adjusting channel fusion in the channel mixing layer. This method enables the linear recurrent neural network large model to handle complex sequence tasks more effectively, improves the performance and generalization ability of the linear recurrent neural network large model, especially when processing sequence data with multidimensional features. By carefully fusing information from different feature channels, the linear recurrent neural network large model can generate a rich memory representation that integrates temporal dynamics and multidimensional features, providing a solid foundation for generating accurate task response results.

[0147] Reference Figure 7 According to some embodiments of the present application, the channel mixing layer includes a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer. According to the channel dependency and the current feature vector, channel fusion adjustment is performed on the incrementally adjusted sequence memory parameters to update the sequence memory parameters, which may include:

[0148] Step S701: In a second data-dependent interpolation layer, a second interpolation conversion process is performed on the incrementally adjusted sequence memory parameters in different feature channels according to a time-dependent relationship to obtain a plurality of second memory parameters.

[0149] Step S702: In a second linear layer, a second linear transformation is performed on the current feature vectors in different feature channels to obtain a plurality of second current linear vectors.

[0150] Step S703: In the element-wise multiplication fusion layer, the second memory parameters between different feature channels and the second current linear vector are element-wise multiplied to update the sequence memory parameters.

[0151] In some embodiments of the present application, the channel mixing layer further processes and fuses information from different feature channels to update sequence memory parameters. The channel mixing layer consists of a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer. Each sublayer has a specific task, and together they implement channel fusion adjustment.

[0152] In step S701 of some embodiments, a second data-dependent interpolation layer uses time dependencies to perform a second interpolation conversion on the incrementally adjusted sequence memory parameters in different feature channels. This step helps smooth temporal variations in the sequence, generating an interpolated sequence memory parameter, i.e., a second memory parameter, for each feature channel. This interpolation process not only helps fill potential time gaps but also helps the large linear recurrent neural network model establish a more coherent sequence representation between different time steps.

[0153] In step S702 of some embodiments, the second linear layer performs a second linear transformation on the current feature vectors in different feature channels. This transformation generates multiple second current linear vectors, each of which represents information about a specific feature channel at the current time step. This step further refines the feature vectors, making them more suitable for subsequent fusion processes.

[0154] In some embodiments of the present application, the second linear layer is responsible for performing a second linear transformation on the current feature vectors in different feature channels to refine and emphasize the key information in these features. Although this step mainly involves linear operations, the introduction of activation functions can significantly enhance the effectiveness of this process.

[0155] The activation function in the second linear layer introduces nonlinearity, enabling large linear recurrent neural network models to learn and represent more complex functions and patterns. Without activation functions, regardless of the number of layers in the network, only linear combinations of the input data can be learned, limiting the expressive power and complexity of large linear recurrent neural network models. By applying an activation function in the second linear layer, the linear transformation results in each feature channel undergo a nonlinear transformation, which helps reveal complex structures and relationships in the data.

[0156] The choice of activation function can be based on a variety of considerations. For example, ReLU (Rectified Linear Unit) is widely used due to its high computational efficiency and ability to alleviate the vanishing gradient problem. When the ReLU activation function is applied, it sets all negative values to zero while leaving positive values unchanged. This operation helps retain important positive features while reducing the influence of unimportant features. In addition, variants of the activation function such as Leaky ReLU or Parametric ReLU can be further improved to allow small gradient values to pass through, thereby avoiding the problem of neuron death.

[0157] In this way, the activation function in the second linear layer not only increases the nonlinear expressiveness of the linear recurrent neural network model but also helps improve its training efficiency and ultimate performance. It enables the linear recurrent neural network model to capture and utilize complex features in the data, thereby achieving better results in various sequence processing tasks. Therefore, although the second linear layer mainly performs linear transformations, the introduction of the activation function is a key step in realizing efficient and powerful linear recurrent neural network models for sequence processing.

[0158] In step S703 of some embodiments, the element-wise multiplication fusion layer performs element-wise multiplication on the second memory parameters and the second current linear vector across different feature channels. This operation is an effective fusion mechanism that allows large linear recurrent neural network models to combine information from different channels at the element level, thereby updating the sequence memory parameters. Element-wise multiplication is particularly suitable for strengthening features that are important across multiple channels while suppressing features that are only significant in a few channels.

[0159] It should be understood that the channel mixing layer not only enhances the linear recurrent neural network model's understanding of the relationships between different feature channels, but also improves its ability to express multidimensional features. This channel fusion adjustment enables the linear recurrent neural network model to more effectively utilize information during feature extraction and sequence processing, generating a rich sequence memory representation that integrates multi-channel information.

[0160] The processing flow described in the embodiments of the present application realizes channel fusion adjustment of sequence memory parameters by performing detailed interpolation conversion, linear transformation and element-wise multiplication fusion in the channel mixing layer. This method enables the large linear recurrent neural network model to process multidimensional feature data more comprehensively, and improves the performance and generalization ability of the large linear recurrent neural network model when processing complex sequence tasks. In this way, the large linear recurrent neural network model can generate a rich memory representation that integrates temporal dynamics and multidimensional features, providing a solid foundation for generating accurate task response results.

[0161] Reference Figure 8 This paper provides a schematic diagram of the architecture of a large linear recurrent neural network model. This large linear recurrent neural network model has a complex structure designed to process target sequence data and capture the dependencies between sequence sub-data corresponding to different time steps. The architecture consists of multiple layers, including an input layer, a core layer, and an output layer. Each layer carries a specific function, which together enable efficient processing of sequence data.

[0162] The input layer receives the input vector x t These vectors represent the feature representation of the target sequence data at each time step (denoted as t), also known as the sub-sequence data. The purpose of the input layer is to convert the sub-sequence data into a form that can be further analyzed by the large linear recurrent neural network model, namely the feature vector. The core layer can be further divided into the time mixing layer and the channel mixing layer.

[0163] The feature vectors corresponding to the sequence sub-data first enter the time mixing layer through residual links. Operations performed here include data-dependent interpolation and incremental rule operations. These operations help the linear recurrent neural network model capture dynamic changes in the time series and update its internal state based on the current input. The time mixing layer further includes a linear layer and a gating mechanism. The linear layer performs a linear transformation on the input data, while the gating mechanism controls the flow of information, allowing the linear recurrent neural network model to selectively focus on or ignore certain information. This selectivity is very useful when processing complex sequence data because it allows the linear recurrent neural network model to focus on the most relevant parts.

[0164] Another component of the core layer is the channel mixing layer, which includes a linear layer and an activation function. The channel mixing layer handles the dependencies between feature channels and fuses information from different channels through operations such as element-wise multiplication. The introduction of activation functions provides nonlinear processing capabilities to the large linear recurrent neural network model, enabling it to capture more complex feature relationships.

[0165] Finally, the data processed by the channel mixing layer is passed to the output layer through residual connections and normalization. A linear layer can also be included between the channel mixing layer and the output layer. This layer is responsible for generating the final output based on the internal state of the linear recurrent neural network model and the processed information. This output can be a classification, prediction, or other task response for the target sequence data.

[0166] It should be understood that Figure 8 The large linear recurrent neural network model presented in [1] enables in-depth analysis and processing of target sequence data through its carefully designed hierarchical structure and various operations. Each component of the large linear recurrent neural network model is optimized for a specific data processing task. From feature extraction at the input layer to processing of temporal and channel dependencies at the core layer, to the final decision at the output layer, the entire architecture works together to achieve efficient sequence data processing.

[0167] In step S106 of some embodiments, the process returns to input the current sequence sub-data into the linear recurrent neural network model until all sequence sub-data of the target sequence data are selected, and generates a task response result according to the current sequence sub-data and sequence memory parameters through the output layer.

[0168] It should be noted that the linear recurrent neural network model will continue to execute the above process of updating the sequence memory parameters until all the sequence sub-data in the target sequence data have been processed. This iterative processing method ensures that the linear recurrent neural network model can process each part of the target sequence data one by one while maintaining a global understanding of the entire target sequence data. When all the sequence sub-data have been processed, the linear recurrent neural network model generates the final task response result through the output layer. This task response result is based on the last processed sequence sub-data and the updated sequence memory parameters. It reflects the linear recurrent neural network model's in-depth understanding of the entire target sequence data, including its long-term dependencies and complex patterns. The design of the output layer enables the linear recurrent neural network model to output its learned information in a way that is meaningful for a specific task. For example, it can be a classification label for text, a transcription of audio, or a predicted value for a sequence.

[0169] It is worth noting that the process of inputting the current sequence sub-data into the large linear recurrent neural network model is the key to achieving efficient processing of long sequence data. This approach allows the large linear recurrent neural network model to process only a portion of the sequence at a time, thereby reducing the demand for memory and enabling the large linear recurrent neural network model to run in environments with limited computing resources. At the same time, by maintaining sequence memory parameters at the core layer, the large linear recurrent neural network model is able to maintain a global understanding of the entire sequence, even if it is processed in parts. This design not only improves the ability of the large linear recurrent neural network model to process long sequence data, but also enables it to flexibly adapt to different tasks and data types.

[0170] Reference Figure 9 According to the second embodiment of the present application, the task response device based on the large language model may include:

[0171] The data acquisition module 901 is used to acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each sequence sub-data corresponds to a time step;

[0172] A data selection module 902 is configured to select current sequence sub-data from a plurality of sequence sub-data in the target sequence data;

[0173] The task processing module 903 is used to input the current sequence sub-data into the linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters; the current sequence sub-data is converted into the corresponding current feature vector through the input layer; the sequence memory parameter is updated according to the current feature vector through the core layer, and the current sequence sub-data is reselected from the multiple sequence sub-data in the target sequence data according to the sorting of time steps; the current sequence sub-data is input into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and the task response result is generated according to the current sequence sub-data and the sequence memory parameters through the output layer.

[0174] It can be seen that the contents of the above-mentioned embodiment of the task response method based on the large language model are all applicable to the embodiment of the task response device based on the large language model. The functions specifically implemented by the embodiment of the task response device based on the large language model are the same as those of the above-mentioned embodiment of the task response method based on the large language model, and the beneficial effects achieved are also the same as those achieved by the above-mentioned embodiment of the task response method based on the large language model.

[0175] Reference Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0176] The processor 1001 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0177] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the task response method based on the large language model in the embodiments of this application;

[0178] Input / output interface 1003, used to implement information input and output;

[0179] Communication interface 1004, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0180] Bus 1005 , which transmits information between various components of the device (e.g., processor 1001 , memory 1002 , input / output interface 1003 , and communication interface 1004 );

[0181] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0182] The present application also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, so that the computer device executes the above-mentioned task response method based on a large language model.

[0183] The terms "first," "second," "third," "fourth," and the like (if any) in the specification of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein, for example, can be implemented in orders other than those illustrated or described herein. In addition, the terms "comprises" and "comprising," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0184] It should be understood that in the present disclosure, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, and may include any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0185] It should be understood that in the description of the embodiments of the present application, multiple (or multiple items) means more than two, greater than, less than, exceed, etc. are understood to exclude the number itself, and above, below, within, etc. are understood to include the number itself.

[0186] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0187] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0188] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and can include a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0190] It should also be understood that the various implementation methods provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0191] The above is a specific description of the implementation methods of the present disclosure, but the present disclosure is not limited to the above implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present disclosure. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A task response method based on a large language model, characterized in that: include: Acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, each of the sequence sub-data corresponds to one time step; Selecting current sequence sub-data from a plurality of sequence sub-data in the target sequence data; Inputting the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer, and an output layer, the core layer is configured with a sequence memory parameter, the core layer includes a plurality of module layers stacked by residual connections, the module layers include a time mixing layer and a channel mixing layer including residual connections, and different current feature vectors and the sequence memory parameters have respective corresponding feature channels; Converting the current sequence sub-data into a corresponding current feature vector through the input layer; determining, in the time mixing layer, a time dependency relationship between the current feature vector and the sequence memory parameter, and incrementally adjusting the sequence memory parameter based on the time dependency relationship and the current feature vector; Determining, in the channel mixing layer, a channel dependency relationship between the current feature vector and the corresponding sequence memory parameter, and performing a channel fusion adjustment on the incrementally adjusted sequence memory parameter according to the channel dependency relationship and the current feature vector to update the sequence memory parameter; Return to execute inputting the current sequence sub-data into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and generate a task response result according to the current sequence sub-data and the sequence memory parameters through the output layer.

2. The task response method according to claim 1, characterized in that: The time mixing layer includes a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer, and a gating layer. The incremental adjustment of the sequence memory parameter according to the time dependency and the current feature vector includes: In the first data-dependent interpolation layer, a first interpolation conversion process is performed on the sequence memory parameter according to the time-dependent relationship to obtain a first memory parameter; In the first linear layer, performing a first linear transformation on the current feature vector to obtain a first current linear vector; In the incremental rule operation layer, the first memory parameter is incrementally adjusted according to the first current linear vector to obtain a transitional memory parameter; In the gating layer, gate adjustment processing is performed on the first current linear vector and the transition memory parameter.

3. The task response method according to claim 2, characterized in that: The incrementally adjusting the first memory parameter according to the first current linear vector to obtain a transition memory parameter includes: determining a memory writing parameter and a memory erasing parameter according to the first current linear vector; The first memory parameter is incrementally adjusted according to the memory writing parameter and the memory erasing parameter to obtain the transition memory parameter.

4. The task response method according to claim 3, characterized in that: The step of incrementally adjusting the first memory parameter according to the memory writing parameter and the memory erasing parameter to obtain the transitional memory parameter includes: Based on the time step corresponding to the first current linear vector, configuring a decay weight for the first memory parameter, configuring a write weight for the memory write parameter, and configuring an erase weight for the memory erase parameter; The transition memory parameter is obtained by performing calculation based on the first memory parameter and the attenuation weight, the memory write parameter and the write weight, and the memory erase parameter and the erase weight.

5. The task response method according to claim 1, characterized in that: The channel mixing layer includes a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer. The channel fusion adjustment is performed on the incrementally adjusted sequence memory parameters according to the channel dependency and the current feature vector to update the sequence memory parameters, including: In the second data-dependent interpolation layer, a second interpolation conversion process is performed on the sequence memory parameters after incremental adjustment in different feature channels according to the time-dependent relationship to obtain a plurality of second memory parameters; In the second linear layer, a second linear transformation is performed on the current feature vectors in different feature channels to obtain multiple second current linear vectors; In the element-wise multiplication fusion layer, the second memory parameter and the second current linear vector between different feature channels are element-wise multiplied to update the sequence memory parameter.

6. A task response device based on a large language model, characterized in that: include: A data acquisition module, configured to acquire target sequence data; wherein the target sequence data comprises a plurality of sequence sub-data sorted according to time steps, each of the sequence sub-data corresponding to one time step; a data selection module, configured to select current sequence sub-data from a plurality of sequence sub-data in the target sequence data; A task processing module is used to input the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, the core layer is configured with a sequence memory parameter, the core layer includes a plurality of module layers stacked by residual connections, the module layer includes a time mixing layer and a channel mixing layer including residual connections, and different current feature vectors and the sequence memory parameters have respective corresponding feature channels; the current sequence sub-data is converted into a corresponding current feature vector through the input layer; the sequence memory parameter is updated according to the current feature vector through the core layer, and the current sequence is reselected from the plurality of sequence sub-data in the target sequence data according to the order of the time steps. column sub-data; determining the time dependency between the current feature vector and the sequence memory parameter in the time mixing layer, and incrementally adjusting the sequence memory parameter according to the time dependency and the current feature vector; determining the channel dependency between the current feature vector and the corresponding sequence memory parameter in the channel mixing layer, and performing channel fusion adjustment on the incrementally adjusted sequence memory parameter according to the channel dependency and the current feature vector to update the sequence memory parameter; returning to execute inputting the current sequence sub-data into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and generating a task response result according to the current sequence sub-data and the sequence memory parameters through the output layer.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the task response method based on a large language model according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the task response method based on a large language model according to any one of claims 1 to 5.

Images