Task response method and device based on large language model, electronic equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a task response method and device based on a large language model, electronic equipment and a medium. The method comprises the steps of firstly obtaining target sequence data; selecting current sequence sub-data from a plurality of sequence sub-data in the target sequence data; converting the current sequence sub-data into a corresponding current feature vector through an input layer; updating the sequence memory parameter according to the current feature vector through the core layer, and reselecting the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the sequence of the time steps; and returning to execute the operation of inputting the current sequence sub-data into the linear recurrent neural network large model until each sequence sub-data of the target sequence data is selected, and generating a task response result through an output layer according to the current sequence sub-data and the sequence memory parameters. In this way, when the large language model faces the processing requirement of long sequence data, small computing power burden can be brought to the computing power terminal.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a task response method and device, electronic device, and medium based on a large language model. Background Art

[0002] Large Language Models (LLMs) are a type of artificial intelligence large model based on deep learning. They are trained on large amounts of text data to learn the complex patterns and structures of language, so that they can understand and generate natural language. Many large language models have been developed based on the Transformer architecture to handle various natural language processing tasks.

[0003] It is worth noting that the large language models of the Transformer architecture perform well in natural language processing tasks, but when they process longer sequence data, the memory and energy consumption problems caused by them on the computing power terminal limit their efficiency and applicability. Therefore, how to reduce the computing power burden when processing long sequence data with large language models has become a problem that needs to be solved urgently in the industry. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a task response method and device based on a large language model, an electronic device, and a medium, which can bring less computing power burden to the computing power terminal when the large language model faces the processing requirements of longer sequence data.

[0005] According to the first aspect of the present application, a task response method based on a large language model includes:

[0006] Acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each of the sequence sub-data corresponds to one of the time steps;

[0007] Selecting current sequence sub-data from a plurality of the sequence sub-data in the target sequence data;

[0008] Inputting the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model comprises an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters;

[0009] Converting the current sequence sub-data into a corresponding current feature vector through the input layer;

[0010] The sequence memory parameter is updated according to the current feature vector by the core layer, and the current sequence sub-data is reselected from the plurality of sequence sub-data in the target sequence data according to the order of the time steps;

[0011] Return to execute inputting the current sequence sub-data into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and generate a task response result through the output layer according to the current sequence sub-data and the sequence memory parameters.

[0012] According to some embodiments of the present application, the core layer includes a plurality of module layers stacked by residual connections, the module layer includes a time mixing layer, the updating of the sequence memory parameter according to the current feature vector by the core layer, and reselecting the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the order of the time steps, including:

[0013] The time dependency relationship between the current feature vector and the sequence memory parameter is determined in the time mixing layer, and the sequence memory parameter is incrementally adjusted according to the time dependency relationship and the current feature vector to update the sequence memory parameter.

[0014] According to some embodiments of the present application, the time mixing layer includes a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer and a gating layer, and the incremental adjustment of the sequence memory parameter according to the time dependency relationship and the current feature vector to update the sequence memory parameter includes:

[0015] In the first data-dependent interpolation layer, a first interpolation conversion process is performed on the sequence memory parameter according to the time-dependent relationship to obtain a first memory parameter;

[0016] In the first linear layer, a first linear transformation is performed on the current feature vector to obtain a first current linear vector;

[0017] In the incremental rule operation layer, the first memory parameter is incrementally adjusted according to the first current linear vector to obtain a transition memory parameter;

[0018] In the gating layer, gated adjustment processing is performed on the first current linear vector and the transition memory parameter to obtain the updated sequence memory parameter.

[0019] According to some embodiments of the present application, the step of incrementally adjusting the first memory parameter according to the first current linear vector to obtain a transition memory parameter includes:

[0020] Determining a memory writing parameter and a memory erasing parameter according to the first current linear vector;

[0021] The first memory parameter is incrementally adjusted according to the memory writing parameter and the memory erasing parameter to obtain the transition memory parameter.

[0022] According to some embodiments of the present application, the step of incrementally adjusting the first memory parameter according to the memory writing parameter and the memory erasing parameter to obtain the transition memory parameter includes:

[0023] Based on the time step corresponding to the first current linear vector, configuring an attenuation weight for the first memory parameter, configuring a write weight for the memory write parameter, and configuring an erase weight for the memory erase parameter;

[0024] The transition memory parameter is obtained by performing calculation based on the first memory parameter and the attenuation weight, the memory write parameter and the write weight, and the memory erase parameter and the erase weight.

[0025] According to some embodiments of the present application, the module layer includes the time mixing layer and the channel mixing layer of the residual connection, different current feature vectors and the sequence memory parameters have their own corresponding feature channels, and after the sequence memory parameters are incrementally adjusted according to the time dependency and the current feature vector, it also includes:

[0026] The channel dependency relationship between the current feature vector and the corresponding sequence memory parameter is determined in the channel mixing layer, and the sequence memory parameter after the incremental adjustment is subjected to channel fusion adjustment according to the channel dependency relationship and the current feature vector to update the sequence memory parameter.

[0027] According to some embodiments of the present application, the channel mixing layer includes a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer, and the step of performing channel fusion adjustment on the incrementally adjusted sequence memory parameters according to the channel dependency and the current feature vector to update the sequence memory parameters includes:

[0028] In the second data-dependent interpolation layer, the sequence memory parameters after the incremental adjustment in different feature channels are subjected to a second interpolation conversion process according to the time-dependent relationship to obtain a plurality of second memory parameters;

[0029] In the second linear layer, a second linear transformation is performed on the current feature vectors in different feature channels to obtain a plurality of second current linear vectors;

[0030] In the element-wise multiplication fusion layer, the second memory parameter and the second current linear vector between different feature channels are element-wise multiplied to update the sequence memory parameter.

[0031] According to the second aspect of the present application, a task response device based on a large language model includes:

[0032] A data acquisition module, used to acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, each of the sequence sub-data corresponds to one time step;

[0033] A data selection module, used for selecting current sequence sub-data from a plurality of sequence sub-data in the target sequence data;

[0034] A task processing module is used to input the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters; the current sequence sub-data is converted into a corresponding current feature vector through the input layer; the sequence memory parameter is updated according to the current feature vector through the core layer, and the current sequence sub-data is reselected from a plurality of the sequence sub-data in the target sequence data according to the order of the time steps; the current sequence sub-data is input into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and a task response result is generated according to the current sequence sub-data and the sequence memory parameters through the output layer.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the task response method based on a large language model as described in any one of the embodiments of the first aspect of the present application.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement a task response method based on a large language model as described in any one of the embodiments of the first aspect of the present application.

[0037] The task response method and device based on the large language model, the electronic device, and the medium according to the embodiments of the present application have at least the following beneficial effects:

[0038] According to the task response method based on the large language model of the present application, it is necessary to first obtain the target sequence data; wherein the target sequence data includes multiple sequence sub-data sorted according to the time step, and each sequence sub-data corresponds to a time step; select the current sequence sub-data from the multiple sequence sub-data in the target sequence data; input the current sequence sub-data into the linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with a sequence memory parameter; the current sequence sub-data is converted into the corresponding current feature vector through the input layer; the sequence memory parameter is updated according to the current feature vector through the core layer, and the current sequence sub-data is reselected from the multiple sequence sub-data in the target sequence data according to the sorting of the time step; return to execute the input of the current sequence sub-data into the linear recurrent neural network large model until each sequence sub-data of the target sequence data is selected, and the task response result is generated according to the current sequence sub-data and the sequence memory parameter through the output layer. In this way, when the large language model faces the processing requirements of longer sequence data, it can bring less computing power burden to the computing power terminal.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0041] Figure 1 A flowchart of a task response method based on a large language model according to an embodiment of the present application;

[0042] Figure 2 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0043] Figure 3 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0044] Figure 4 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0045] Figure 5 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0046] Figure 6 This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0047] Figure 7This is another flowchart of the task response method based on a large language model according to an embodiment of the present application;

[0048] Figure 8 A schematic diagram of the model architecture of a large linear recurrent neural network model provided in an embodiment of the present application;

[0049] Fig. 9 It is a structural diagram of a task response device based on a large language model provided in an embodiment of the present application;

[0050] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0052] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0053] In the description of the present application, it should be understood that descriptions involving orientation, such as up, down, left, right, front, back, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0054] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0055] In the description of this application, it should be noted that, unless otherwise clearly defined, the terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application in combination with the specific content of the technical solution. In addition, the identification of specific steps below does not represent a limitation on the order of steps and execution logic. The execution order and execution logic between each step should be understood and inferred with reference to the contents described in the embodiment.

[0056] Large Language Models (LLMs) are a type of artificial intelligence large model based on deep learning. They are trained on large amounts of text data to learn the complex patterns and structures of language, so that they can understand and generate natural language. These large language models have made revolutionary progress in the field of natural language processing (NLP) and are widely used in many fields such as text generation, translation, summarization, and question-answering systems.

[0057] The Transformer architecture is one of the core technologies in the field of natural language processing. Based on this architecture, many large language models have been developed to handle various natural language processing tasks. Large language models based on the Transformer architecture include BERT, GPT and other models. It is worth noting that large language models with the Transformer architecture perform well in handling natural language processing tasks, but they face a series of challenges when processing longer sequence data. These challenges mainly stem from their basic design and computational requirements.

[0058] First, the Transformer architecture relies on a self-attention mechanism, which allows the Transformer architecture to compute the relationship between each element of a sequence and all other elements. This mechanism can theoretically capture long-distance dependencies in a sequence, but in practice, the amount of computation grows exponentially as the length of the sequence increases. This means that for a long text containing thousands of words, the Transformer architecture needs to calculate millions of attention scores, which requires not only a lot of computing resources, but also a lot of memory to store these calculations.

[0059] Secondly, since the Transformer architecture needs to process the entire sequence at each time step, it consumes a lot of memory. When processing long sequences, the Transformer architecture must retain information about all elements in the sequence at the same time, which requires a lot of memory space. As the length of the sequence increases, the memory requirements also increase, which can become a problem in resource-limited environments.

[0060] Furthermore, as the amount of computation and memory consumption increases, energy consumption also increases. During training and inference, a large number of computational operations require energy consumption, especially when using high-performance computing devices such as GPUs or TPUs. This high energy consumption not only increases operating costs, but also has a certain impact on the environment.

[0061] Finally, the low efficiency when processing long sequences also limits the use of the Transformer architecture in some application scenarios. For example, in applications that require fast response, such as real-time translation or speech recognition, the inference speed of large models is crucial. However, due to the above-mentioned computational and memory limitations, the Transformer architecture may not be able to meet the requirements of real-time processing when processing long sequences.

[0062] In summary, although the Transformer architecture is very effective in many aspects, its memory consumption and energy consumption issues on computing terminals limit its efficiency and applicability when processing very long sequences.

[0063] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a task response method and device based on a large language model, an electronic device, and a medium, which can bring less computing power burden to the computing power terminal when the large language model faces the processing requirements of longer sequence data.

[0064] Further explanation is given below based on the accompanying drawings.

[0065] Reference Figure 1 According to the first aspect of the present application, the task response method based on the large language model may include:

[0066] Step S101, obtaining target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each sequence sub-data corresponds to a time step;

[0067] Step S102, selecting current sequence sub-data from a plurality of sequence sub-data in the target sequence data;

[0068] Step S103, inputting the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters;

[0069] Step S104, converting the current sequence sub-data into the corresponding current feature vector through the input layer;

[0070] Step S105, updating the sequence memory parameter according to the current feature vector through the core layer, and reselecting the current sequence sub-data from the multiple sequence sub-data in the target sequence data according to the order of the time steps;

[0071] Step S106, returning to execute inputting the current sequence sub-data into the linear recurrent neural network large model until all sequence sub-data of the target sequence data are selected, and generating a task response result through the output layer according to the current sequence sub-data and sequence memory parameters.

[0072] In the face of the memory consumption and energy consumption problems faced by large language models when processing long sequence data, a task response method based on a large language model is proposed through steps S101 to S106 of this application, aiming to reduce the computing power burden caused by task response, so that the large linear recurrent neural network model has the ability to process long sequence data. The method first obtains target sequence data, which includes multiple sequence sub-data, each sequence sub-data corresponding to a time step. This method allows the large linear recurrent neural network model to process only one sequence sub-data at a time, instead of processing the entire long sequence at the same time, thereby reducing the amount of data loaded into the memory at one time.

[0073] When processing each sequence sub-data, the linear recurrent neural network model first selects a current sequence sub-data from multiple sequence sub-data and then inputs it into the linear recurrent neural network model. This linear recurrent neural network model consists of an input layer, a core layer, and an output layer, where the core layer is configured with sequence memory parameters for storing and tracking important information in the sequence. Through the input layer, the current sequence sub-data is converted into a corresponding feature vector, which captures the key information of the sequence sub-data.

[0074] Next, the core layer uses this feature vector to update the sequence memory parameters. This step is crucial because it allows the large linear recurrent neural network model to retain and update knowledge about the state of the entire sequence as it processes each sequence sub-data. After the update is completed, the large linear recurrent neural network model reselects the next current sequence sub-data based on the order of the time steps, and this process is repeated until all sequence sub-data are processed.

[0075] Finally, when all the sequence sub-data are processed, the large linear recurrent neural network model generates a task response result through the output layer. This result is based on the current sequence sub-data and the updated sequence memory parameters, which reflects the information of the entire sequence.

[0076] In general, this task response-based approach effectively reduces the memory and computational requirements of a large linear recurrent neural network model at any given point in time by processing sequence data in steps. This approach not only reduces the computing burden, but also maintains the processing capability of a large linear recurrent neural network model for long sequence data, allowing it to work effectively in resource-constrained environments. In this way, large language models can process long sequence data in a more efficient and energy-efficient manner while still providing accurate task response results.

[0077] In some embodiments, step S101 is to obtain target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each sequence sub-data corresponds to a time step;

[0078] In some embodiments, the target sequence data contains all the information that needs to be processed and analyzed by the large linear recurrent neural network model. The target sequence data includes multiple sequence sub-data, each of which contains information associated with a specific time step, so that the large linear recurrent neural network model can focus on the context of that specific time point without having to process the entire sequence at once. The benefits of doing so are multifaceted: first, it reduces the memory consumption of the large linear recurrent neural network model during processing, because only one sequence sub-data needs to be processed at a time instead of the entire sequence; second, it helps capture time dependencies in the sequence, because each sequence sub-data can be analyzed independently while maintaining the coherence of the sequence through references across sequence sub-data; finally, this method can improve the flexibility and efficiency of the large linear recurrent neural network model in processing long sequence data, because it allows the large linear recurrent neural network model to selectively focus on specific parts of the sequence as needed.

[0079] In addition, decomposing the target sequence data into sequence sub-data ordered by time steps also helps the linear recurrent neural network model to better understand and predict patterns and trends in the sequence. For example, when processing a piece of text, each sequence sub-data can represent a word or phrase, and the entire sequence represents the entire paragraph or document. By analyzing each sequence sub-data separately, the linear recurrent neural network model can learn the basic language structure and semantic information that constitute the entire sequence. Then, by synthesizing the information of all sequence sub-data, the linear recurrent neural network model can generate a comprehensive understanding of the entire sequence, thereby making accurate responses and predictions in various natural language processing tasks. Therefore, obtaining and segmenting the target sequence data is a key step in implementing an efficient and effective linear recurrent neural network model for language processing.

[0080] In some more specific embodiments, the target sequence data and the sequence sub-data may vary according to different actual application scenarios. In the application scenarios indicated below, the target sequence data and the sequence sub-data may specifically refer to the following:

[0081] Targeted at long document processing applications in natural language processing. Target sequence data: the entire long document, which can be a complete research paper, a technical report, or the full text of an e-book. Sequence sub-data: chapters, paragraphs, or individual sentences in a document. During processing, the linear recurrent neural network model can analyze paragraph by paragraph or sentence by sentence, and each paragraph or sentence is a sequence sub-data.

[0082] Applications for time series forecasting. Target sequence data: continuous data points collected over a period of time, such as a stock market price change sequence within a day, a temperature change record for a month, or a year's rainfall data. Sequence sub-data: smaller data segments in the target sequence divided in time order, such as a single day's stock market price change, a week's temperature change, or a month's rainfall.

[0083] Applications targeting audio processing. Target sequence data: a complete audio recording, such as an entire music track, a speech, or a conversation recording. Sequence sub-data: smaller segments of audio, such as a single bar in a music track, a single sentence in a speech, or the utterances of a single speaker in a conversation.

[0084] Applications for gene sequence analysis in bioinformatics. Target sequence data: complete genome sequence or protein sequence, which may come from the genome of a species or a protein family. Sequence subdata: specific regions in the genome, such as a single gene or regulatory region, or specific domains in a protein sequence.

[0085] Applications for resource-constrained embedded systems. Target sequence data: A sequence of sensor data collected over a long period of time, such as temperature and humidity records over a month, machine performance indicators over a month, or health monitoring data over a few weeks. Sequence sub-data: A segment of sensor data, such as temperature and humidity records over a single day, machine performance indicators over a single week, or a single health monitoring cycle.

[0086] In these scenarios, the target sequence data represents the complete data set that needs to be analyzed, while the sequence sub-data are smaller, manageable pieces of data that make up the entire sequence. By processing the sequence sub-data in steps, the large linear recurrent neural network model can effectively manage memory consumption and computational load while still capturing key information and long-term dependencies in the sequence data. This implementation is particularly suitable for processing long sequence data that cannot be loaded into memory all at once, or for efficient data processing in environments with limited computing resources.

[0087] In step S102 of some embodiments, current sequence sub-data is selected from a plurality of sequence sub-data in the target sequence data;

[0088] In some embodiments, a key step in processing target sequence data is to select the sequence sub-data that needs to be processed currently from the entire sequence. This step is the basis for achieving effective sequence processing, especially when facing long sequence data, it is difficult to process the entire sequence at once, so it is necessary to decompose the sequence into smaller, manageable parts, namely, sequence sub-data.

[0089] The process of selecting the current sequence sub-data can select each sequence sub-data one by one in chronological order. This method is suitable for situations where the sequence data has a clear chronological order and the information of each time step is relatively independent. After selecting the current sequence sub-data, the linear recurrent neural network large model will conduct an in-depth analysis on it, extract useful features, and update the state of the linear recurrent neural network large model. For example, in natural language processing, this may involve semantic analysis of a paragraph or sentence; in time series analysis, it may involve the identification of trends and patterns in data within a certain period of time. In this way, the linear recurrent neural network large model can gradually build an understanding of the entire sequence.

[0090] It should be understood that by processing only one sequence sub-data at a time, the amount of data that needs to be loaded into memory at the same time can be reduced, thereby reducing the demand for computing resources. This is particularly important in resource-constrained environments or scenarios where particularly long sequence data needs to be processed.

[0091] In general, selecting the current sequence sub-data from the target sequence data is a key step to achieve efficient and scalable sequence processing. This step not only determines how the large linear recurrent neural network model processes and analyzes sequence data, but also affects the computational efficiency and resource usage of the large linear recurrent neural network model, which can enable the large linear recurrent neural network model to adapt to different application scenarios and resource constraints while maintaining high performance.

[0092] In step S103 of some embodiments, the current sequence sub-data is input into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters;

[0093] In some embodiments, step S103 involves inputting the selected current sequence sub-data into a linear recurrent neural network large model. This linear recurrent neural network large model is specially designed to process sequence data, and it can not only process a single data point, but also understand and remember the previous data points in the sequence, so as to capture the long-term dependencies in the sequence.

[0094] The linear recurrent neural network model consists of three main parts: input layer, core layer and output layer.

[0095] It should be noted that the input layer is responsible for converting the sequence sub-data into a form that can be processed by the linear recurrent neural network model, for example, converting text into word vectors or converting audio signals into feature representations. Since the current sequence sub-data usually needs to be converted into a specific form (such as a feature vector), this step allows the linear recurrent neural network model to extract useful information from the current sequence sub-data.

[0096] It should be noted that the core layer is the main body of the linear recurrent neural network model, which is configured with sequence memory parameters, which act as the "memory" of the linear recurrent neural network model and store information from previous time steps. The sequence memory parameters enable the linear recurrent neural network model to take into account past context when processing the current sequence sub-data, which is essential for understanding patterns and dependencies in sequence data. The core layer integrates the newly input information by updating these memory parameters, thereby maintaining an understanding of the sequence history at each step. It should be understood that the sequence memory parameters, as the "memory" of the linear recurrent neural network model, refer to the information about the previous sequence sub-data accumulated by the linear recurrent neural network model when processing the current sequence sub-data. Specifically, when the linear recurrent neural network model processes sequence data, the sequence memory parameters are used to store and represent the internal state of the previous time step information. This internal state is the linear recurrent neural network model's encoding of past input information, which enables the linear recurrent neural network model to take into account previous context information at the current time step.

[0097] It should be noted that the output layer generates a response based on the current sequence sub-data and the updated sequence memory parameters. In natural language processing tasks, the specific responses generated by the output layer can include various types of task responses such as classification, summary or translation.

[0098] In step S104 of some embodiments, the current sequence sub-data is converted into a corresponding current feature vector through an input layer;

[0099] It should be noted that the role of the input layer can be to convert the current sequence sub-data into a feature vector. This step involves extracting key information that can represent the current sequence sub-data from the original data and encoding this information into a vector in a high-dimensional space. For example, when processing text data, this may involve converting words or phrases into their embedded representations; when processing audio data, it may involve extracting the features of sound clips. This feature vector provides a compact and information-rich representation for subsequent processing, allowing the large linear recurrent neural network model to capture the important features in the current sequence sub-data.

[0100] In step S105 of some embodiments, the core layer updates the sequence memory parameter according to the current feature vector, and reselects the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the order of the time steps;

[0101] It should be noted that the core layer uses the current feature vector obtained from the input layer to update the sequence memory parameters. The sequence memory parameter is a state inside the linear recurrent neural network model, which saves the linear recurrent neural network model's memory of previously processed data. By updating these sequence memory parameters, the linear recurrent neural network model can combine the newly processed sequence sub-data with the previous context information to capture the long-term dependencies in the sequence. After this sequence memory parameter is updated, the linear recurrent neural network model will reselect the next sequence sub-data to be processed based on the order of the time steps. This dynamic selection mechanism allows the linear recurrent neural network model to determine the next processing target based on the updated memory state and time order after processing the current sub-sequence, further optimizing the processing flow and enabling the linear recurrent neural network model to cope with long sequence data more flexibly.

[0102] Reference Figure 2 According to some embodiments of the present application, the core layer includes a plurality of module layers stacked by residual connections, the module layer includes a time mixing layer, the core layer updates the sequence memory parameter according to the current feature vector, and reselects the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the order of the time steps, which may include:

[0103] Step S201, determining the time dependency between the current feature vector and the sequence memory parameter in the time mixing layer;

[0104] Step S202: incrementally adjust the sequence memory parameters according to the time dependency and the current feature vector to update the sequence memory parameters.

[0105] It should be noted that the core layer of the embodiment of the present application is composed of multiple module layers, which are stacked through residual connections, thereby promoting effective information flow and gradient propagation. Each module layer carries a specific function, among which the time mixing layer plays a vital role in processing sequence data.

[0106] In the time mixing layer, the time dependency between the current feature vector and the sequence memory parameters must first be determined. This step is key to capturing long-term dependencies in the sequence. The time dependency reflects the correlation between feature vectors corresponding to different time steps, which allows the linear recurrent neural network model to understand the influence relationship between each sequence sub-data in the target sequence data in the order of time steps. By analyzing the interaction between the current feature vector and the sequence memory parameters, the linear recurrent neural network model can identify important patterns and structures in the sequence.

[0107] After determining the temporal dependencies, the next step is to incrementally adjust the sequence memory parameters based on the temporal dependencies and the current feature vector. Incremental adjustment is a sophisticated update method that makes small, gradual changes to the sequence memory parameters based on the currently processed sequence sub-data. This update method not only reflects the impact of the newly processed sequence sub-data, but also retains the knowledge gained in the previous processing steps. Incremental adjustment allows the linear recurrent neural network model to dynamically adapt to new inputs while having historical memory of previous sequence sub-data in the linear recurrent neural network model.

[0108] In addition, the process of updating the sequence memory parameters also involves reselecting the current sequence sub-data from multiple sequence sub-data in the target sequence data according to the order of time steps. This dynamic selection mechanism enables the large linear recurrent neural network model to flexibly determine the next sequence sub-data to be processed based on the updated sequence memory parameters and time dependencies, thereby more efficiently processing the entire target sequence data.

[0109] In general, the processing flow described in the embodiments of the present application determines the time dependency in the time mixing layer and incrementally adjusts the sequence memory parameters accordingly, so that the large linear recurrent neural network model can effectively process long sequence data and maintain a global understanding of the entire sequence. This method not only improves the ability of the large linear recurrent neural network model to process more complex target sequence data, but also reduces the demand for computing resources, enabling it to perform well in various sequence processing tasks while adapting to different application scenarios and resource constraints.

[0110] Reference Figure 3According to some embodiments of the present application, the time mixing layer includes a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer, and a gating layer. According to the time dependency relationship and the current feature vector, the sequence memory parameter is incrementally adjusted to update the sequence memory parameter, which may include:

[0111] Step S301, in a first data-dependent interpolation layer, performing a first interpolation conversion process on a sequence memory parameter through a time-dependent relationship to obtain a first memory parameter;

[0112] Step S302, in a first linear layer, performing a first linear transformation on the current feature vector to obtain a first current linear vector;

[0113] Step S303, in the incremental rule operation layer, incrementally adjusting the first memory parameter according to the first current linear vector to obtain a transition memory parameter;

[0114] Step S304: In the gated layer, gated adjustment processing is performed on the first current linear vector and the transition memory parameter to obtain updated sequence memory parameters.

[0115] In some embodiments of the present application, the time mixing layer is a key component of the core layer in the large linear recurrent neural network model, which is responsible for processing the time dependency in the time series data. The time mixing layer consists of multiple sublayers, including a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer, and a gating layer. These sublayers work together to achieve incremental adjustments to the sequence memory parameters, thereby updating these parameters and capturing the time dynamics in the sequence data.

[0116] In step S301 of some embodiments, in a first data-dependent interpolation layer, the time dependency is used to perform a first interpolation conversion process on the sequence memory parameters. The purpose of this step is to generate a smooth sequence memory parameter representation that can reflect the time-dependent structure in the sequence. Through this interpolation conversion, the linear recurrent neural network large model can fill in the blanks or missing data points in the sequence while maintaining the continuity of the time series data.

[0117] In step S302 of some embodiments, in the first linear layer, the current feature vector is used to perform a first linear transformation. This transformation maps the feature vector to a new space to highlight the features that are most important to the subsequent processing steps. This linear transformation usually involves matrix multiplication and bias addition, which provides a way for the linear recurrent neural network large model to extract and emphasize the key information in the current sequence sub-data.

[0118] In step S303 of some embodiments, in the incremental rule operation layer, the first current linear vector is used to incrementally adjust the first memory parameter. This step is the key to updating the sequence memory parameters according to the new feature information. The incremental adjustment ensures that the large linear recurrent neural network model can gradually integrate new data while avoiding the instability that may be caused by a one-time large-scale update. The incremental rule can take different forms, for example, by adding, replacing or accumulating new information to update the memory parameter.

[0119] In step S304 of some embodiments, in the gating layer, the first current linear vector and the transition memory parameter are used to perform a gating adjustment process. The function of the gating layer is to control the information flow, which determines which information should be retained and which should be forgotten. This adjustment process is implemented through a gating mechanism, which can be a learned parameter or a dynamic calculation result based on the current context. The output of the gating layer is an updated sequence memory parameter, which reflects the understanding of the current sequence sub-data by the linear recurrent neural network large model, while integrating the previous time step information.

[0120] It should be understood that the processing flow described in the embodiments of the present application realizes an incremental update of the sequence memory parameters by performing a series of delicate operations in the time mixing layer. This method not only improves the ability of the large linear recurrent neural network model to process time series data, but also enables it to flexibly adapt to new inputs while maintaining continuous memory of the sequence history. In this way, the large linear recurrent neural network model can more effectively process long sequence data, capture time dependencies, and provide accurate responses in various sequence processing tasks.

[0121] Reference Figure 4 According to some embodiments of the present application, incrementally adjusting the first memory parameter according to the first current linear vector to obtain the transition memory parameter may include:

[0122] Step S401, determining a memory writing parameter and a memory erasing parameter according to the first current linear vector;

[0123] Step S402, incrementally adjusting the first memory parameter according to the memory writing parameter and the memory erasing parameter to obtain a transition memory parameter.

[0124] In some embodiments of the present application, the process of incrementally adjusting the first memory parameter is achieved by carefully manipulating the parameters of memory writing and memory erasing.

[0125] In step S401 of some embodiments, the linear recurrent neural network large model determines a memory write parameter and a memory erase parameter according to the first current linear vector. The memory write parameter is used to determine the extent to which new information is updated to the memory parameter, which reflects the content worth remembering in the current sequence sub-data. In contrast, the memory erase parameter is used to determine which old information is removed from the memory, which identifies information that is no longer relevant or outdated based on the current context and the previous memory state.

[0126] In step S402 of some embodiments, after determining the memory write parameter and the memory erase parameter, the linear recurrent neural network large model then performs incremental adjustment, that is, adjusting the first memory parameter according to the memory write parameter and the memory erase parameter to calculate the transition memory parameter. Incremental adjustment is a dynamic update mechanism that allows the linear recurrent neural network large model to gradually integrate new information and forget old information. This adjustment is not a simple replacement of memory content, but selectively retains and updates the information in memory based on the importance and relevance of the current sequence sub-data.

[0127] In this way, the large linear recurrent neural network model can continuously update its internal state to reflect the latest understanding of the entire target sequence data. The delicate operation of memory writing and erasing enables the large linear recurrent neural network model to not only capture the characteristics of the current sequence sub-data, but also maintain its understanding of the target sequence data over a longer time frame. This incremental adjustment mechanism is very effective when processing long sequence data because it allows the large linear recurrent neural network model to adaptively update its memory state when processing each sequence sub-data, thereby better capturing the time dependencies and complex patterns in the sequence data.

[0128] It should be understood that the incremental adjustment process described in the embodiments of the present application provides a flexible and effective method for updating sequence memory parameters by combining memory write parameters and memory erase parameters. This method enables a large linear recurrent neural network model to process long sequence data in a controlled and step-by-step manner while maintaining a comprehensive understanding of the sequence history and current context, laying the foundation for generating accurate task response results.

[0129] Reference Figure 5 According to some embodiments of the present application, incrementally adjusting the first memory parameter according to the memory writing parameter and the memory erasing parameter to obtain the transition memory parameter may include:

[0130] Step S501, based on the time step corresponding to the first current linear vector, configuring an attenuation weight for the first memory parameter, configuring a write weight for the memory write parameter, and configuring an erase weight for the memory erase parameter;

[0131] Step S502, calculating according to the first memory parameter and the attenuation weight, the memory writing parameter and the writing weight, and the memory erasing parameter and the erasing weight, to obtain the transition memory parameter.

[0132] In some embodiments of the present application, the method of incrementally adjusting the first memory parameter controls the memory update process by finely configuring weights. This method involves configuring specific weights for different parameters to reflect the importance and role of each parameter in the update process.

[0133] In step S501 of some embodiments, based on the time step corresponding to the first current linear vector, a decay weight is configured for the first memory parameter, a write weight is configured for the memory write parameter, and an erase weight is configured for the memory erase parameter. Among them, the decay weight is used to control the influence of the old memory information in the new memory update. By configuring the decay weight for the first memory parameter, the linear recurrent neural network large model can determine to what extent the memory information in the previous time step is retained. This weight reflects the natural decay of memory over time, helping the linear recurrent neural network large model to adapt to new input data while preventing important information from being forgotten too quickly. The write weight and the erase weight are combined with the memory write parameter and the memory erase parameter to determine the extent to which new information is written into the memory and the extent to which old information is forgotten. The write weight enhances the memory information associated with the current feature vector, while the erase weight reduces the memory information that is no longer relevant or outdated. This mechanism enables the linear recurrent neural network large model to process sequence data more flexibly and dynamically adjust its memory content according to the current context.

[0134] In step S502 of some embodiments, calculations are performed based on these weighted parameters to obtain transition memory parameters. This calculation process combines the effects of decay weights, write weights, and erase weights to generate a new memory state that contains both updated memory information and outdated information. The transition memory parameters become the basis for the linear recurrent neural network model to make predictions or classifications at the next time step, providing the linear recurrent neural network model with an updated memory representation that is relevant to the current context.

[0135] It should be understood that the incremental adjustment process described in these embodiments provides a mechanism for controlling memory updates by configuring weights for memory parameters and performing calculations. This method enables the large linear recurrent neural network model to process time series data more accurately, capture key information, and forget content that is no longer relevant. In this way, the large linear recurrent neural network model can maintain a dynamically updated memory state, providing strong support for processing complex sequence tasks.

[0136] In some more specific embodiments, if the time step is represented as t, the first memory parameter is represented as The memory write parameters are expressed as The memory erase parameter is expressed as The transition memory parameter Memory is calculated based on the first memory parameter and the attenuation weight, the memory write parameter and the write weight, and the memory erase parameter and the erase weight. t+1 , which can be expressed as:

[0137]

[0138] in, is the decay weight, α t is the erasure weight, β t is the write weight. It should be noted that λ t To control the speed at which memory decays over time, we can choose a suitable t The large linear recurrent neural network model can maintain sensitivity to key information over a longer time span, thereby effectively capturing long-distance dependencies. In addition, λ t , α t and β t All depend on the time step t. It should be noted that the memory write parameter and memory wipe parameters It is also dynamically calculated based on the current time step t.

[0139] Reference Figure 6 According to some embodiments of the present application, the module layer includes a residual connected time mixing layer and a channel mixing layer, different current feature vectors and sequence memory parameters have their own corresponding feature channels, and after the sequence memory parameters are incrementally adjusted according to the time dependency and the current feature vector, the following may also be included:

[0140] Step S601, determining the channel dependency between the current feature vector and the corresponding sequence memory parameter in the channel mixing layer;

[0141] Step S602 , performing channel fusion adjustment on the incrementally adjusted sequence memory parameters according to the channel dependency and the current feature vector, so as to update the sequence memory parameters.

[0142] In some embodiments of the present application, the design of the module layer includes a time mixing layer and a channel mixing layer of residual connections. Such an architecture is intended to enhance the understanding and processing capabilities of the linear recurrent neural network large model for target sequence data through the fusion of feature channels. Under this framework, different current feature vectors and sequence memory parameters are assigned to their respective feature channels, so that the linear recurrent neural network large model can process information from different sources in parallel. After completing the incremental adjustment of the sequence memory parameters based on the time dependency and the current feature vector, the linear recurrent neural network large model enters the processing stage of the channel mixing layer.

[0143] In step S601 of some embodiments, the primary task of the channel mixing layer is to determine the channel dependency between the current feature vector and the corresponding sequence memory parameters. This channel dependency reveals the interaction and influence between different feature channels, which may involve the correlation or complementarity between different types of features. By identifying these channel dependencies, the linear recurrent neural network large model can understand how different features jointly affect the final task output.

[0144] In step S602 of some embodiments, the linear recurrent neural network large model performs channel fusion adjustment on the incrementally adjusted sequence memory parameters according to the channel dependency and the current feature vector. This process involves integrating information from different channels to generate an updated sequence memory parameter that integrates multi-channel information. Channel fusion adjustment enables the linear recurrent neural network large model to not only capture dynamic changes in time series, but also fuse multi-dimensional information at the feature level, thereby obtaining a more comprehensive and rich sequence representation.

[0145] It should be understood that the process of channel fusion adjustment in the embodiment of the present application can be implemented in a variety of ways, such as by weighted summation, feature interpolation or attention mechanism to integrate the contributions of different channels. In this way, the large linear recurrent neural network model can adaptively adjust the influence of different feature channels, strengthen those features that are most helpful to the current task, and suppress those less relevant or redundant features.

[0146] The processing flow described in the embodiments of the present application further enhances the understanding and processing capabilities of the linear recurrent neural network large model for sequence data by determining channel dependencies and adjusting channel fusion in the channel mixing layer. This method enables the linear recurrent neural network large model to handle complex sequence tasks more effectively, improves the performance and generalization capabilities of the linear recurrent neural network large model, especially when processing sequence data with multidimensional features. By carefully fusing information from different feature channels, the linear recurrent neural network large model can generate a rich memory representation that integrates temporal dynamics and multidimensional features, providing a solid foundation for generating accurate task response results.

[0147] Reference Figure 7 According to some embodiments of the present application, the channel mixing layer includes a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer, and according to the channel dependency and the current feature vector, performing channel fusion adjustment on the incrementally adjusted sequence memory parameters to update the sequence memory parameters may include:

[0148] Step S701, in a second data-dependent interpolation layer, performing a second interpolation conversion process on the incrementally adjusted sequence memory parameters in different feature channels through a time-dependent relationship to obtain a plurality of second memory parameters;

[0149] Step S702, in a second linear layer, performing a second linear transformation on current feature vectors in different feature channels to obtain a plurality of second current linear vectors;

[0150] Step S703, in the element-wise multiplication fusion layer, the second memory parameters between different feature channels and the second current linear vector are element-wise multiplied to update the sequence memory parameters.

[0151] In some embodiments of the present application, the channel mixing layer is used to further process and fuse the information in different feature channels to update the sequence memory parameters. The channel mixing layer is composed of a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer, each of which has a specific task to jointly achieve channel fusion adjustment.

[0152] In step S701 of some embodiments, the second data-dependent interpolation layer uses the time dependency to perform a second interpolation conversion process on the incrementally adjusted sequence memory parameters in different feature channels. This step helps to smooth the time changes in the sequence and generate an interpolated sequence memory parameter, i.e., the second memory parameter, for each feature channel. This interpolation process not only helps to fill in potential time intervals, but also helps the large linear recurrent neural network model to establish a more coherent sequence representation between different time steps.

[0153] In step S702 of some embodiments, the second linear layer performs a second linear transformation on the current feature vectors in different feature channels. This transformation generates multiple second current linear vectors, each of which represents the information of a specific feature channel in the current time step. This step further refines the feature vector to make it more suitable for the subsequent fusion process.

[0154] In some embodiments of the present application, the second linear layer is responsible for performing a second linear transformation on the current feature vectors in different feature channels to extract and emphasize the key information in these features. Although this step mainly involves linear operations, the introduction of activation functions can significantly enhance the effect of this process.

[0155] The role of the activation function in the second linear layer is to introduce nonlinearity, which enables the large linear recurrent neural network model to learn and represent more complex functions and patterns. Without the activation function, no matter how many layers the network has, it can only learn a linear combination of the input data, which limits the expressive power and complexity of the large linear recurrent neural network model. By applying the activation function in the second linear layer, the linear transformation results in each feature channel will undergo a nonlinear transformation, which helps to reveal the complex structure and relationship in the data.

[0156] The choice of activation function can be based on a variety of considerations. For example, ReLU (Rectified Linear Unit) is widely used because of its high computational efficiency and ability to alleviate the gradient vanishing problem. When the ReLU activation function is applied, it sets all negative values ​​to zero while keeping positive values ​​unchanged. This operation helps retain important positive features while reducing the influence of unimportant features. In addition, variants of the activation function such as Leaky ReLU or ParametricReLU can be further improved to allow small gradient values ​​to pass through, thereby avoiding the problem of neuron death.

[0157] In this way, the activation function in the second linear layer not only increases the nonlinear expression ability of the linear recurrent neural network model, but also helps to improve the training efficiency and final performance of the linear recurrent neural network model. It enables the linear recurrent neural network model to capture and utilize complex features in the data, thereby achieving better results in various sequence processing tasks. Therefore, although the second linear layer mainly performs linear transformations, the introduction of the activation function is a key step in achieving efficient and powerful sequence processing linear recurrent neural network models.

[0158] In step S703 of some embodiments, the element-wise multiplication fusion layer performs element-wise multiplication of the second memory parameter and the second current linear vector between different feature channels. This operation is an effective fusion mechanism that allows the linear recurrent neural network large model to combine information from different channels at the element level, thereby updating the sequence memory parameters. Element-wise multiplication is particularly suitable for strengthening features that are important in multiple channels while suppressing features that are only significant in a few channels.

[0159] It should be understood that the channel mixing layer not only enhances the linear recurrent neural network model's understanding of the relationship between different feature channels, but also improves the linear recurrent neural network model's ability to express multi-dimensional features. This channel fusion adjustment enables the linear recurrent neural network model to more effectively utilize information during feature extraction and sequence processing, generating a rich sequence memory representation that integrates multi-channel information.

[0160] The processing flow described in the embodiments of the present application realizes channel fusion adjustment of sequence memory parameters by performing detailed interpolation conversion, linear transformation and element-wise multiplication fusion in the channel mixing layer. This method enables the large linear recurrent neural network model to process multidimensional feature data more comprehensively, and improves the performance and generalization ability of the large linear recurrent neural network model when processing complex sequence tasks. In this way, the large linear recurrent neural network model can generate a rich memory representation that integrates temporal dynamics and multidimensional features, providing a solid foundation for generating accurate task response results.

[0161] Reference Figure 8 , provides a schematic diagram of the architecture of a large linear recurrent neural network model. The architecture of a large linear recurrent neural network model has a complex structure, which is designed to process target sequence data and capture the dependencies between sequence sub-data corresponding to different time steps. The architecture consists of multiple layers, including input layer, core layer, and output layer. Each layer carries a specific function and together achieves efficient processing of sequence data.

[0162] The input layer receives the input vector x t , these vectors represent the feature representation of the target sequence data at each time step (the time step is represented as t), that is, the sequence sub-data. The purpose of the input layer is to convert the sequence sub-data into a form that can be further analyzed by the linear recurrent neural network model, that is, the feature vector. The core layer can be further subdivided into the time mixing layer and the channel mixing layer.

[0163] The feature vectors corresponding to the sequence sub-data first enter the time mixing layer through the residual link. The operations performed here include data-dependent interpolation and incremental rule operations, which help the linear recurrent neural network model capture dynamic changes in the time series and update its internal state according to the current input. The time mixing layer further includes a linear layer and a gating mechanism. The linear layer performs a linear transformation on the input data, while the gating mechanism controls the flow of information, allowing the linear recurrent neural network model to selectively focus on or ignore certain information. This selectivity is very useful when processing complex sequence data because it allows the linear recurrent neural network model to focus on the most relevant parts.

[0164] Another part of the core layer is the channel mixing layer, which contains a linear layer and an activation function. The purpose of the channel mixing layer is to process the dependencies between feature channels and fuse information from different channels through operations such as element-wise multiplication. The introduction of the activation function provides nonlinear processing capabilities for the linear recurrent neural network model, enabling the linear recurrent neural network model to capture more complex feature relationships.

[0165] Finally, the data processed by the channel mixing layer is passed to the output layer through residual connections and normalization operations. A linear layer can also be included between the channel mixing layer and the output layer, which is responsible for generating the final output based on the internal state of the linear recurrent neural network model and the processed information. This output can be the classification, prediction or other types of task responses of the target sequence data.

[0166] It should be understood that Figure 8 The large linear recurrent neural network model shown in Figure 1 enables in-depth analysis and processing of target sequence data through its carefully designed hierarchical structure and various operations. Each part of the large linear recurrent neural network model is optimized for specific data processing tasks, from feature extraction at the input layer to processing of time and channel dependencies at the core layer to the final decision at the output layer. The entire architecture works together to achieve efficient sequence data processing.

[0167] In step S106 of some embodiments, the execution returns to input the current sequence sub-data into the linear recurrent neural network large model until all sequence sub-data of the target sequence data are selected, and a task response result is generated through the output layer according to the current sequence sub-data and sequence memory parameters.

[0168] It should be noted that the linear recurrent neural network model will continue to execute the above process of updating the sequence memory parameters until all the sequence sub-data in the target sequence data are processed. This iterative processing method ensures that the linear recurrent neural network model can process each part of the target sequence data one by one while maintaining a global understanding of the entire target sequence data. When all the sequence sub-data are processed, the linear recurrent neural network model generates the final task response result through the output layer. This task response result is based on the last processed sequence sub-data and the updated sequence memory parameters. It reflects the linear recurrent neural network model's in-depth understanding of the entire target sequence data, including its long-term dependencies and complex patterns. The design of the output layer enables the linear recurrent neural network model to output its learned information in a way that is meaningful to a specific task. For example, it can be a classification label for text, a transcription of audio, or a predicted value for a sequence.

[0169] It is worth noting that the process of inputting the current sequence sub-data into the large linear recurrent neural network model is the key to achieving efficient processing of long sequence data. This approach allows the large linear recurrent neural network model to process only part of the sequence at a time, thereby reducing the demand for memory and enabling the large linear recurrent neural network model to run in an environment with limited computing resources. At the same time, by maintaining the sequence memory parameters at the core layer, the large linear recurrent neural network model is able to maintain a global understanding of the entire sequence even if it is processed in parts. This design not only improves the ability of the large linear recurrent neural network model to process long sequence data, but also enables it to flexibly adapt to different tasks and data types.

[0170] Reference Fig. 9 According to the second aspect of the present application, the task response device based on the large language model may include:

[0171] The data acquisition module 901 is used to acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each sequence sub-data corresponds to a time step;

[0172] A data selection module 902 is used to select current sequence sub-data from a plurality of sequence sub-data in the target sequence data;

[0173] The task processing module 903 is used to input the current sequence sub-data into the linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters; the current sequence sub-data is converted into the corresponding current feature vector through the input layer; the sequence memory parameters are updated according to the current feature vector through the core layer, and the current sequence sub-data is reselected from multiple sequence sub-data in the target sequence data according to the order of time steps; the current sequence sub-data is input into the linear recurrent neural network large model until all sequence sub-data of the target sequence data are selected, and the task response result is generated according to the current sequence sub-data and the sequence memory parameters through the output layer.

[0174] It can be seen that the contents of the above-mentioned task response method embodiment based on a large language model are all applicable to the embodiment of the task response device based on a large language model. The functions specifically implemented by the present task response device embodiment based on a large language model are the same as those of the above-mentioned task response method embodiment based on a large language model, and the beneficial effects achieved are also the same as those achieved by the above-mentioned task response method embodiment based on a large language model.

[0175] Reference Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0176] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0177] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the task response method based on a large language model in the embodiment of this application;

[0178] Input / output interface 1003, used to implement information input and output;

[0179] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0180] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0181] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0182] The embodiment of the present application also provides a computer program product, which includes a computer program. The processor of the computer device reads and executes the computer program, so that the computer device executes the task response method based on the large language model.

[0183] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0184] It should be understood that in the present disclosure, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, which may include any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0185] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.

[0186] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, which may include several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0190] It should also be understood that the various implementations provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0191] The above is a specific description of the implementation methods of the present disclosure, but the present disclosure is not limited to the above implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the present disclosure. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A task response method based on a large language model, characterized in that: include: Acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, and each of the sequence sub-data corresponds to one of the time steps; Selecting current sequence sub-data from a plurality of the sequence sub-data in the target sequence data; Inputting the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model comprises an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters; Converting the current sequence sub-data into a corresponding current feature vector through the input layer; The sequence memory parameter is updated according to the current feature vector by the core layer, and the current sequence sub-data is reselected from the plurality of sequence sub-data in the target sequence data according to the order of the time steps; Return to execute inputting the current sequence sub-data into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and generate a task response result through the output layer according to the current sequence sub-data and the sequence memory parameters.

2. The task response method according to claim 1, characterized in that: The core layer includes a plurality of module layers stacked by residual connections, the module layers include a time mixing layer, the core layer updates the sequence memory parameter according to the current feature vector, and reselects the current sequence sub-data from the plurality of sequence sub-data in the target sequence data according to the order of the time steps, including: The time dependency relationship between the current feature vector and the sequence memory parameter is determined in the time mixing layer, and the sequence memory parameter is incrementally adjusted according to the time dependency relationship and the current feature vector to update the sequence memory parameter.

3. The task response method according to claim 2, characterized in that: The time mixing layer includes a first data-dependent interpolation layer, a first linear layer, an incremental rule operation layer and a gating layer, and the incremental adjustment of the sequence memory parameter according to the time dependency relationship and the current feature vector to update the sequence memory parameter includes: In the first data-dependent interpolation layer, a first interpolation conversion process is performed on the sequence memory parameter according to the time-dependent relationship to obtain a first memory parameter; In the first linear layer, a first linear transformation is performed on the current feature vector to obtain a first current linear vector; In the incremental rule operation layer, the first memory parameter is incrementally adjusted according to the first current linear vector to obtain a transition memory parameter; In the gating layer, gated adjustment processing is performed on the first current linear vector and the transition memory parameter to obtain the updated sequence memory parameter.

4. The task response method according to claim 3, characterized in that: The step of incrementally adjusting the first memory parameter according to the first current linear vector to obtain a transition memory parameter includes: Determining a memory writing parameter and a memory erasing parameter according to the first current linear vector; The first memory parameter is incrementally adjusted according to the memory writing parameter and the memory erasing parameter to obtain the transition memory parameter.

5. The task response method according to claim 4, characterized in that: The step of incrementally adjusting the first memory parameter according to the memory writing parameter and the memory erasing parameter to obtain the transition memory parameter comprises: Based on the time step corresponding to the first current linear vector, configuring an attenuation weight for the first memory parameter, configuring a write weight for the memory write parameter, and configuring an erase weight for the memory erase parameter; The transition memory parameter is obtained by performing calculation based on the first memory parameter and the attenuation weight, the memory write parameter and the write weight, and the memory erase parameter and the erase weight.

6. The task response method according to any one of claims 2 to 5, characterized in that: The module layer includes the time mixing layer and the channel mixing layer of the residual connection, different current feature vectors and the sequence memory parameters have their own corresponding feature channels, and after the sequence memory parameters are incrementally adjusted according to the time dependency and the current feature vector, it also includes: The channel dependency relationship between the current feature vector and the corresponding sequence memory parameter is determined in the channel mixing layer, and the sequence memory parameter after the incremental adjustment is subjected to channel fusion adjustment according to the channel dependency relationship and the current feature vector to update the sequence memory parameter.

7. The task response method according to claim 6, characterized in that: The channel mixing layer includes a second data-dependent interpolation layer, a second linear layer, and an element-wise multiplication fusion layer, and the channel fusion adjustment is performed on the incrementally adjusted sequence memory parameters according to the channel dependency and the current feature vector to update the sequence memory parameters, including: In the second data-dependent interpolation layer, the sequence memory parameters after the incremental adjustment in different feature channels are subjected to a second interpolation conversion process according to the time-dependent relationship to obtain a plurality of second memory parameters; In the second linear layer, a second linear transformation is performed on the current feature vectors in different feature channels to obtain a plurality of second current linear vectors; In the element-wise multiplication fusion layer, the second memory parameter and the second current linear vector between different feature channels are element-wise multiplied to update the sequence memory parameter.

8. A task response device based on a large language model, characterized in that: include: A data acquisition module, used to acquire target sequence data; wherein the target sequence data includes a plurality of sequence sub-data sorted according to time steps, each of the sequence sub-data corresponds to one time step; A data selection module, used for selecting current sequence sub-data from a plurality of sequence sub-data in the target sequence data; A task processing module is used to input the current sequence sub-data into a linear recurrent neural network large model; wherein the linear recurrent neural network large model includes an input layer, a core layer and an output layer, and the core layer is configured with sequence memory parameters; the current sequence sub-data is converted into a corresponding current feature vector through the input layer; the sequence memory parameter is updated according to the current feature vector through the core layer, and the current sequence sub-data is reselected from a plurality of the sequence sub-data in the target sequence data according to the order of the time steps; the current sequence sub-data is input into the linear recurrent neural network large model until all the sequence sub-data of the target sequence data are selected, and a task response result is generated according to the current sequence sub-data and the sequence memory parameters through the output layer.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the task response method based on a large language model as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the task response method based on a large language model as described in any one of claims 1 to 7.

Images