Long text processing method and related device

Abstract

The invention belongs to the field of artificial intelligence, and particularly relates to a long text processing method and related device.The method comprises the steps that for long text data to be processed in a current session, context information corresponding to all text fragments is extracted from the long text data; the context information is stored in different storage areas of an external memory module; the external memory module comprises a short-term memory area and a long-term memory area; the short-term memory area is used for storing first context information of which the importance reaches a preset condition in the current session; the long-term memory area is used for storing second context information repeatedly appearing in a plurality of sessions; the plurality of sessions comprise a current session and / or a historical session; the first context information and the second context information are input into a large language model, and natural language processing of the long text data is achieved through the large language model. According to the method, the long text processing efficiency and accuracy can be improved, and the context memory ability and the flexible adaptability of the model are enhanced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a long text processing method and related devices. Background Art

[0002] At present, large language models have made significant progress in the field of natural language processing (NLP) and are widely used in tasks such as machine translation, text generation, and dialogue systems.

[0003] With the explosive growth of Internet content, processing and understanding long texts has become increasingly important. Users want to be able to process ultra-long texts such as long articles, complete books, and cross-session dialogues. In related technologies, large language models mainly rely on the Self-Attention Mechanism to capture dependencies in input sequences. However, as the input length increases, the computational complexity and memory requirements of the model increase exponentially, which leads to technical problems such as inefficient long text processing and loss of contextual information. It can be seen that existing large language models have obvious limitations when processing long texts, and often fail to meet the above needs of users.

[0004] Therefore, it is urgent to design a new technical solution to overcome at least one of the above technical problems. Summary of the invention

[0005] The present application provides a long text processing method and related devices to improve the efficiency and accuracy of long text processing and enhance the context memory ability and flexible adaptability of the model.

[0006] In a first aspect, the present application provides a long text processing method, comprising: For the long text data to be processed in the current session, extracting context information corresponding to each text segment from the long text data; Each piece of context information is stored in different storage areas of an external memory module; the external memory module includes a short-term memory area and a long-term memory area; the short-term memory area is used to store first context information whose importance reaches a preset condition in the current session; the long-term memory area is used to store second context information that appears repeatedly in multiple sessions; the multiple sessions include the current session and / or historical sessions; The first context information and the second context information are input into a large language model, and natural language processing of the long text data is performed through the large language model.

[0007] In a second aspect, an embodiment of the present application provides a long text processing device, comprising at least the following units: An extraction unit, configured to extract context information corresponding to each text segment from the long text data to be processed in the current session; A storage unit is configured to store each context information in different storage areas of an external memory module; the external memory module includes a short-term memory area and a long-term memory area; the short-term memory area is used to store first context information whose importance reaches a preset condition in a current session; the long-term memory area is used to store second context information that appears repeatedly in multiple sessions; the multiple sessions include a current session and / or a historical session; An execution unit is configured to input the first context information and the second context information into a large language model, and implement natural language processing of the long text data through the large language model.

[0008] In a third aspect, an embodiment of the present application provides a chip for implementing the long text processing method described in the first aspect.

[0009] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the long text processing method described in the first aspect.

[0010] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed, the long text processing method described in the first aspect is implemented.

[0011] In an embodiment of the present application, first, for the long text data to be processed in the current session, the context information corresponding to each text fragment is extracted from the long text data. Then, each context information is stored in different storage areas of the external memory module respectively; the external memory module includes a short-term memory area and a long-term memory area; the short-term memory area is used to store the first context information whose importance reaches a preset condition in the current session; the long-term memory area is used to store the second context information that appears repeatedly in multiple sessions; multiple sessions include the current session and / or historical sessions. Finally, the first context information and the second context information are input into the large language model, and the natural language processing of the long text data is realized through the large language model. In the embodiment of the present application, by introducing an external memory module and a multi-level memory structure, the efficiency and accuracy of long text processing are significantly improved, while the context memory ability and flexible adaptability of the model are enhanced, providing more powerful and comprehensive technical support for natural language processing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a long text processing method according to an embodiment of the present application; Figure 2 is a structural block diagram of a long text processing device according to an embodiment of the present application; Figure 3 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] The embodiment of the present application provides a long text processing method and related devices. First, for the long text data to be processed in the current session, the context information corresponding to each text segment is extracted from the long text data. Then, each context information is stored in different storage areas of the external memory module; the external memory module includes a short-term memory area and a long-term memory area; the short-term memory area is used to store the first context information whose importance reaches a preset condition in the current session; the long-term memory area is used to store the second context information that appears repeatedly in multiple sessions; the multiple sessions include the current session and / or the historical session. Finally, the first context information and the second context information are input into the large language model, and the natural language processing of the long text data is realized by the large language model. The embodiment of the present application, by introducing an external memory module and a multi-level memory structure, significantly improves the efficiency and accuracy of long text processing, while enhancing the context memory ability and flexible adaptability of the model, providing more powerful and comprehensive technical support for natural language processing tasks.

[0014] Specifically, in the embodiment of the present application, by introducing an external memory module and a multi-level memory structure, the effect of long text processing is significantly optimized. When processing long text data, the context information of each text fragment is first extracted from the long text, and the information is stored in different storage areas of the external memory module respectively. The external memory module includes a short-term memory area and a long-term memory area. The short-term memory area stores the first context information whose importance reaches a preset condition in the current session, while the long-term memory area stores the second context information that appears repeatedly in multiple sessions. This multi-level memory structure effectively solves the common context loss problem in traditional models, making the generated content more coherent and in line with context logic. Secondly, by storing the context information in the external memory module, the computational overhead of the model when processing long texts is reduced. In particular, the separation of the short-term memory area and the long-term memory area enables the model to quickly access and utilize relevant context information, thereby improving the reasoning speed. This design avoids the arduous task of recalculating or storing a large amount of context information in each reasoning process, significantly improving the overall computing efficiency. Thirdly, the dynamic adjustment and optimization mechanism of the external memory module can ensure that the model can obtain more accurate and relevant context information when generating and understanding long texts. By inputting the first context information and the second context information into the large language model, the model can make full use of this context information for natural language processing, thereby generating more accurate and relevant results. This mechanism not only improves the accuracy of the model, but also enhances the adaptability and generalization ability of the model. Finally, the design of the multi-level memory structure is not only suitable for long text processing, but also can play a role in other tasks that require contextual memory. The separation of short-term memory and long-term memory enables the model to flexibly adapt to different types of task requirements. Whether it is key information in the current session or common information in historical sessions, it can be effectively managed and utilized in different storage areas.

[0015] The long text processing solution provided in the embodiments of the present application can also be executed by an electronic device, which can be a server, a server cluster, or a cloud server. The electronic device can also be a terminal device such as a mobile phone, a computer, a tablet computer, a wearable device, or a dedicated device (such as a dedicated terminal device with a long text processing system). These electronic devices can also be equipped with the chips introduced in the above embodiments. Alternatively, these electronic devices can also be installed with a service program for executing the long text processing solution.

[0016] The long text processing method and related devices provided in the embodiments of the present application are described below in conjunction with the accompanying drawings. Figure 1 A long text processing method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes: S101, extracting context information corresponding to each text segment from the long text data to be processed in the current session; S102, storing each piece of context information in different storage areas of an external memory module; S103: Input the first context information and the second context information into a large language model, and implement natural language processing of the long text data through the large language model.

[0017] The above three steps provide significant technical optimization and beneficial effects for long text data processing by introducing external memory modules and multi-level memory structures. Specifically, first, by extracting the contextual information of each text fragment, the model can more deeply understand the key content in the long text, avoiding the problem of context loss that traditional models are prone to when processing long texts. Decomposing long texts into multiple text fragments and extracting their contextual information helps the model better capture the specific meaning and logical relationship of each fragment, and improves the level of processing refinement. When generating text, the contextual information of each fragment can be more accurately referenced, thereby generating more coherent and logical text content.

[0018] Then, each piece of context information is stored in different storage areas of the external memory module. In this way, the external memory module (including the short-term memory area and the long-term memory area) can reasonably store and manage context information of different importance and timeliness, avoiding information redundancy and confusion. By storing important context information in the current session in the short-term memory area, the model can quickly access this key information, improving the reasoning and processing speed. The long-term memory area is used to store context information that appears repeatedly in multiple sessions, which helps the model accumulate and utilize long-term knowledge and enhances the model's knowledge base and adaptability.

[0019] Finally, the first context information and the second context information are input into the large language model for natural language processing. By integrating the context information of the current session (short-term memory) and the historical session (long-term memory), the large language model can obtain more comprehensive and coherent context support when generating and understanding text, and the generated content is more natural and logical. Combining the context information of short-term and long-term memory, the model can more accurately understand the content of long texts and generate outputs that are highly relevant to the context, significantly improving the accuracy and relevance of the model. The design of the multi-level memory structure makes this technology not only suitable for long text processing, but also can play a role in other tasks that require context memory, enhancing the versatility and adaptability of the model.

[0020] In summary, these three steps significantly improve the contextual understanding ability, computational efficiency and result accuracy of long text processing by introducing external memory modules and multi-level memory structures, while enhancing the flexible adaptability of the model and providing more comprehensive and efficient technical support for natural language processing tasks.

[0021] In an embodiment of the present application, the external memory module includes a short-term memory area and a long-term memory area. The division of these two areas enables the model to flexibly respond to different types of context information and improve overall processing efficiency and accuracy.

[0022] Among them, the short-term memory area is used to store the first context information in the current session whose importance meets the preset conditions. That is, the information stored in the short-term memory area is important content in the recent period of time and can be frequently accessed in the current session. The short-term memory area is mainly used to store context information that appears frequently in the current session or is crucial to understanding the current content. This information may be mentioned or referenced many times in the current session, so it needs to be able to be quickly accessed in a short time. In order to avoid storing too much unimportant information, the short-term memory area only stores context information whose importance meets the preset conditions. This screening mechanism ensures that the stored information is of high value, thereby avoiding information redundancy and confusion. Since the short-term memory area stores key information in the current session, the model can frequently access this information when processing the current session, thereby improving reasoning speed and processing efficiency.

[0023] The long-term memory area is used to store secondary context information that recurs in multiple sessions. The long-term memory area is mainly used to store context information that recurs in multiple sessions. This information may appear multiple times in historical sessions, or may be common background knowledge or topics across sessions. By storing this recurring context information, the long-term memory area helps the model accumulate long-term knowledge. This knowledge can be referenced as background information in the current session, enhancing the model's understanding and generation of current content. The storage content of the long-term memory area can include context information in the current session and historical sessions, allowing the model to maintain context coherence and consistency when processing content across sessions.

[0024] It is understandable that multiple sessions include current sessions and / or historical sessions. The current session refers to the currently ongoing conversation or processed text content. The short-term memory area mainly stores key information in the current session, while the long-term memory area may also contain contextual information that recurs in the current session. The historical session refers to a conversation that has been conducted or text content that has been processed before. The contextual information stored in the long-term memory area may come from historical sessions, and this information can be referenced and used in the current session.

[0025] In summary, through the division of short-term memory and long-term memory areas, the external memory module achieves efficient storage and management of contextual information. The short-term memory area ensures that key information in the current session can be quickly accessed and utilized, improving the real-time and efficiency of processing. The long-term memory area enhances the model's long-term knowledge accumulation and contextual coherence by storing repeated information across sessions, making the model more accurate and efficient when processing complex and multi-session content. Therefore, through the reasonable division of short-term memory and long-term memory areas, the model's contextual understanding ability and processing efficiency when processing long text data are significantly improved, providing more comprehensive and powerful technical support for natural language processing tasks.

[0026] As an optional embodiment, in 102, storing each piece of context information in different storage areas of the external memory module can be implemented as the following steps: 201. Obtaining importance evaluation values ​​corresponding to each piece of context information in the current session through a self-attention mechanism; the importance evaluation values ​​are used to indicate the correlation between the context information and the long text data; the higher the correlation between the context information and the long text data, the higher the importance evaluation value corresponding to the context information; 202. Selecting first context information whose importance evaluation value meets a preset condition in the current session, and storing the first context information in the short-term memory area; 203. Match each context information in the current session with the second context information stored in the long-term memory area, and use the context information that appears repeatedly in the current session and the historical session as newly added second context information, and update it into the long-term memory area.

[0027] Specifically, in 201, the importance evaluation value corresponding to each context information in the current session is obtained through the self-attention mechanism. Among them, the self-attention mechanism is a technology for calculating the correlation between each element in the input sequence. In this step, the self-attention mechanism is used to evaluate the correlation between each context information and the long text data in the current session. Specifically, through the self-attention mechanism, the model can calculate an importance evaluation value for each context information, which reflects the correlation between the context information and the long text data. The higher the correlation, the higher the importance evaluation value. In this way, the self-attention mechanism can accurately calculate the correlation between each context information and the long text data, thereby providing an accurate importance evaluation basis for subsequent steps. Through the importance evaluation value, the model can quickly filter out context information that is highly relevant to the current long text data, reduce the interference of irrelevant information, and improve the efficiency of storage and processing.

[0028] In 202, the first context information whose importance evaluation value meets the preset conditions in the current session is selected, and the first context information is stored in the short-term memory area. Specifically, according to the pre-set conditions (such as the threshold of the importance evaluation value), the first context information that meets the conditions is screened out. Thus, the context information whose importance evaluation value reaches or exceeds the preset threshold is stored in the short-term memory area so that these key information can be quickly accessed and utilized in the current session. In this way, through the screening of preset conditions, the short-term memory area only stores the context information that is crucial to understanding the current long text, avoiding information redundancy and confusion. The information in the short-term memory area can be quickly accessed, which improves the reasoning and processing speed of the model in the current session.

[0029] It is understandable that the setting of preset conditions is the key to achieving efficient storage and management of short-term memory areas. Preset conditions can be set based on a variety of factors to ensure that contextual information that is critical to understanding the current long text is stored. For example, according to actual needs, multiple conditions such as importance evaluation value, access frequency, timeliness, semantic relevance and information entropy can be combined to set multi-dimensional preset conditions. For example, only contextual information that meets the importance evaluation value threshold, access frequency threshold and semantic similarity threshold at the same time will be stored in the short-term memory area. In this way, according to the progress of the session and the change of contextual information, the threshold of each preset condition is dynamically adjusted to ensure that the stored information is always the latest and high-value. Here, the stored information has a high access frequency and timeliness, which improves the access speed and processing efficiency of the short-term memory area. Through the combination and dynamic adjustment of multiple conditions, the performance and adaptability of the model are further improved. In summary, through the reasonable setting of multiple preset conditions, efficient storage and management of contextual information can be achieved, providing more accurate and efficient technical support for the processing of long text data.

[0030] Exemplarily, the importance evaluation value of each context information is calculated using the self-attention mechanism. The self-attention mechanism evaluates the correlation between the context information and the current long text data by calculating the attention score between them. The attention score of each context information and all positions in the current long text data is calculated, and the attention score is normalized by the Softmax function to obtain the importance evaluation value of each context information. A threshold for the importance evaluation value is set in advance. For example, setting the threshold to 0.8 means that only those context information with an importance evaluation value of or greater than 0.8 will be stored in the short-term memory area. The self-attention mechanism can accurately evaluate the correlation between the context information and the current long text data, ensure that the stored information is of high value, avoid storing unimportant information, and improve the efficiency of storage and processing. The threshold can be adjusted according to the actual application scenario and requirements, and the storage strategy can be optimized to further improve the performance and efficiency of the model. For example, when processing highly complex conversations, the threshold can be appropriately lowered to store more relevant information; when processing simple conversations, the threshold can be appropriately increased to reduce the storage of redundant information.

[0031] Exemplarily, in the current session, the number of times each context information is accessed is recorded in real time. The access frequency indicates the frequency with which a certain context information is mentioned or referenced in the current session. The access frequency of each context information is initialized to 0, and its access frequency is updated whenever the context information is mentioned or referenced. An access frequency threshold is set in advance. For example, setting the threshold to 5 means that only those context information with an access frequency of 5 or more times will be stored in the short-term memory area. Context information with a high access frequency is usually highly relevant to the content of the current session, ensuring that only important context information that is frequently mentioned in the current session is stored, and avoiding the storage of unimportant information. The access frequency threshold can be dynamically adjusted according to the progress of the session and the changes in the context information to ensure that the stored information is always up-to-date and high-value. For example, a lower threshold can be set at the beginning of the session to quickly accumulate important information; in the later stage of the session, the threshold can be appropriately increased to filter out the truly important information.

[0032] Exemplarily, the information entropy value of each context information is calculated. The information entropy value reflects the amount of information contained in the context information. The higher the information entropy value, the greater the amount of information contained in the context information. The information entropy value of each context information is calculated. The calculation formula is: .in, represents the probability of each possible state in the context information, Represents the probability Take the logarithm, Represents the self-information value of all probability items for i from 0 to n Sum them up. The uncertainty of the entire data set is quantified by calculating the expected value of the self-information of all possible results. The larger the information entropy, the higher the uncertainty of the data set; conversely, the smaller the information entropy, the lower the uncertainty of the data set. Calculate the information entropy value of all context information and sort them. Further optionally, set an information entropy threshold in advance. For example, setting the threshold to 0.7 means that only context information with an information entropy value of 0.7 or more will be stored in the short-term memory area. In this way, it is ensured that the stored information has a high amount of information and avoids storing redundant and low-value information. Context information with high information entropy values ​​usually contains more key information, which helps to improve the understanding and reasoning ability of the model.

[0033] In practical applications, we can further adjust the information entropy threshold according to the actual application scenarios and requirements, optimize the storage strategy, and further improve the performance and efficiency of the model. For example, when processing highly complex conversations, we can appropriately lower the threshold to store more information containing rich information; when processing simple conversations, we can appropriately increase the threshold to reduce the storage of redundant information.

[0034] Exemplarily, a time window is set and only the context information that appears within the time window is stored. For example, only the context information that appears in the past 10 minutes is stored. A time decay function is used to evaluate the timeliness of the context information. The closer the context information is to the current time, the higher its timeliness is and the priority is stored. Thus, it is ensured that the stored information has high timeliness and outdated information is avoided. The time window can be adjusted according to the duration of the session to ensure that the stored information is always up-to-date and relevant.

[0035] For example, a semantic similarity algorithm (such as cosine similarity) is used to calculate the semantic relevance between the context information and the current long text data. A semantic similarity threshold is set in advance. Only those context information whose semantic similarity reaches or exceeds the threshold will be stored in the short-term memory area. Thus, it is ensured that the stored information has a high semantic relevance to the current long text data, enhancing the ability to understand. The threshold can be adjusted according to the actual application scenario and needs, and the storage strategy can be optimized to further improve the performance and efficiency of the model.

[0036] In 203, each context information in the current session is matched with the second context information stored in the long-term memory area, and the context information that appears repeatedly in the current session and the historical session is used as the newly added second context information and updated to the long-term memory area. Specifically, each context information in the current session is matched with the second context information stored in the long-term memory area to find out the context information that appears repeatedly in multiple sessions. Then, the matched repeatedly appearing context information is used as the newly added second context information and updated to the long-term memory area. This information may come from the current session and / or the historical session. Thus, through matching and updating, the long-term memory area can accumulate repeated context information across sessions, enhancing the long-term knowledge base and adaptability of the model. The updated long-term memory area can provide more comprehensive background information for the current session, enhance the model's ability to understand and generate current content, and maintain the coherence and consistency of the context. The information in the long-term memory area is not only applicable to the current session, but can also be referenced and used in future sessions, improving the generalization ability and application scope of the model.

[0037] Similar to step 202, step 203 can be implemented by using a method similar to that illustrated in step 202 in combination with the storage requirements of the long-term memory area for context information in actual application scenarios, which will not be elaborated here.

[0038] Through the implementation of steps 201 to 203, the efficient storage and management capabilities of context information are significantly improved. The importance of context information is accurately evaluated through the self-attention mechanism, providing an accurate evaluation basis for subsequent steps. By pre-setting conditions to filter and store key information in the current session, the access speed and processing efficiency of the short-term memory area are improved. By matching and updating the long-term memory area, the long-term knowledge accumulation and context coherence of the model are enhanced, and the generalization ability and application scope of the model are improved. In summary, optional embodiment 102 significantly optimizes the storage and management of context information through the self-attention mechanism and reasonable storage strategy, and provides more efficient and accurate technical support for the processing of long text data.

[0039] In another optional embodiment, in 102, each piece of context information is stored in a different storage area of ​​the external memory module, and the access frequency corresponding to each piece of context information in the current session can be further obtained. Then, the first context information whose access frequency in the current session is higher than the set access frequency threshold is selected, and the first context information is stored in the short-term memory area.

[0040] For example, in the current session, the number of times each context information is accessed is recorded. The access frequency indicates how often a certain context information is mentioned or referenced in the current session. By monitoring the access behavior in the current session in real time, the model can dynamically calculate the access frequency of each context information. A threshold of access frequency is set in advance as a condition for filtering context information. Only those context information with an access frequency higher than or equal to the threshold are considered important. According to the access frequency threshold, the context information with a higher access frequency in the current session is filtered out and marked as the first context information. The filtered first context information is stored in the short-term memory area. This information may be frequently mentioned or referenced in the current session, so it needs to be able to be quickly accessed in a short time.

[0041] It is understandable that the context information in the short-term memory area has a high access frequency, which means that this information has a high importance in the current session. The model can quickly access this key information, which improves the reasoning and processing speed. The information in the short-term memory area can be dynamically updated according to the real-time access of the current session, ensuring that the stored information is always up-to-date and high-value. Frequently accessed context information is usually highly relevant to the content of the current session. Storing this information in the short-term memory area helps the model maintain the coherence and consistency of the context when processing the current session. When generating text, the model can more accurately refer to the context information in the short-term memory area, thereby generating more coherent and logical text content.

[0042] Further optionally, access modes corresponding to different levels may be further set based on access frequency to improve the response speed of access requests.

[0043] As an optional embodiment, in 103, inputting the first context information and the second context information into a large language model may be implemented as the following steps: 301. Obtain the target text information to be processed from the current session; 302. Generate a corresponding search request based on the target text information; the search request at least includes any one or more of the text content information, text type, and text subject in the target text information; 303. Retrieve first context information and second context information matching the target text information from the external memory module based on the retrieval request; 304. Fusing the retrieved first context information and the second context information, and inputting the fusion processing result into the large language model.

[0044] For example, in this optional embodiment, the first context information and the second context information are input into the large language model through multiple steps to achieve efficient processing of the target text information. Assume an application scenario: a customer service robot intelligently answers questions raised by users. Assume that the target text information is a question raised by a user, "How do I upgrade my account membership?" In 301, the customer service robot extracts the question raised by the user from the current user's conversation record: "How do I upgrade my account membership?" Here, the target text information = "How do I upgrade my account membership?" In 302, a search request is generated according to the target text information, and the request contains the characteristics of the target text, such as text content, type, and topic. For example, the text content information is "How do I upgrade my account membership?", the text type is question, and the text topic is account upgrade. Thus, a corresponding search request is generated, namely {"text": "How do I upgrade my account membership?", "type": "question", "topic": "account upgrade"}.

[0045] In 303, according to the generated search request, context information related to the target text information is retrieved from an external memory module (such as a knowledge base or database). For example, the first context information is the steps for account upgrade. The second context information is the section about account upgrade in the FAQ. Then, the first context information = "You can upgrade your account membership by following the steps: log in to your account->select a membership package->complete payment"; the second context information = "Have problems upgrading your account? Please refer to item 4 in the FAQ" In 304, the retrieved first and second context information are fused to form a more comprehensive and detailed information set. For example, the fusion processing result = "You can upgrade your account membership by following the steps below: log in to your account->select a membership package->complete payment. If you encounter any problems during the upgrade process, please refer to Article 4 in the FAQ." In this way, the fusion processing result is input into the large language model, and the model will generate an intelligent answer. Here, the answer generated by the large language model can be, "To upgrade your account membership, please log in to your account, select your preferred membership package, and then complete the payment. If you encounter any problems during the process, please feel free to refer to Article 4 in our FAQ." Through the above example steps, the customer service robot can extract relevant information from the external memory module and combine this information to generate an accurate and comprehensive answer. This method not only improves the accuracy of the answer, but also enhances the user experience.

[0046] As an optional embodiment, in 303, retrieving the first context information and the second context information matching the target text information from the external memory module based on the retrieval request can be implemented as the following steps: 401. Extracting a target text segment from the target text information; 402. Obtain the first context information and the second context information in the external memory module, and the cosine similarity between each of the first context information and the target text segment; 403 . Select the first context information and the second context information whose cosine similarity meets a preset condition as the first context information and the second context information that match the target text segment.

[0047] Among them, cosine similarity is a measurement method used to measure the angle between two vectors. In text processing and information retrieval, it can be used to evaluate the similarity between two text vectors. The value range of cosine similarity is [-1, 1]. The larger the value, the more similar the two vectors are, and the smaller the value, the less similar the two vectors are. In text processing, the Bag of Words model or Word Embedding is used to convert text into vector form, and then the cosine similarity between these vectors is calculated.

[0048] Exemplarily, an external memory module is set up, in which context information related to account upgrade is stored. Assume that the context information related to the user question "How do I upgrade my account membership?" is retrieved by cosine similarity. In 401, a key segment is extracted from the target text information "How do I upgrade my account membership?" Assume that the target text segment is "upgrade my account membership".

[0049] In 402, the first context information and the second context information are obtained from the external memory module. For example, the first context information is, "You can upgrade your account membership by following the steps below: Log in to your account->Select a membership package->Complete payment." The second context information is, "Have problems upgrading your account? Please refer to Article 4 in the FAQ." Then, the target text segment "Upgrade my account membership" and the context information are converted into vector form. The cosine similarity calculation formula is used to calculate the cosine similarity between the target text segment and each context information. The target text segment vector is [0.1, 0.2, 0.3, 0.4]. The first context information vector is [0.15, 0.25, 0.35, 0.45]. The second context information vector is [0.2, 0.3, 0.4, 0.5]. The calculation result is that the cosine similarity between the target text segment and the first context information is 0.98, and the cosine similarity between the target text segment and the second context information is 0.95.

[0050] In 403, a threshold value of cosine similarity is set, such as 0.9. Context information with a cosine similarity greater than or equal to 0.9 is selected. The cosine similarity of the first context information is 0.98 (qualified), and the cosine similarity of the second context information is 0.95 (qualified). The first context information and the second context information are selected as context information matching the target text segment.

[0051] In this way, by using the cosine similarity algorithm, contextual information that is highly relevant to the target text information can be efficiently retrieved from the external memory module. This method not only improves the accuracy of retrieval, but also reduces the interference of irrelevant information, thereby improving the performance of the overall system and user experience. Cosine similarity can accurately measure the similarity between texts and ensure that the retrieved contextual information is highly relevant to the target text. The calculation of cosine similarity is relatively simple and suitable for large-scale text retrieval tasks. The threshold of cosine similarity can be adjusted according to actual needs to optimize the retrieval strategy. If there is a lot of noise in the text vector, it may affect the accuracy of cosine similarity. In high-dimensional space, the calculation of cosine similarity may be affected by sparsity problems.

[0052] In summary, the cosine similarity algorithm has a wide range of applications in text processing and information retrieval. It can efficiently measure the similarity between texts and filter out the most relevant context information based on the set conditions. The retrieval effect can be further improved by reasonably setting the threshold and optimizing the vector representation method.

[0053] Further optionally, it is assumed that the first context information and the second context information in the external memory module are respectively associated with the corresponding dynamic index according to their respective text content characteristics and / or text format types. Based on this, before 403, the text attribute features of the target text segment can also be identified. The text attribute features are used to represent the text content characteristics and / or text format type of the target text segment. Furthermore, the text retrieval precondition of the target text segment is constructed based on the text content characteristics and / or text format type. Then, based on the text retrieval precondition, the target dynamic index in the external memory module that matches the target text segment is determined, and the first context information and the second context information associated with the target dynamic index are obtained as the first context information and the second context information to be screened.

[0054] In this embodiment, the context information in the external memory module is associated with the corresponding dynamic index according to the respective text content characteristics and / or text format type. By identifying the text attribute characteristics of the target text segment, the text retrieval precondition of the target text segment is constructed, thereby more accurately determining the target dynamic index in the external memory module that matches the target text segment. This step can significantly improve the efficiency and accuracy of retrieval.

[0055] Assume that there is an external memory module that stores a large amount of contextual information related to account upgrades. The information in the external memory module is associated with different dynamic indexes according to content characteristics and format types. The target text information is the question raised by the user, "How do I upgrade my account membership?", and the keyword is "upgrade account membership". Based on this, the key words and topics in the target text segment can be analyzed to determine the type of the target text segment (such as questions, statements, commands, etc.). The text content characteristics are, keywords = "upgrade", "account", "membership"; the text format type is question.

[0056] According to the text attribute characteristics of the target text segment, a search condition is constructed to filter relevant information from the external memory module. The keywords include "upgrade", "account", "membership", and the text format is question. According to the text search precondition, the dynamic index matching the target text segment in the external memory module is determined. Assume dynamic index 1: keywords = "upgrade", "account", "membership", text format = question; dynamic index 2: keywords = "membership", "account", text format = statement; dynamic index 3: keywords = "upgrade", "account", text format = command. Based on the above assumptions and implementation principles, it can be determined that the dynamic index matching the target text segment is dynamic index 1.

[0057] Then, relevant context information is obtained from the determined target dynamic index. The first context information associated with the target dynamic index 1 is, "You can upgrade your account membership by following the steps below: Log in to your account -> Select a membership package -> Complete payment." The second context information is, "Have problems upgrading your account? Please refer to item 4 in the FAQ." In this way, dynamic indexing can quickly locate relevant context information based on the text attribute characteristics of the target text segment, reduce unnecessary full-text retrieval, and improve retrieval speed. Identifying the text attribute characteristics of the target text segment and constructing retrieval preconditions can reduce the interference of irrelevant information and ensure that the retrieved context information is highly relevant to the target text. More accurate and efficient context information retrieval can improve user satisfaction, especially in application scenarios such as customer service robots, which can provide users with the information they need in a timely manner. Dynamic indexing can be dynamically adjusted according to the characteristics and format type of text content, adapt to different types of text information, and improve the flexibility and adaptability of the system. In short, by identifying the text attribute characteristics of the target text segment and constructing retrieval preconditions, the dynamic index matching the target text segment in the external memory module can be more accurately determined, thereby improving the efficiency and accuracy of context information retrieval. This method not only optimizes the performance of the system, but also significantly improves the user experience.

[0058] Then, in 304, the retrieved first context information and the second context information are fused. Specifically, the retrieved first context information and the second context information are sorted from high to low according to the cosine similarity, so as to select the first context information and the second context information whose similarity is at a preset position or reaches a preset similarity threshold. Then, the selected first context information and the second context information are subjected to multi-level memory fusion processing to obtain the fusion result.

[0059] In step 304, the retrieved first context information and second context information are sorted from high to low according to cosine similarity, and context information with similarity at a preset position or reaching a preset similarity threshold is selected. Then, the selected context information is subjected to multi-level memory fusion processing to obtain a final fusion result.

[0060] For example, a hybrid concatenation method is used. Specifically, the retrieved context information is sorted according to cosine similarity. The context information whose similarity reaches a preset threshold or is in a preset position is selected. The selected context information is concatenated to form a complete answer.

[0061] Exemplarily, the first context information (similarity 0.98): You can upgrade your account membership by following the steps: Log in to your account->Select a membership package->Complete payment. The second context information (similarity 0.95): Have problems upgrading your account? Please refer to Article 4 in the FAQ. The third context information (similarity 0.88): After the account is successfully upgraded, you can enjoy more membership benefits. The preset similarity threshold is 0.9, and the first context information and the second context information are filtered out. The filtered context information is spliced ​​to form a complete answer. Therefore, the final fusion result is, "You can upgrade your account membership by following the steps: Log in to your account->Select a membership package->Complete payment. If you encounter problems during the process, please refer to Article 4 in the FAQ." Alternatively, an information superposition method may be used. Specifically, the retrieved context information is sorted according to cosine similarity. Context information whose similarity reaches a preset threshold or is at a preset position is selected. The information content of the selected context information is superimposed to form a more detailed and comprehensive answer.

[0062] Exemplarily, the first context information (similarity 0.98): You can upgrade your account membership by following the steps: Log in to your account->Select a membership package->Complete payment. The second context information (similarity 0.95): Have problems upgrading your account? Please refer to Article 4 in the FAQ. The third context information (similarity 0.88): After the account is successfully upgraded, you can enjoy more membership benefits. The preset similarity threshold is 0.9, and the first context information and the second context information are filtered out. The information content of the filtered context information is superimposed to form a detailed answer. Therefore, the final fusion result is, "You can upgrade your account membership by following the steps: Log in to your account->Select a membership package->Complete payment. If you encounter problems during the process, please refer to Article 4 in the FAQ. After the account is successfully upgraded, you can enjoy more membership benefits." Alternatively, an information synthesis method may be used. Specifically, the retrieved context information is sorted according to cosine similarity, context information with a similarity reaching a preset threshold or being in a preset position is selected, and a natural language processing technology (such as a sentence generation model) is used to synthesize a natural and fluent answer.

[0063] Exemplarily, the first context information (similarity 0.98): You can upgrade your account membership by following the steps: log in to your account->select a membership package->complete payment. The second context information (similarity 0.95): Having problems upgrading your account? Please refer to Article 4 in the FAQ. The third context information (similarity 0.88): After your account is successfully upgraded, you can enjoy more membership benefits. The preset similarity threshold is 0.9, and the first context information and the second context information are filtered out. The sentence generation model is used to synthesize the filtered context information into a natural and fluent answer. Therefore, the final fusion result is, "To upgrade your account membership, you need to log in to your account, select a suitable membership package, and complete the payment. If you encounter any problems during the upgrade process, you can refer to Article 4 in our FAQ." Alternatively, an information screening method can also be used. Specifically, the retrieved context information is sorted according to the cosine similarity, and the context information with a similarity reaching a preset threshold or being in a preset position is selected. According to the content of the context information and the user's intention, the most relevant information is further screened out. Finally, the screened information is spliced ​​or synthesized to form an accurate answer. For example, the first context information (similarity 0.98): You can upgrade your account membership by following the steps: log in to your account->select a membership package->complete payment. The second context information (similarity 0.95): Have problems upgrading your account? Please refer to Article 4 in the FAQ. The third context information (similarity 0.88): After the account is successfully upgraded, you can enjoy more membership benefits. The preset similarity threshold is 0.9, and the first context information and the second context information are screened out. Further optionally, according to the user's intention (that is, the user is mainly concerned about how to upgrade the account), the most relevant information is further screened out. The screened information is spliced ​​or synthesized. Therefore, the final fusion result is, "To upgrade your account membership, please log in to your account, select a membership package, and complete the payment. If you encounter problems during the upgrade process, please refer to Article 4 in the FAQ." In summary, through multi-level memory fusion processing, multiple related context information can be integrated together to provide more accurate and comprehensive answers. The information synthesis method can generate natural and fluent text to improve the user experience. The information screening method can further eliminate irrelevant information to ensure the accuracy of the answer. Through cosine similarity sorting and preset threshold screening, the amount of context information that needs to be processed can be significantly reduced, improving the processing speed and performance of the system.

[0064] In practical applications, the multi-level memory fusion process can select different fusion methods according to specific needs, such as hybrid splicing, information superposition, information synthesis and information screening. These methods not only improve the accuracy and comprehensiveness of the answers, but also enhance the naturalness and fluency, thereby significantly improving user experience and system performance.

[0065] As an optional embodiment, after storing each context information in different storage areas of the external memory module in 103, corresponding memory decay coefficients may be set for the first context information and the second context information in the external memory module; wherein the lower the access frequency of the context information, the greater the corresponding memory decay coefficient. Then, a least recently used (Least Recently Used, LRU) strategy is adopted to perform memory management on the first context information and the second context information stored in the external memory module based on the memory decay coefficient.

[0066] Among them, the LRU strategy is a common cache eviction algorithm used to manage data in the cache. In this strategy, the least recently used data will be deleted first to make room for new data. The LRU strategy is widely used in various fields of computer science, including operating systems, databases, and various cache systems. The LRU strategy maintains a data structure that records the access time or access frequency of data.

[0067] When the cache is full, the least recently used data is removed first. The LRU strategy is usually implemented using a combined data structure called a doubly linked list and a hash table. A doubly linked list is used to record the order in which data is accessed. Newly accessed data is added to the head of the linked list, while the tail of the linked list is the least recently used data. A hash table is used to quickly find data. The key of a hash table is the unique identifier of the data, and the value is a pointer to the corresponding node in the doubly linked list.

[0068] Specifically, create a doubly linked list and a hash table. If the cache is not full, insert the new data into the head of the linked list and add the corresponding key-value pair in the hash table. If the cache is full, delete the node at the tail of the linked list (that is, the least recently used data), then insert the new data into the head of the linked list and add the corresponding key-value pair in the hash table. When accessing a piece of data, quickly find the node of the data through the hash table. Remove the node from the linked list and reinsert it into the head of the linked list. When data needs to be deleted, delete the node from the tail of the linked list and delete the corresponding key-value pair in the hash table.

[0069] In the external memory module, the LRU strategy can be combined with the memory decay coefficient to manage the context information. The corresponding memory decay coefficients are set for the first context information and the second context information in the external memory module. That is, the memory decay coefficient is set according to the access frequency of the context information. The lower the access frequency, the larger the corresponding memory decay coefficient. For example, if a certain context information has not been accessed in the past month, its memory decay coefficient can be set to 1.5, while the frequently accessed context information can be set to 1.0.

[0070] The least recently used strategy LRU is adopted to manage the memory of the first context information and the second context information stored in the external memory module based on the memory decay coefficient. The access record is to maintain a bidirectional linked list and a hash table to record the access time and frequency of each context information. Each time the context information is accessed, its access frequency is updated and the memory decay coefficient is adjusted. When the cache is full, the comprehensive score of each context information is calculated, and the comprehensive score = access time + memory decay coefficient. The context information with the lowest comprehensive score is selected for elimination.

[0071] Assume that the following context information is stored in the external memory module. The content of the first context information is, "You can upgrade your account membership by following the steps below: Log in to your account -> Select a membership package -> Complete payment". The access frequency is 10 times, and the memory decay coefficient is 1.0. The content of the second context information is, "Have problems upgrading your account? Please refer to item 4 in the FAQ". The access frequency is 5 times, and the memory decay coefficient is 1.2. The content of the third context information is, "After the account is successfully upgraded, you can enjoy more membership benefits". The access frequency is 2 times, and the memory decay coefficient is 1.5. When the cache is full, a context information needs to be eliminated. Calculate the comprehensive score of each context information, for example, the first context information: comprehensive score = last access time + memory decay coefficient (assuming that the last access time is the same, both are 0); the second context information: comprehensive score = last access time + memory decay coefficient; the third context information: comprehensive score = last access time + memory decay coefficient. Since the third context information has the lowest access frequency and the largest memory decay coefficient, it has the lowest comprehensive score and will be eliminated first.

[0072] In this way, through the LRU strategy, it can be ensured that the cache retains the context information that has been frequently accessed recently, thereby improving the cache hit rate. The use of the memory decay coefficient can better manage the storage space and give priority to retaining the context information that is frequently accessed and of high importance. The improvement of the cache hit rate can reduce the number of accesses to the external memory module, thereby improving the overall performance of the system. The memory decay coefficient can be dynamically adjusted according to actual conditions to adapt to different usage scenarios and requirements. Therefore, by setting the memory decay coefficient for the context information in the external memory module and managing it with the LRU strategy, the efficiency of the cache and the utilization of the storage space can be effectively improved, thereby improving the performance of the system and the user experience. This strategy is not only applicable to text information management, but can also be widely used in other scenarios that require cache management.

[0073] Here, the update management of context information can be divided into two modes: batch management and scattered management. Batch management is applicable to a large amount of information stored in the same period. By setting a unified update management cycle, the information is updated or eliminated uniformly at the end of each cycle to simplify the management process. Scattered management formulates different update strategies for each tag according to the characteristics and importance of context information, and regularly checks and updates the information under each tag to achieve more refined and dynamic management.

[0074] Exemplarily, assume that the following context information is deposited between 10:00 and 11:00 on November 19, 2024. Further assume that context information 1 is, deposit time = 10:05 on November 19, 2024, content = "How to add a bank card", access frequency = 5 times, memory decay coefficient = 1.0. Context information 2 is, deposit time = 10:15 on November 19, 2024, content = "What to do if you forget your password", access frequency = 3 times, memory decay coefficient = 1.2. Context information 3: deposit time = 10:30 on November 19, 2024, content = "Introduction to member benefits", access frequency = 8 times, memory decay coefficient = 1.5. The update management cycle is 1 month. Based on the above assumptions, the update operation can be to check this batch of context information at 11:00 on December 19, 2024. Context information 2 has the lowest access frequency (3 times), and its comprehensive score is: storage time + memory decay coefficient = 10:15 + 1.2 = 11:15.2. The comprehensive score of context information 1 is: 10:05 + 1.0 = 10:06. The comprehensive score of context information 3 is: 10:30 + 1.5 = 10:31.5. Context information 2 is selected for elimination.

[0075] Assume that there is the following context information with different labels in the external memory module, such as: Context information 1: Label = Account Upgrade, Deposit Time = 10:00, November 19, 2024, Content = "How to upgrade account membership", Access Frequency = 10 times, Memory Decay Coefficient = 1.0.

[0076] Context information 2: Label = FAQ, storage time = November 19, 2024 11:00, content = "What to do if you forget your password", access frequency = 5 times, memory decay coefficient = 1.2.

[0077] Context information 3: Label = Membership Benefits, Deposit Time = November 19, 2024 12:00, Content = "Introduction to Membership Benefits", Access Frequency = 12 times, Memory Attenuation Coefficient = 1.5.

[0078] Based on the above settings, Tag 1 (Account Upgrade): Update cycle is 1 week, access frequency threshold is 5 times, memory decay coefficient is 1.0. Tag 2 (FAQ): Update cycle is 1 month, access frequency threshold is 10 times, memory decay coefficient is 1.2. Tag 3 (Membership Benefits): Update cycle is 3 months, access frequency threshold is 15 times, memory decay coefficient is 1.5.

[0079] The tag-based update strategy can be, Tag 1 (account upgrade): Check at 10:00 on the 19th of each week. The access frequency of context information 1 is 10 times, which is 5 times higher than the threshold, and is retained.

[0080] Label 2 (FAQ): Check at 11:00 on the 19th of each month. The access frequency of context information 2 is 5 times, which is lower than the threshold of 10 times. The comprehensive score is: 11:00+1.2=11:01.2, and it is eliminated.

[0081] Tag 3 (membership benefits): Check at 12:00 on the 19th day of every three months. The access frequency of context information 3 is 12 times, which is higher than the threshold of 15 times and is retained.

[0082] In this way, batch management simplifies the management process and is suitable for a large amount of information stored in the same period. Scattered management is more flexible and accurate, and different update strategies can be formulated according to the characteristics and importance of context information. Tag-based scattered management can perform refined management for different types of context information to ensure that important information is retained. By setting reasonable update cycles and access frequency thresholds, storage resources can be optimized and unnecessary storage usage can be reduced. Both batch management and scattered management can be dynamically adjusted according to system operation and user needs to ensure that the system always runs efficiently. By managing context information in batches and in pieces, the system performance and user experience can be effectively improved, ensuring that the information in the external memory module is always kept up to date and most relevant.

[0083] Further optionally, in the above steps, an LRU strategy is adopted to perform memory management on the first context information and the second context information stored in the external memory module based on the memory decay coefficient. Specifically, if the external memory module meets the management conditions, the information replacement strategy of the external memory module is determined based on the model application scenario in which the external memory module is deployed. Among them, the management conditions include at least one of the following: the storage capacity of the external memory module reaches the upper limit, the preset update cycle of the external memory module is reached, and the preset update event of the external memory module is triggered. Furthermore, based on the information replacement strategy, any one or more operations of updating, deleting, modifying, and merging are performed on the first context information and the second context information stored in the external memory module. Similar to the specific implementation example of the above-mentioned LRU strategy, it will not be repeated here.

[0084] Further optionally, in the external memory module, in order to better adapt to different application scenarios, the application scenario of the model can be identified before determining the information replacement strategy, and the information replacement strategy can be dynamically configured and updated according to the model operation parameters and scene environment parameters. Specifically, based on the model application scenario deployed by the external memory module, before determining the information replacement strategy of the external memory module, the model application scenario deployed by the external memory module can also be identified; the model operation parameters and / or scene environment parameters in the model application scenario are obtained; based on the model operation parameters and / or scene environment parameters, the information replacement strategy of the external memory module is parameterized and parameter update management is performed to dynamically maintain any one or more of the context information type, storage capacity, and storage space attributes to be managed in the external memory module.

[0085] For example, the model application scenario deployed by the external memory module is identified through the model configuration information, log records or user-defined methods. The model operation parameters and scenario environment parameters in the model application scenario are obtained. Assume that the model application scenario is a customer service chat robot, and the model operation parameters are request processing rate, conversation rounds, user activity, etc. Assume that the scenario environment parameters are user access volume, peak time period, system resource utilization, etc. Exemplarily, the request processing rate is the number of requests processed per second. Conversation rounds are the average conversation rounds between users and chat robots. User activity is the user's active time period and active frequency. User access volume is the number of user visits per day. Peak time period is the peak time period of user access. System resource utilization is the utilization rate of the system's CPU, memory and other resources.

[0086] For example, the request processing rate is 100 requests per second. The number of conversation rounds is 5 rounds per conversation on average. The user activity is that the user is most active from 2 pm to 6 pm. The number of user visits is 1,000 users per day. The peak time period is from 4 pm to 6 pm every day. The system resource utilization is that the CPU utilization reaches 80% during the peak period.

[0087] The types of context information to be managed are determined based on user activity and peak time periods, such as frequently asked questions (FAQs). The storage capacity is determined based on user access volume and request processing rate, such as storing 10,000 pieces of context information. The storage space attribute is optimized based on system resource utilization, such as reducing the frequency of eliminating expired information during peak hours.

[0088] Periodic updates are to regularly check model operation parameters and scenario environment parameters, and adjust information replacement strategies according to new parameters. Event-triggered updates are to immediately update information replacement strategies when the system detects a specific event (such as a sudden increase in user visits).

[0089] For example, the application scenario is a customer service chatbot. The initial configuration is that the types of context information to be managed are FAQs, account management, and member benefits. The storage capacity is 10,000 pieces of context information. The storage space attribute is to reduce the frequency of elimination of expired information during peak hours.

[0090] Based on this, the parameters obtained are: the request processing rate is 100 requests per second, the number of conversation rounds is an average of 5 rounds per conversation, the user activity is that users are most active from 2 pm to 6 pm, the user access volume is 1,000 users per day, the peak time period is from 4 pm to 6 pm every day, and the system resource utilization is that the CPU utilization reaches 80% during the peak period.

[0091] In the dynamic adjustment strategy, the types of context information to be managed can be analyzed, and it can be found that users are most active from 2 pm to 6 pm, and the peak time period is from 4 pm to 6 pm, and the frequency of access to FAQs and account management information is high. Alternatively, FAQs and account management information can be managed first to reduce the frequency of elimination of these types of information.

[0092] The storage capacity is 1,000 user visits per day, 100 requests per second, and an average of 5 rounds per conversation. The storage capacity is adjusted to 15,000 pieces of context information to cope with the high visit volume during peak hours.

[0093] Storage space attributes: The system resource utilization rate reaches 80% during peak hours. It is necessary to ensure the stable operation of the system during peak hours. Adjust to reduce the frequency of eliminating expired information during peak hours (4 pm to 6 pm) and keep more context information in the cache. During non-peak hours, increase the frequency of eliminating expired information to free up storage space.

[0094] Periodic update means setting the update cycle to update the model operation parameters and scene environment parameters once a week. The update operation is that at 10:00 on the 19th of each week, the system automatically obtains the latest model operation parameters and scene environment parameters. Readjust the information replacement strategy according to the new parameters. For example, if the number of user visits increases to 1,500, the storage capacity can be further increased. If the user active time period is adjusted to 3 pm to 7 pm, the management strategy for peak hours can be adjusted accordingly.

[0095] The event trigger is updated to set the trigger condition to update the information replacement policy immediately when the number of user visits exceeds 1,200 or the system resource utilization exceeds 90%. The update operation is that the user visit trigger is that the system detects that the number of user visits exceeds 1,200, and immediately increases the storage capacity and reduces the frequency of information elimination. The system resource utilization trigger is that the system detects that the CPU utilization exceeds 90%, and immediately reduces the frequency of information elimination to reduce the system burden.

[0096] Assume that on November 19, 2024, the context information in the external memory module is as follows: Context information 1 is label = Account upgrade, deposit time = November 19, 2024 10:00, content = "How to upgrade account membership", access frequency = 100 times, memory decay coefficient = 1.0. Context information 2 is label = FAQ, deposit time = November 19, 2024 11:00, content = "What to do if you forget your password", access frequency = 80 times, memory decay coefficient = 1.2. Context information 3 is label = Membership benefits, deposit time = November 19, 2024 12:00, content = "Introduction to member benefits", access frequency = 20 times, memory decay coefficient = 1.5.

[0097] The initial information replacement strategy is that the types of context information to be managed are FAQs, account management, and member benefits, the storage capacity is 10,000 pieces of context information, and the storage space attribute is to reduce the frequency of elimination of expired information during peak hours.

[0098] The new model operation parameters and scenario environment parameters obtained are: user activity is that users are most active from 3 pm to 7 pm, user visits are 1,200 users per day, the peak time period is from 5 pm to 7 pm every day, and the system resource utilization is that the CPU utilization reaches 95% during the peak period.

[0099] The dynamic adjustment operation is to prioritize the types of context information to be managed, such as FAQs and account management information, to maintain a high access frequency for this information. The storage capacity is adjusted to 12,000 pieces of context information to cope with higher user access. The storage space attribute is to reduce the frequency of eliminating expired information during peak hours (5pm to 7pm) and keep more context information in the cache. During non-peak hours, increase the frequency of eliminating expired information to free up storage space.

[0100] In short, by identifying the application scenarios of the model and obtaining the model operation parameters and scenario environment parameters, the information replacement strategy of the external memory module can be dynamically configured and updated. This dynamic management method not only improves the flexibility and adaptability of the system, but also optimizes the utilization of storage resources, ensuring that the information in the external memory module is always up to date and most relevant in different application scenarios, improving the system performance and user experience.

[0101] As an optional embodiment, after storing each context information in different storage areas of the external memory module in 103, the storage level of the context information in different storage areas is also dynamically adjusted. For example, based on different categories of sessions, the hierarchical index is managed, and different indexes correspond to the context information of different types of sessions. Specifically, by dynamically adjusting the storage level of different storage areas in the external memory module, the access efficiency can be significantly improved, the storage resource utilization can be optimized, the needs of different users can be adapted, and the flexibility of the system can be improved. Specifically, by classifying and storing the context information and setting different hierarchical indexes, the system can dynamically adjust the priority of the storage level according to the activity, importance, user needs, etc. of the session. For example, assuming that the access frequency of FAQs continues to increase, the system dynamically upgrades the level of storage area 1 to level 0 (highest priority) to ensure that the information of FAQs can be quickly accessed, thereby improving the user experience. At the same time, for context information with higher importance, the system can dynamically allocate more storage space, such as when the access frequency of account management information increases, the level is upgraded and more storage space is allocated to avoid performance degradation. In addition, the system can dynamically adjust the storage level according to the personalized needs of users. For example, when a user frequently queries personal preference information, the level of storage area 4 is upgraded and the personalized preference information of the user is stored first, providing more accurate services. This dynamic management method not only enhances the performance of the system and user experience, but also ensures that in different application scenarios, the external memory module can efficiently and intelligently manage context information, quickly respond to changes in user needs, and provide the best service.

[0102] As an optional embodiment, after storing each context information in different storage areas of the external memory module in 103, the storage level of different categories of context information can also be adjusted based on the predicted value of the business development trend. For example, the storage level of context information corresponding to information content with a high search frequency is higher, so that it can be quickly called during the session processing. Specifically, by dynamically adjusting the storage level of different categories of context information in the external memory module based on the predicted value of the business development trend, the user's search experience can be significantly improved, the utilization of storage resources can be optimized, the intelligence of the system can be enhanced, and marketing activities can be effectively supported. Specifically, when "healthy diet" is searched more than a preset number of times, the system will dynamically upgrade the storage level of context information related to healthy diet (such as healthy diet suggestions, healthy recipes, etc.) to a high priority, ensuring that users can quickly obtain relevant information during the session and improve user experience. This dynamic management method not only improves the performance and user experience of the system, but also enables the external memory module to manage context information more intelligently and efficiently, cope with different application scenarios and user needs, quickly respond to changes in user needs, and provide the best service.

[0103] As an optional embodiment, after storing each type of context information in different storage areas of the external memory module in 103, the storage levels of different types of context information may be adjusted based on the user's historical behavior.

[0104] For example, in the external memory module, after the context information is stored in different storage areas, the storage level of different categories of context information can be dynamically adjusted based on the user's historical behavior. This adjustment can increase the priority of context information that users prefer or frequently access, thereby enabling rapid call-up during session processing, bringing many beneficial effects. User behavior analysis is to collect and analyze user search history, click records, purchase behavior and other data to identify user preferences and commonly used context information. Dynamically adjusting the storage level is to adjust the storage level of different categories of context information based on the user's historical behavior, and increase the priority of context information that users prefer.

[0105] For example, the user history behavior is that user A frequently searches for information such as "healthy diet" and "fitness plan". The context information categories are healthy diet suggestions, fitness plans, and nutritional knowledge. The effect is to ensure that the user can quickly obtain personalized information during the session by increasing the priority of the context information preferred by the user, thereby improving the user experience.

[0106] For example, the system dynamically adjusts the storage level of "healthy diet suggestions" and "fitness plans" to high priority based on the search history of user A. Quick response means that when user A asks about relevant content, the system can quickly call up high-priority context information and provide immediate and accurate answers. The effect is that by adjusting the storage level, the use of storage resources can be optimized, ensuring that the context information preferred by the user occupies more storage space and computing resources.

[0107] For example, if user A frequently accesses information related to "fitness plan", the system dynamically adjusts storage space allocation to ensure that there is enough storage space for this information. Resource optimization ensures that user A's preferred "fitness plan" related information has enough storage space during peak hours to avoid performance degradation due to insufficient storage space. The effect is that by dynamically adjusting the storage level, the system can intelligently adjust the storage strategy according to user behavior, improving the intelligence level of the system.

[0108] For example, dynamic adjustment means that the system can automatically adjust the storage level according to the behavior data of user A without manual intervention. Intelligent management means that the system can intelligently identify and respond to the preferences of user A, improving the intelligence level of the system. The effect is that by providing personalized services and improving the priority of context information of user preferences, user satisfaction and stickiness can be effectively improved. Dynamic adjustment means that if user A frequently queries "healthy diet suggestions", the system dynamically raises the storage level of this context information to high priority. User stickiness means that when user A visits again, the system can quickly provide detailed healthy diet suggestions to improve user satisfaction and stickiness. Assume that on November 19, 2024, the context information in the external memory module is as follows: Context information 1 is category = healthy diet suggestions, storage area = storage area 1, level = level 1, content = "How to make a healthy diet plan", access frequency = 50 times. Context information 2 is category = fitness plan, storage area = storage area 2, level = level 2, content = "personal fitness plan recommendation", access frequency = 30 times. Context information 3 is category = Nutritional knowledge, storage area = Storage area 3, level = Level 3, content = "Nutrition knowledge popularization", and access frequency = 20 times. The historical behavior analysis of user A is as follows: the search history is that user A frequently searches for "healthy diet" and "fitness plan" related content. The click record is that user A clicks on "healthy diet suggestions" and "fitness plan" related links many times. The purchase behavior is that user A purchases multiple fitness-related products.

[0109] Accordingly, the dynamic adjustment operation is: the user behavior analysis is: healthy diet suggestions are frequently searched and clicked by user A, and the access frequency increases to 80 times. Fitness plans are frequently searched and purchased by user A, and the access frequency increases to 60 times. The access frequency of nutritional knowledge for user A remains stable at 20 times. The dynamic adjustment of storage levels is: Storage area 1 (healthy diet suggestions): the level is upgraded to level 0 (highest priority). Storage area 2 (fitness plans): the level is upgraded to level 1. Storage area 3 (nutritional knowledge): the level remains at level 3.

[0110] In the above example, the personalized user experience is improved when user A asks for healthy diet advice or fitness plans, and the system can quickly provide detailed answers to improve the user experience. Optimizing resource utilization is to have enough storage space for healthy diet advice and fitness plan related information to avoid performance degradation. Enhancing system intelligence is to automatically identify and adjust the storage level of context information preferred by user A, and improve the intelligence level of the system. Improving user stickiness is to effectively improve user satisfaction and stickiness by providing personalized services and increasing the priority of context information preferred by user A.

[0111] In short, by dynamically adjusting the storage level of different categories of context information in the external memory module based on user historical behavior, we can significantly improve personalized user experience, optimize the use of storage resources, enhance the intelligence of the system, and effectively improve user satisfaction and stickiness. This dynamic management method not only improves the performance and user experience of the system, but also enables the external memory module to manage context information more intelligently and efficiently to cope with different application scenarios and user needs.

[0112] As an optional embodiment, after storing each context information in different storage areas of the external memory module in 103, the space capacity of different storage areas in the external memory module can also be dynamically adjusted. Specifically, by dynamically adjusting the space capacity of different storage areas in the external memory module, the allocation of storage resources can be significantly optimized, the performance and user experience of the system can be improved, and the adaptability and flexibility of the system can be enhanced. This dynamic management method not only improves the overall performance and user experience of the system, but also enables the external memory module to manage storage resources more intelligently and efficiently to cope with different application scenarios and user needs. Specifically, by identifying the access frequency and storage requirements of each storage area through data analysis, the system can dynamically adjust the space capacity to ensure that important or frequently accessed context information has sufficient storage space, thereby avoiding performance degradation and improving access efficiency. For example, assuming that the access frequency of healthy diet advice and fitness plans increases significantly, the system can automatically increase the space capacity of the corresponding storage area to ensure that this information can respond quickly during peak hours. This dynamic adjustment not only optimizes the allocation of storage resources and improves the performance of the system, but also enhances the adaptability of the system, enabling it to automatically adjust the storage strategy according to actual needs without manual intervention. Ultimately, this intelligent management approach significantly improves the user experience, allowing users to get faster and more accurate responses when searching for important information, improving satisfaction and overall usage experience.

[0113] In the embodiment of the present application, the above three steps provide significant technical optimization and beneficial effects for long text data processing by introducing external memory modules and multi-level memory structures. Specifically, first, by extracting the context information of each text fragment, the model can more deeply understand the key content in the long text, avoiding the context loss problem that traditional models are prone to when processing long texts. Decomposing a long text into multiple text fragments and extracting their context information helps the model better capture the specific meaning and logical relationship of each fragment, and improves the degree of refinement of the processing. When generating text, the context information of each fragment can be more accurately referenced, thereby generating more coherent and logical text content. Then, each context information is stored in different storage areas of the external memory module. In this way, the context information of different importance and timeliness can be reasonably stored and managed through the external memory module (including short-term memory area and long-term memory area), avoiding information redundancy and confusion. By storing the important context information in the current session in the short-term memory area, the model can quickly access these key information, improving the reasoning and processing speed. The long-term memory area is used to store context information that recurs in multiple sessions, which helps the model accumulate and utilize long-term knowledge and enhances the knowledge base and adaptability of the model. Finally, the first context information and the second context information are input into the large language model for natural language processing. By integrating the context information of the current session (short-term memory) and the historical session (long-term memory), the large language model can obtain more comprehensive and coherent context support when generating and understanding text, and the generated content is more natural and logical. Combining the context information of short-term and long-term memory, the model can understand the content of long texts more accurately and generate outputs that are highly relevant to the context, significantly improving the accuracy and relevance of the model. The design of the multi-level memory structure makes this technology not only suitable for long text processing, but also can play a role in other tasks that require context memory, enhancing the versatility and adaptability of the model. In summary, these three steps significantly improve the context understanding ability, computational efficiency and result accuracy of long text processing by introducing external memory modules and multi-level memory structures, while enhancing the flexible adaptability of the model, providing more comprehensive and efficient technical support for natural language processing tasks.

[0114] Based on the same implementation principle, the embodiment of the present application also provides a long text processing device, which is used to implement the long text processing method in the above embodiment. Figure 2 This is a structural block diagram of the long text processing device provided in the embodiment of the present application. Figure 2 As shown, the device comprises at least the following units: An extraction unit, configured to extract context information corresponding to each text segment from the long text data to be processed in the current session; A storage unit is configured to store each context information in different storage areas of an external memory module; the external memory module includes a short-term memory area and a long-term memory area; the short-term memory area is used to store first context information whose importance reaches a preset condition in a current session; the long-term memory area is used to store second context information that appears repeatedly in multiple sessions; the multiple sessions include a current session and / or a historical session; An execution unit is configured to input the first context information and the second context information into a large language model, and implement natural language processing of the long text data through the large language model.

[0115] In an optional embodiment, when the storage unit stores each piece of context information in different storage areas of the external memory module, the storage unit is configured as follows: Obtaining, through a self-attention mechanism, an importance evaluation value corresponding to each context information in the current session; the importance evaluation value is used to indicate the correlation between the context information and the long text data; the higher the correlation between the context information and the long text data, the higher the importance evaluation value corresponding to the context information; Selecting first context information whose importance evaluation value meets a preset condition in the current session, and storing the first context information in the short-term memory area; Each context information in the current session is matched with the second context information stored in the long-term memory area, and the context information that appears repeatedly in the current session and the historical session is used as the newly added second context information and updated to the long-term memory area.

[0116] In an optional embodiment, the storage unit stores each piece of context information in different storage areas of the external memory module, and is further configured to: Get the access frequency corresponding to each context information in the current session; The first context information whose access frequency in the current session is higher than the set access frequency threshold is selected, and the first context information is stored in the short-term memory area.

[0117] In an optional embodiment, when the execution unit inputs the first context information and the second context information into the large language model, it is configured to: Get the target text information to be processed from the current session; Generate a corresponding search request based on the target text information; the search request at least includes any one or more of the text content information, text type, and text subject in the target text information; Retrieving first context information and second context information matching the target text information from the external memory module based on the retrieval request; The retrieved first context information and the second context information are fused, and the fusion result is input into the large language model.

[0118] In an optional embodiment, when the execution unit retrieves the first context information and the second context information matching the target text information from the external memory module based on the retrieval request, it is configured to: Extracting a target text segment from the target text information; Obtaining the first context information and the second context information in the external memory module, and the cosine similarity between each of the first context information and the target text segment; The first context information and the second context information whose cosine similarity meets a preset condition are selected as the first context information and the second context information that match the target text segment.

[0119] In an optional embodiment, when the execution unit performs fusion processing on the retrieved first context information and the second context information, it is configured to: Sort the retrieved first context information and second context information from high to low according to the cosine similarity, so as to select the first context information and the second context information whose similarity is in a preset position or reaches a preset similarity threshold; The selected first context information and the second context information are subjected to multi-level memory fusion processing to obtain the fusion result.

[0120] In an optional embodiment, the first context information and the second context information in the external memory module are respectively associated with corresponding dynamic indexes according to their respective text content characteristics and / or text format types; the execution unit, before obtaining the first context information and the second context information in the external memory module and the cosine similarity between each of them and the target text segment, is further configured as follows: Identifying text attribute features of the target text segment; the text attribute features are used to represent text content characteristics and / or text format type of the target text segment; Constructing a text search precondition for the target text segment based on text content characteristics and / or text format type; Based on the text retrieval precondition, a target dynamic index in the external memory module that matches the target text segment is determined, and first context information and second context information associated with the target dynamic index are obtained as the first context information and second context information to be screened. In an optional embodiment, after the storage unit stores each piece of context information in different storage areas of the external memory module, it is further configured to: Setting corresponding memory decay coefficients for the first context information and the second context information in the external memory module, respectively; wherein the lower the access frequency of the context information, the greater the corresponding memory decay coefficient; The least recently used strategy LRU is adopted to perform memory management on the first context information and the second context information stored in the external memory module based on the memory decay coefficient.

[0121] In an optional embodiment, the storage unit, when adopting the least recently used strategy LRU and performing memory management on the first context information and the second context information stored in the external memory module based on the memory decay coefficient, is configured as follows: If the external memory module meets the management conditions, then based on the model application scenario deployed by the external memory module, determine the information replacement strategy of the external memory module; wherein the management conditions include at least one of the following: the storage capacity of the external memory module reaches the upper limit, the preset update cycle of the external memory module is reached, and the preset update event of the external memory module is triggered; Based on the information replacement strategy, any one or more operations of updating, deleting, modifying, and merging are performed on the first context information and the second context information stored in the external memory module.

[0122] In an optional embodiment, the storage unit, before determining the information replacement strategy of the external memory module based on the model application scenario deployed by the external memory module, is further configured to: Identifying a model application scenario in which the external memory module is deployed; Obtaining model operation parameters and / or scenario environment parameters in the model application scenario; Based on the model operation parameters and / or scene environment parameters, the information replacement strategy of the external memory module is parameter configured and parameter update management is performed to dynamically maintain any one or more of the context information type, storage capacity, and storage space attributes to be managed in the external memory module.

[0123] It should be noted that, regarding the long text processing device provided by the implementation of this application, the specific functions and implementation details of each module therein have the same implementation principles as the implementation process of the corresponding steps in the aforementioned method embodiment. For details, please refer to the description of the corresponding parts in the aforementioned method embodiment, which will not be repeated here.

[0124] Based on the same implementation principle, the embodiment of the present application also provides an electronic device, Figure 3 is a structural block diagram of the electronic device 300, such as Figure 3 As shown, the electronic device 300 includes: a processor 301, a memory 302, a communication interface 303, a communication bus 304 and a controller 305; wherein the processor 301, the memory 302, and the communication interface 303 communicate with each other via the communication bus 304; the memory 302 is used to store computer programs; the processor 301 is used to execute the programs stored in the memory 302 to implement corresponding processing functions; the communication bus 304 is used for communication between the electronic device 300 and other devices; and the controller 305 is used to implement the long text processing method described in the above method embodiment.

[0125] In the embodiment of the present application, the communication bus 304 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 304 may be divided into an address bus, a data bus, a control bus, etc., and the specific form is not limited. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0126] The memory 302 may include a random access memory (RAM) or a non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far away from the processor 301 .

[0127] The processor 301 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other editable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the specific form can be determined according to actual needs.

[0128] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0129] One or more embodiments of the present application are described above in conjunction with optional implementation methods, but these implementation methods are only exemplary and serve only as an illustration. On this basis, multiple replacements and improvements can be made to one or more embodiments of the present application, all of which fall within the scope of protection of one or more embodiments of the present application.

Claims

1. A long text processing method, characterized in that: The method comprises: For the long text data to be processed in the current session, extracting context information corresponding to each text segment from the long text data; Each piece of context information is stored in different storage areas of an external memory module; the external memory module includes a short-term memory area and a long-term memory area; the short-term memory area is used to store first context information whose importance reaches a preset condition in the current session; the long-term memory area is used to store second context information that appears repeatedly in multiple sessions; the multiple sessions include the current session and / or historical sessions; The first context information and the second context information are input into a large language model, and natural language processing of the long text data is performed through the large language model.

2. The method according to claim 1, characterized in that The storing of each piece of context information in different storage areas of the external memory module respectively includes: Obtaining, through a self-attention mechanism, an importance evaluation value corresponding to each context information in the current session; the importance evaluation value is used to indicate the correlation between the context information and the long text data; the higher the correlation between the context information and the long text data, the higher the importance evaluation value corresponding to the context information; Selecting first context information whose importance evaluation value meets a preset condition in the current session, and storing the first context information in the short-term memory area; Each context information in the current session is matched with the second context information stored in the long-term memory area, and the context information that appears repeatedly in the current session and the historical session is used as the newly added second context information and updated to the long-term memory area.

3. The method according to claim 2, characterized in that The storing of each context information into different storage areas of the external memory module respectively also includes: Get the access frequency corresponding to each context information in the current session; The first context information whose access frequency in the current session is higher than the set access frequency threshold is selected, and the first context information is stored in the short-term memory area.

4. The method according to claim 1, characterized in that: The step of inputting the first context information and the second context information into a large language model includes: Get the target text information to be processed from the current session; Generate a corresponding search request based on the target text information; the search request at least includes any one or more of the text content information, text type, and text subject in the target text information; Retrieving first context information and second context information matching the target text information from the external memory module based on the retrieval request; The retrieved first context information and the second context information are fused, and the fusion result is input into the large language model.

5. The method according to claim 4, characterized in that The retrieving first context information and second context information matching the target text information from the external memory module based on the retrieval request includes: Extracting a target text segment from the target text information; Obtaining the first context information and the second context information in the external memory module, and the cosine similarity between each of the first context information and the target text segment; The first context information and the second context information whose cosine similarity meets a preset condition are selected as the first context information and the second context information that match the target text segment.

6. The method according to claim 5, characterized in that The fusing the retrieved first context information and the second context information includes: Sort the retrieved first context information and second context information from high to low according to the cosine similarity, so as to select the first context information and the second context information whose similarity is in a preset position or reaches a preset similarity threshold; The selected first context information and the second context information are subjected to multi-level memory fusion processing to obtain the fusion result.

7. The method according to claim 5, characterized in that The first context information and the second context information in the external memory module are respectively associated with corresponding dynamic indexes according to their respective text content characteristics and / or text format types; Before obtaining the cosine similarity between the first context information and the second context information in the external memory module and the target text segment, the method further includes: Identifying text attribute features of the target text segment; the text attribute features are used to represent text content characteristics and / or text format type of the target text segment; Constructing a text search precondition for the target text segment based on text content characteristics and / or text format type; Based on the text retrieval precondition, a target dynamic index in the external memory module that matches the target text segment is determined, and first context information and second context information associated with the target dynamic index are obtained as the first context information and second context information to be screened.

8. The method according to claim 1, characterized in that After storing each piece of context information in different storage areas of the external memory module respectively, the method further includes: Setting corresponding memory decay coefficients for the first context information and the second context information in the external memory module, respectively; wherein the lower the access frequency of the context information, the greater the corresponding memory decay coefficient; The least recently used strategy LRU is adopted to perform memory management on the first context information and the second context information stored in the external memory module based on the memory decay coefficient.

9. The method according to claim 8, characterized in that The method adopts the least recently used strategy LRU and performs memory management on the first context information and the second context information stored in the external memory module based on the memory decay coefficient, including: If the external memory module meets the management conditions, then based on the model application scenario deployed by the external memory module, determine the information replacement strategy of the external memory module; wherein the management conditions include at least one of the following: the storage capacity of the external memory module reaches the upper limit, the preset update cycle of the external memory module is reached, and the preset update event of the external memory module is triggered; Based on the information replacement strategy, any one or more operations of updating, deleting, modifying, and merging are performed on the first context information and the second context information stored in the external memory module.

10. The method according to claim 9, characterized in that Before determining the information replacement strategy of the external memory module based on the model application scenario deployed by the external memory module, the method further includes: Identifying a model application scenario in which the external memory module is deployed; Obtaining model operation parameters and / or scenario environment parameters in the model application scenario; Based on the model operation parameters and / or scene environment parameters, the information replacement strategy of the external memory module is parameter configured and parameter update management is performed to dynamically maintain any one or more of the context information type, storage capacity, and storage space attributes to be managed in the external memory module.

11. A long text processing device, characterized in that: The device comprises at least the following units: An extraction unit, configured to extract context information corresponding to each text segment from the long text data to be processed in the current session; A storage unit is configured to store each context information in different storage areas of an external memory module; the external memory module includes a short-term memory area and a long-term memory area; the short-term memory area is used to store first context information whose importance reaches a preset condition in a current session; the long-term memory area is used to store second context information that appears repeatedly in multiple sessions; the multiple sessions include a current session and / or a historical session; An execution unit is configured to input the first context information and the second context information into a large language model, and implement natural language processing of the long text data through the large language model.

12. A chip, characterized in that: The chip includes a processor coupled to a transceiver, and is used to execute the long text processing method according to any one of claims 1 to 10.

13. An electronic device, characterized in that: The invention comprises a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the long text processing method according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed, implement the long text processing method described in any one of claims 1 to 10.

Images