A long dialogue key information extraction method for a 5g new call environment

Abstract

The application relates to a long dialogue information extraction method for a 5G new call, which comprises the following steps: one is voice signal collection and preprocessing: the system collects voice signals between a first user and a second user through a 5G call device; two is automatic speech recognition: an automatic speech recognition technology converts the preprocessed voice signals into text data; three is key information extraction through an iterative mode; and four is output result analysis. Through the multi-level key information extraction and iterative optimization mechanism, the information extraction capability of a large language model (LLM) in a long dialogue scene is significantly enhanced. The method effectively alleviates the complexity of long text understanding and the limitation of computing resources.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and specifically to a method for extracting key information from long conversations in a new 5G call environment. Background Art

[0002] Currently, information extraction technologies for long conversations in the new 5G call environment primarily involve key technologies such as automatic speech recognition (ASR), text summarization, keyword extraction, and to-do list extraction. Since ASR is only the initial step in the information processing pipeline, this article does not discuss it in depth. Text summarization methods include statistical extractive summarization and generative summarization. Statistical extractive summarization methods generate summaries by extracting key sentences or phrases from the text, while generative summarization methods use language models to generate new sentences to construct summaries. While extractive summarization offers certain guarantees regarding grammar and syntax, it also faces certain challenges, such as incorrect content selection, poor coherence, and limited flexibility. Generative summarization allows for the inclusion of new words or phrases in summaries, offering greater flexibility. Large language models (LLMs), due to their powerful semantic understanding and flexible generation methods, generate summaries that exhibit high fluency and coherence. Keywords are words that express the central content of a document. Traditional keyword extraction methods, including frequency-based, graph-based, and statistical methods, have played a foundational role in the field of natural language processing (NLP). However, the emergence of LLMs has ushered in a new era of keyword extraction, accurately capturing contextual nuances, semantic relationships, and the complexities of specific domains. LLMs' enhanced ability to capture context and semantics makes them more suitable for keyword extraction in complex linguistic domains. A to-do list is a list of tasks or activities that need to be completed at a certain point in the future. Before the rise of LLMs, to-do list extraction employed rule-based and text classification methods. Rule-based methods are suitable for simple, structured scenarios, with limited applicability but strong controllability. Text classification methods, such as BERT and Fastext, use pre-trained models, but their performance relies heavily on high-quality training data and are sensitive to new data. LLMs, with their ability to understand context, are optimal for extracting complex and flexible to-do lists.

[0003] When designing LLMs, developers typically set an upper limit on the input text length to balance the model's capabilities with computational performance. Therefore, when processing long texts, it is often necessary to truncate or split the text to meet the input length constraint. As LLMs continue to evolve, the maximum number of tokens supported has gradually increased, from the initial 4K and 8K to the current 16K, 32K, and even 128K. This advancement provides stronger support for long text processing, but it also raises a series of new challenges:

[0004] 1. Computing resource limitations

[0005] While current LLMs can theoretically support longer token sequences, computing resources (especially GPU memory) remain a major bottleneck in practical applications. For example, even if the input text is within the 128K token range, the system may not be able to process the entire sample in one go due to limited GPU memory. Therefore, GPU memory capacity becomes a key limiting factor in long text processing.

[0006] 2. Insufficient model understanding

[0007] Even with sufficient resources, many open-source LLMs still struggle to understand and summarize long texts. Inputting too many tokens at once can make it difficult for the model to accurately capture key information, or even cause "hallucinations." Therefore, ensuring that models can fully understand each fragment and effectively extract key information when processing long texts is an important research direction. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for extracting key information of long conversations in the new 5G call environment to solve the problems raised in the above background technology.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for extracting key information from long conversations in a 5G new call environment, the method comprising the following steps:

[0010] Step 1: Voice signal collection and preprocessing: The system collects the voice signal between the first and second users through the 5G call device;

[0011] Step 2: Automatic speech recognition: Automatic speech recognition technology converts the pre-processed speech signal into text data;

[0012] Step 3: Extract key information, which includes the following sub-steps:

[0013] Sub-step 31: Split the long text according to a preset segmentation standard to obtain multiple short text segments;

[0014] Sub-step 32: Input the short text segment and the first prompt word Prompt 1 into the large language model LLM to generate key information: the extraction results generated by the large language model LLM include to-do items, keywords and text summary;

[0015] Sub-step 33: Extract results through iteration: take part of the results of the first stage and splice the second prompt word Prompt2, input it into the large language model LLM again, and generate the key information extraction results of the second stage. After splicing, input LLM with Prompt 2 to generate the key information extraction results of the second stage . And so on, After splicing, input LLM with Prompt 2 to get the result of the second stage .

[0016]

[0017] In each iteration, the output of the previous stage ( ) will be used as the input for the next stage, and will be concatenated with the second prompt word Prompt2 and then input into the large language model LLM for processing; this process will continue to iterate until the total length of the input text and the second prompt word Prompt2 can be input into the large language model LLM at one time, generating the final key information extraction result ;

[0018] Step 4: Output result analysis.

[0019] Preferably, the length of the short text segment is limited by the difference between the input length and the length of the first prompt word Prompt 1 .

[0020] Preferably, the splicing length is also limited by the difference between the input token length set by the system and the token length of Prompt2.

[0021] Preferably, the large language model LLM includes an open source model and / or a closed source model.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] This invention can enhance the ability to understand long conversations, effectively overcoming the problem of insufficient understanding faced by large language models (LLMs) when processing long conversation inputs. Through a multi-level information extraction and optimization mechanism, it improves the processing accuracy of complex long texts.

[0024] At the same time, it optimizes computing resource utilization, solving the challenge of significantly increasing GPU computing resource requirements in the LLM inference phase as the number of input tokens increases, thereby improving the system's resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of the present invention;

[0026] Figure 2 A specific example of Prompt 1 provided by the present invention;

[0027] Figure 3 A specific example of Prompt2 provided by the present invention; DETAILED DESCRIPTION

[0028] First, the terms used in the embodiments of this specification are explained.

[0029] Text summarization: The goal of text summarization is to extract the main ideas from one or more text sources and create a short, coherent descriptive text that remains consistent with the original text.

[0030] Keyword extraction: Keyword extraction aims to identify and extract important words or phrases from a text that best represent the theme or core content of the text.

[0031] To-do item extraction in the field of natural language processing (NLP) refers to the process of automatically identifying and extracting clear tasks, action items, or to-do items from unstructured text.

[0032] A large language model (LLM) is an artificial intelligence model designed to understand and generate human language. LLMs typically refer to language models containing tens of billions (or more) of parameters. They are trained on massive amounts of text data to achieve a deep understanding of language.

[0033] The technical solution in one embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiment of the present invention.

[0034] See also Figure 1-3 : A method for extracting key information from long conversations in a 5G new call environment, the method comprising the following steps:

[0035] Step 1: Speech signal acquisition and preprocessing:

[0036] The system collects voice signals between the first and second users via 5G communication equipment. During the signal acquisition phase, a series of preprocessing techniques, including noise reduction, echo cancellation, and gain control, are applied to improve the accuracy of the subsequent automatic speech recognition (ASR) process. The preprocessed voice signals are then input into the ASR module for further processing.

[0037] Step 2: Automatic Speech Recognition:

[0038] This paper uses automatic speech recognition technology to convert preprocessed speech signals into text data. In the ASR module, the Paraformer-large non-autoregressive end-to-end speech recognition framework is used. This framework consists of an encoder, a predictor, a sampler, a decoder, and a loss function, with the predictor and sampler being the core components.

[0039] First: Predictor: It consists of a two-layer feedforward neural network (FFN) and is responsible for extracting the acoustic feature vector corresponding to the target text and accurately predicting the length of the target text.

[0040] Second: Sampler: This has no learnable parameters and converts acoustic feature vectors into feature vectors containing semantic information through sampling. This is then combined with the target text feature vector to enhance the context modeling capabilities of the bidirectional decoder.

[0041] Step 3: Extract key information

[0042] First, the long dialogue text generated by the ASR module is removed from the long text, and irrelevant special characters, punctuation marks, etc. are removed to keep only the useful text content. After the text preprocessing is completed, the long text is split according to the preset segmentation criteria to obtain multiple short text segments. The split short text segments will serve as the basic processing unit in the subsequent key information extraction and generation process. By splitting long text into multiple short text segments, it is ensured that the subsequent input of the text into the large language model will not be restricted due to excessive length, and the input requirements are met. The specific segmentation criteria are as follows:

[0043] 3.1: The length of the short text segment is limited by the difference between the input length and the length of Prompt 1. That is, the length of the short text segment is not greater than the difference between the input length and the length of Prompt 1;

[0044] 3.2: Split by conversation turn, ensuring that a short text contains a complete conversation between the first and second users;

[0045] 3.3: If the current paragraph length does not reach the short text limit, continue to add the next round of dialogue content; if it exceeds the limit, keep the current length.

[0046] Subsequently, the short text fragment and Prompt 1 are input into the large language model LLM in sequence to generate key information:

[0047] The extraction results generated by LLM include to-do items, keywords and text summaries. Next, take the results of the first stage , and input LLM again with the second prompt word Prompt 2 to generate the key information extraction results of the second stage By analogy, input the concatenated result and Prompt 2 into LLM to get the result of the second stage. .

[0048]

[0049] It should be noted that the concatenation length is also limited by the difference between the input token length and the Prompt2 token length set by the system. In each iteration, the output of the previous stage ( ) will be used as the input for the next stage, combined with Prompt2 and input into LLM again for processing. This process continues iteratively until the total length of the input text and Prompt2 can be input into LLM at one time, generating the final key information extraction result .

[0050] The present invention is not limited to a specific LLM and is applicable to open source models such as Qwen 2.5, LLaMA 3.1, GLM-4, and closed source models such as ChatGPT, ChatGLM, and Wenxinyiyan. As can be seen from the previous detailed description, since the iterative information extraction process is divided into input types of short text segments and extracted key information of the segments, Prompt1 and Prompt2 are designed separately. Specific examples of Prompt 1 and Prompt 2 are as follows: Figure 2 and Figure 3 shown.

[0051] The above are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for extracting key information from long conversations in a 5G new call environment, characterized by: The method comprises the following steps: Step 1: Voice signal collection and preprocessing: The system collects the voice signal between the first and second users through the 5G call device; Step 2: Automatic speech recognition: Automatic speech recognition technology converts the pre-processed speech signal into text data; Step 3: Extract key information, which includes the following sub-steps: Sub-step 31: Split the long text according to a preset segmentation standard to obtain multiple short text segments; Sub-step 32: Input the short text segment and the first prompt word Prompt 1 into the large language model LLM to generate key information: the extraction results generated by the large language model LLM include to-do items, keywords and text summary; Sub-step 33: Extract the results in an iterative manner: take the partial results of the first stage and concatenate them with the second prompt word Prompt2, input them into the large language model LLM again, and generate the key information extraction results of the second stage; After splicing, input LLM with Prompt 2 to generate the key information extraction results of the second stage ; and so on, After splicing, input LLM with Prompt 2 to get the result of the second stage ; In each iteration, the output of the previous stage It will be used as the input for the next stage, concatenated with the second prompt word Prompt2, and then input into LLM again for processing; This process continues iteratively until the total length of the input text and the second prompt word Prompt2 can be input into the LLM at one time, generating the final key information extraction result. ; Step 4: Output result analysis.

2. The method for extracting key information of long conversations in a 5G new call environment according to claim 1, characterized in that: The length of the short text segment is limited by the difference between the input length and the length of the first prompt word Prompt 1 .

3. The method for extracting key information of long conversations in a 5G new call environment according to claim 1, characterized in that: The splicing length is limited by the difference between the input length set by the system and the length of Prompt2.

4. The method for extracting key information from long conversations in a 5G new call environment according to claim 1, wherein: The large language model LLM includes an open source model and / or a closed source model.

Images