Data processing method, device, electronic device and computer program product

Abstract

The present invention discloses a data processing method, device, electronic device, and computer program product. The method comprises: obtaining subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating the voice stream data to be identified, and the subtitle stream data comprises: multiple text frames, and the multiple text frames are used to constitute the natural language text to be identified; using a preset back-end point model to identify the back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the position of the semantic end, and the preset back-end point model is a pre-trained semantic model used to identify the back-end point in the natural language text; using the back-end point as the position point for intercepting the text frame, intercepting multiple text frames in the subtitle stream data to obtain the data to be processed; and generating reply stream data for the data to be processed. The present invention solves the technical problem of slow response speed in voice dialogues.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a data processing method, device, electronic device and computer program product. Background Art

[0002] The emergence of large language models has revolutionized the traditional field of natural language processing (NLP). These models, such as GPT-3 and BERT, are trained on massive amounts of data, not only improving the accuracy of language understanding and generation but also significantly enhancing their ability to handle complex language tasks. They capture the deep semantics and contextual relationships of language, enabling machines to more fluently understand and respond to human speech.

[0003] Large language models make voice interaction systems more intelligent and natural, capable of handling multiple conversations, maintaining coherence, and even, to a certain extent, understanding and responding to user emotions. The advantage of large language models also lies in their strong generalization capabilities. They can perform well on a variety of language tasks without requiring extensive customized training for each task. This makes large language models promising for broad application in fields such as search engine optimization, content generation, and machine translation.

[0004] Traditional voice interaction is broadly divided into several modules: automatic speech recognition (ASR), natural language understanding (NLU), answer generation (NLG), and text-to-speech (TTS). The emergence of large language models has made NLU and NLG more intelligent in traditional voice interaction chains. However, the increased scale of large language models also brings challenges to voice interaction, such as performance loss, changes in interaction patterns, and increased inference costs. How to leverage the intelligent capabilities of large language models and overcome the engineering challenges they present is a pressing issue for the implementation of large language model voice interaction.

[0005] Currently, when using a large language model for voice conversations, the traditional approach is to convert speech into text and then feed the large text into the large language model for analysis to generate a response. However, this approach requires waiting for the speech input to complete before the large language model can analyze it, which increases the input latency. Furthermore, feeding a large text into the large language model requires a large amount of data to analyze, and the waiting time for generating a response is also long, resulting in slow response times for voice conversations.

[0006] Currently, no effective solution has been proposed to the problem of slow response time in the above-mentioned voice conversations. Summary of the Invention

[0007] Embodiments of the present invention provide a data processing method, apparatus, electronic device, and computer program product to at least solve the technical problem of slow response speed in voice conversations.

[0008] According to one aspect of an embodiment of the present invention, a data processing method is provided, comprising: obtaining subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating voice stream data to be identified, and the subtitle stream data comprises: a plurality of text frames, wherein the plurality of text frames are used to constitute a natural language text to be identified; using a preset rear endpoint model to identify a rear endpoint in the plurality of text frames of the subtitle stream data, wherein the rear endpoint is used to indicate a position where semantics ends, and the preset rear endpoint model is a pre-trained semantic model for identifying the rear endpoint in a natural language text; using the rear endpoint as a position point for intercepting text frames, intercepting a plurality of text frames in the subtitle stream data to obtain data to be processed; and generating reply stream data for the data to be processed.

[0009] Optionally, after obtaining the subtitle stream data to be identified, the method further includes: determining the number of blank frames of consecutive blank frames in the subtitle stream data, wherein the blank frames are text frames representing blank semantics; when the number of blank frames exceeds a preset number threshold, using the blank frames of the number of blank frames as position points for intercepting text frames, intercepting multiple text frames in the subtitle stream data to obtain the data to be processed.

[0010] Optionally, the method further includes: inputting the subtitle stream data into the preset back-end endpoint model when the number of blank frames does not exceed a preset number threshold.

[0011] Optionally, obtaining subtitle stream data to be recognized includes: obtaining voice stream data to be recognized, wherein the voice stream data includes: multiple voice frames, and a voice timestamp corresponding to each voice frame; translating the voice stream data to obtain translated text data, wherein the translated text data includes: multiple texts to be recognized, and the multiple texts to be recognized are used to constitute a natural language text to be recognized; determining a matching relationship between each text to be recognized and the voice frame, wherein the text to be recognized and the voice frame having the matching relationship express the same semantics; according to the voice timestamp of the voice frame, adding a subtitle timestamp to the text to be recognized corresponding to the voice frame to obtain the subtitle stream data, wherein each text frame data in the subtitle stream data is the text to be recognized with the subtitle timestamp added.

[0012] Optionally, generating reply stream data for the data to be processed includes: using a preset large language model to process the data to be processed to obtain reply text data for the data to be processed, wherein the preset large language model can understand and generate the natural language text, and the data to be processed and the reply text data are the natural language text; using the preset back-end point model to identify the back-end point in the reply text data; using the back-end point as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; translating the text data to be translated to obtain the reply stream data.

[0013] Optionally, the method further includes: when the reply text data includes multiple back-end points, taking the first back-end point as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; after intercepting the text data to be translated in the reply text data, updating the reply text data, wherein, in the updated reply text data, the ranking order of each back-end point is promoted.

[0014] Optionally, after translating the text data to be translated and obtaining the reply stream data, the method further includes: adding the reply stream data to a preset audio frame cache queue, wherein the preset audio frame cache queue includes: a plurality of the reply stream data arranged in a queue order; and playing each of the reply stream data in the preset audio frame cache queue in sequence according to the queue order.

[0015] According to another aspect of an embodiment of the present invention, a data processing device is further provided, including: an acquisition module for acquiring subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating the voice stream data to be identified, and the subtitle stream data includes: multiple text frames, and the multiple text frames are used to constitute the natural language text to be identified; an identification module for using a preset back-end point model to identify the back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the position of the semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in the natural language text; a interception module for intercepting multiple text frames in the subtitle stream data with the back-end point as the position point for intercepting the text frame to obtain data to be processed; and a generation module for generating response stream data for the data to be processed.

[0016] According to another aspect of an embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the data processing method through the computer program.

[0017] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, which implement the steps of the data processing method when executed by a processor.

[0018] In an embodiment of the present invention, subtitle stream data to be recognized is obtained, wherein the subtitle stream data is obtained by translating voice stream data to be recognized, and the subtitle stream data includes: multiple text frames, the multiple text frames being used to constitute natural language text to be recognized; a preset back-end point model is used to identify a back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the location of a semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in natural language text; the back-end point is used as a location point for intercepting the text frame, and multiple text frames are intercepted from the subtitle stream data to obtain data to be processed; response stream data for the data to be processed is generated, so that the voice stream data represented in voice form can be converted into subtitle stream data represented in text form, and then the back-end point indicating the semantic end in the subtitle stream data is identified, and based on the back-end point, the subtitle stream data is intercepted into data to be processed including less text content, thereby achieving the purpose of generating response stream data based on the data to be processed including less text content. Since the text content of the data to be processed required to be analyzed to generate the response stream data is less, the analysis speed is relatively fast, thereby achieving the technical effect of improving the generation speed of the response stream data, thereby solving the technical problem of slow response speed in voice dialogue. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of a general streaming voice interaction solution according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of a cyclic control process of a process control module according to an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of a back-end point model construction according to an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of a data processing device according to an embodiment of the present invention;

[0025] Figure 6It is a structural block diagram of a computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0029] TTS: The full name is Text To Speech, which means text to speech in Chinese.

[0030] ASR: The full name is Automatic Speech Recognition, which means automatic speech recognition in Chinese.

[0031] RTC: The full name is Real-time Communications, which means real-time communication in Chinese.

[0032] IVR: The full name is Interactive Voice Response, which means interactive voice question and answer in Chinese.

[0033] NLP: The full name is Natural Language Processing, which means natural language processing in Chinese.

[0034] VAD: The full name is Voice Activity Detection, which means voice activity detection in Chinese.

[0035] PCM: The full name is Pulse Code Modulation, which means pulse code modulation in Chinese.

[0036] LLM: The full name is Large Language Model, which means large language model in Chinese.

[0037] CTC: The full name is Connectionist Temporal Classification, which means continuous temporal classification in Chinese.

[0038] Large language models: Deep learning-based AI models that can understand and generate natural language text. The "large" in large language models generally refers to the model's size, including the number of parameters and the amount of training data, which together determine the model's capabilities and performance.

[0039] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0041] Step S102: obtaining subtitle stream data to be recognized, wherein the subtitle stream data is obtained by translating the voice stream data to be recognized, and the subtitle stream data includes: a plurality of text frames, and the plurality of text frames are used to constitute the natural language text to be recognized;

[0042] Step S104, using a preset end-point model to identify the end-point in multiple text frames of the subtitle stream data, wherein the end-point is used to indicate the location of the end of the semantics, and the preset end-point model is a pre-trained semantic model for identifying the end-point in natural language text;

[0043] Step S106, taking the subsequent endpoint as the position point for intercepting text frames, intercepting multiple text frames in the subtitle stream data to obtain data to be processed;

[0044] Step S108: Generate response stream data of the data to be processed.

[0045] In an embodiment of the present invention, subtitle stream data to be recognized is obtained, wherein the subtitle stream data is obtained by translating voice stream data to be recognized, and the subtitle stream data includes: multiple text frames, the multiple text frames being used to constitute natural language text to be recognized; a preset back-end point model is used to identify a back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the location of a semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in natural language text; the back-end point is used as a location point for intercepting the text frame, and multiple text frames are intercepted from the subtitle stream data to obtain data to be processed; response stream data for the data to be processed is generated, so that the voice stream data represented in voice form can be converted into subtitle stream data represented in text form, and then the back-end point indicating the semantic end in the subtitle stream data is identified, and based on the back-end point, the subtitle stream data is intercepted into data to be processed including less text content, thereby achieving the purpose of generating response stream data based on the data to be processed including less text content. Since the text content of the data to be processed required to be analyzed to generate the response stream data is less, the analysis speed is relatively fast, thereby achieving the technical effect of improving the generation speed of the response stream data, thereby solving the technical problem of slow response speed in voice dialogue.

[0046] In the above step S102, the voice stream data can be obtained by collecting user voice conversations. In the case of collecting user voice conversations, the collected voice file can be recorded by multiple voice frames, and each voice frame represents audio data per unit time.

[0047] Optionally, in the voice stream data, multiple voice frames are arranged sequentially in the form of a queue. When translating the voice stream data into subtitle stream data, the voice frames at the front end of the queue can be translated first, and then the voice frames at the back end of the queue can be translated in the queue order. When translating the voice frames, the user voice conversation can be collected synchronously, and then the newly collected voice frames can be placed at the end of the queue.

[0048] In the above step S102, the subtitle stream data is a text sequence obtained by translating the voice stream data. The subtitle stream data includes multiple text frames. The text frames in the subtitle stream data correspond to the voice frames in the voice stream data. Each text frame represents text data per unit time.

[0049] In the above step S104, the preset rear endpoint model is used to identify the rear endpoint indicating the end of semantics in the natural language text. The rear endpoint is the position of the text frame indicating the end of semantics.

[0050] In the above step S106, the data to be processed is a portion of the subtitle stream data intercepted from the beginning of the subtitle stream data as the starting point and the text frame where the endpoint is located as the ending point. The intercepted data to be processed includes: multiple intercepted text frames.

[0051] In the above step S108, the reply stream data may be reply files in a stream form, and each file may be a voice frame of the reply.

[0052] As an optional embodiment, after obtaining the subtitle stream data to be identified, the method further includes: determining the number of blank frames of consecutive blank frames in the subtitle stream data, wherein the blank frames are text frames representing blank semantics; when the number of blank frames exceeds a preset number threshold, using the blank frames having the same number of blank frames as the position points for intercepting the text frames, intercepting multiple text frames in the subtitle stream data to obtain data to be processed.

[0053] In the above embodiment of the present application, when there is a pause in the voice uttered by the user, the text frame corresponding to the pause period in the subtitle stream data is a blank frame representing blank semantics. The user may have appropriate pauses during the conversation, and the duration of the pause can be represented by the number of consecutive blank frames. Then, by counting the number of blank frames of the consecutive blank frames, the pause duration of the user's voice can be determined. If the pause duration is too long, it is more likely to indicate the end of semantics. Then, before determining the end point according to the semantics, the interception position of the data to be processed from the subtitle stream data can be determined by comparing whether the number of blank frames in the subtitle stream data exceeds the preset number threshold.

[0054] As an optional embodiment, the method further includes: inputting the subtitle stream data into a preset back-end point model when the number of blank frames does not exceed a preset number threshold.

[0055] In the above-mentioned embodiment of the present application, the priority of intercepting the data to be processed based on the number of blank frames and the preset number threshold is higher than the priority of intercepting the data to be processed based on the rear endpoint. Therefore, when the number of blank frames exceeds the preset number threshold, the data to be processed is intercepted based on the number of blank frames and the preset number threshold; when the number of blank frames does not exceed the preset number threshold, the data to be processed is intercepted based on the rear endpoint.

[0056] As an optional example, in the above embodiment of the present application, if the pause time reaches the longest confirmation time (that is, the number of blank frames exceeds the preset number threshold), even if the preset rear endpoint model determines that the input is not yet completed, the input is terminated; if the maximum pause time is not reached (that is, the number of blank frames does not exceed the preset number threshold), the preset rear endpoint model gives an output judgment.

[0057] As an optional embodiment, obtaining subtitle stream data to be recognized includes: obtaining voice stream data to be recognized, wherein the voice stream data includes: multiple voice frames and a voice timestamp corresponding to each voice frame; translating the voice stream data to obtain translated text data, wherein the translated text data includes: multiple texts to be recognized, and the multiple texts to be recognized are used to constitute a natural language text to be recognized; determining a matching relationship between each text to be recognized and the voice frame, wherein the text to be recognized and the voice frame with a matching relationship express the same semantics; according to the voice timestamp of the voice frame, adding a subtitle timestamp to the text to be recognized corresponding to the voice frame to obtain subtitle stream data, wherein each text frame data in the subtitle stream data is a text to be recognized with a subtitle timestamp added.

[0058] In the above-mentioned embodiment of the present application, the subtitle stream data is obtained by translating the voice stream data. The voice stream data before translation includes multiple voice frames, each voice frame is audio with a voice timestamp added, and the subtitle stream data after translation includes multiple text frames, each text frame is text with a subtitle timestamp added. The text frames in the subtitle stream data correspond to the voice frames in the voice stream data. Therefore, after translating the voice stream data to obtain translated text data for constituting natural language text, a corresponding subtitle timestamp can be added to the text to be recognized in the translated text data to obtain subtitle stream data.

[0059] As an optional embodiment, generating reply stream data for the data to be processed includes: using a preset large language model to process the data to be processed to obtain reply text data for the data to be processed, wherein the preset large language model can understand and generate natural language text, and the data to be processed and the reply text data are natural language text; using a preset back-end point model to identify the back-end point in the reply text data; using the back-end point as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; translating the text data to be translated to obtain reply stream data.

[0060] In the above-mentioned embodiment of the present application, reply stream data of the data to be processed can be generated by a preset large language model. The preset large language model can generate reply text data in text form based on the data to be processed in text form, and then identify the back-end point of the reply text data, and intercept the text data to be transferred in text form from the reply text data based on the back-end point, so as to realize the splitting of the reply text data, split the reply text data with a larger amount of text into the text data to be transferred with a smaller amount of text, and then translate the text data to be translated into reply stream data, which can reduce the amount of text required to be translated each time and improve the translation speed of the reply stream data.

[0061] As an optional embodiment, the method also includes: when the reply text data includes multiple back-end points, taking the first back-end point as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; after intercepting the text data to be translated in the reply text data, updating the reply text data, wherein, in the updated reply text data, the ranking order of each back-end point is promoted.

[0062] In the above embodiment of the present application, when the reply text data can be split into multiple text data to be translated, the text data to be translated at the first position of the reply text data can be preferentially intercepted based on the first back-end point, and then after intercepting the text data to be translated, such as when translating the text data to be translated into reply stream data, the reply text data can be updated, and the ranking order of the back-end point in the reply text data can be promoted so as to intercept the next text data to be translated, and then the text data to be translated intercepted for the first time can be preferentially translated into reply stream data and played, thereby speeding up the generation and playback speed of the reply stream data and reducing the waiting time of the conversation.

[0063] As an optional embodiment, after translating the text data to be translated and obtaining reply stream data, the method also includes: adding the reply stream data to a preset audio frame cache queue, wherein the preset audio frame cache queue includes: multiple reply stream data arranged in a queue order; and playing each reply stream data in the preset audio frame cache queue in sequence according to the queue order.

[0064] In the above-mentioned embodiment of the present application, multiple reply stream data can constitute streaming voice data in the form of a preset audio frame cache queue. Each time the text data to be translated is translated into reply stream data, the reply stream data can be added to the preset audio frame cache queue to constitute a new voice stream. The translated reply stream data can then be played in sequence according to the queue order of the reply stream data in the preset audio frame cache queue, thereby realizing a reply to the voice stream data.

[0065] Optionally, adding the reply stream data to the preset audio frame buffer queue includes: adding the reply stream data to the end of the preset audio frame buffer queue, or inserting the reply stream data into a specified position in the preset audio frame buffer queue, such as inserting it into the head of the preset audio frame buffer queue.

[0066] The present invention also provides an optional embodiment, which provides a general streaming voice interaction solution that can adapt to a large language model for interaction, so as to give full play to the experience brought by streaming interaction and the intelligent advantages of the large model, and provide a more accurate and natural interaction experience.

[0067] Due to the large amount of computation required for large models and the characteristics of generative returns one after another, as well as to save the computational cost of large models, more accurate input control is needed, such as general voice input and output control solutions and large-model-based conversational robot streaming request processing solutions.

[0068] It should be noted that the advantages of streaming large-model interaction mainly include two aspects: first, streaming interaction, which is more efficient than traditional interaction modes; second, the fusion of large models, which is more accurate and intelligent than traditional interaction modes.

[0069] Figure 2 is a schematic diagram of a general streaming voice interaction solution according to an embodiment of the present invention, such as Figure 2 As shown, it includes: a process control module, an input processing module and an output processing module, among which the process control module is mainly responsible for checking the input data, judging the input timing and content, issuing output instructions and performing state control; the input processing module is mainly responsible for the text / vector conversion, caching and simple management of the input voice signal, and is responsible for the pre-processing and content conversion of multimodal data; the output processing module is mainly responsible for the voice conversion, caching and simple management of the text / vector signal, and is responsible for the pre-processing and content conversion of multimodal data, as well as the execution of playback interruption.

[0070] As an alternative example, Figure 2 As shown, after the voice stream is input, each audio frame in the input voice stream can be put into the audio frame cache queue, and then the process control module controls to obtain the audio stream from the audio frame cache queue and input it into the ASR model. The ASR model returns the callback stream event of the audio stream to obtain the text cache queue, wherein the text cache queue can represent the segmented text, segmented frame id, round text and round frame id. For example, the text is "How is the weather tomorrow?", and its segmented text is "How is the weather tomorrow?", the segmented frame id corresponding to "tomorrow" is "2-4", the segmented frame id corresponding to "weather" is "5-8", the segmented frame id corresponding to "how?" is "5-8", the round text is "How is the weather tomorrow?", and the round frame id is "2-12".

[0071] As another alternative example, Figure 2 As shown, the process control module is used to perform text input checking, text output checking, and process control loops, and then input the input text (such as the text cache queue) into the large model dialogue system for analysis to obtain text output.

[0072] As another optional example, Figure 2As shown, for the text "What's the weather like tomorrow?", the output text is "The weather tomorrow will be sunny", and a text cache queue with the segmented text "The weather tomorrow will be sunny" is obtained. The text cache queue is then input into the audio frame cache queue for sequential playback, and finally a voice stream is output.

[0073] Alternatively, as Figure 2 As shown, in the process of inputting the text buffer queue into the audio frame buffer queue, the process control module can decide whether to interrupt the voice stream being broadcast.

[0074] Figure 3 FIG. 1 is a schematic diagram of a cyclic control process of a process control module according to an embodiment of the present invention. Figure 3 As shown, the specific steps include:

[0075] Step S31: wake up cyclically according to a certain step length, such as 100ms.

[0076] Step S32: Check the cache of the input module to see if there is any new input content. In theory, the input content is incremental. If it is inconsistent with the content of the previous step, it is considered to be updated.

[0077] Step S33: The decision module determines whether the current input needs to be sent to the large model dialogue system. The basis for the input is whether the user has finished speaking. If the dialogue system can accept word-by-word input, the module directly sends the content.

[0078] Step S34, checking whether the broadcast needs to be interrupted or paused. Interrupting will directly clear the broadcast content, and pausing the broadcast only stops sending voice packets or sending silent packets.

[0079] Step S35: Check whether there is an audio package that has not been broadcast, and push it if there is one.

[0080] The most important part of this solution lies in the coordination between process control and several other modules, mainly involving the judgment of input timing and the audio processing of large model streaming output.

[0081] As an optional example, the input timing may be a timing of sending to the large model dialogue system.

[0082] It's important to note that the biggest challenge with audio streaming input is determining input content and output timing. Traditionally, this is achieved through ASR segmentation. Each segmented sentence counts as one input, corresponding to one output. However, in real-world interactions, there can be issues like pauses and interruptions. If segmentation and input are performed during a pause, the user might submit the conversation before they've finished speaking, or multiple inputs might correspond to one output, disrupting the interaction flow. There are two approaches to addressing this issue. The first is to use a back-end module to determine the input. This module can reference audio data and ASR text information to determine whether the voice input is complete. The other approach is to bypass segmentation and simply pass all information to the dialogue system, which then determines when to output. For currently common large-scale model interfaces, the input is a fixed text segment, and the output is multiple text segments, making the second approach unsuitable.

[0083] As an optional embodiment, the input of the large model dialogue system is actually judged by an extensible back-end point module, which provides a default configuration version. The judgment of the back-end point should be based not only on audio, but also on semantics, and should take into account real-time performance. The default solution regards the ASR translated text as a subtitle stream, and derives the longest input pause time through the difference between the current audio frame time and the corresponding subtitle time. A back-end point model is trained by aligning audio and text data. If the pause reaches the longest confirmation time, even if the back-end point determines that the input is not complete, the input is terminated. If the maximum pause time is not reached, the back-end point module will give an output judgment.

[0084] Figure 4 is a schematic diagram of a back-end point model construction according to an embodiment of the present invention, such as Figure 4 As shown, the default back-end module needs to be built in conjunction with an ASR model, effectively acting as a plug-in for a specific model. First, the output must be restored to CTC mode to align with the speech data. Next, a corpus must be constructed, using actual online transcribed audio data. Time-aligning each word is performed and the sentence is split into multiple sentences according to the ASR results. A classification task is constructed by marking the semantic completeness of the segmented content. When the ASR segmentation is performed, the back-end model is called once to determine whether the input is semantically complete. The back-end's judgment is also influenced by the current scenario's intent. The same sentence may have a certain input in some scenarios but not in others.

[0085] As an optional implementation, the progressive generation of large language models can make traditional TTS half-duplex mode, which waits for complete output before converting to speech, particularly slow in first-packet response. Without changing the TTS service interface, adopting a short-sentence, multi-output approach can address this audio latency issue, achieving speeds roughly comparable to existing human-machine systems.

[0086] As an example, the above solution was implemented in a customer center scenario, using WebSocket as the input and output transport protocol for voice streams. A back-end endpoint model was built based on self-developed ASR and TTS. The interaction control module used a Linux server with four cores and 16GB of storage, deployed on two machines, and six ASR and TTS service machines, supporting 100 concurrent calls and an average daily active user base of 50,000.

[0087] The above-mentioned embodiments of the present application solve the input determination problem and output streaming broadcast problem of streaming input in the streaming voice interaction scenario combined with a large model, and have achieved certain results in practice.

[0088] According to an embodiment of the present invention, a data processing device embodiment is also provided. It should be noted that the data processing device can be used to execute the data processing method in the embodiment of the present invention, and the data processing method in the embodiment of the present invention can be executed in the data processing device.

[0089] Figure 5 is a schematic diagram of a data processing device according to an embodiment of the present invention. Figure 5 As shown, the device may include: an acquisition module 52, used to acquire subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating the voice stream data to be identified, and the subtitle stream data includes: multiple text frames, and the multiple text frames are used to constitute the natural language text to be identified; an identification module 54, used to use a preset back-end point model to identify the back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the position of the semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in the natural language text; an interception module 56, used to intercept multiple text frames in the subtitle stream data with the back-end point as the position point for intercepting the text frame to obtain data to be processed; a generation module 58, used to generate response stream data for the data to be processed.

[0090] It should be noted that the acquisition module 52 in this embodiment can be used to execute step S102 in the embodiment of the present application, the identification module 54 in this embodiment can be used to execute step S104 in the embodiment of the present application, the interception module 56 in this embodiment can be used to execute step S106 in the embodiment of the present application, and the generation module 58 in this embodiment can be used to execute step S108 in the embodiment of the present application. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments.

[0091] In an embodiment of the present invention, subtitle stream data to be recognized is obtained, wherein the subtitle stream data is obtained by translating voice stream data to be recognized, and the subtitle stream data includes: multiple text frames, the multiple text frames being used to constitute natural language text to be recognized; a preset back-end point model is used to identify a back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the location of a semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in natural language text; the back-end point is used as a location point for intercepting the text frame, and multiple text frames are intercepted from the subtitle stream data to obtain data to be processed; response stream data for the data to be processed is generated, so that the voice stream data represented in voice form can be converted into subtitle stream data represented in text form, and then the back-end point indicating the semantic end in the subtitle stream data is identified, and based on the back-end point, the subtitle stream data is intercepted into data to be processed including less text content, thereby achieving the purpose of generating response stream data based on the data to be processed including less text content. Since the text content of the data to be processed required to be analyzed to generate the response stream data is less, the analysis speed is relatively fast, thereby achieving the technical effect of improving the generation speed of the response stream data, thereby solving the technical problem of slow response speed in voice dialogue.

[0092] As an optional embodiment, the device also includes: a determination submodule, which is used to determine the number of blank frames in the subtitle stream data after obtaining the subtitle stream data to be identified, wherein the blank frame is a text frame representing blank semantics; and a capture submodule, which is used to capture multiple text frames in the subtitle stream data with the blank frames of the number of blank frames as the position points for capturing the text frames when the number of blank frames exceeds a preset threshold, so as to obtain data to be processed.

[0093] As an optional embodiment, the apparatus further includes: an input submodule, configured to input the subtitle stream data into a preset back-end point model when the number of blank frames does not exceed a preset number threshold.

[0094] As an optional embodiment, the acquisition module includes: an acquisition unit, used to acquire voice stream data to be recognized, wherein the voice stream data includes: multiple voice frames, and a voice timestamp corresponding to each voice frame; a first translation unit, used to translate the voice stream data to obtain translated text data, wherein the translated text data includes: multiple texts to be recognized, and the multiple texts to be recognized are used to constitute the natural language text to be recognized; a determination unit, used to determine the matching relationship between each text to be recognized and the voice frame, wherein the text to be recognized and the voice frame with a matching relationship express the same semantics; an adding unit, used to add a subtitle timestamp to the text to be recognized corresponding to the voice frame according to the voice timestamp of the voice frame, to obtain subtitle stream data, wherein each text frame data in the subtitle stream data is a text to be recognized with a subtitle timestamp added.

[0095] As an optional embodiment, the generation module includes: a processing unit, used to use a preset large language model to process the data to be processed to obtain reply text data of the data to be processed, wherein the preset large language model can understand and generate natural language text, and the data to be processed and the reply text data are natural language text; an identification unit, used to use a preset rear endpoint model to identify the rear endpoint in the reply text data; a second interception unit, used to intercept the text data to be translated in the reply text data with the rear endpoint as the position point for intercepting the reply text data; a second translation unit, used to translate the text data to be translated to obtain reply stream data.

[0096] As an optional embodiment, the device also includes: an interception subunit, which is used to intercept the text data to be translated in the reply text data with the first back endpoint as the position point for intercepting the reply text data when the reply text data includes multiple back endpoints; an updating subunit, which is used to update the reply text data after intercepting the text data to be translated in the reply text data, wherein the ranking order of each back endpoint in the updated reply text data is promoted.

[0097] As an optional embodiment, the device also includes: a queue subunit, which is used to add the reply stream data to a preset audio frame cache queue after translating the text data to be translated and obtaining the reply stream data, wherein the preset audio frame cache queue includes: multiple reply stream data arranged in a queue order; a playing subunit, which is used to play each reply stream data in the preset audio frame cache queue in sequence according to the queue order.

[0098] An embodiment of the present invention may provide an electronic device, which may be a computer terminal, and the computer terminal may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0099] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0100] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the data processing method: obtaining subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating the voice stream data to be identified, and the subtitle stream data includes: multiple text frames, and the multiple text frames are used to constitute the natural language text to be identified; using a preset back-end point model to identify the back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the position of the semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in the natural language text; using the back-end point as the position point for intercepting the text frame, intercepting multiple text frames in the subtitle stream data to obtain data to be processed; generating reply stream data for the data to be processed.

[0101] Figure 6 is a structural block diagram of a computer terminal according to an embodiment of the present invention, such as Figure 6 As shown, the computer terminal 60 may include: one or more (only one is shown in the figure) processors 62 and a memory 64.

[0102] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiments of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the above-mentioned data processing method. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal 60 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0103] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtaining subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating the voice stream data to be identified, and the subtitle stream data includes: multiple text frames, and the multiple text frames are used to constitute the natural language text to be identified; using a preset back-end point model to identify the back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the position of the semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in the natural language text; using the back-end point as the position point for intercepting the text frame, intercepting multiple text frames in the subtitle stream data to obtain data to be processed; generating reply stream data for the data to be processed.

[0104] Optionally, the processor may also execute the program code of the following steps: obtaining subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating the voice stream data to be identified, and the subtitle stream data includes: multiple text frames, and the multiple text frames are used to constitute the natural language text to be identified; using a preset back-end point model to identify the back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the position of the semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in the natural language text; using the back-end point as the position point for intercepting the text frame, intercepting multiple text frames in the subtitle stream data to obtain data to be processed; generating reply stream data for the data to be processed.

[0105] Optionally, the processor may also execute the program code of the following steps: determining the number of consecutive blank frames in the subtitle stream data, wherein the blank frames are text frames representing blank semantics; when the number of blank frames exceeds a preset threshold, using the blank frames having the same number of blank frames as the position points for intercepting the text frames, intercepting multiple text frames in the subtitle stream data to obtain data to be processed.

[0106] Optionally, the processor may further execute program code of the following steps: when the number of blank frames does not exceed a preset number threshold, inputting the subtitle stream data into a preset back-end point model.

[0107] Optionally, the processor may also execute the program code of the following steps: obtaining voice stream data to be recognized, wherein the voice stream data includes: multiple voice frames, and a voice timestamp corresponding to each voice frame; translating the voice stream data to obtain translated text data, wherein the translated text data includes: multiple texts to be recognized, and the multiple texts to be recognized are used to constitute the natural language text to be recognized; determining the matching relationship between each text to be recognized and the voice frame, wherein the text to be recognized and the voice frame with a matching relationship express the same semantics; adding a subtitle timestamp to the text to be recognized corresponding to the voice frame according to the voice timestamp of the voice frame to obtain subtitle stream data, wherein each text frame data in the subtitle stream data is a text to be recognized with a subtitle timestamp added.

[0108] Optionally, the processor may also execute the program code of the following steps: using a preset large language model to process the data to be processed to obtain reply text data of the data to be processed, wherein the preset large language model can understand and generate natural language text, and the data to be processed and the reply text data are natural language text; using a preset back-end point model to identify the back-end point in the reply text data; using the back-end point as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; translating the text data to be translated to obtain reply stream data.

[0109] Optionally, the processor may also execute the program code of the following steps: when the reply text data includes multiple back-end points, the first back-end point is used as the position point for intercepting the reply text data, and the text data to be translated is intercepted in the reply text data; after intercepting the text data to be translated in the reply text data, the reply text data is updated, wherein, in the updated reply text data, the ranking order of each back-end point is promoted.

[0110] Optionally, the above-mentioned processor can also execute the program code of the following steps: adding the reply stream data to the preset audio frame cache queue, wherein the preset audio frame cache queue includes: multiple reply stream data arranged in queue order; playing each reply stream data in the preset audio frame cache queue in sequence according to the queue order.

[0111] An embodiment of the present invention provides a data processing solution. In an embodiment of the present invention, subtitle stream data to be recognized is obtained, wherein the subtitle stream data is obtained by translating voice stream data to be recognized, and the subtitle stream data includes: multiple text frames, the multiple text frames being used to constitute natural language text to be recognized; a preset back-end point model is used to identify a back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the location of a semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in natural language text; the back-end point is used as a location point for intercepting the text frame, and multiple text frames are intercepted from the subtitle stream data to obtain data to be processed; response stream data for the data to be processed is generated, so that the voice stream data represented in voice form can be converted into subtitle stream data represented in text form, and then the back-end point indicating the semantic end in the subtitle stream data is identified, and based on the back-end point, the subtitle stream data is intercepted into data to be processed including less text content, thereby achieving the purpose of generating response stream data based on the data to be processed including less text content. Since the text content of the data to be processed required to be analyzed to generate the response stream data is less, the analysis speed is relatively fast, thereby achieving the technical effect of improving the generation speed of the response stream data, thereby solving the technical problem of slow response speed in voice dialogue.

[0112] It can be understood by those skilled in the art that Figure 6 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 6 It does not limit the structure of the above electronic device. For example, the computer terminal 60 may also include Figure 6 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 6 Different configurations shown.

[0113] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a computer program. The computer program can be stored in a non-volatile medium. The non-volatile storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0114] The embodiment of the present invention further provides a non-volatile storage medium. Optionally, in this embodiment, the non-volatile storage medium can be used to store the program code executed by the data processing method provided in the above embodiment.

[0115] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0116] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: obtaining subtitle stream data to be identified, wherein the subtitle stream data is obtained by translating the voice stream data to be identified, and the subtitle stream data includes: multiple text frames, and the multiple text frames are used to constitute the natural language text to be identified; using a preset back-end point model to identify the back-end point in the multiple text frames of the subtitle stream data, wherein the back-end point is used to indicate the position of the semantic end, and the preset back-end point model is a pre-trained semantic model for identifying the back-end point in the natural language text; using the back-end point as the position point for intercepting the text frame, intercepting multiple text frames in the subtitle stream data to obtain data to be processed; generating reply stream data for the data to be processed.

[0117] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: in the subtitle stream data, determining the number of blank frames of consecutive blank frames, wherein the blank frames are text frames representing blank semantics; when the number of blank frames exceeds a preset number threshold, using the blank frames having the same number of blank frames as position points for intercepting the text frames, intercepting multiple text frames in the subtitle stream data to obtain data to be processed.

[0118] Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for executing the following steps: if the number of blank frames does not exceed a preset number threshold, inputting the subtitle stream data into a preset rear endpoint model.

[0119] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: obtaining voice stream data to be recognized, wherein the voice stream data includes: multiple voice frames, and a voice timestamp corresponding to each voice frame; translating the voice stream data to obtain translated text data, wherein the translated text data includes: multiple texts to be recognized, and the multiple texts to be recognized are used to constitute a natural language text to be recognized; determining a matching relationship between each text to be recognized and the voice frame, wherein the text to be recognized and the voice frame with a matching relationship express the same semantics; according to the voice timestamp of the voice frame, adding a subtitle timestamp to the text to be recognized corresponding to the voice frame to obtain subtitle stream data, wherein each text frame data in the subtitle stream data is a text to be recognized with a subtitle timestamp added.

[0120] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: using a preset large language model to process the data to be processed to obtain reply text data of the data to be processed, wherein the preset large language model can understand and generate natural language text, and the data to be processed and the reply text data are natural language text; using a preset back-end point model to identify the back-end point in the reply text data; using the back-end point as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; translating the text data to be translated to obtain reply stream data.

[0121] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: when the reply text data includes multiple back-end points, taking the first back-end point as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; after intercepting the text data to be translated in the reply text data, updating the reply text data, wherein, in the updated reply text data, the ranking order of each back-end point is promoted.

[0122] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for performing the following steps: adding the reply stream data to a preset audio frame cache queue, wherein the preset audio frame cache queue includes: multiple reply stream data arranged in a queue order; and playing each reply stream data in the preset audio frame cache queue in sequence according to the queue order.

[0123] An embodiment of the present invention further provides a computer program product, including a computer program.

[0124] Optionally, in this embodiment, when the computer program is executed by the processor, the steps of the data processing method provided in the above embodiment are implemented.

[0125] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0126] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0128] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0129] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a non-volatile storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0131] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: include: Acquire subtitle stream data to be recognized, wherein the subtitle stream data is obtained by translating the voice stream data to be recognized, and the subtitle stream data includes: a plurality of text frames, wherein the plurality of text frames are used to constitute the natural language text to be recognized; Identifying a rear endpoint in the plurality of text frames of the subtitle stream data using a preset rear endpoint model, wherein the rear endpoint is used to indicate a semantic end position, and the preset rear endpoint model is a pre-trained semantic model for identifying the rear endpoint in a natural language text; Taking the rear endpoint as a position point for intercepting text frames, intercepting multiple text frames in the subtitle stream data to obtain data to be processed; generating response stream data of the data to be processed; The step of generating the response stream data of the data to be processed includes: Processing the data to be processed using a preset large language model to obtain reply text data for the data to be processed, wherein the preset large language model is capable of understanding and generating the natural language text, and the data to be processed and the reply text data are the natural language text; identifying the back-end point in the reply text data using the preset back-end point model; Taking the rear endpoint as the position point for intercepting the reply text data, intercepting the text data to be translated in the reply text data; The text data to be translated is translated to obtain the response stream data.

2. The method according to claim 1, characterized in that After obtaining the subtitle stream data to be recognized, the method further includes: In the subtitle stream data, determining the number of blank frames of consecutive blank frames, wherein the blank frames are text frames representing blank semantics; When the number of blank frames exceeds a preset threshold, the blank frames of the number of blank frames are used as positions for intercepting text frames, and a plurality of the text frames are intercepted from the subtitle stream data to obtain the data to be processed.

3. The method according to claim 2, characterized in that The method also includes: When the number of blank frames does not exceed a preset number threshold, the subtitle stream data is input into the preset back-end endpoint model.

4. The method according to claim 1, wherein Obtaining the subtitle stream data to be recognized includes: Acquire voice stream data to be recognized, wherein the voice stream data includes: a plurality of voice frames and a voice timestamp corresponding to each of the voice frames; Translating the voice stream data to obtain translated text data, wherein the translated text data includes: a plurality of texts to be recognized, the plurality of texts to be recognized are used to constitute a natural language text to be recognized; Determining a matching relationship between each of the to-be-recognized texts and the speech frame, wherein the to-be-recognized texts and the speech frame having the matching relationship express the same semantics; According to the speech timestamp of the speech frame, a subtitle timestamp is added to the text to be recognized corresponding to the speech frame to obtain the subtitle stream data, wherein each text frame data in the subtitle stream data is the text to be recognized with the subtitle timestamp added.

5. The method according to claim 1, characterized in that The method further comprises: In the case where the reply text data includes a plurality of rear endpoints, taking the first rear endpoint as the position point for intercepting the reply text data, intercepting the text data to be translated from the reply text data; After the text data to be translated is intercepted from the reply text data, the reply text data is updated, wherein the ranking order of each of the back-end nodes is promoted in the updated reply text data.

6. The method according to claim 1, characterized in that After translating the text data to be translated to obtain the response stream data, the method further includes: Adding the reply stream data to a preset audio frame buffer queue, wherein the preset audio frame buffer queue includes: a plurality of the reply stream data arranged in a queue order; Each of the reply stream data in the preset audio frame buffer queue is played in sequence according to the queue order.

7. A data processing device, characterized in that: include: An acquisition module is configured to acquire subtitle stream data to be recognized, wherein the subtitle stream data is obtained by translating the voice stream data to be recognized, and the subtitle stream data includes: a plurality of text frames, wherein the plurality of text frames are used to constitute the natural language text to be recognized; an identification module, configured to identify a rear endpoint in the plurality of text frames of the subtitle stream data using a preset rear endpoint model, wherein the rear endpoint is used to indicate a semantic end position, and the preset rear endpoint model is a pre-trained semantic model for identifying the rear endpoint in natural language text; An interception module, configured to intercept a plurality of text frames from the subtitle stream data with the rear endpoint as a position point for intercepting text frames, to obtain data to be processed; A generating module, configured to generate response stream data of the data to be processed; Wherein, the generation module includes: a processing unit, configured to process the data to be processed using a preset large language model to obtain reply text data for the data to be processed, wherein the preset large language model is capable of understanding and generating the natural language text, and the data to be processed and the reply text data are the natural language text; an identification unit, configured to identify the rear endpoint in the reply text data using the preset rear endpoint model; a second interception unit, configured to intercept the text data to be translated from the reply text data, using the rear endpoint as a position point for intercepting the reply text data; The second translation unit is used to translate the text data to be translated to obtain the response stream data.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the data processing method according to any one of claims 1 to 6 through the computer program.

9. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

Images