Data transmission method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a data transmission method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: a sending end obtains a voice data stream, and converts the voice data stream into characters word by word to obtain a first text data stream; under different data transmission modes, coding the first text data stream by adopting different coding modes to obtain a coding result; generating a to-be-sent data packet according to the character serial number of the starting character of the first text data stream, the first text data stream, the character serial number of the character missing at the receiving end and the character serial number of the character missing at the sending end; the sending end packages the coding result into a data packet to be sent to obtain the data packet, and sends the data packet to a receiving end; and the receiving end decodes the coded content in the data packet to obtain a text data stream. The technical problem that the voice call quality under weak signals is poor due to the fact that the probability of voice interruption in the voice call service of the weak signals is large is solved.

Description

Technical Field

[0001] The present application relates to the field of communications, and more specifically, to a data transmission method and device, a non-volatile storage medium, and an electronic device. Background Art

[0002] Voice calls are a traditional service for operators, and the bandwidth required for voice calls depends on the encoding method and network configuration. In existing networks, a standard uncompressed VOIP call typically requires 80Kbps of bandwidth, and even after compression, it still requires over 20Kbps of transmission bandwidth. Even though cellular voice calls utilize more efficient compression algorithms, they still typically require 10Kbps-30Kbps of transmission bandwidth.

[0003] In the field of voice AI coding, the AVS3P10 voice coding industry standard uses deep learning-based AI coding by invoking end-side computing power. Its 6kbps bitrate is close to the 20kbps quality of the international mainstream OPUS standard, achieving standard-definition voice calls at a lower bandwidth and reducing the voice bitrate by 60%.

[0004] However, for voice calls at lower bandwidths, data packet transmission is significantly impacted by unstable wireless links. Increased packet loss, latency, and bit error rates directly disrupt voice call continuity, leading to frequent interruptions and severely impacting call quality and user experience.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] The present application provides a data transmission method and apparatus, a non-volatile storage medium, and an electronic device to at least solve the technical problem of poor voice call quality under weak signals due to a high probability of voice interruption in voice call services with weak signals.

[0007] According to one aspect of the present application, a data transmission method is provided, including: a sending end obtains a local voice data stream, uses a pre-trained voice recognition model to convert the voice data stream into characters word by word, and uses a pre-trained text error correction model to perform error correction processing on the characters to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; the sending end determines different transmission modes adopted by the sending end and the receiving end for data transmission based on a target fault tolerance rate of a packet loss rate when the sending end and the receiving end perform data transmission, wherein the target fault tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; under different transmission modes, the first text data stream is encoded using different encoding methods to obtain an encoding result; the sending end receives the character sequence number of the missing character sent by the receiving end, the second text data stream, and each character in the second text data stream. a second character sequence number corresponding to the second character; determining the character sequence number of the missing character at the sending end based on the second text data stream and the second character sequence number; generating a data packet to be sent based on the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the missing character at the receiving end, and the character sequence number of the missing character at the sending end; the sending end encapsulates the encoding result into the data packet to be sent, obtains the data packet, and sends the data packet to the receiving end; the receiving end decodes the encoded content in the data packet to obtain a text data stream; uses a pre-trained text error correction model to correct the text data stream, and uses a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, converts the processed text data stream into a voice data stream, and outputs the voice data stream through the voice output module.

[0008] Optionally, based on the target fault tolerance rate of the packet loss rate when the sending end and the receiving end perform data transmission, different transmission modes adopted by the sending end and the receiving end for data transmission are determined, including: obtaining a preconfigured initial packet loss rate fault tolerance rate; detecting the packet loss rate when the sending end and the receiving end perform data transmission in real time; if the packet loss rate is in an increasing state within a preset period, increasing the initial packet loss rate fault tolerance rate to obtain the target fault tolerance rate; if the packet loss rate is in a decreasing state within the preset period, reducing the initial packet loss rate fault tolerance rate to obtain the target fault tolerance rate; obtaining the target fault tolerance rate at the target moment, and when the target fault tolerance rate is less than a first preset threshold, determining to adopt the first transmission mode for data transmission with the receiving end; when the target fault tolerance rate is greater than the first preset threshold, determining to adopt the second transmission mode for data transmission with the receiving end.

[0009] Optionally, in different transmission modes, different encoding methods are used to encode the first text data stream to obtain an encoding result, including: creating a text queue, wherein the text queue includes: a first queue, a second queue and a third queue, the first queue is used to store character information in the first text data stream, the second queue is used to store character information of missing characters on the receiving end, and the third queue is used to store character sequence numbers of missing characters on the sending end; both the first queue and the second queue include: character sequence number, character code, latest sending time of the character, G character code slices corresponding to the character code, latest sending time of the character code slice, and character code slice sequence number to be sent, wherein G is a positive integer greater than 1; obtaining an unencoded target element combination in the first queue, wherein the target element combination includes: target character sequence number, target character code and latest sending time of the target character; when using the first transmission mode to transmit data to the receiving end, using a forward error correction coding algorithm to process the target character code in the target element combination to obtain a first character code slice set, wherein the first character code slice set includes G character code slices, each character The character code slice is a code segment of the target character code; the first character code slice set is added to the target element combination; and, according to the target time, the latest sending time of the target character, G and K, the latest sending time of each character code slice in the first character code slice set is determined to obtain the first character code slice latest sending time set, wherein K is a positive integer not greater than G; the first character code slice latest sending time set is added to the target element combination to obtain a first coding result; when the second transmission mode is adopted to transmit data with the receiving end, the target character code in the target element combination is copied to obtain a second character code slice set, wherein the second character code slice set includes G target character codes; the second character code slice set is added to the target element combination; and, according to the target time, the latest sending time of the target character, G and K, the latest sending time of each character code slice in the second character code slice set is determined to obtain the second character code slice latest sending time set, and the second character code slice latest sending time set is added to the target element combination to obtain a second coding result.

[0010] Optionally, before obtaining the unencoded target element combination in the first queue, the method also includes: obtaining a target triple corresponding to the target character in the text data stream at time T, wherein the target triple includes: a target character sequence number, a target character code, and a target time; T is the target time, and T is a positive integer greater than 1; obtaining the latest sending time of the character code slice at time T-1 in the first queue, and determining the latest sending time of the target character based on the latest sending time of the character code slice at time T-1, a preconfigured time sliding window, and the target time; adding the target character sequence number, the target character code, the latest sending time of the target character, the target character code slice, the latest sending time of the target character code slice, and the target sequence number to the first queue, wherein the target character code slice and the latest sending time of the target character code slice are empty arrays, and the target sequence number is 0.

[0011] Optionally, the transmitting end encapsulates the encoding result into a data packet to be sent, obtains a data packet, and sends the data packet to the receiving end, including: step S1, determining the transmission mode adopted by the transmitting end and the receiving end for data transmission as the first field of the data to be sent; determining the number of characters of the missing characters of the receiving end and the number of retransmitted characters as the second field of the data to be sent; determining the number of characters of the missing characters of the transmitting end as the third field of the data to be sent; determining the character sequence number of the starting character of the first text data stream as the fourth field of the data to be sent; determining the first text data stream as the fifth field of the data to be sent; determining the number of characters of the missing characters of the receiving end as the fifth field of the data to be sent The character sequence number is determined as the sixth field of the data to be sent; the character sequence number of the missing character at the sending end is determined as the seventh field of the data to be sent; step S2, determine the maximum length corresponding to the data packet to be sent; step S3, from the first queue, starting from the first position, sequentially take out the first character code slice whose latest sending time is within the first interval, wherein the first position is the position of the first character of the multiple characters that are not completely sent at the target time in the first queue, the first interval is from the target time to the first time, and the first time is the sum of the target time and the preconfigured packet sending interval; add the first character code slice Add to the fifth field, and determine the first serial number of the character corresponding to the first character code slice and the second serial number of the character corresponding to the last character code slice in the fifth field; step S4, from the second queue, starting from the first character code to be sent, sequentially take out the second character code slice whose latest sending time is within the second interval, wherein the second interval is from the target moment to the second moment, and the second moment is the next adjacent moment of the target moment; add the second character code slice to the sixth field; step S5, take out all character serial numbers from the third queue, add all character serial numbers to the seventh field, and in the third queue Delete all character serial numbers; Step S6, take out the third character code slice after the second serial number from the first queue, and add the third character code slice to the fifth field; Step S7, take out the fourth character code slice before the first serial number from the first queue, and add the fourth character code slice to the fifth field; Step S8, after executing all steps from Step S3 to Step S7 or the length of the data packet to be sent reaches the maximum length, obtain the data packet and add the data packet to the queue to be sent; Step S9, based on the preconfigured packet sending interval, obtain the data packet from the queue to be sent, and send the data packet to the receiving end.

[0012] Optionally, determining the maximum length corresponding to the data packet to be sent includes: obtaining a preconfigured initial bit rate; detecting in real time the packet loss rate during data transmission between the sending end and the receiving end, and if the packet loss rate is in an increasing state within a preset period, reducing the initial bit rate to obtain a target bit rate; detecting in real time the amount of text in the data cache area on which the sending end and the receiving end rely for data transmission, and if the amount of text is greater than a preset second preset threshold, increasing the initial bit rate to obtain a target bit rate; determining the maximum length corresponding to the data packet to be sent based on the target bit rate, the preconfigured packet sending interval, and the protocol packet header length in the preconfigured rules.

[0013] Optionally, the receiving end decodes the encoded content in the data packet to obtain a text data stream, including: obtaining a text queue; taking the character sequence number of the starting character in the data packet as the starting point, and sequentially parsing the character code slices in the retransmitted characters or the character code slices in the continuous characters in the data packet; when the first field indicates that the transmission mode adopted by the sending end and the receiving end for data transmission is the first transmission mode, if the number of character code slices obtained by parsing is G / 2+1, using a forward error correction coding algorithm to decode the character code from the (G / 2+1) character code slices to obtain a first decoding result, wherein the first decoding result includes the character code and the character sequence number corresponding to the character code; when the first field indicates that the transmission mode adopted by the sending end and the receiving end for data transmission is the second transmission mode, decoding the character code from any character code slice to obtain a second decoding result, wherein the second decoding result includes the character code and the character sequence number corresponding to the character code.

[0014] Optionally, the method also includes: parsing the missing characters in the data packet, determining the triplet corresponding to the missing characters in the first queue, wherein the triplet includes: character sequence number, character encoding, and latest character sending time; adding the triplet corresponding to the missing characters to the second queue; and deleting the missing characters in the third queue when the decoded character encoding is the character encoding corresponding to the missing characters in the third queue.

[0015] According to another aspect of the present application, a data transmission device is also provided, including: an acquisition module, used to control the sending end to obtain a local voice data stream, use a pre-trained voice recognition model to convert the voice data stream into characters word by word, and use a pre-trained text error correction model to perform error correction on the characters to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; an encoding module, used to control the sending end to determine different transmission modes adopted by the sending end and the receiving end for data transmission according to a target fault tolerance rate of a packet loss rate when the sending end and the receiving end perform data transmission, wherein the target fault tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; under different transmission modes, the first text data stream is encoded using different encoding methods to obtain an encoding result; a generation module, used to control the sending end to receive the character sequence number of the missing character of the receiving end, the second text data stream, and the second text data sent by the receiving end a second character sequence number corresponding to each second character in the stream; determining the character sequence number of the missing character at the sending end according to the second text data stream and the second character sequence number; generating a data packet to be sent according to the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the missing character at the receiving end, and the character sequence number of the missing character at the sending end; an encapsulation module for controlling the sending end to encapsulate the encoding result into the data packet to be sent, obtain the data packet, and send the data packet to the receiving end; a decoding module for controlling the receiving end to decode the encoded content in the data packet to obtain a text data stream; using a pre-trained text error correction model to correct the text data stream, and using a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, converting the processed text data stream into a voice data stream, and outputting the voice data stream through the voice output module.

[0016] According to another aspect of the present application, a non-volatile storage medium is provided, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above data transmission method.

[0017] According to another aspect of the present application, an electronic device is provided, including: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the above data transmission method is executed when the program is run.

[0018] According to yet another aspect of the present application, a computer program is provided, wherein when the computer program is executed by a processor, the above data transmission method is implemented.

[0019] According to another aspect of the present application, a computer program product is provided, which includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above data transmission method is implemented.

[0020] In the present application, a sending end is used to obtain a local voice data stream, a pre-trained voice recognition model is used to convert the voice data stream word by word into characters, and a pre-trained text error correction model is used to perform error correction processing on the characters to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; the sending end determines the different transmission modes adopted by the sending end and the receiving end for data transmission according to the target fault tolerance rate of the packet loss rate when the sending end and the receiving end perform data transmission, wherein the target fault tolerance rate is a parameter periodically adjusted according to the change of the packet loss rate; under different transmission modes, different encoding methods are used to encode the first text data stream to obtain an encoding result; the sending end receives the character sequence number of the missing character of the receiving end sent by the receiving end, the second text data stream and the second character sequence number corresponding to each second character in the second text data stream; according to the second text data stream and the second character sequence number, the character sequence number of the missing character of the sending end is determined; according to the character sequence number of the starting character of the first text data stream, the character sequence number of the first text data stream, the character sequence number of the first character ... The invention relates to a method for generating a data packet to be sent by combining the data stream, the character sequence number of the missing characters at the receiving end, and the character sequence number of the missing characters at the sending end; the sending end encapsulates the encoding result into the data packet to be sent to obtain a data packet, and sends the data packet to the receiving end; the receiving end decodes the encoded content in the data packet to obtain a text data stream; the text data stream is corrected using a pre-trained text error correction model, and the missing text in the text data stream is predicted and filled using a pre-trained text prediction model; based on the pre-stored default voiceprint features or the voiceprint features of the target object, the processed text data stream is converted into a voice data stream, and the voice data stream is output through the voice output module, thereby achieving the purpose of performing low-bitrate voice encoding in the voice call service process of weak signal, thereby reducing the probability of voice interruption, thereby achieving the technical effect of improving the voice call quality under weak signal, and thus solving the technical problem of poor voice call quality under weak signal due to the high probability of voice interruption in the voice call service of weak signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 is a flow chart of a data transmission method according to an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of a data transmission system according to an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of a coding field according to an embodiment of the present application;

[0025] Figure 4 is a structural diagram of a data transmission device according to an embodiment of the present application;

[0026] Figure 5 This is a hardware structure block diagram of a computer terminal according to a data transmission method of an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] In related technologies, the low-rate AI voice coding standard AVS3P10 uses AI algorithms to encode and decode voice. It obtains audio feature vectors and uses AI algorithms to predict and reconstruct audio with error correction, making it applicable to any audio call.

[0030] Some related methods for voice transmission based on ASR / TTS, such as "Instant Messaging System Based on Multi-language Conversion" (Publication No. CN106453043A), collect voice on the terminal side, send the voice to an intermediate server for voice-to-text transcription, language translation, and voice synthesis, and then send it to another terminal in the form of a text message. It is essentially a semantic coding technology based on the source, and the communication mode is text message. The smallest unit of communication is a whole sentence / paragraph, and there is no need to consider the real-time problem of word-by-word transmission in a call scenario.

[0031] Some related FEC coding error-tolerance-based technologies, such as "A Low-Delay Real-Time Video FEC Coding Transmission Control Method" (Publication No. CN115189810A) and "A System and Method for Dynamic Adaptive Forward Error Control in IPTV Networks" (Publication No. CN 200710095954), sample the audio and video at a specific bit rate and then perform real-time forward error correction (FEC) coding by setting the coding delay D, the number of redundant packets S, and controlling the number of packets and groups in the buffer, to ensure that all sampled video frames (data packets) can be transmitted within the specified delay after generation.

[0032] According to an embodiment of the present application, a method embodiment of a data transmission method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 1 is a flow chart of a data transmission method according to an embodiment of the present application, such as Figure 1 As shown, the method includes the following steps:

[0034] In step S101, the sending end obtains a local voice data stream, converts the voice data stream into characters word by word using a pre-trained voice recognition model, and performs error correction on the characters using a pre-trained text error correction model to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream.

[0035] In step S101, the first character sequence in the first text data stream is, for example, "Have you eaten today?", and the first character sequence is, for example, "1234567."

[0036] In step S101, the transmitting end continuously captures the voice data stream input by the user through its internal voice receiving module. Subsequently, in order to improve the accuracy of speech recognition, the voice data stream is preprocessed, wherein the preprocessing includes noise reduction, echo cancellation, audio gain adjustment, etc., to remove or reduce the interference of environmental noise and improve speech clarity. The preprocessed voice data stream is fed into a pre-trained speech recognition model, such as a model based on deep learning (e.g. Whisper), and converted into a text data stream word by word in real time. The converted text data stream contains multiple characters, each of which is accompanied by its relative position information in the voice stream to maintain the original semantic order of the speech.

[0037] Furthermore, the recognized text data stream may contain certain errors due to misrecognition by the speech recognition model or network transmission errors. In order to improve the accuracy of the text, a pre-trained text error correction model is used to perform real-time error correction processing on the text data stream. The text error correction model can be based on an N-gram model, a language model, or a more advanced deep learning model (such as a Transformer-based model). The text error correction model can correct text deviations caused by recognition errors or transmission errors based on contextual information. The text error correction model not only corrects spelling errors, but also performs semantic corrections based on contextual understanding, ensuring that even small errors in the recognition process can be repaired through semantic understanding, thereby outputting text content that is more consistent with the speaker's intention.

[0038] Through the above step S101, not only can the voice signal be converted into a text data stream efficiently and accurately, but also the errors that may occur during the conversion process can be corrected in real time, ensuring that the communication content between users remains accurate and clear even when the network signal is unstable or the voice recognition conditions are poor.

[0039] In step S102, the transmitting end determines different transmission modes to be adopted for data transmission between the transmitting end and the receiving end based on a target error tolerance rate of a packet loss rate during data transmission between the transmitting end and the receiving end, wherein the target error tolerance rate is a parameter that is periodically adjusted according to changes in the packet loss rate; and under different transmission modes, different encoding methods are used to encode the first text data stream to obtain an encoding result.

[0040] In step S102, the network packet loss rate between the transmitting end and the receiving end can be monitored in real time by the coding rate controller, and the target fault tolerance rate can be periodically adjusted based on the change of the packet loss rate. Specifically, if the packet loss rate rises within a preset period, the target fault tolerance rate is increased, otherwise the target fault tolerance rate is reduced. It is worth noting that the adjustment of the target fault tolerance rate follows the following principles: 1. When the target fault tolerance rate is less than a first preset threshold (for example, 0.4), the coding rate controller determines to adopt the first transmission mode (fault-tolerant mode), i.e., mode A, for data transmission. 2. When the target fault tolerance rate is greater than or equal to the first preset threshold, the coding rate controller determines to adopt the second transmission mode (non-fault-tolerant mode), i.e., mode B, for data transmission. This dynamic adjustment mechanism allows the coding strategy to adapt to network conditions, ensuring that high voice call quality can be maintained under different packet loss rates.

[0041] For Mode A (fault-tolerant mode), redundant encoding, such as Reed-Solomon encoding, is used to segment the GBK encoding of each character and add error correction symbols to form G character encoding slices, where G is the number of character encoding slices. For Mode B (non-fault-tolerant mode), the GBK encoding of each character is repeated G times to form G identical slices, increasing transmission redundancy.

[0042] In step S103, the sending end receives the character sequence number of the missing character of the receiving end, the second text data stream, and the second character sequence number corresponding to each second character in the second text data stream sent by the receiving end; determines the character sequence number of the missing character of the sending end based on the second text data stream and the second character sequence number; and generates a data packet to be sent based on the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the missing character of the receiving end, and the character sequence number of the missing character of the sending end.

[0043] In step S104, the transmitting end encapsulates the encoding result into a data packet to be sent, obtains a data packet, and sends the data packet to the receiving end.

[0044] In step S105, the receiving end decodes the encoded content in the data packet to obtain a text data stream; uses a pre-trained text error correction model to correct the text data stream, and uses a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, the processed text data stream is converted into a voice data stream, and the voice data stream is output through the voice output module.

[0045] According to the above steps, a sending end is used to obtain a local voice data stream, a pre-trained voice recognition model is used to convert the voice data stream word by word into characters, and a pre-trained text error correction model is used to perform error correction processing on the characters to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; the sending end determines different transmission modes adopted by the sending end and the receiving end for data transmission based on a target fault tolerance rate of a packet loss rate when the sending end and the receiving end perform data transmission, wherein the target fault tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; under different transmission modes, different encoding methods are used to encode the first text data stream to obtain an encoding result; the sending end receives the character sequence number of the missing character of the receiving end sent by the receiving end, the second text data stream and the second character sequence number corresponding to each second character in the second text data stream; the character sequence number of the missing character of the sending end is determined based on the second text data stream and the second character sequence number. number; generate a data packet to be sent according to the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the missing character at the receiving end, and the character sequence number of the missing character at the sending end; the sending end encapsulates the encoding result into the data packet to be sent to obtain a data packet, and sends the data packet to the receiving end; the receiving end decodes the encoded content in the data packet to obtain a text data stream; uses a pre-trained text error correction model to correct the text data stream, and uses a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, the processed text data stream is converted into a voice data stream, and the voice data stream is output through the voice output module, thereby achieving the purpose of performing low-bitrate voice encoding in the voice call service process with weak signals, thereby reducing the probability of voice interruption, thereby achieving the technical effect of improving the voice call quality under weak signals.

[0046] The following Figure 1 The steps shown are exemplified and explained.

[0047] According to some optional embodiments of the present application, different transmission modes adopted by the sending end and the receiving end for data transmission are determined based on the target fault tolerance rate of the packet loss rate when the sending end and the receiving end perform data transmission, which can be achieved by the following method: obtaining a preconfigured initial packet loss rate fault tolerance rate; detecting the packet loss rate when the sending end and the receiving end perform data transmission in real time; if the packet loss rate is in an increasing state within a preset period, increasing the initial packet loss rate fault tolerance rate to obtain the target fault tolerance rate; if the packet loss rate is in a decreasing state within the preset period, reducing the initial packet loss rate fault tolerance rate to obtain the target fault tolerance rate; obtaining the target fault tolerance rate at the target moment, and when the target fault tolerance rate is less than a first preset threshold, determining to adopt the first transmission mode for data transmission with the receiving end; when the target fault tolerance rate is greater than the first preset threshold, determining to adopt the second transmission mode for data transmission with the receiving end.

[0048] In the above embodiment, when the transmitting end initializes, the coding rate controller obtains an initial packet loss rate tolerance R from preconfigured rules. The initial packet loss rate tolerance is set based on expected network conditions. For example, in a 4G weak coverage scenario, the initial packet loss rate tolerance can be set to 0.3 or lower to accommodate potential high packet loss environments. The coding rate controller continuously monitors data transmission between the transmitting and receiving ends, obtaining the current network packet loss rate in real time. Based on the obtained packet loss rate data, the coding rate controller analyzes its changing trend within a preset period. If the packet loss rate shows an upward trend within the period, the coding rate controller increases the initial packet loss rate tolerance to obtain a new target tolerance, thereby increasing coding redundancy and improving transmission robustness. Conversely, if the packet loss rate decreases, the coding rate controller decreases the initial packet loss rate tolerance to reduce redundancy and optimize bandwidth utilization. The adjustment range should be determined based on network conditions and voice quality requirements to avoid voice quality degradation caused by excessive redundancy or compression.

[0049] After adjusting the packet loss rate tolerance, the code rate controller determines the target error tolerance at the target time. If the target error tolerance is less than a first preset threshold (e.g., 0.4), the first transmission mode (Mode A) is adopted, which has a low probability of code loss. If the target error tolerance is greater than the first preset threshold, the code rate controller determines to adopt the second transmission mode (Mode B) to cope with more severe network conditions, such as high packet loss and unstable connections.

[0050] According to other optional embodiments of the present application, encoding the first text data stream using different encoding methods in different transmission modes to obtain encoding results can be achieved by the following method:

[0051] A text queue is created, wherein the text queue includes: a first queue, a second queue and a third queue, the first queue is used to store character information in the first text data stream, the second queue is used to store character information of missing characters on the receiving end, and the third queue is used to store character sequence numbers of missing characters on the sending end.

[0052] Both the first queue and the second queue include: character sequence number, character code, latest sending time of the character, G character code slices corresponding to the character code, latest sending time of the character code slice, and sequence number of the character code slice to be sent, where G is a positive integer greater than 1.

[0053] Obtain an uncoded target element combination in the first queue, wherein the target element combination includes: a target character sequence number, a target character code, and a target character latest sending time; in the case of adopting a first transmission mode to transmit data with a receiving end, use a forward error correction coding algorithm to process the target character code in the target element combination to obtain a first character code slice set, wherein the first character code slice set includes G character code slices, each character code slice being a coding segment of the target character code; add the first character code slice set to the target element combination; and determine the latest sending time of each character code slice in the first character code slice set according to the target time, the target character latest sending time, G, and K, to obtain a first character code slice latest sending time set. Wherein, K is a positive integer not greater than G; the latest sending time set of the first character code slice is added to the target element combination to obtain a first coding result; when the second transmission mode is adopted to transmit data with the receiving end, the target character code in the target element combination is copied to obtain a second character code slice set, wherein the second character code slice set includes G target character codes; the second character code slice set is added to the target element combination; and according to the target moment, the latest sending time of the target character, G and K, the latest sending time of each character code slice in the second character code slice set is determined to obtain the second character code slice latest sending time set, and the second character code slice latest sending time set is added to the target element combination to obtain a second coding result.

[0054] It's worth explaining that, depending on network conditions and a preset minimum time window Tmin (e.g., 2 seconds), each character has a minimum send time (or "character minimum send time"). Furthermore, each character code slice has a specific minimum send time (or "character code slice minimum send time") to ensure that even in the event of network packet loss, the receiver can still receive at least one valid character code slice within a certain timeframe.

[0055] In the above embodiment, three text queues are first created, namely the first queue Q1, the second queue Q2 and the third queue Q3, wherein the first queue Q1 is used to store the character related information generated by the sender in a streaming manner, and the first queue Q1 includes: character sequence number i, character code C i , the latest time a character is sent T imax , G character code slices CM corresponding to the character code i [cm i1 ,…,cm iG ], the latest sending time of character encoding slice T i [t i1 ,…,t iG], the character code slice number j to be sent, where G is a positive integer greater than 1. Q1 can be expressed as [(i,C i ,T imax ,CM i ,T i ,j),…,()].

[0056] The second queue Q2 is used to store the information about the characters missing at the receiving end. Its format and maintenance method are the same as those of the first queue Q1. Q2 can also be expressed as [(i,C i ,T imax ,CM i ,T i ,j),…,()].

[0057] The third queue Q3 is used to store the list of missing character sequence numbers at the sender, and Q3 can be expressed as [i....].

[0058] Furthermore, the uncoded target element combination (i, C i ,T imax ), and process it in the following way, where i is the target character number, C i is the target character encoding, T imax The latest sending time of the target character.

[0059] When in the first transmission mode (transmission mode A), the 2-byte GBK-encoded C i Each byte in is used as a coding symbol (a total of 2 symbols) as the input of the forward error correction coding algorithm, and G-2 error correction codes are added to obtain G character codes. Let CM i For [C i1 ~C iG ](G elements in total, each element is C i A coded fragment, occupies 1 byte), CM i Fill the queue back. Then, the latest sending time T of each character encoding slice is i Determine as array [T+T imax / G*k], where k=0,…,G; T is the target time (current time), T imax is the latest sending time of the target character. Let j = 0, that is, the 0th code is transmitted at the initial time. i Fill it back into the first queue to complete the encoding.

[0060] When in the second transmission mode (transmission mode B), C i Copy the same G copies and make CM i For [C i ,...,C i](G elements in total, each element is C i , occupies 2 bytes), CM i Fill the queue back. Then, the latest sending time T of each character encoding slice is i Determine as array [T+T imax / G*k], where k=0,…,G; T is the target time (current time), T imax is the latest sending time of the target character. Let j = 0, that is, the 0th code is transmitted at the initial time. i Fill it back into the first queue to complete the encoding.

[0061] In some optional embodiments of the present application, before obtaining the unencoded target element combination in the first queue, the following steps may also be performed: obtain the target triple corresponding to the target character in the text data stream at time T, wherein the target triple includes: target character sequence number, target character encoding, target time; T is the target time, and T is a positive integer greater than 1; obtain the latest sending time of the character encoding slice at time T-1 in the first queue, and determine the latest sending time of the target character based on the latest sending time of the character encoding slice at time T-1, the preconfigured time sliding window, and the target time; add the target character sequence number, target character encoding, target character latest sending time, target character encoding slice, target character encoding slice latest sending time, and target sequence number to the first queue, wherein the target character encoding slice and the target character encoding slice latest sending time are empty arrays, and the target sequence number is 0.

[0062] In the above embodiment, the target triple corresponding to the target character in the text data stream at time T is obtained, wherein the target triple can be expressed as (i, C i ,T), i is the character count number in the text (target character number), C i is the GBK encoding of character i (target character encoding), T is the current time (target time). Get the target triple corresponding to the character i-1 before character i from the first queue Q1: (i-1, C i -1,T imax -1), let T of character i imax =max(T imax -1+T min ,T), where T imax -1 is the latest sending time of the character encoding slice at time T-1, T min is a preconfigured time sliding window (e.g. 2 seconds). i ,T imax ,CM i ,T i ,j) put into the first queue Q1, where CM i ,Ti It is an empty array and j is 0.

[0063] In a preferred embodiment of the present application, the transmitting end encapsulates the encoding result into a data packet to be sent, obtains a data packet, and sends the data packet to the receiving end, which can be achieved by the following method:

[0064] Step S1: The transmission mode used by the transmitting end and the receiving end for data transmission is determined as the first field S of the data to be sent. a The number of characters of the missing characters at the receiving end and the number of retransmitted characters are determined as the second field S of the data to be sent b The number of characters missing from the sending end is determined as the third field S of the data to be sent c The character sequence number of the starting character of the first text data stream is determined as the fourth field S of the data to be sent d The first text data stream is determined as the fifth field S of the data to be sent e ; The character sequence number of the missing character at the receiving end is determined as the sixth field S of the data to be sent f ; The character sequence number of the missing character at the sending end is determined as the seventh field S of the data to be sent g .

[0065] Step S2, determine the maximum length of the data packet to be sent. a It is a required field, the second field S b It is a required field, the third field S c It is a required field, the fourth field S d It is a required field, the fifth field S e Optional field, sixth field S f Optional field, seventh field S g This is an optional field. The fifth field (continuous character sequence) S e The format is: character 1 character encoding slice | character 2 character encoding slice | .... The sixth field (retransmission character sequence) S f The format is: sequence number 1 | character 1 character encoding slice | sequence number 2 | character 2 character encoding slice | .... The seventh field (missing sequence number) S g The format is: sequence number 1 | sequence number 2 | ...

[0066] Step S3, from the first queue, starting from the first position, sequentially take out the first character code slice whose latest sending time is within the first interval, wherein the first position is the position of the first character of the multiple characters that are not completely sent at the target time in the first queue, the first interval is from the target time to the first time, and the first time is the sum of the target time and the preconfigured packet sending interval; add the first character code slice to the fifth field, and determine the first serial number of the character corresponding to the first character code slice and the second serial number of the character corresponding to the last character code slice in the fifth field.

[0067] Step S4: From the second queue, starting from the first character code to be sent, sequentially take out the second character code slice whose latest sending time is within the second interval, where the second interval is from the target moment to the second moment, and the second moment is the next adjacent moment of the target moment; add the second character code slice to the sixth field.

[0068] Step S5: Take out all character serial numbers from the third queue, add all character serial numbers to the seventh field, and delete all character serial numbers in the third queue.

[0069] Step S6: Take out the third character code slice after the second sequence number from the first queue, and add the third character code slice to the fifth field.

[0070] Step S7: Take out the fourth character code slice before the first sequence number from the first queue, and add the fourth character code slice to the fifth field.

[0071] Step S8: After all steps from step S3 to step S7 are executed or the length of the data packet to be sent reaches the maximum length, a data packet is obtained and added to a queue to be sent.

[0072] Step S9: based on the preconfigured packet sending interval, obtain the data packet from the queue to be sent, and send the data packet to the receiving end.

[0073] Specifically, step S2 can be implemented by the following method: obtaining a preconfigured initial bit rate; detecting the packet loss rate during data transmission between the sending end and the receiving end in real time, if the packet loss rate is in an increasing state within a preset period, reducing the initial bit rate to obtain the target bit rate; detecting the amount of text in the data cache area on which the sending end and the receiving end rely for data transmission in real time, if the amount of text is greater than a preset second preset threshold, increasing the initial bit rate to obtain the target bit rate; determining the maximum length corresponding to the data packet to be sent based on the target bit rate, the preconfigured packet sending interval and the protocol packet header length in the preconfigured rules.

[0074] It should be noted that the maximum length corresponding to the initial data packet S is M*dt - H, where M is the target code rate, d is the pre-configured packet sending interval, and H is the protocol packet header length in the pre-configured rule. For example, after enabling ROHC header compression, the RTP packet header length is 24 bit.

[0075] Furthermore, each time an encoded S is assembled, when S reaches the maximum length, stop assembling and recreate S. The initial S a is the data transmission mode determined according to the target error tolerance, S b 、S c 、S d is 0, S e 、S f 、S g is empty, and assemble in the following order.

[0076] In the first queue Q1, save the position P1 of the first character that has not been completely sent currently. After each character encoding slice of character i is sent, j is automatically incremented by 1. When j of character i is G, it means that character i has been sent. If P1 is i at this time, then P1 is incremented by 1 to change the first character position of the queue, and j = 0 is set, indicating that if retransmission is required, start transmitting from the 0th slice.

[0077] In the first queue Q1, starting from i = P1, incrementally retrieve the latest sending time T of all character encoding slices ij of the elements in the range of T to T + dt, and append them to S e . Update j to j + 1, and record the serial number of the first character appended as i a , and the serial number of the last character as i b . Only take 1 character encoding slice for each character.

[0078] Starting from the first tuple to be sent recorded in the second queue Q2, sequentially retrieve the encoding segments j of all elements where T < Timax <= T + 1, and append them to S f . Update j to j + 1, and record the number of characters added as S b .

[0079] Sequentially retrieve all serial numbers i from the third queue Q3 and append them to S g . Delete this element from Q3. Record the number of missing character serial numbers added as S c .

[0080] Sequentially retrieve all elements j of the encoding segments after i b from the first queue Q1 and append them to S in character order e . Update j to j + 1, and update i b .

[0081] Sequentially take out all i from the first queue Q1 a The previous encoding segment j is appended to S in character order e Update j to j+1, update i a , and let S d =i a .

[0082] Finally, when S reaches its maximum length or all the above steps are completed, S is placed in the queue to be sent. Every fixed time interval dt, a data packet is retrieved from the queue and sent. If the queue is empty, S in the buffer is sent.

[0083] In other preferred embodiments of the present application, under different transmission modes, different decoding methods are used to decode the encoded content in the data packet to obtain a decoding result, which can be achieved by the following method: obtaining a text queue; taking the character sequence number of the starting character in the data packet as the starting point, sequentially parsing the character code slices in the retransmitted characters or the character code slices in the continuous characters in the data packet; when the first field indicates that the transmission mode adopted by the sending end and the receiving end for data transmission is the first transmission mode, if the number of character code slices obtained by parsing is G / 2+1, using the forward error correction coding algorithm to decode the character code from the (G / 2+1) character code slices to obtain a first decoding result, wherein the first decoding result includes the character code and the character sequence number corresponding to the character code; when the first field indicates that the transmission mode adopted by the sending end and the receiving end for data transmission is the second transmission mode, decoding the character code from any character code slice to obtain a second decoding result, wherein the second decoding result includes the character code and the character sequence number corresponding to the character code.

[0084] Furthermore, the missing characters are parsed out in the data packet, and the triples corresponding to the missing characters are determined in the first queue, wherein the triples include: character sequence number, character encoding, and latest character sending time; the triples corresponding to the missing characters are added to the second queue; and when the decoded character encoding is the character encoding corresponding to the missing characters in the third queue, the missing characters in the third queue are deleted.

[0085] In the above embodiment, the receiving end parses the received data packet to obtain the number S of characters missing from the sending end. c , and decode the missing character number at the corresponding position, and find the corresponding (i, C i ,T imax ), join the second queue Q2.

[0086] Furthermore, the transmission mode S is parsed from the received data packet. a , the number of retransmitted characters Sb , the first character number of the continuous character sequence, and decode the corresponding retransmitted character code slice or a slice of the continuous character code at the corresponding position. ij ). Among them, the sequence number is fixed to 2 bytes. When the transmission mode is mode B, the character encoding slice is 2 bytes; when the transmission mode is mode A, the character encoding slice is 1 byte. Then cache (i, CM ij ).

[0087] When the transmission mode is A, after accumulating G / 2+1 character code slices, the forward error correction decoding algorithm is used to decode the character from (i, CM ij ) decodes C i When the transmission mode is B, any CM ij Can be used as C i 。 Will (i,C i ) is stored in the queue to be played. i If it is a missing character in the third queue Q3, delete Q3(i). If the time from Q3 to playing is less than the configured threshold, delete Q3(i).

[0088] Furthermore, an LLM-based text error correction module can be used to correct the text being played based on the context. An LLM-based text prediction module can be used to predict missing text in the text array based on the context. The missing text sequence number can also be periodically checked, and if i is not included in Q3, i is added to Q3.

[0089] Finally, the pre-stored default voiceprint feature or the other party's voiceprint feature is obtained from the voiceprint feature memory, synthesized into speech through the speech synthesis module, and output through the speech output module.

[0090] It is worth noting that the above Figure 1 The data transmission method shown can theoretically achieve low-bit-rate error-tolerant speech coding of about 0.5 kbps through AI encoding and decoding algorithms and adaptive dynamic bit rate error-tolerant coding control.

[0091] It should be noted that the above Figure 1The usage scenarios of the data transmission method shown include: 1. Low-bandwidth mobile phone communication scenarios. In some environments with extremely harsh network conditions, such as remote mountainous areas, offshore operating platforms, or ruins areas after natural disasters, the network bandwidth may be very limited. 0.5kbps voice calls can achieve basic real-time voice communication functions. Compared with the existing 4-6kbps technology, it ensures that emergency contacts can still be made in extreme situations and reduces the incidence of voice interruptions. 2. Satellite communication scenarios. In satellite communication scenarios, due to the limitation of uplink bandwidth, a low-orbit satellite (LEO) can support a maximum of about a thousand people making simultaneous voice calls. After applying this solution, the number of people making simultaneous calls can be increased by about 10 times. 3. Wireless network / wifi weak signal area scenarios. When the user is in an area with weak mobile phone signals, the communication bandwidth is limited and the network packet loss is severe. This application can achieve real-time voice communication in areas with weak signals. 4. Low-cost communication demand scenarios. The data volume of a 0.5kbps voice call is extremely small, which can significantly reduce the line cost of communication.

[0092] above Figure 1 The data transmission method shown uses the UE's AI computing power to perform real-time encoding, decoding, error correction, and data transmission and reception. It mainly includes input and output parts, such as Figure 2 After the UE inputs a call, it undergoes AI-powered transcription and error correction before being stored in a queue. While maintaining the minimum transmission rate, it utilizes dynamic, fault-tolerant coding and transmission across the source and channel, taking into account factors such as channel bandwidth and packet loss. Upon receiving the message from the other party, it decodes it and uses AI text error correction, AI comprehension prediction, and AI speech synthesis for voice playback. Fault-tolerant feedback is then provided through queue synchronization.

[0093] It should be noted that Figure 1 The prerequisites for executing the steps shown include: the caller and the called party have established a connection, the header compression protocol such as ROHC or IPHC is enabled, and it is agreed to use the encoding proposed in this application for voice calls.

[0094] It should be further explained that Figure 1The sending end and the receiving end in the steps shown can perform data transmission based on the transfer device. The transfer device is used to perform the following method: receiving a call request message sent by a first terminal device to a second terminal device, wherein the call request message includes at least a first list corresponding to the first terminal device, and the first list includes at least the codec types supported by the first terminal device; in the case where the first list includes pre-configured non-universal codec types, constructing a first call request message including the first list, and sending the first call request message to the second terminal device; receiving a first call response message sent by the second terminal device in response to the first call request message, wherein the first call response message includes at least a second list corresponding to the second terminal device, and the second list includes at least the codec types supported by the second terminal device. The codec types supported in the first list and the priorities corresponding to the codec types; if the second list includes non-universal codec types, the non-universal codec types have the highest priority; if the second list does not include non-universal codec types, add non-universal codec types in the second list, wherein the newly added non-universal codec types have the lowest priority; determine the target codec type as the codec type used for data transmission between the transit device and the first terminal device and the second terminal device, wherein the target codec type is the codec type with the highest priority in the second list; forward the first call response message to the first terminal device, and establish a voice call with the first terminal device and the second terminal device.

[0095] Furthermore, after determining the target codec type as the codec type used for data transmission between the transit device and the first terminal device and the second terminal device, the method also includes: receiving an update message sent by the first terminal device, wherein the update message is used to change the codec type used for data transmission between the first terminal device and the transit device from a universal codec type to a non-universal codec type, and the update message is sent when the codec type used for data transmission between the first terminal device and the transit device is a universal codec type and the network performance parameter is less than a preset threshold; when confirming that the codec type used for data transmission between the first terminal device and the transit device is changed to a non-universal codec type, data transmission is performed with the first terminal device via the non-universal codec type.

[0096] Furthermore, upon confirming that the codec type used for data transmission between the first terminal device and the transit device is changed to a non-universal codec type, if the second terminal device supports the non-universal codec type, the update message is forwarded to the second terminal device; upon receiving the first confirmation message sent by the second terminal device, the codec type used for data transmission between the second terminal device and the transit device is changed to a non-universal codec type, wherein the first confirmation message is used to indicate that the second terminal device confirms that the codec type used for data transmission between the second terminal device and the transit device is changed to a non-universal codec type.

[0097] Furthermore, after establishing a voice call between the first terminal device and the second terminal device, the method also includes: receiving a coding change request sent by the first terminal device, wherein the coding change request is used to request that the codec type used for data transmission between the first terminal device and the transit device be changed from a universal codec type to a non-universal codec type, and the coding change request is sent when the codec type used for data transmission between the first terminal device and the transit device is a universal codec type and the network performance parameter is less than a preset threshold; upon confirming that the codec type used for data transmission between the first terminal device and the transit device is changed to a non-universal codec type, sending a first response message to the first terminal device; upon successfully changing the codec type used for data transmission between the first terminal device and the transit device to a non-universal codec type, sending a second response message to the first terminal device, and sending a second notification to the second terminal device via the data surface, wherein the second response message is used to instruct the first terminal device to continue the call, and the second notification is used to instruct the second terminal device to continue the call.

[0098] Furthermore, the method also includes: when the second terminal device supports a non-universal codec type, sending a coding type change request to the second terminal device, wherein the coding type change request is used to request that the codec type used for data transmission between the second terminal device and the transit device be changed from a universal codec type to a non-universal codec type; after receiving a second confirmation message from the second terminal device, starting to change the codec type used for data transmission between the second terminal device and the transit device from a universal codec type to a non-universal codec type, wherein the second confirmation message is used to indicate that the second terminal device confirms that the codec type used for data transmission between the second terminal device and the transit device will be changed to a non-universal codec type; when the codec type used for data transmission between the first terminal device and the transit device is successfully changed to a non-universal codec type, and when the codec type used for data transmission between the second terminal device and the transit device is successfully changed to a non-universal codec type, sending a second response message to the first terminal device, and sending a second notification to the second terminal device through the data surface, wherein the second response message is used to instruct the first terminal device to continue the call, and the second notification is used to instruct the second terminal device to continue the call.

[0099] Furthermore, after sending the first response message to the first terminal device, the method also includes: suspending data plane communication with the first terminal device until the codec type used for data transmission between the first terminal device and the transit device is successfully changed to a non-universal codec type.

[0100] Furthermore, the method also includes: receiving a session initiation protocol end message sent by the first terminal device, forwarding the session initiation protocol end message to the second terminal device, and ending the voice call between the first terminal device and the second terminal device; or, receiving a session initiation protocol end message sent by the second terminal device, forwarding the session initiation protocol end message to the first terminal device, and ending the voice call between the first terminal device and the second terminal device.

[0101] It is worth noting that the above non-universal encoding types include Figure 1 The encoding method shown.

[0102] The following combination Figure 1 and Figure 2 , summarizes the main steps of the data transmission method proposed in this application.

[0103] Transmitter encoding and control method.

[0104] Step 1: Get the initial encoding parameters. The initial encoding parameters include: the protocol packet header length H in the pre-configured rule (for example, after ROHC header compression is enabled, the RTP packet header length is 24 bits). The pre-set minimum time sliding window Tmin (For example, 2 seconds.) The preset initial packet loss rate tolerance rate R (when the packet loss rate is less than R, decoding is not affected), the preset initial bit rate M, and the preset packet sending interval are preset to dt.

[0105] Step 2: Dynamic network monitoring. The coding rate controller periodically obtains and evaluates the network's maximum bandwidth, packet loss rate, and congestion to determine the coding rate and related parameters.

[0106] Step 3: Dynamically adjust encoding parameters. The encoding rate controller periodically adjusts initial parameters based on network conditions and the number of packets in the buffer to be sent. When the packet loss rate increases, the loss tolerance error ratio (R) is increased to reduce the retransmission rate. When network congestion occurs, the bit rate (M) is reduced to increase the encoded information capacity. When the buffer size increases, the bit rate (M) is increased to increase transmission speed, within the configured or measured maximum bandwidth and assuming no increase in network congestion.

[0107] Step 4: Queue creation. Create and maintain three queues, Q1, Q2, and Q3, and record the current location of each queue.

[0108] Step 4.1, create and maintain streaming text queue Q1: used to store the character-related information generated by the sender, including character sequence number i, character code Ci, and the latest sending time of the character T imax , G character encoding slices CM i [cm i1 ,…,cm iG ], the latest sending time of G symbol encoding slices T i [t i1 ,…,t iG ], the current slice number j to be sent: Q1=[(i,C i ,T imax ,CM i ,T i ,j),…,()]. And save the position P1 of the first character that is not fully sent. Every time a slice of character i is sent, j is automatically increased by 1. When j of character i is G, it means that character i has been sent. If P1 is i at this time, P1 is increased by 1, becoming the first character position of the queue, and setting j=0, indicating that if retransmission is required, transmission will start from the 0th slice.

[0109] Step 4.2, create and maintain the peer missing text queue Q2: used to store the information related to the characters currently missing on the receiving end. The format and maintenance method are the same as 4.1: Q2 = [(i,C i ,T imax ,CM i ,T i ,j),…,()]. And save the first character position P2 that is not fully sent.

[0110] Step 4.3, create and maintain the local missing sequence number text queue Q3: used to store the current missing character sequence number list of the sending end: Q3 = [i....].

[0111] Step 5: Determine the coding mode. The transmitter obtains the loss tolerance error rate from the coding rate controller. When the loss tolerance error rate R is less than the first threshold (according to the forward error correction coding performance, the first threshold is usually less than or equal to 0.4), the error-tolerant transmission mode (mode A) is adopted, in which the probability of coding loss is small. When the loss tolerance error rate R is greater than the first threshold, the error correction coding performance is too low, and the non-error-tolerant transmission mode (mode B) is switched. When a change occurs between the two modes, the code CM of all the incomplete character slices in Q1 and Q2 is changed. i 、T i ; j is set to 0.

[0112] Step 6: Voice input and preprocessing: Use the voice receiving module to continuously record the user's input voice and perform preprocessing such as noise reduction.

[0113] Step 7, speech transcription. Use a speech recognition model (such as Whisper, etc.) to convert speech into text word by word in real time, and use a pre-trained N-gram model to perform real-time text error correction. Obtain the corrected character and sequence number triples (i, C i ,T), where i is the character count number in the text, C i is the GBK code of character i, and T is the current time.

[0114] Step 8, calculate the latest sending time. Get the previous word (i-1, C i -1,T imax -1), let T imax =max(T imax -1+T min ,T). Modify the triple T in step 7 to T imax , (i,C i ,T imax ,CM i ,T i ,j) put into queue Q1. Among them, CM i ,T i It is an empty array, j is 0

[0115] Step 9: Queue element encoding. Get the unencoded elements (i, C i ,T imax ), and process it as follows.

[0116] Step 9.1, copy. When in transfer mode B, copy C iCopy the same G copies and make CM i For [C i ,...,C i ](G elements in total, each element is C i , occupies 2 bytes), CM i Fill back the queue.

[0117] Step 9.2, redundant encoding. When in transmission mode A, the 2-byte GBK-encoded C i , each byte is used as a coding symbol, a total of 2 symbols, as the input of the forward error correction coding algorithm (such as the Reed-Solomon algorithm), add G-2 error correction codes, and obtain G character codes. Let CM i For [C i1 ~C iG ](G elements in total, each element is C i A coded fragment, occupies 1 byte), CM i Fill back the queue.

[0118] Step 9.3, calculate the sending time of each coded slice. Let each slice Ti be the array [T+T imax / G*k] where k=0...G; let j=0, that is, the 0th code is transmitted initially. Put Ti back into the queue to complete the encoding.

[0119] Step 10: Create a data packet to be sent. Create an empty data packet S, which contains the following fields: S a , Transmission mode (required), S b Number of retransmitted characters (required), S c Number of missing characters (required), S d Continuous character starting number (required), S e Continuous character sequence (optional), S f Retransmission character sequence (optional), S g Missing sequence number sequence (optional) Field in data packet S such as Figure 3 shown.

[0120] Step 11: Code assembly. Assemble a code S each time. The maximum length of S is M*dt-H. When S reaches the maximum length, stop assembling and recreate S. Initially, S a , is the transmission mode determined by the coding rate controller, S b 、S c 、S d is 0, S e 、S f 、S g Empty, assemble in the following order.

[0121] Step 11.1, in the first queue Q1, starting from i = P1 and incrementing, sequentially retrieve all the latest transmission times T of the character encoding slices. ij For the character encoding slice j of the elements within T to T+dt, append it to S. e Update j to j + 1, and record the starting character serial number of the appended as i. a and the ending character serial number as i. b Only take 1 character encoding slice for each character.

[0122] Step 11.2, starting from the first tuple to be transmitted recorded in the second queue Q2, sequentially retrieve all the encoding segments j of the elements where T < Timax <= T + 1, and append them to S. f Update j to j + 1, and record the number of characters added as S. b .

[0123] Step 11.3, sequentially retrieve all the serial numbers i from the third queue Q3, append them to S, g and delete the element from Q3. Record the number of missing character serial numbers added as S. c .

[0124] Step 11.4, sequentially retrieve all the encoding segments j of the elements after i from the first queue Q1, and append them to S in character order. b Update j to j + 1, and update i. e . b .

[0125] Step 11.5, sequentially retrieve all the encoding segments j before i from the first queue Q1, and append them to S in character order. a Update j to j + ۱, update i, e and set S a = i d . a .

[0126] Step 11.6, when S reaches the maximum length or after completing all the steps from 11.1 to 11.5, put S into the queue to be transmitted.

[0127] Step 12, data transmission. Every fixed time interval dt, obtain a data packet from the queue to be transmitted and send it. If the queue is empty, send S in the buffer.

[0128] Receiver decoding and control method.

[0129] Step 1, missing character decoding. From the received data packet, parse out the number of missing characters S c sent by the sender, and decode the missing character serial numbers at the corresponding positions. Search in the first queue Q1 for the (i, C i , Timax ), join the second queue Q2.

[0130] Step 2: parse the transmission mode S from the received data packet a , the number of retransmitted characters S b , the first character number of the continuous character sequence, and decode the corresponding retransmitted character code slice or a slice of the continuous character code at the corresponding position. ij ). Among them, the sequence number is fixed to 2 bytes. When the transmission mode is mode B, the character encoding slice is 2 bytes; when the transmission mode is mode A, the character encoding slice is 1 byte. Then cache (i, CM ij ).

[0131] Step 3: Code slice decoding. When the transmission mode is A, after accumulating G / 2+1 character code slices, the forward error correction decoding algorithm is used to decode the code slices from (i, CM ij ) decodes C i When the transmission mode is B, any CM ij Can be used as C i 。 Will (i,C i ) is stored in the queue to be played.

[0132] Step 4, when the decoded C i If it is a missing character in the third queue Q3, delete Q3(i). If the time from Q3 to playing is less than the configured threshold, delete Q3(i).

[0133] Step 5: Text error correction: Use the LLM-based text error correction module to correct the text to be played based on the context.

[0134] Step 6: Text prediction: Use the LLM-based text prediction module to predict the missing text in the text array based on the context.

[0135] Step 7: Check for missing characters. You can periodically check for missing character numbers and add i to Q3 if it does not contain i.

[0136] Step 8: Voice synthesis and playback: Obtain the pre-stored default voiceprint features or the other party's voiceprint features from the voiceprint feature memory, synthesize them into voice through the voice synthesis module, and output them through the voice output module.

[0137] In summary, this application has the following advantages:

[0138] 1. This solution combines semantic communication technology with source-channel coding to recognize speech word by word into semantic symbols such as text and sequence numbers on the client side. Dynamic error-tolerant encoding and transmission are then performed based on channel network quality, and decoding is performed on the other end. This solution addresses bandwidth limitations and packet loss in weak signal conditions.

[0139] 2. A single-character splitting and forward error correction coding control mechanism based on loss tolerance and error rate was designed. This error correction method changes the traditional error correction coding scheme that requires multiple groups of data to be packaged and transmitted, and improves the real-time performance of streaming forward error correction.

[0140] 3. Through the design of time sliding windows, latest sending time, and cache queue management mechanisms, a real-time streaming error correction transmission mechanism with different rates at the data generation end, data consumption end, and data transmission process is implemented under the condition of a specified bit rate, thereby improving transmission stability.

[0141] 4. By allocating variable-length content to fixed-length data packets, the coding space is fully utilized and the retransmission efficiency is improved.

[0142] 5. A transmission mechanism based on coded fragmentation and distributed sending has been designed. While ensuring timeliness, it can to a certain extent resist the situation where continuous packet loss caused by network congestion results in incomplete transmission information.

[0143] Figure 4 is a structural diagram of a data transmission device according to an embodiment of the present application, such as Figure 4 As shown, the device includes:

[0144] The acquisition module 41 is used to control the sending end to obtain the local voice data stream, use the pre-trained speech recognition model to convert the voice data stream into characters word by word, and use the pre-trained text error correction model to correct the characters, so as to obtain the first text data stream and the first character sequence number corresponding to each first character in the first text data stream.

[0145] The encoding module 42 is used to control the transmitter to determine different transmission modes used for data transmission between the transmitter and the receiver based on a target error tolerance rate for the packet loss rate during data transmission between the transmitter and the receiver, wherein the target error tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; and to encode the first text data stream using different encoding methods under different transmission modes to obtain an encoding result.

[0146] The generation module 43 is used to control the sending end to receive the character sequence number of the missing character of the receiving end, the second text data stream, and the second character sequence number corresponding to each second character in the second text data stream sent by the receiving end; determine the character sequence number of the missing character of the sending end based on the second text data stream and the second character sequence number; and generate a data packet to be sent based on the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the missing character of the receiving end, and the character sequence number of the missing character of the sending end.

[0147] The encapsulation module 44 is used to control the sending end to encapsulate the encoding result into the data packet to be sent, obtain the data packet, and send the data packet to the receiving end.

[0148] The decoding module 45 is used to control the receiving end to decode the encoded content in the data packet to obtain a text data stream; use a pre-trained text error correction model to correct the text data stream, and use a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, convert the processed text data stream into a voice data stream, and output the voice data stream through the voice output module.

[0149] Optionally, the encoding module 42 is also used to perform the following steps: obtaining a preconfigured initial packet loss rate tolerance rate; detecting the packet loss rate during data transmission between the sending end and the receiving end in real time; if the packet loss rate is in an increasing state within a preset period, increasing the initial packet loss rate tolerance rate to obtain a target fault tolerance rate; if the packet loss rate is in a decreasing state within the preset period, reducing the initial packet loss rate tolerance rate to obtain a target fault tolerance rate; obtaining the target fault tolerance rate at the target moment, and when the target fault tolerance rate is less than a first preset threshold, determining to adopt a first transmission mode for data transmission with the receiving end; when the target fault tolerance rate is greater than the first preset threshold, determining to adopt a second transmission mode for data transmission with the receiving end.

[0150] Optionally, the encoding module 42 is also used to perform the following steps: creating a text queue, wherein the text queue includes: a first queue, a second queue and a third queue, the first queue is used to store character information in the first text data stream, the second queue is used to store character information of missing characters on the receiving end, and the third queue is used to store character sequence numbers of missing characters on the sending end; the first queue and the second queue both include: character sequence number, character code, latest sending time of the character, G character code slices corresponding to the character code, latest sending time of the character code slice, and sequence number of the character code slice to be sent, wherein G is a positive integer greater than 1; obtaining an unencoded target element combination in the first queue, wherein the target element combination includes: target character sequence number, target character code and latest sending time of the target character; when the first transmission mode is adopted to transmit data to the receiving end, the target character code in the target element combination is processed by using a forward error correction coding algorithm to obtain a first character code slice set, wherein the first character code slice set includes G character code slices, each character code slice is a target character code The method comprises the following steps: adding a first character code slice set to a target element combination; and determining, according to the target time, the latest sending time of the target character, G and K, the latest sending time of each character code slice in the first character code slice set to obtain a first character code slice latest sending time set, wherein K is a positive integer not greater than G; adding the first character code slice latest sending time set to the target element combination to obtain a first coding result; in the case of adopting a second transmission mode to transmit data with the receiving end, copying the target character code in the target element combination to obtain a second character code slice set, wherein the second character code slice set includes G target character codes; adding the second character code slice set to the target element combination; and determining, according to the target time, the latest sending time of the target character, G and K, the latest sending time of each character code slice in the second character code slice set to obtain a second character code slice latest sending time set, and adding the second character code slice latest sending time set to the target element combination to obtain a second coding result.

[0151] Optionally, the encoding module 42 is also used to perform the following steps before obtaining the unencoded target element combination in the first queue: obtain the target triple corresponding to the target character in the text data stream at time T, wherein the target triple includes: target character sequence number, target character code, target time; T is the target time, and T is a positive integer greater than 1; obtain the latest sending time of the character code slice at time T-1 in the first queue, and determine the latest sending time of the target character based on the latest sending time of the character code slice at time T-1, the preconfigured time sliding window, and the target time; add the target character sequence number, target character code, target character latest sending time, target character code slice, target character code slice latest sending time, and target sequence number to the first queue, wherein the target character code slice and the target character code slice latest sending time are empty arrays, and the target sequence number is 0.

[0152] Optionally, the encapsulation module 44 is further configured to perform the following steps: Step S1, determining the transmission mode used by the transmitting end and the receiving end for data transmission as the first field of the data to be sent; determining the number of characters of the missing characters and the number of retransmitted characters of the receiving end as the second field of the data to be sent; determining the number of characters of the missing characters of the transmitting end as the third field of the data to be sent; determining the character sequence number of the starting character of the first text data stream as the fourth field of the data to be sent; determining the first text data stream as the fifth field of the data to be sent; determining the character sequence number of the missing characters of the receiving end as the sixth field of the data to be sent; segment; determine the character sequence number of the missing character at the sending end as the seventh field of the data to be sent; step S2, determine the maximum length corresponding to the data packet to be sent; step S3, from the first queue, starting from the first position, sequentially take out the first character code slice whose latest sending time of the character code slice is within the first interval, wherein the first position is the position of the first character of the multiple characters that are not completely sent at the target time in the first queue, the first interval is from the target time to the first time, and the first time is the sum of the target time and the preconfigured packet sending interval; add the first character code slice to the fifth field, and in the fifth The first sequence number of the character corresponding to the first character code slice and the second sequence number of the character corresponding to the last character code slice are determined in the field; step S4, from the second queue, starting from the first character code to be sent, the second character code slice with the latest character sending time within the second interval is sequentially taken out, wherein the second interval is from the target time to the second time, and the second time is the next adjacent time of the target time; the second character code slice is added to the sixth field; step S5, all character sequence numbers are taken out from the third queue, all character sequence numbers are added to the seventh field, and all character sequence numbers are deleted from the third queue; step S6, the third character code slice after the second sequence number is taken out from the first queue, and the third character code slice is added to the fifth field; step S7, the fourth character code slice before the first sequence number is taken out from the first queue, and the fourth character code slice is added to the fifth field; step S8, after all steps from step S3 to step S7 are executed or the length of the data packet to be sent reaches the maximum length, a data packet is obtained and added to the queue to be sent; step S9, based on the preconfigured packet sending interval, the data packet is obtained from the queue to be sent and sent to the receiving end.

[0153] Optionally, the encapsulation module 44 is also used to perform the following steps: obtaining a preconfigured initial bit rate; detecting in real time the packet loss rate during data transmission between the sending end and the receiving end, and if the packet loss rate is in an increasing state within a preset period, reducing the initial bit rate to obtain a target bit rate; detecting in real time the amount of text in the data cache area on which the sending end and the receiving end rely for data transmission, and if the amount of text is greater than a preset second preset threshold, increasing the initial bit rate to obtain a target bit rate; determining the maximum length corresponding to the data packet to be sent based on the target bit rate, the preconfigured packet sending interval, and the protocol packet header length in the preconfigured rules.

[0154] Optionally, the decoding module 45 is also used to perform the following steps: obtaining a text queue; taking the character sequence number of the starting character in the data packet as the starting point, sequentially parsing the character code slices in the retransmitted characters or the character code slices in the continuous characters in the data packet; when the first field indicates that the transmission mode adopted by the sending end and the receiving end for data transmission is the first transmission mode, if the number of character code slices obtained by parsing is G / 2+1, using the forward error correction coding algorithm to decode the character code from the (G / 2+1) character code slices to obtain a first decoding result, wherein the first decoding result includes the character code and the character sequence number corresponding to the character code; when the first field indicates that the transmission mode adopted by the sending end and the receiving end for data transmission is the second transmission mode, decoding the character code from any character code slice to obtain a second decoding result, wherein the second decoding result includes the character code and the character sequence number corresponding to the character code.

[0155] Optionally, the data transmission device is also used to perform the following steps: parse out the missing characters in the data packet, determine the triplet corresponding to the missing characters in the first queue, wherein the triplet includes: character sequence number, character code, and latest character sending time; add the triplet corresponding to the missing characters to the second queue; and delete the missing characters in the third queue when the decoded character code is the character code corresponding to the missing characters in the third queue.

[0156] It should be noted that the above Figure 4 The modules in the embodiment can be program modules (for example, a set of program instructions that implement a specific function) or hardware modules. For the latter, they can be expressed in the following forms, but are not limited to these: the expression form of each of the above modules is a processor, or the functions of each of the above modules are implemented by a processor.

[0157] It should be noted that Figure 4 The preferred implementation of the embodiment shown can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0158] Figure 5FIG1 shows a hardware structure block diagram of a computer terminal for implementing a data transmission method. Figure 5 As shown, the computer terminal 50 may include one or more (502a, 502b, ..., 502n are shown in the figure) processors 502 (the processor 502 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 504 for storing data, and a transmission module 506 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 5 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 5 More or fewer components than shown, or with Figure 5 Different configurations shown.

[0159] It should be noted that the one or more processors 502 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 50. As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0160] The memory 504 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data transmission method in the embodiment of the present application. The processor 502 executes various functional applications and data processing by running the software programs and modules stored in the memory 504, that is, implementing the above-mentioned data transmission method. The memory 504 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 504 may further include a memory remotely located relative to the processor 502, and these remote memories may be connected to the computer terminal 50 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The transmission module 506 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 50. In one embodiment, the transmission module 506 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission module 506 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0162] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 50 .

[0163] It should be noted that, in some optional embodiments, the above Figure 5 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 5 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0164] It should be noted that Figure 5 The computer terminal shown is used to execute Figure 1 The data transmission method shown, therefore the relevant explanations in the execution method of the above command are also applicable to the electronic device and will not be repeated here.

[0165] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above data transmission method.

[0166] A program for a non-volatile storage medium to perform the following functions: a sending end obtains a local voice data stream, uses a pre-trained voice recognition model to convert the voice data stream into characters word by word, and uses a pre-trained text error correction model to perform error correction on the characters to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; the sending end determines different transmission modes used by the sending end and the receiving end for data transmission based on a target fault tolerance rate of a packet loss rate when the sending end and the receiving end perform data transmission, wherein the target fault tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; under different transmission modes, the first text data stream is encoded using different encoding methods to obtain an encoding result; the sending end receives the character sequence number of the missing character sent by the receiving end, the second text data stream, and each second character in the second text data stream. a corresponding second character sequence number; determining the character sequence number of the missing character at the sending end according to the second text data stream and the second character sequence number; generating a data packet to be sent according to the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the missing character at the receiving end, and the character sequence number of the missing character at the sending end; the sending end encapsulates the encoding result into the data packet to be sent, obtains the data packet, and sends the data packet to the receiving end; the receiving end decodes the encoded content in the data packet to obtain a text data stream; uses a pre-trained text error correction model to correct the text data stream, and uses a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, converts the processed text data stream into a voice data stream, and outputs the voice data stream through the voice output module.

[0167] An embodiment of the present application further provides an electronic device, comprising: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the above data transmission method is executed when the program is run.

[0168] The processor is used to run a program that performs the following functions: the sending end obtains a local voice data stream, uses a pre-trained voice recognition model to convert the voice data stream into characters word by word, and uses a pre-trained text error correction model to perform error correction on the characters to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; the sending end determines different transmission modes used by the sending end and the receiving end for data transmission based on a target fault tolerance rate of a packet loss rate when the sending end and the receiving end perform data transmission, wherein the target fault tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; under different transmission modes, different encoding methods are used to encode the first text data stream to obtain an encoding result; the sending end receives the character sequence number of the missing character of the receiving end sent by the receiving end, the second text data stream, and each second character sequence number in the second text data stream. the corresponding second character sequence number; according to the second text data stream and the second character sequence number, determining the character sequence number of the missing character at the sending end; generating a data packet to be sent according to the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the missing character at the receiving end, and the character sequence number of the missing character at the sending end; the sending end encapsulates the encoding result into the data packet to be sent, obtains the data packet, and sends the data packet to the receiving end; the receiving end decodes the encoded content in the data packet to obtain a text data stream; uses a pre-trained text error correction model to correct the text data stream, and uses a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, converts the processed text data stream into a voice data stream, and outputs the voice data stream through the voice output module.

[0169] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0170] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0171] In the above-mentioned embodiments of the present application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary protection measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0173] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0174] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0176] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data transmission method, characterized in that: include: The transmitting end obtains a local voice data stream, converts the voice data stream into characters word by word using a pre-trained voice recognition model, and performs error correction processing on the characters using a pre-trained text error correction model to obtain a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; The transmitting end determines, based on a target error tolerance rate of a packet loss rate during data transmission between the transmitting end and the receiving end, different transmission modes adopted by the transmitting end and the receiving end for data transmission, wherein the target error tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; and encoding the first text data stream using different encoding methods under different transmission modes to obtain encoding results; The transmitting end receives the character sequence number of the receiving end's missing character, the second text data stream, and the second character sequence number corresponding to each second character in the second text data stream, sent by the receiving end; determines the character sequence number of the sending end's missing character based on the second text data stream and the second character sequence number; and generates a data packet to be sent based on the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the receiving end's missing character, and the character sequence number of the sending end's missing character; The transmitting end encapsulates the encoding result into the data packet to be sent to obtain a data packet, and sends the data packet to the receiving end; The receiving end decodes the encoded content in the data packet to obtain a text data stream; uses a pre-trained text error correction model to correct the text data stream, and uses a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on the pre-stored default voiceprint features or the voiceprint features of the target object, the processed text data stream is converted into a voice data stream, and the voice data stream is output through a voice output module.

2. The method according to claim 1, characterized in that Determining different transmission modes adopted by the transmitting end and the receiving end for data transmission according to a target error tolerance rate of a packet loss rate during data transmission between the transmitting end and the receiving end includes: Get the pre-configured initial packet loss rate tolerance; Real-time detection of a packet loss rate during data transmission between the transmitting end and the receiving end; if the packet loss rate is increasing within a preset period, increasing the initial packet loss rate tolerance to obtain the target fault tolerance; if the packet loss rate is decreasing within the preset period, decreasing the initial packet loss rate tolerance to obtain the target fault tolerance; Obtain a target fault tolerance rate at a target moment. When the target fault tolerance rate is less than a first preset threshold, determine to adopt a first transmission mode for data transmission with the receiving end; when the target fault tolerance rate is greater than the first preset threshold, determine to adopt a second transmission mode for data transmission with the receiving end.

3. The method according to claim 2, characterized in that Under different transmission modes, encoding the first text data stream using different encoding methods to obtain encoding results includes: Creating a text queue, wherein the text queue includes: a first queue, a second queue, and a third queue, the first queue being used to store character information in the first text data stream, the second queue being used to store character information of missing characters on the receiving end, and the third queue being used to store character sequence numbers of missing characters on the sending end; The first queue and the second queue both include: a character sequence number, a character code, a latest character sending time, G character code slices corresponding to the character code, a latest character code slice sending time, and a sequence number of a character code slice to be sent, where G is a positive integer greater than 1; Obtaining an uncoded target element combination in the first queue, wherein the target element combination includes: a target character sequence number, a target character code, and a latest sending time of the target character; In the case of adopting the first transmission mode to transmit data with the receiving end, processing the target character code in the target element combination using a forward error correction coding algorithm to obtain a first character code slice set, wherein the first character code slice set includes G character code slices, each of which is a coding segment of the target character code; adding the first character code slice set to the target element combination; and Determining the latest sending time of each character code slice in the first character code slice set based on the target time, the latest sending time of the target character, G, and K, to obtain a first character code slice latest sending time set, where K is a positive integer not greater than G; adding the first character code slice latest sending time set to the target element combination to obtain a first encoding result; In the case of using the second transmission mode to perform data transmission with the receiving end, copying the target character code in the target element combination to obtain a second character code slice set, wherein the second character code slice set includes G target character codes; adding the second character code slice set to the target element combination; and According to the target time, the latest sending time of the target character, G and K, determine the latest sending time of each character code slice in the second character code slice set, obtain the second character code slice latest sending time set, add the second character code slice latest sending time set to the target element combination, and obtain the second encoding result.

4. The method according to claim 3, characterized in that Before obtaining the unencoded target element combination in the first queue, the method further includes: Obtain a target triple corresponding to a target character in the text data stream at time T, wherein the target triple includes: a target character sequence number, a target character code, and the target time; T is the target time, and T is a positive integer greater than 1; Obtaining the latest sending time of the character code slice at time T-1 in the first queue, and determining the latest sending time of the target character according to the latest sending time of the character code slice at time T-1, a preconfigured time sliding window, and the target time; Add the target character sequence number, the target character encoding, the latest sending time of the target character, the target character encoding slice, the latest sending time of the target character encoding slice, and the target sequence number to the first queue, wherein the target character encoding slice and the latest sending time of the target character encoding slice are empty arrays, and the target sequence number is 0.

5. The method according to claim 3, characterized in that The transmitting end encapsulates the encoding result into the data packet to be sent to obtain a data packet, and sends the data packet to the receiving end, including: Step S1, determining the transmission mode adopted by the transmitting end and the receiving end for data transmission as the first field of the data to be sent; determining the number of characters of missing characters and the number of retransmitted characters of the receiving end as the second field of the data to be sent; determining the number of characters of missing characters of the transmitting end as the third field of the data to be sent; determining the character sequence number of the starting character of the first text data stream as the fourth field of the data to be sent; determining the first text data stream as the fifth field of the data to be sent; determining the character sequence number of the missing characters of the receiving end as the sixth field of the data to be sent; determining the character sequence number of the missing characters of the transmitting end as the seventh field of the data to be sent; Step S2, determining the maximum length corresponding to the data packet to be sent; Step S3: From the first queue, starting from the first position, sequentially extract the first character code slice whose latest sending time is within the first interval, wherein the first position is the position of the first character among multiple characters that are not completely sent at the target time in the first queue, the first interval is from the target time to the first time, and the first time is the sum of the target time and a preconfigured packet sending interval; add the first character code slice to the fifth field, and determine the first sequence number of the character corresponding to the first character code slice and the second sequence number of the character corresponding to the last character code slice in the fifth field; Step S4: From the second queue, starting with the first character code to be sent, sequentially extract the second character code slices whose latest character sending time is within a second interval, where the second interval is from the target time to a second time, and the second time is the next adjacent time to the target time; and add the second character code slices to the sixth field. Step S5, taking out all the character serial numbers from the third queue, adding all the character serial numbers to the seventh field, and deleting all the character serial numbers from the third queue; Step S6: taking out the third character code slice after the second sequence number from the first queue, and adding the third character code slice to the fifth field; Step S7: Take out the fourth character code slice before the first sequence number from the first queue, and add the fourth character code slice to the fifth field; Step S8, after all steps S3 to S7 are executed or the length of the data packet to be sent reaches the maximum length, obtaining the data packet and adding the data packet to a queue to be sent; Step S9: based on the preconfigured packet sending interval, obtain the data packet from the queue to be sent, and send the data packet to the receiving end.

6. The method according to claim 5, characterized in that Determining the maximum length of the data packet to be sent includes: Obtaining a preconfigured initial bit rate; detecting in real time a packet loss rate during data transmission between the transmitting end and the receiving end, and if the packet loss rate is increasing within a preset period, reducing the initial bit rate to obtain a target bit rate; detecting in real time the amount of text in a data cache area relied upon for data transmission between the transmitting end and the receiving end, and if the amount of text is greater than a preset second threshold, increasing the initial bit rate to obtain a target bit rate; The maximum length of the data packet to be sent is determined according to the target bit rate, the preconfigured packet sending interval, and the protocol packet header length in the preconfigured rule.

7. The method according to claim 5, characterized in that The receiving end decodes the encoded content in the data packet to obtain a text data stream, including: Obtain the text queue; Taking the character sequence number of the starting character in the data packet as the starting point, sequentially parsing the character code slices in the retransmitted characters or the character code slices in the continuous characters in the data packet; When the first field indicates that a transmission mode adopted by the transmitting end and the receiving end for data transmission is a first transmission mode, if the number of character code slices obtained by parsing is G / 2+1, decoding a character code from the (G / 2+1) character code slices using a forward error correction coding algorithm to obtain a first decoding result, wherein the first decoding result includes the character code and a character sequence number corresponding to the character code; When the first field indicates that the transmission mode adopted for data transmission between the sending end and the receiving end is the second transmission mode, the character code is decoded from any character code slice to obtain a second decoding result, wherein the second decoding result includes the character code and the character sequence number corresponding to the character code.

8. The method according to claim 7, characterized in that The method further comprises: Parsing the missing characters in the data packet, determining a triplet corresponding to the missing characters in the first queue, wherein the triplet includes: a character sequence number, a character code, and a latest character sending time; and adding the triplet corresponding to the missing characters to the second queue; When the decoded character code is the character code corresponding to the missing character in the third queue, the missing character in the third queue is deleted.

9. A data transmission device, characterized in that: include: an acquisition module, configured to control the transmitting end to acquire a local voice data stream, convert the voice data stream word by word into characters using a pre-trained voice recognition model, and perform error correction processing on the characters using a pre-trained text error correction model, thereby obtaining a first text data stream and a first character sequence number corresponding to each first character in the first text data stream; an encoding module, configured to control the transmitting end to determine different transmission modes adopted by the transmitting end and the receiving end for data transmission based on a target error tolerance rate of a packet loss rate during data transmission between the transmitting end and the receiving end, wherein the target error tolerance rate is a parameter periodically adjusted according to changes in the packet loss rate; and to encode the first text data stream using different encoding methods under different transmission modes to obtain encoding results; a generating module configured to control the transmitting end to receive the character sequence number of the receiving end's missing character, the second text data stream, and the second character sequence number corresponding to each second character in the second text data stream, sent by the receiving end; determine the character sequence number of the sending end's missing character based on the second text data stream and the second character sequence number; and generate a data packet to be sent based on the character sequence number of the starting character of the first text data stream, the first text data stream, the character sequence number of the receiving end's missing character, and the character sequence number of the sending end's missing character; An encapsulation module, configured to control the transmitting end to encapsulate the encoding result into the data packet to be sent, obtain a data packet, and send the data packet to the receiving end; A decoding module is used to control the receiving end to decode the encoded content in the data packet to obtain a text data stream; use a pre-trained text error correction model to correct the text data stream, and use a pre-trained text prediction model to predict and fill in the missing text in the text data stream; based on pre-stored default voiceprint features or the voiceprint features of the target object, convert the processed text data stream into a voice data stream, and output the voice data stream through a voice output module.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the data transmission method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the data transmission method according to any one of claims 1 to 8 is executed when the program is run.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the data transmission method according to any one of claims 1 to 8 is implemented.

Images