A data processing method and related device
By identifying the packet loss scenario type and generating compensation packets, the problem of poor robustness of the waveform compensation method in the prior art is solved, and better speech recovery effect and adaptability are achieved.
Patent Information
- Application Number
- CN201910980635.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-10-14
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-10-14
AI Technical Summary
The existing waveform compensation method is poorly robust in packet loss scenarios, making it difficult to adapt to various packet loss scenarios, resulting in poor voice recovery effect.
By determining the packet loss scenario type and generating compensation packets to compensate for the lost data packets when specific conditions are met, the lost data is directly used to recover the lost data, so as to achieve data recovery effect that is better than traditional PLC methods.
It improves adaptability in various packet loss scenarios, significantly improves voice recovery effect, and reduces complexity.
Smart Images

Figure CN112669858B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology, and in particular to a data processing method and related devices. Background Art
[0002] In a communication system, to achieve voice transmission between two devices, the sending device usually needs to encode the voice to obtain a voice packet, then transmit the voice packet to the receiving device through the network element device in the communication system, and finally the receiving device decodes the voice packet to obtain the voice.
[0003] During the transmission of voice packets, voice packets may be lost due to network congestion and other problems. At present, in the case of voice packet loss, it is usually necessary to compensate the waveform of the lost voice packet by a waveform compensation method during the decoding of the voice packet by the receiving device, so as to ensure the continuity of the voice waveform. Current waveform compensation methods, such as packet loss concealment (PLC) methods, mainly reconstruct the waveform of the lost voice packet based on multiple voice packets before the lost voice packet.
[0004] However, in some packet loss scenarios, for example, when the waveform corresponding to the lost voice packet is significantly different from the waveforms of the previous multiple voice packets, the waveform compensated by PLC will be significantly distorted, causing the sound quality to degrade and the voice recovery effect to be poor. Therefore, the waveform compensation method currently used has poor robustness and is difficult to adapt to various packet loss scenarios. Summary of the invention
[0005] The embodiment of the present application provides a data processing method and related devices, which determine the type of packet loss scenario and then generate a compensation data packet to compensate for the lost data packet when the packet loss scenario type meets the data packet compensation condition. Because in the scenario where the data packet compensation condition is met, the data directly recovered by the compensation data packet is closer to the lost data, it is possible to achieve a data recovery effect that is better than the PLC method with lower complexity, thereby improving adaptability in various different packet loss scenarios.
[0006] In view of this, the first aspect of the present application provides a data processing method, comprising:
[0007] During the transmission of data packets, when it is detected that a first data packet is lost, the packet loss scenario type corresponding to the first data packet is determined; if the packet loss scenario type corresponding to the first data packet meets the data packet compensation condition, a second data packet is generated based on the data packets adjacent to the first data packet; finally, the second data packet is added to the target position corresponding to the first data packet before it was lost, that is, the second data packet is directly used to compensate for the lost first data packet.
[0008] In an embodiment of the present application, by determining the packet loss scenario type corresponding to the lost data packet, and then generating a compensation data packet to compensate for the lost data packet when the packet loss scenario type satisfies the data packet compensation condition, because in the scenario where the data packet compensation condition is met, the data directly recovered by the compensation data packet is closer to the lost data, it is possible to achieve a data recovery effect that is better than the PLC method with lower complexity, thereby improving adaptability in various different packet loss scenarios.
[0009] Optionally, in some possible implementations, the packet loss scenario type may include a first packet loss scenario, the data packet compensation condition may include a first compensation condition, and the first packet loss scenario satisfies the first compensation condition; if the packet loss scenario type corresponding to the first data packet satisfies the data packet compensation condition, generating a second data packet based on a data packet adjacent to the first data packet, including: if the packet loss scenario type corresponding to the first data packet satisfies the first compensation condition, that is, the packet loss scenario type corresponding to the first data packet is the first packet loss scenario, then generating a second data packet based on a data packet adjacent to the first data packet.
[0010] In an embodiment of the present application, by determining, based on the packet loss scenario type corresponding to the lost data packet, that a data packet adjacent to the lost data packet is used to compensate for the lost data packet, it is possible to achieve a data recovery effect that is better than the PLC method with lower complexity in a scenario where a data packet before the lost data packet is closer to the lost data packet, thereby improving adaptability in various different packet loss scenarios.
[0011] Optionally, in some possible implementations, the packet loss scenario type may include a second packet loss scenario, the data packet compensation condition may include a second compensation condition, and the second packet loss scenario satisfies the second compensation condition; if the packet loss scenario type corresponding to the first data packet satisfies the data packet compensation condition, generating a second data packet based on a data packet adjacent to the first data packet, including: if the packet loss scenario type corresponding to the first data packet satisfies the second compensation condition, that is, the packet loss scenario type corresponding to the first data packet is the second packet loss scenario, then generating a second data packet based on a data packet adjacent to the first data packet.
[0012] In an embodiment of the present application, by determining, based on the packet loss scenario type corresponding to the lost data packet, a data packet adjacent to the lost data packet to compensate for the lost data packet, it is possible to achieve a data recovery effect better than the PLC method with lower complexity in a scenario where a data packet after the lost data packet is closer to the lost data packet, thereby improving adaptability in various different packet loss scenarios.
[0013] Optionally, in some possible implementations, the packet loss scenario type includes a third packet loss scenario, and the third packet loss scenario does not meet the data packet compensation condition; the method also includes: if the packet loss scenario type does not meet the data packet compensation condition, that is, the packet loss scenario type is the third packet loss scenario, then the waveform of the first data packet is reconstructed by the PLC method, thereby achieving recovery of the first data packet.
[0014] In an embodiment of the present application, when it is determined that the packet loss scenario type corresponding to the lost data packet is a scenario that does not meet the data packet compensation conditions, that is, the effect of compensating the lost data packet using a data packet adjacent to the lost data packet is worse than the effect of recovering the lost data packet using the PLC method, the PLC method is used to achieve recovery of the lost data packet, thereby ensuring the recovery effect of the lost data packet.
[0015] Optionally, in some possible implementations, when the loss of a first data packet is detected, determining a packet loss scenario type corresponding to the first data packet includes: when the loss of the first data packet is detected, determining a packet loss scenario type corresponding to the first data packet based on associated data packets corresponding to the first data packet, wherein the associated data packets may be M adjacent data packets before the first data packet, or the associated data packets may be N adjacent data packets after the first data packet, or the associated data packets may also be M adjacent data packets before the first data packet and N adjacent data packets after the first data packet, M is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.
[0016] In the embodiment of the present application, by using associated data packets corresponding to the lost data packets to realize the identification of the packet loss scenario type, the recognition rate of the packet loss scenario type can be improved, thereby ensuring the recovery effect of the lost data packets in the subsequent process.
[0017] Optionally, in some possible implementations, the method further includes: obtaining an associated data packet corresponding to the first data packet; obtaining encoding parameters of the associated data packet; determining a packet loss scenario type based on the associated data packet, including: inputting the encoding parameters of the associated data packet into a pre-trained packet loss scenario judgment model; and then determining the packet loss scenario type corresponding to the first data packet based on an output result of the packet loss scenario judgment model.
[0018] In an embodiment of the present application, by adopting a pre-trained packet loss scenario judgment model and encoding parameters of associated data packets to realize the recognition of packet loss scenario types, the recognition rate and recognition speed of packet loss scenario types can be improved, thereby ensuring the recovery effect of lost data packets in subsequent processes.
[0019] Optionally, in some possible implementations, the first data packet and the corresponding associated data packet may be voice data packets; before the encoding parameters of the associated data packet are input into the packet loss scenario judgment model, the method further includes: first obtaining a sample voice packet segment, the sample voice packet segment including multiple first voice packets, and at least one second voice packet in the sample voice packet segment has been removed, that is, the sample voice packet segment is a voice packet segment in which the second voice packet has been lost; then generating a sample to be trained based on the sample voice packet segment, wherein the sample to be trained includes the sample voice packet segment and a packet loss scenario label corresponding to the sample voice packet segment; finally, training the classification model using the sample to be trained to obtain a packet loss scenario judgment model, that is, the trained classification model.
[0020] In an embodiment of the present application, a sample voice packet segment in which voice packets are lost is used to train a classification model, thereby obtaining a packet loss scene judgment model. By using a large number of samples to train the model, the recognition rate of the packet loss scene judgment model for the packet loss scene can be improved, thereby ensuring the recovery effect of lost data packets in subsequent processes.
[0021] Optionally, in some possible implementations, generating samples to be trained based on sample voice packet fragments may specifically include: performing packet compensation on a second voice packet by using a first voice packet adjacent to the second voice packet before the second voice packet to obtain a first restored voice; performing packet compensation on the second voice packet by using a first voice packet adjacent to the second voice packet after the second voice packet to obtain a second restored voice; performing voice recovery on the sample voice packet fragment by using a PLC method to obtain a third restored voice; obtaining a first voice quality assessment value corresponding to the first restored voice, a second voice quality assessment value corresponding to the second restored voice, and a third voice quality assessment value corresponding to the third restored voice; and determining a packet loss scenario label corresponding to the sample voice packet fragment based on the first voice quality assessment value, the second voice quality assessment value, and the third voice quality assessment value.
[0022] In an embodiment of the present application, different voice packet compensation methods are used to compensate for lost voice packets, and the packet loss scene labels of sample voice packet segments are determined based on the voice quality corresponding to the recovered voices obtained based on the different voice packet compensation methods. This can effectively improve the accuracy of the packet loss scene labels corresponding to the sample voice packet segments, thereby improving the recognition rate of the packet loss scene by the obtained packet loss scene judgment model.
[0023] Optionally, in some possible implementations, determining the packet loss scenario label corresponding to the sample voice packet segment according to the first voice quality evaluation value, the second voice quality evaluation value and the third voice quality evaluation value may include: if the first voice quality evaluation value is greater than the second voice quality evaluation value and the third voice quality evaluation value, marking the packet loss scenario of the sample voice packet segment as the first packet loss scenario; if the second voice quality evaluation value is greater than the first voice quality evaluation value and the third voice quality evaluation value, marking the packet loss scenario of the sample voice packet segment as the second packet loss scenario; if the third voice quality evaluation value is greater than the first voice quality evaluation value and the second voice quality evaluation value, marking the packet loss scenario of the sample voice packet segment as the first packet loss scenario.
[0024] In an embodiment of the present application, after using different speech recovery methods to perform speech recovery on a sample voice packet segment, the packet loss scene label of the sample voice packet segment is determined by comparing the speech qualities between the recovered speech obtained by different speech recovery methods. This can effectively improve the accuracy of the packet loss scene label corresponding to the sample voice packet segment, thereby improving the recognition rate of the packet loss scene by the obtained packet loss scene judgment model.
[0025] Optionally, in some possible implementations, obtaining a first voice quality assessment value corresponding to the first restored speech, a second voice quality assessment value corresponding to the second restored speech, and a third voice quality assessment value corresponding to the third restored speech includes: obtaining a first mean opinion score (MOS) value corresponding to the first restored speech, the first MOS value being the first voice quality assessment value; obtaining a second MOS value corresponding to the second restored speech, the second MOS value being the second voice quality assessment value; obtaining a third MOS value corresponding to the third restored speech, the third MOS value being the third voice quality assessment value.
[0026] In the embodiment of the present application, the speech restoration quality corresponding to the restored speech is determined by determining the MSO value corresponding to the restored speech, which can effectively improve the flexibility of the solution.
[0027] The second aspect of the present application provides a data processing device, which has the function of implementing the data processing method of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions.
[0028] A third aspect of an embodiment of the present application provides a data processing device, including:
[0029] A processor, a memory and an input-output interface, wherein the processor and the memory are connected to the input-output interface; the memory is used to store program code; when the processor calls the program code in the memory, the steps of the method provided in the first aspect of the present application or any embodiment of the first aspect are executed.
[0030] A fourth aspect of the embodiments of the present application provides a base station, which may include:
[0031] A processor, a memory and an input-output interface, wherein the processor and the memory are connected to the input-output interface; the memory is used to store program code; when the processor calls the program code in the memory, the steps of the method provided in the first aspect of the present application or any embodiment of the first aspect are executed.
[0032] A fifth aspect of the embodiment of the present application provides a core network device, which may include:
[0033] A processor, a memory and an input-output interface, wherein the processor and the memory are connected to the input-output interface; the memory is used to store program code; when the processor calls the program code in the memory, the steps of the method provided in the first aspect of the present application or any embodiment of the first aspect are executed.
[0034] A sixth aspect of the embodiments of the present application provides a terminal, which may include:
[0035] A processor, a memory and an input-output interface, wherein the processor and the memory are connected to the input-output interface; the memory is used to store program code; when the processor calls the program code in the memory, the steps of the method provided in the first aspect of the present application or any embodiment of the first aspect are executed.
[0036] In a seventh aspect of the embodiments of the present application, a data processing device is provided, which can be applied to a base station, a core network device, a terminal or other equipment, and the data processing device is coupled to a memory, and is used to read and execute instructions stored in the memory, so that the data processing device implements the steps of the method provided in the first aspect of the present application or any embodiment of the first aspect. In one possible design, the data processing device is a chip or a system on a chip.
[0037] An eighth aspect of an embodiment of the present application provides a chip system, which includes a processor for supporting a base station, a core network device or a terminal to implement the functions involved in the above aspects, for example, for processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the base station, the core network device or the terminal. The chip system can be composed of chips, or it can include chips and other discrete devices.
[0038] Among them, the processor mentioned in any of the above places can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the data search method of the first aspect mentioned above.
[0039] The ninth aspect of the embodiments of the present application provides a storage medium. It should be noted that the technical solution of the embodiments of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium for storing computer software instructions used for the above-mentioned device, which includes a program designed for executing any optional implementation method of the first aspect above for a data processing device, such as a base station, a core network device or a terminal.
[0040] The storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0041] The tenth aspect of the embodiments of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute a method as described in the first aspect of the present application or any optional implementation of the first aspect.
[0042] According to an eleventh aspect of the present application, a communication system is provided, which may include a base station, a core network device, and a terminal;
[0043] The base station may be the base station provided by the fourth aspect of the embodiment of the present application;
[0044] The core network device may be the core network device provided by the fifth aspect of the embodiment of the present application;
[0045] The terminal may be the terminal provided in the sixth aspect of the embodiment of the present application.
[0046] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
[0047] The embodiment of the present application provides a data processing method and related devices, which determine the type of packet loss scenario and then generate a compensation data packet to compensate for the lost data packet when the packet loss scenario type meets the data packet compensation condition. Because in the scenario where the data packet compensation condition is met, the data directly recovered by the compensation data packet is closer to the lost data, it is possible to achieve a data recovery effect that is better than the PLC method with lower complexity, thereby improving adaptability in various different packet loss scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A schematic diagram of an application scenario of the data processing method provided in an embodiment of the present application;
[0049] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;
[0050] Figure 3a An example diagram of an application of the data processing method provided in an embodiment of the present application;
[0051] Figure 3b Another example diagram of application of the data processing method provided in the embodiment of the present application;
[0052] Figure 3c Another example diagram of application of the data processing method provided in the embodiment of the present application;
[0053] Figure 3d Another example diagram of application of the data processing method provided in the embodiment of the present application;
[0054] Figure 3e Another example diagram of application of the data processing method provided in the embodiment of the present application;
[0055] Figure 4 Another schematic diagram of a data processing method according to an embodiment of the present invention;
[0056] Figure 5 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0057] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0058] Figure 7 A schematic diagram of the structure of a base station or core network device provided in an embodiment of the present application;
[0059] Figure 8 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION
[0060] The embodiments of the present application provide a data processing method and related devices for achieving a data recovery effect that is better than the PLC method with lower complexity, and improving adaptability in various packet loss scenarios.
[0061] The data processing method provided in the present application can be applied to various communication systems or communication networks, for example, fifth generation mobile communication technology (5G) system, new radio (NR) communication system, long term evolution (LTE) system, global system for mobile communication (GSM) or code division multiple access (CDMA) network, wideband code division multiple access (WCDMA) network, etc., and can also be other communication networks or communication systems such as world-wide interoperability for microwave access (WiMAX) or wireless local area network (WLAN).
[0062] For example, the specific application scenarios of the embodiments of the present application can be as follows: Figure 1 Specifically, the application scenario may include one or more terminals, one or more base stations, or one or more core network devices. Figure 1 As shown, the terminal can access the base station, and the base station can access the core network device. In the process of uplink data transmission from the terminal to the base station (for example Figure 1 In the process of transmitting data from terminal 1 to base station 1 uplink), if data packet loss occurs, the base station can use the data processing method provided in the embodiment of the present application to process the data to achieve data recovery. It can also be in the process of forwarding data from the base station to the core network device (for example Figure 1 In the process of forwarding data from the core network device to the base station 1, if data packet loss occurs, the core network device can use the data processing method provided in the embodiment of the present application to process the data to achieve data recovery. Figure 1The core network device in the middle forwards data to base station 2, where base station 1 and base station 2 can be the same base station or different base stations). If data packet loss occurs, the base station can use the data processing method provided in the embodiment of the present application to process the data to achieve data recovery. It can also be in the process of downlink data transmission from the base station to the terminal (for example Figure 1 In the case where the base station 2 transmits data downlink to the terminal 2, if data packet loss occurs, the terminal can use the data processing method provided in the embodiment of the present application to process the data to achieve data recovery.
[0063] Therefore, the data processing method provided in the embodiment of the present application can be executed by various data processing devices, and the data processing device can be a network device such as a base station, a core network device or a terminal, or the data processing device can be included in a network device such as a base station, a core network device or a terminal. The base station can be various forms of macro base stations, micro base stations (also called small stations), relay stations, access points, etc. In different communication systems, the name of the base station may also be different. For example, the base station can be a base transceiver station (BTS) in a GSM or CDMA network, a NB (NodeB) in a WCDMA, a long-term evolution node (evolutional nodeB, eNB or eNodeB) in an LTE system, or a base station device in a 5G network or a communication device in a future evolved public land mobile network (PLMN) network, for example, a 5G base station (next generation NodeB, gNB). The terminal can be various handheld devices including communication functions, wearable devices, computing devices, or other processing devices connected to a wireless modem, etc. For example, it can be a mobile station (MS), a subscriber unit, a cellular phone, a smart phone, a wireless data card, a personal digital assistant (PDA), a tablet computer, a wireless modem, a handset, a laptop computer, a machine type communication (MTC) terminal, etc.
[0064] It should be understood that the data processing method provided in the embodiments of the present application can be executed by the various independent data processing devices mentioned above, or data processing devices in devices such as base station terminals. The specific process of the data processing method provided in the embodiments of the present application is described in detail below based on the data processing device.
[0065] like Figure 2 As shown, Figure 2 The data processing method provided in the embodiment of the present application is a flowchart of the data processing method provided in the embodiment of the present application, which may include:
[0066] 201. When a first data packet is detected to be lost, determine a packet loss scenario type corresponding to the first data packet;
[0067] Usually, during the data transmission process, the network device sending the data will divide the data into several small pieces, and each small piece of data is used as a transmission unit to send the data. Such a transmission unit is often called a data packet, that is, the network device receiving the data can usually receive data in the form of several data packets. However, during the data transmission process, due to the influence of various factors such as the hardware of the network device or the network environment, data packet loss may occur, that is, the data packets received by the network device receiving the data are not continuous and complete, but there may be one or more data packets that are not received. Therefore, in the case of data packet loss, in order to ensure the integrity of the data received by the network device as much as possible, the network device receiving the data can usually recover the data corresponding to the lost data packet after detecting the lost data packet, thereby ensuring the integrity of the data.
[0068] Because the network device that sends data usually marks each data packet with a serial number in a certain order before sending the data packet, that is, each data packet has a unique serial number, and the serial numbers corresponding to the continuously sent data packets are also continuous, therefore, the network device that receives data can detect whether there is a data packet loss by detecting the serial numbers corresponding to the received data packets. For example, during the data transmission process, the network device that receives data receives 5 data packets, and the serial numbers corresponding to the 5 data packets are 001, 002, 004, 005 and 006 respectively. Obviously, there is a data packet with a serial number of 003 missing between the data packet with a serial number of 002 and the data packet with a serial number of 004, and the network device has not received the data packet with a serial number of 003. If the network device has not received the data packet with a serial number of 003 within a certain period of time, it can be considered that the data packet with a serial number of 003 has been lost, that is, the network device detects that the data packet with a serial number of 003 has been lost.
[0069] In some optional embodiments, the above-mentioned first data packet can specifically be a voice data packet, that is, a data packet used to transmit voice information. For example, during a real-time call between terminals, the sending terminal encodes and compresses the voice signal obtained in real time to obtain a voice data packet for transmission, and then the sending terminal forwards the voice data packet to the receiving terminal through network devices such as base stations and core network devices. The receiving terminal can receive the voice data packet sent by the sending terminal, and obtain the corresponding voice signal after processing the voice data packet. In addition, the first data packet can also be a data packet such as a video data packet or an audio and video data packet, that is, a data packet used to transmit video information or audio and video information. The specific transmission process is similar to the transmission process of the above-mentioned voice data packet, which will not be repeated here. In this embodiment, the first data packet can include but is not limited to data packets such as voice data packets, video data packets, and audio and video data packets. For the sake of convenience, the following will take the first data packet as a voice data packet as an example to describe the data processing method provided in the embodiment of the present application in detail.
[0070] In some optional embodiments, if during the data transmission process, the network device receiving the data detects that the first data packet is lost, the packet loss scenario type corresponding to the first data packet can be determined based on the associated data packet corresponding to the first data packet. Specifically, the associated data packet can be the M data packets adjacent to the first data packet, or the associated data packet can be the N data packets adjacent to the first data packet, or the associated data packet can also be the M data packets adjacent to the first data packet and the N data packets adjacent to the first data packet, wherein M is an integer greater than or equal to 1, and N is an integer greater than or equal to 1, and M and N can be the same integer or different integers, which is not specifically limited here.
[0071] For example, when a network device receiving data receives data packets with sequence numbers of 001, 002, 004, 005, and 006, the packet loss scenario type corresponding to the lost data packet (i.e., the data packet with sequence number 003) can be determined according to the associated data packets corresponding to the lost data packet. The associated data packets can specifically be the M data packets adjacent to the lost data packet, such as a data packet with sequence number 002, or two data packets with sequence numbers 002 and 001; the associated data packets can also be the N data packets adjacent to the lost data packet, such as a data packet with sequence number 004, or two data packets with sequence numbers 004 and 005, or three data packets with sequence numbers 004, 005, and 006; in addition, the associated data packets can also be the M data packets adjacent to the lost data packet and the N data packets adjacent to the lost data packet, such as two data packets with sequence numbers 002 and 004, or five data packets with sequence numbers 001, 002, 004, 005, and 006, etc., which will not be repeated here.
[0072] In addition, in some scenarios of continuous packet loss, the corresponding packet loss scenario type can also be determined according to different associated data packets, that is, in the case where multiple continuous data packets are lost, for multiple lost data packets, the associated data packets corresponding to each lost data packet may be different. For example, when the network device receiving data receives data packets with sequence numbers 001, 002, 006 and 007, it can be determined that the lost data packets are data packets with sequence numbers 003, 004 and 005. For the lost data packet with sequence number 003, its corresponding associated data packet may be the M adjacent data packets before, for example, it may be a data packet with sequence number 002, or two data packets with sequence numbers 002 and 001. For the lost data packet with sequence number 005, its associated data packet may be the N adjacent data packets after, for example, it may be a data packet with sequence number 006, or two data packets with sequence numbers 006 and 007. For the lost data packet with sequence number 004, since the two adjacent data packets of the data packet have been lost (i.e., the two data packets with sequence numbers 003 and 004), the data packet cannot be compensated by data packet compensation. Therefore, the packet loss scenario type corresponding to the data packet can be directly determined as a packet loss scenario that does not require packet compensation, and the data packet will no longer be compensated.
[0073] In some optional embodiments, determining the packet loss scenario type according to the associated data packet corresponding to the first data packet may specifically be to determine the corresponding packet loss scenario type through a packet loss scenario judgment model and coding parameters of the associated data packet. Specifically, after detecting the loss of the first data packet, the associated data packet corresponding to the first data packet may be determined, and the corresponding coding parameters of the associated data packet may be obtained. Wherein, in the case where the associated data packet is a voice data packet, the corresponding voice coding parameters of the associated data packet may be obtained. For example, in the case where the associated data packet is encoded using an algebraic code-excited linear prediction (ACELP) coding scheme, the adaptive codebook index, adaptive codebook gain parameter, and fixed codebook gain parameter of the associated data packet may be obtained. It is worth noting that in the case where the associated data packet is encoded using other voice coding schemes, the corresponding voice coding parameters of the associated data packet under the coding scheme may be obtained, which will not be repeated here. Since each voice data packet has corresponding voice coding parameters, when there are multiple associated data packets, the voice coding parameters of each associated data packet can be obtained to form a parameter list, and then the voice coding parameters in the parameter list are input into a pre-trained packet loss scenario model. Finally, the packet loss scenario type corresponding to the first data packet is determined based on the output result of the packet loss scenario judgment model.
[0074] 202. If the packet loss scenario type satisfies the data packet compensation condition, generate a second data packet according to a data packet adjacent to the first data packet;
[0075] In an embodiment of the present application, after the network device receiving data detects that the first data packet is lost, if it is determined that the packet loss scenario type corresponding to the first data packet satisfies the data packet compensation condition, a second data packet can be generated based on the data packet adjacent to the first data packet. Wherein, the data packet adjacent to the first data packet refers to the data packet that is adjacent to the first data packet in the order of the received data packets. For example, when the network device receives data packets with serial numbers of 001, 002, 004, 005 and 006, the data packet with serial number 003 is the data packet that has been lost (i.e., the first data packet), and the data packet with serial number 002 and the data packet with serial number 004 are adjacent to the data packet that has been lost and have been received. Therefore, the second data packet can be generated based on the data packet with serial number 002 or the data packet with serial number 004.
[0076] In some optional embodiments, generating a second data packet according to a data packet adjacent to a first data packet may be specifically by copying a data packet adjacent to the first data packet, and modifying some parameters of the copied data packet to be the same as the parameters of the first data packet itself, thereby obtaining the second data packet. Specifically, for a data packet, its data structure is usually composed of two parts, namely, a header and a body. The information carried in the header is mainly parameter information related to the data packet, such as information used to declare the data type in the data packet, the size of the data packet, the serial number corresponding to the data packet, and other parameter information; and the information carried in the body is the data content corresponding to the data packet. For example, for a voice data packet, the information carried in the body is the voice data content obtained after encoding processing. Therefore, in this embodiment, after copying the data packet adjacent to the first data packet, some parameter information in the header of the copied data packet can be modified so that the parameter information in the header of the data packet is the same as the parameter information in the header of the first data packet. For example, when a network device receives data packets with serial numbers 001, 002, 004, 005, and 006, it can be determined that the serial number of the lost first data packet is 003. Then, the copied data packet can be obtained by copying the data packet with serial number 002 or the data packet with serial number 004, and then the serial number information carried in the header of the copied data packet is modified to 003, thereby obtaining a second data packet, which can be used to replace the lost first data packet.
[0077] It is worth noting that in some optional embodiments, for voice data packets, in addition to using a sequence number to identify the order between voice data packets, a timestamp may also be used to identify the order between voice data packets. For example, for a voice of several seconds, multiple data packets will be used to send this voice, wherein each data packet carries voice data within a fixed time period, for example, the first data packet carries voice data within 0 to 20ms, the second data packet carries voice data within 20ms to 40ms, and the third data packet carries voice data within 40ms to 60ms, and so on, different data packets carry voice data in different time periods, and a complete voice can be obtained by combining all data packets. In this case, each data packet often carries its corresponding timestamp information on the packet header, for example, the first data packet often carries information with a timestamp of 0 to 20ms, and the second data packet often carries information with a timestamp of 20 to 40ms. Therefore, after obtaining the copied data packet, in addition to changing the sequence number corresponding to the copied data packet, the timestamp corresponding to the data packet can also be modified so that the second data packet finally obtained can be used to replace the lost first data packet. For example, when a network device receives data packets with serial numbers of 001, 002, 004, 005, and 006, it can be determined that the serial number of the lost first data packet is 003. Then, a copied data packet can be obtained by copying the data packet with serial number 002 or the data packet with serial number 004, and then the serial number carried in the header of the copied data packet can be modified to 003. Secondly, when the data packet carries timestamp information, the timestamp information corresponding to the copied data packet can be determined based on the data packet with serial number 002 and the data packet with serial number 004. For example, when the timestamp information corresponding to the data packet with serial number 002 is 20 to 40ms, and the timestamp information corresponding to the data packet with serial number 004 is 60 to 80ms, the timestamp information corresponding to the copied data packet can be determined to be 40 to 60ms. Therefore, the timestamp information corresponding to the copied data packet can be modified to 40 to 60ms, thereby obtaining a second data packet, which can be used to replace the lost first data packet.
[0078] In some optional embodiments, the packet loss scenario types may specifically include a first packet loss scenario, a second packet loss scenario, and a third packet loss scenario, and the data packet compensation conditions may specifically include a first compensation condition and a second compensation condition, wherein the first packet loss scenario satisfies the first compensation condition, and the second packet loss scenario satisfies the second compensation condition, that is, when the packet loss scenario type is the first packet loss scenario or the second packet loss scenario, the data packet compensation condition is met, and when the packet loss scenario type is the third packet loss scenario, the data packet compensation condition is not met.
[0079] When the packet loss scenario type is the first packet loss scenario, the second data packet can be specifically generated based on a data packet adjacent to the first data packet. For example, when the serial number of the lost first data packet is 003, the second data packet is generated based on a data packet adjacent to the first data packet (i.e., a data packet with a serial number of 002). Among them, the first packet loss scenario can specifically be a scenario in which the voice parameter content of the first data packet is relatively close to the voice parameter content of the previous data packet, but is significantly different from the voice parameter content of the subsequent data packet. Therefore, in the first packet loss scenario, generating a second data packet to replace the first data packet based on a data packet adjacent to the first data packet can make the finally recovered voice closer to the original voice, and can obtain a better voice recovery effect.
[0080] When the packet loss scenario type is the second packet loss scenario, the second data packet can be generated specifically based on a data packet adjacent to the first data packet. For example, when the serial number of the lost first data packet is 003, the second data packet is generated based on a data packet adjacent to the first data packet (i.e., a data packet with a serial number of 004). Among them, the second packet loss scenario can specifically be a scenario in which the voice parameter content of the first data packet is relatively close to the voice parameter content of the data packet after it, but is quite different from the voice parameter content of the data packet before it. Therefore, in the second packet loss scenario, generating a second data packet to replace the first data packet based on a data packet adjacent to the first data packet can make the finally recovered voice closer to the original voice, and can obtain a better voice recovery effect.
[0081] When the packet loss scenario type is the third packet loss scenario, if the network device receiving the data is a base station or a core network device, the base station or the core network device does not need to perform packet compensation on the first data packet, and can forward the received data packet in a normal processing manner after the data packet is lost. For example, when the base station or the core network device receives data packets with sequence numbers of 001, 002, 004, 005 and 006, the base station or the core network device first forwards the data packets with sequence numbers of 001 and 002. After waiting for a certain period of time, if the data packet with sequence number 003 is still not received, the data packets with sequence numbers 004 to 006 are continuously forwarded in sequence. In addition, if the network device receiving the data is a terminal, since the terminal needs to decode the received voice data packets to obtain the corresponding voice signals, when the terminal receives the data packets with sequence numbers of 001, 002, 004, 005 and 006, the voice waveform corresponding to the data packet with sequence number 003 can be reconstructed by the PLC method during the decoding process, thereby realizing the waveform compensation of the data packet with sequence number 003.
[0082] 203. Add the second data packet to the target position corresponding to the first data packet before it is lost.
[0083] In this embodiment, after the network device generates the second data packet, the second data packet can be added to the target position corresponding to the first data packet in the entire data packet sequence before the loss of the first data packet, which is equivalent to using the second data packet to replace the first data packet before the loss, thereby achieving packet compensation for the first data packet.
[0084] When the network device is a base station, after adding the second data packet to the target position corresponding to the first data packet before it was lost, the second data packet can be sent to the core network device or terminal in the order in which the data packets were sent. For example, when the network device receives data packets with sequence numbers of 001, 002, 004, 005, and 006, the second data packet can be generated according to the data packet with sequence number 002 or 004, and then the second data packet can be added between the data packet with sequence number 002 and the data packet with sequence number 004, and then the data packets are forwarded to the core network device or terminal in sequence according to the sequence numbers of the received data packets, so that the core network device or terminal can receive the complete sequence of data packets.
[0085] Similarly, when the network device is a core network device, after adding the second data packet to the target position corresponding to the first data packet before it was lost, the second data packet can be sent to the base station in the order in which the data packets were sent, so that the base station can receive a complete sequence of data packets.
[0086] In addition, when the network device is a terminal, after the second data packet is added to the target position corresponding to the first data packet before it is lost, the entire complete data packet sequence can be decoded and processed in the order of the data packets to obtain the corresponding voice data, without the need to reconstruct the waveform of the first data packet through the PLC method.
[0087] It is worth noting that, since voice data packets are related to each other in voice encoding and decoding, the voice waveform obtained after decoding the voice data packets is essentially recovered based on the contents of multiple voice data packets. Therefore, in the embodiment of the present application, although two voice data packets containing the same content are obtained after the second data packet is generated based on the data packet adjacent to the first data packet, the waveform parts corresponding to the two identical voice data packets are actually different in the waveform finally generated. Moreover, in the case where the voice parameter content contained in the lost first data packet and the voice parameter content contained in the second data packet are relatively close, the second data packet is used to implement packet compensation for the first data packet, so that the final voice waveform can be closer to the original voice waveform before packet loss, thereby achieving a good voice recovery effect.
[0088] In addition, since the PLC method mainly reconstructs the waveform of the lost voice packet based on the interpolation filtering of some important parameters of multiple voice packets before the lost voice packet, and then reconstructs the waveform of the lost voice packet based on these parameters and the cached voice waveform, its complexity is relatively high. In the embodiment of the present application, packet compensation is achieved by copying the data packets adjacent to the first data packet to generate a second data packet for replacing the first data packet, and it is not necessary to perform interpolation filtering of some important parameters of multiple voice packets and to achieve reconstruction of the voice waveform based on parameters and audio synthesis, so that voice recovery can be achieved with lower complexity.
[0089] In addition, in some speech coding methods, such as Adaptive Multi-Rate Wideband (AMR), Adaptive Multi-Rate Wideband (AMR-WB) and other coding methods, since the PLC method mainly generates the speech parameters of the lost speech packet by extrapolating and interpolating the corresponding coding parameters of multiple speech packets before the lost speech packet and weakening them to a certain extent, and then restores the lost waveform according to the generated speech parameters. When the speech parameters of the lost speech packet are greatly different from the speech parameters of the multiple speech packets before it, and the speech parameters of the speech packet after it are relatively small, after compensating the lost speech packet by copying a data packet after the lost speech packet, the recovered speech is better than the speech recovered by the PLC method. And, in the case where the parameters of the lost speech packet are relatively close to the speech parameters of the previous speech packet, since the PLC method mainly performs extrapolation and interpolation on the corresponding coding parameters of multiple speech packets before the lost speech packet and weakens them to a certain extent, therefore, after compensating the lost speech packet by directly copying a data packet before the lost speech packet, the recovered speech is also better than the speech recovered by the PLC method.
[0090] In an embodiment of the present application, when it is detected that a first data packet is lost during data transmission, a packet loss scenario type corresponding to the first data packet is determined, and then a compensation data packet is generated to compensate for the lost data packet when the packet loss scenario type satisfies a data packet compensation condition. Because the data directly recovered from the compensation data packet is closer to the lost data in the scenario where the data packet compensation condition is met, it is possible to achieve a data recovery effect that is better than the PLC method with lower complexity, thereby improving adaptability in various packet loss scenarios.
[0091] For ease of understanding, the following will describe in detail the data processing method provided by the embodiment of the present application in conjunction with a specific scenario example diagram. Figures 3a to 3d , Figure 3a An example diagram of an application of the data processing method provided in an embodiment of the present application; Figure 3b Another example diagram of application of the data processing method provided in the embodiment of the present application; Figure 3c Another example diagram of application of the data processing method provided in the embodiment of the present application; Figure 3d Another example diagram of application of the data processing method provided in an embodiment of the present application.
[0092] in, Figure 3a hereinafter represents a scenario in which, when a voice data packet is lost during uplink transmission of a voice data packet by the terminal 1 to the base station 1, the base station 1 executes the data processing method provided in the embodiment of the present application; Figure 3b It represents a scenario in which the core network device executes the data processing method provided in the embodiment of the present application when a voice data packet is lost during the process in which the base station 1 forwards the voice data packet transmitted uplink by the terminal 1 to the core network device; Figure 3c It represents a scenario in which the base station 2 executes the data processing method provided in the embodiment of the present application when a voice data packet is lost during the process of the core network device forwarding the voice data packet to the base station 2 downlink; Figure 3d It represents a scenario in which, when a voice data packet is lost during the process of base station 2 sending a voice data packet downlink to terminal 2, terminal 2 executes the data processing method provided in the embodiment of the present application.
[0093] For the sake of convenience, the following will be Figure 3a Taking the scenario represented in as an example, the data processing method provided in the embodiment of the present application is described in detail.
[0094] During the real-time call between terminal 1 and terminal 2, terminal 1 first encodes and encapsulates the acquired voice signal to obtain a voice data packet, and then terminal 1 sends the voice data packet to base station 1 in a certain order. Base station 1 then delivers the received voice data packet uplink to the core network device. After receiving the voice data packet sent by base station 1, the core network device forwards the voice data packet downlink to base station 2 to which terminal 2 is connected, and base station 2 sends the voice data packet to terminal 2, thereby realizing the transmission of voice data packets between terminal 1 and terminal 2; wherein, base station 1 and base station 2 can be the same base station or different base stations.
[0095] In the process of transmitting voice data packets from terminal 1 to base station 1, if base station 1 detects that voice data packet k transmitted by terminal 1 is lost, the packet loss scenario type corresponding to voice data packet k is determined. Among them, the specific method for base station 1 to detect the loss of voice data packet k transmitted by terminal 1 can be that after base station 1 receives voice data packet k-1 (i.e., the previous voice data packet of voice data packet k) transmitted by terminal 1, if base station 1 waits for a certain period of time and still does not receive voice data packet k, it can be considered that voice data packet k has been lost, so base station 1 can detect the loss of voice data packet k. After detecting that voice data packet k is lost, base station 1 can obtain the voice coding parameters of the associated data packets corresponding to voice data packet k, for example, obtain the voice coding parameters of voice data packets such as voice data packet k-1, voice data packet k-2, voice data packet k+1 and voice data packet k+2, and then input these obtained voice coding parameters into the pre-configured packet loss scenario judgment model, and determine the packet loss scenario type corresponding to voice data packet k according to the output result of the packet loss scenario judgment model.
[0096] If base station 1 determines that the packet loss scenario type corresponding to voice data packet k is the first packet loss scenario, a copied voice data packet is obtained by copying voice data packet k-1, and the serial number information and / or timestamp information in the packet header of the copied voice data packet is modified to information matching voice data packet k, thereby obtaining a new voice data packet k, which can be used to replace the originally lost voice data packet k; finally, base station 1 sends the new voice data packet k and its subsequent voice data packets to the core network device according to the order of the voice data packets, so that the core network device can receive a continuous and complete sequence of voice data packets.
[0097] Similarly, if base station 1 determines that the packet loss scenario type corresponding to voice data packet k is the second packet loss scenario, it obtains a copied voice data packet by copying voice data packet k+1, and modifies the serial number information and / or timestamp information in the packet header of the copied voice data packet to information matching voice data packet k, thereby obtaining a new voice data packet k, which can be used to replace the original lost voice data packet k; finally, base station 1 sends the new voice data packet k and its subsequent voice data packets to the core network device according to the order of the voice data packets, so that the core network device can receive a continuous and complete sequence of voice data packets.
[0098] In addition, if base station 1 determines that the packet loss scenario type corresponding to voice data packet k is the third packet loss scenario, packet compensation will no longer be performed for the lost voice data packet k. After sending voice data packet k-1, base station 1 directly sends voice data packet k+1 and its subsequent voice data packet k+1 to the core network device. That is, the core network device finally receives a voice data packet sequence in which voice data packet k is missing.
[0099] Similarly, in the scenario where the core network device and the base station 2 execute the data processing method provided in the embodiment of the present application, the steps executed by the core network device and the base station 2 are similar to the steps executed by the above-mentioned base station 1, and will not be repeated here. It is worth noting that in the scenario where the terminal 2 executes the data processing method provided in the embodiment of the present application, after the terminal 2 determines that the packet loss scenario type corresponding to the voice data packet k is the third packet loss scenario, the steps executed by the terminal 2 are different from the steps executed by the above-mentioned base station 1, and the other steps are similar to the steps executed by the above-mentioned base station 1. Specifically, if the terminal 2 determines that the packet loss scenario type corresponding to the voice data packet k is the third packet loss scenario, the voice waveform corresponding to the voice data packet k can be reconstructed by the PLC method during the process of decoding the voice data packet by the terminal 2, thereby realizing the waveform reconstruction of the voice data packet k and realizing the recovery of the voice.
[0100] Also, see Figure 3e , Figure 3e Another example diagram of the application of the data processing method provided in the embodiment of the present application. Figure 3e The scene represented in Figure 3a The scenarios shown in are similar to those shown in the figure, both of which show that when a voice data packet is lost during the process of uplink transmission of a voice data packet by the terminal 1 to the base station 1, the base station 1 executes the data processing method provided in the embodiment of the present application; however, Figure 3e and Figure 3a The difference is that Figure 3e It indicates that during the process of uplink transmission of voice data packets by terminal 1 to base station 1, multiple continuous voice data packets are lost.
[0101] In the process of transmitting voice data packets from terminal 1 to base station 1, if base station 1 detects that voice data packet k and voice data packet k+1 transmitted by terminal 1 are lost, the packet loss scenario types corresponding to voice data packet k and voice data packet k+1 are determined respectively. Specifically, firstly, one or more adjacent voice data packets before voice data packet k are determined as associated data packets corresponding to voice data packet k, such as voice data packet k-1 and voice data packet k-2, and then the voice coding parameters of the corresponding associated data packets are obtained, and these obtained voice coding parameters are input into a pre-configured packet loss scenario judgment model, and the packet loss scenario type corresponding to voice data packet k is determined according to the output result of the packet loss scenario judgment model. In addition, one or more adjacent voice data packets after voice data packet k+1 can also be determined as associated data packets corresponding to voice data packet k+1, such as voice data packet k+2 and voice data packet k+3, and then the voice coding parameters of the corresponding associated data packets are obtained, and these obtained voice coding parameters are input into a pre-configured packet loss scenario judgment model, and the packet loss scenario type corresponding to voice data packet k+1 is determined according to the output result of the packet loss scenario judgment model.
[0102] It is worth noting that, since the adjacent voice data packets after voice data packet k have been lost, the adjacent voice data packets after voice data packet k cannot be used to compensate voice data packet k. Therefore, after determining the packet loss scenario type corresponding to voice data packet k, a voice data packet adjacent to voice data packet k can be used to compensate voice data packet k, or voice data packet k can be not compensated. Therefore, the packet loss scenario judgment model can be preprocessed so that when the packet loss scenario judgment model outputs the packet loss scenario corresponding to voice data packet k, it only outputs one of the first packet loss scenario and the third packet loss scenario, and does not output the second packet loss scenario. Similarly, for voice data packet k+1, the adjacent voice data packets before voice data packet k+1 have been lost, and the adjacent voice data packets before voice data packet k+1 cannot be used to compensate voice data packet k+1. Therefore, the packet loss scenario judgment model can also be preprocessed so that when the packet loss scenario judgment model outputs the packet loss scenario corresponding to voice data packet k+1, it only outputs one of the second packet loss scenario and the third packet loss scenario, and does not output the first packet loss scenario. Specifically, after the packet loss scenario type corresponding to the voice data packet is determined by the packet loss scenario judgment model, the steps performed are the same as those described above. Figure 3a The steps described in are similar and will not be repeated here.
[0103] The foregoing is a detailed description of the process of the data processing method provided in the embodiment of the present application. For ease of understanding, the following will describe in detail the training process of the packet loss scenario judgment model used in the foregoing data processing method. Figure 4 , Figure 4 This is another flow chart of the data processing method provided in the embodiment of the present application. The data processing method provided in the embodiment of the present application may also include:
[0104] 401. Obtain a sample voice packet segment;
[0105] In an embodiment of the present application, before training a packet loss scenario judgment model, a large number of sample voice package segments are first obtained. In these sample voice package segments, for each sample voice package segment, the voice packages in the sample voice package segment are not continuous and complete. Specifically, the sample voice package segment may include multiple first voice packages, and at least one second voice package in the sample voice package segment has been removed, that is, the sample voice package segment is a voice package segment that has lost the second voice package. Exemplarily, the method of obtaining the sample voice package segments can be specifically to collect voice segments with lost voice packages from existing voice package data to obtain corresponding sample voice package segments; or it can be to collect continuous and complete voice segments from existing voice package data, and then discard or remove one or more voice packages in these continuous and complete voice segments to obtain corresponding sample voice package segments.
[0106] In some optional embodiments, the speech length corresponding to each sample voice package segment may be 6 to 8 seconds. Of course, in other embodiments, sample voice package segments with longer or shorter speech lengths may also be used, and no limitation is made here. In the case where the speech length corresponding to the sample voice package segment is 6 to 8 seconds, since the speech length corresponding to each voice package is about 0.02 seconds, each sample voice package segment may include 300 to 400 voice packages. In order to ensure the accuracy of the trained packet loss scenario model, in the actual training process, specifically about 100,000 sample voice package segments may be collected to train the packet loss scenario model, wherein the sample voice package segments used for training may be specifically determined according to the needs of the actual application, and no specific limitation is made here.
[0107] 402. Generate a sample to be trained according to the sample voice packet segment, wherein the sample to be trained includes the sample voice packet segment and a packet loss scenario label corresponding to the sample voice packet segment;
[0108] In this embodiment, after obtaining the sample voice packet segment, the packet loss scenario label corresponding to the sample voice packet segment can also be determined, and the sample voice packet segment and its corresponding packet loss scenario label can be used as training samples for training the packet loss scenario judgment model.
[0109] In some optional embodiments, generating samples to be trained based on sample voice package segments can specifically include: by copying a first voice package adjacent to the second voice package in the sample voice package segment to replace the lost second voice package, thereby obtaining a complete and continuous sample voice package segment, and then obtaining the first restored voice corresponding to this sample voice package segment through operations such as decoding processing; similarly, by copying a first voice package adjacent to the second voice package in the sample voice package segment to replace the lost second voice package, thereby obtaining another complete and continuous sample voice package segment, and then obtaining the second restored voice corresponding to the other sample voice package segment through operations such as decoding processing; in addition, the third restored voice corresponding to the sample voice package segment can also be directly obtained through the PLC method; that is, three different restored voices can be obtained through three different compensation methods. Secondly, the voice quality evaluation values corresponding to the first restored voice, the second restored voice, and the third restored voice are respectively obtained, that is, the voice qualities corresponding to the three restored voices are respectively evaluated. The voice quality evaluation values corresponding to the first restored voice, the second restored voice, and the third restored voice can be obtained specifically by MOS scoring, that is, the MOS scores corresponding to the first restored voice, the second restored voice, and the third restored voice are obtained, and then the MOS corresponding to the three restored voices are used as their corresponding voice quality evaluation values. The MOS score is an evaluation standard used in international standards to evaluate the quality of compressed voice. In practical applications, the MOS scores corresponding to the first restored voice, the second restored voice, and the third restored voice can be obtained specifically through corresponding MOS score testing equipment. After obtaining the voice quality evaluation values corresponding to the three restored voices, the restored voice with the largest voice quality evaluation value among the three restored voices can be determined, and then the packet loss scenario label corresponding to the sample voice packet segment can be determined based on the compensation method adopted by this restored voice. For example, when the voice quality evaluation value corresponding to the first restored voice is the largest, the packet loss scenario label corresponding to the sample voice packet segment is determined to be the first packet loss scenario; when the voice quality evaluation value corresponding to the second restored voice is the largest, the packet loss scenario label corresponding to the sample voice packet segment is determined to be the second packet loss scenario; when the voice quality evaluation value corresponding to the third restored voice is the largest, the packet loss scenario label corresponding to the sample voice packet segment is determined to be the third packet loss scenario. In other words, the first packet loss scenario refers to a packet loss scenario that can have the best voice recovery effect after compensating for the lost voice packet by a voice packet adjacent to the previous lost voice packet; the second packet loss scenario refers to a packet loss scenario that can have the best voice recovery effect after compensating for the lost voice packet by a voice packet adjacent to the subsequent lost voice packet; the third packet loss scenario refers to a packet loss scenario that can have the best voice recovery effect after compensating for the lost voice packet by the PLC method.
[0110] 403. Train the classification model using the samples to be trained, thereby obtaining a packet loss scenario judgment model.
[0111] In this embodiment, the process of training the classification model through the sample to be trained can be specifically to use the speech coding parameters corresponding to the sample voice package segment in the sample to be trained as the input of the classification model, and use the packet loss scene label corresponding to the sample voice package segment as the output of the classification model to train the classification model. Among them, the speech coding parameters corresponding to the sample voice package segment specifically refer to the speech coding parameters of the associated voice package of the second voice package in the sample voice package segment.
[0112] In some optional embodiments, the associated voice packets of the second voice packet in the sample voice packet segment may be the M data packets adjacent to the second voice packet, or may be the N data packets adjacent to the second voice packet, or may also be the M data packets adjacent to the second voice packet and the N data packets adjacent to the first data packet, wherein M is an integer greater than or equal to 1, and N is an integer greater than or equal to 1, and M and N may be the same integer or different integers, and are not specifically limited here.
[0113] Usually, for a voice packet, the voice coding parameters contained in a complete voice packet include an adaptive codebook index (4 in each voice packet), a fixed codebook index vector (4 to 32 in each voice packet), an adaptive codebook gain parameter (4 in each voice packet), a fixed codebook gain parameter (4 in each voice packet), and some unique parameters, etc. Specifically, in this embodiment, some parameters in the associated voice packet can be selected to form a detection vector as the input of the classification model. Among them, some parameters in the associated voice packet that can be used to form the detection vector can specifically include parameters such as an adaptive codebook index, an adaptive codebook gain parameter, and a fixed codebook gain parameter. More other parameters can also be selected according to the needs in the actual application process, which will not be repeated here.
[0114] In some optional embodiments, the classification model can be a convolutional neural network (CNN) model, a long short-term memory network (LSTM) model in deep learning, or a traditional machine learning model, such as a random forest, Adaboost, etc. In practical applications, the corresponding classification model can be selected according to specific actual needs, and no specific limitation is made here.
[0115] The above describes in detail the data processing method provided in the embodiment of the present application. The following will describe the data processing device provided in the embodiment of the present application.
[0116] See also Figure 5 , Figure 5 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application; the data processing device provided in an embodiment of the present application may include:
[0117] A determining unit 501 is configured to determine a packet loss scenario type corresponding to the first data packet when a first data packet loss is detected;
[0118] A generating unit 502, configured to generate a second data packet according to a data packet adjacent to the first data packet if the packet loss scenario type satisfies a data packet compensation condition;
[0119] The adding unit 503 is configured to add the second data packet to the target position corresponding to the first data packet before it is lost.
[0120] In some optional embodiments, the packet loss scenario type includes a first packet loss scenario, the data packet compensation condition includes a first compensation condition, and the first packet loss scenario satisfies the first compensation condition; the generation unit 502 is specifically used to generate a second data packet based on a data packet adjacent to the first data packet.
[0121] In some optional embodiments, the packet loss scenario type includes a second packet loss scenario, the data packet compensation condition includes a second compensation condition, and the second packet loss scenario satisfies the second compensation condition; the generation unit 502 is specifically used to generate a second data packet based on a data packet adjacent to the first data packet.
[0122] In some optional embodiments, the packet loss scenario type includes a third packet loss scenario, and the third packet loss scenario does not meet the data packet compensation condition; the device also includes: a reconstruction unit 504, which is used to reconstruct the waveform of the first data packet through a packet loss hiding PLC method if the packet loss scenario type does not meet the data packet compensation condition.
[0123] In some optional embodiments, the determination unit 501 is specifically used to: when a first data packet is detected to be lost, determine the packet loss scenario type corresponding to the first data packet based on associated data packets corresponding to the first data packet, wherein the associated data packets are the M adjacent data packets before the first data packet, or the associated data packets are the N adjacent data packets after the first data packet, or the associated data packets are the M adjacent data packets before the first data packet and the N adjacent data packets after the first data packet, M is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.
[0124] In some optional embodiments, the device also includes: an acquisition unit 505, used to obtain encoding parameters of the associated data packet; a determination unit 501, specifically used to: input the encoding parameters of the associated data packet into a packet loss scenario judgment model; and determine the packet loss scenario type corresponding to the first data packet based on the output result of the packet loss scenario judgment model.
[0125] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application, the data processing device 600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 622 (or other types of processors) and a storage medium 630, the storage medium 630 being used to store one or more application programs 642 or data 644. Among them, the storage medium 630 may be a temporary storage or a permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), each module may include a series of instruction operations in the data processing device. Furthermore, the central processing unit 622 may be configured to communicate with the storage medium 630, and execute a series of instruction operations in the storage medium 630 on the data processing device 600.
[0126] The central processing unit 622 can execute any of the aforementioned method embodiments according to the instruction operation.
[0127] The data processing device 600 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input and output interfaces 658, and / or one or more operating systems 641, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0128] The data processing device provided in this application may include a base station, a core network device, and a terminal.
[0129] Exemplarily, when the data processing device is a base station or a core network device, or the data processing device is included in a base station or a core network device, the structure of the base station or the core network device may be as follows: Figure 7 shown.
[0130] Figure 7A schematic diagram of the structure of a base station or core network device provided in an embodiment of the present application, the base station or core network device 700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 722 (or other types of processors) and a storage medium 730, the storage medium 730 being used to store one or more application programs 742 or data 744. Among them, the storage medium 730 may be a temporary storage or a permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), each module may include a series of instruction operations in the base station. Furthermore, the central processor 722 may be configured to communicate with the storage medium 730, and execute a series of instruction operations in the storage medium 730 on the base station or core network device 700.
[0131] The central processing unit 722 can execute any of the aforementioned method embodiments according to the instruction operation.
[0132] The base station or core network device 700 may also include one or more power supplies 726, one or more wired or wireless network interfaces 750, one or more input and output interfaces 758, and / or one or more operating systems 741, such as Windows Server 2008. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.
[0133] The data processing device provided in the embodiment of the present application may be various terminals, or it may be understood that the data processing device may be included in a terminal, for example, a mobile phone, a tablet computer, a laptop computer, a television, a smart wearable device or other electronic device with a display screen, etc. In the above embodiment, there is no restriction on the specific form of the terminal. Among them, the system that the terminal can carry may include Or other operating systems, etc., the embodiments of the present application do not impose any limitations on this.
[0134] The terminal can be applicable to various communication systems. Specifically, the communication system, for example, is CDMA, TDMA, FDMA, OFDMA, SC-FDMA and other systems. The term "system" can be interchangeable with "network". The CDMA system can implement wireless technologies such as UTRA, CDMA2000, etc. UTRA can include WCDMA technology and other CDMA variant technologies. CDMA2000 can cover interim standard (IS) 2000 (IS-2000), IS-95 and IS-856 standards. The TDMA system can implement wireless technologies such as global system for mobile communication (GSM). The OFDMA system can implement wireless technologies such as evolved universal radio terrestrial access (E-UTRA), ultra mobile broadband (UMB), IEEE 802.11 (Wi-Fi), IEEE 802.7 (WiMAX), IEEE 802.20, Flash OFDMA, etc. UTRA and E-UTRA are UMTS and UMTS evolution versions. 3GPP's long term evolution (LTE) and various versions based on LTE evolution are new versions of UMTS using E-UTRA. In addition, the communication system can also be applied to future-oriented communication technologies, which can be applied to the technical solutions provided in the embodiments of the present application.
[0135] For example, Taking the terminal 100 of the operating system as an example, Figure 8 As shown, the terminal 100 can be logically divided into a hardware layer 21, an operating system 161, and an application layer 31. The hardware layer 21 includes hardware resources such as an application processor 101, a microcontroller unit 103, a modem 107, a Wi-Fi module 111, a sensor 114, a positioning module 150, and a memory. The application layer 31 includes one or more application programs, such as an application 163, which can be any type of application program such as a social application, an e-commerce application, a browser, etc. The operating system 161, as a software middleware between the hardware layer 21 and the application layer 31, is a computer program that manages and controls hardware and software resources.
[0136] In one embodiment, the operating system 161 includes a kernel 23, a hardware abstraction layer (HAL) 25, libraries and runtime 27, and a framework 29. The kernel 23 is used to provide underlying system components and services, such as power management, memory management, thread management, hardware drivers, etc.; hardware drivers include Wi-Fi drivers, sensor drivers, positioning module drivers, etc. The hardware abstraction layer 25 is an encapsulation of the kernel driver, providing an interface to the framework 29 and shielding the low-level implementation details. The hardware abstraction layer 25 runs in the user space, while the kernel driver runs in the kernel space.
[0137] The library and runtime 27 is also called the runtime library, which provides the required library files and execution environment for the executable program at runtime. The library and runtime 27 includes the Android Runtime (ART) 271 and the library 273. ART271 is a virtual machine or virtual machine instance that can convert the bytecode of the application into machine code. The library 273 is a program library that provides support for the executable program at runtime, including a browser engine (such as webkit), a script execution engine (such as a JavaScript engine), a graphics processing engine, etc.
[0138] The framework 29 is used to provide various basic common components and services for the application programs in the application layer 31, such as window management, location management, etc. The framework 29 may include a phone manager 291, a resource manager 293, a location manager 295, and the like.
[0139] The functions of the various components of the operating system 161 described above can be implemented by the application processor 101 executing the program stored in the memory.
[0140] Those skilled in the art will appreciate that the terminal 100 may include Figure 8 Fewer or more components than shown, Figure 8 The terminal shown only includes components that are more relevant to the multiple implementations disclosed in the embodiments of the present application.
[0141] The terminal usually supports the installation of multiple applications (Application, APP), such as word processing application, telephone application, email application, instant messaging application, photo management application, web browsing application, digital music player application, and / or digital video player application.
[0142] The present application provides a chip system, which includes a processor for supporting a data processing device to implement the functions involved in the above aspects, such as sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The chip system can be composed of a chip, or it can include a chip and other discrete devices.
[0143] In another possible design, when the data processing device is a chip in a terminal or base station, the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute computer-executable instructions stored in the storage unit, so that the chip in the terminal or base station may execute the steps of the network-executed method in any of the above method embodiments. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the terminal or base station, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.
[0144] The embodiment of the present application also provides a chip, including: a processing module and a communication interface, the processing module can execute the method flow related to the data processing device in any of the above method embodiments. Further, the chip can also include a storage module (such as a memory), the storage module is used to store instructions, the processing module is used to execute the instructions stored in the storage module, and the execution of the instructions stored in the storage module enables the processing module to execute the method flow related to the data processing device in any of the above method embodiments.
[0145] The embodiment of the present application also provides a communication system, which may include a terminal and a base station. The terminal may be as follows: Figure 8 The terminal shown in FIG. 1 can be used to perform any step in any of the above-mentioned method embodiments. The base station can be as follows: Figure 7 The base station or core network device shown can be used for any step in any of the aforementioned method embodiments.
[0146] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the method flow related to the data processing device in any of the above method embodiments is implemented. Correspondingly, the computer can be the above data processing device.
[0147] The present application also provides a computer program or a computer program product including the computer program, which, when executed on a computer, enables the computer to implement the method flow related to the data processing device in any of the above method embodiments. Correspondingly, the computer may be the above data processing device.
[0148] In any of the above method embodiments, all or part of the method can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the method can be implemented in the form of a computer program product.
[0149] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0150] It should be understood that the processor mentioned in the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0151] It should also be understood that the number of processors in the present application can be one or more, which can be adjusted according to the actual application scenario, which is only an exemplary description and not limited here. The number of memories in the embodiment of the present application can be one or more, which can be adjusted according to the actual application scenario, which is only an exemplary description and not limited here.
[0152] It should also be noted that when the data processing device includes a processor (or processing module) and a memory, the processor in this application can be integrated with the memory, or the processor and the memory can be connected through an interface. The specific adjustment can be made according to the actual application scenario and is not limited.
[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0154] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0155] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0156] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or other network device, etc.) to execute the present application. Figure 2 as well as Figure 4 The aforementioned storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc., which can store program codes.
[0158] It should be understood that the storage medium or memory mentioned in the present application may include volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be ROM, programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) or flash memory. The volatile memory may be RAM, which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (Direct Rambus RAM, DR RAM).
[0159] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0160] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A data processing method, characterized in that: include: When a first data packet is detected to be lost, acquiring associated data packets corresponding to the first data packet, the associated data packets including data packets adjacent to the first data packet and / or data packets adjacent to the first data packet; Inputting the encoding parameters of the associated data packets into a packet loss scenario determination model; Determining a packet loss scenario type corresponding to the first data packet according to an output result of the packet loss scenario judgment model; If the packet loss scenario type satisfies the data packet compensation condition, a second data packet is generated according to a data packet adjacent to the first data packet, and the packet loss scenario type is used to determine a data packet from two data packets adjacent to the first data packet to generate the second data packet, and the two data packets adjacent to the first data packet include a data packet adjacent to before the first data packet and a data packet adjacent to after the first data packet; The second data packet is added to the target location corresponding to the first data packet before it is lost.
2. The data processing method according to claim 1, characterized in that: The packet loss scenario type includes a first packet loss scenario, the data packet compensation condition includes a first compensation condition, and the first packet loss scenario satisfies the first compensation condition; The step of generating a second data packet according to a data packet adjacent to the first data packet comprises: The second data packet is generated according to a data packet adjacent to the first data packet.
3. The data processing method according to claim 1, characterized in that: The packet loss scenario type includes a second packet loss scenario, the data packet compensation condition includes a second compensation condition, and the second packet loss scenario satisfies the second compensation condition; The step of generating a second data packet according to a data packet adjacent to the first data packet comprises: The second data packet is generated according to a data packet adjacent to the first data packet.
4. The data processing method according to claim 1, characterized in that: The packet loss scenario type includes a third packet loss scenario, and the third packet loss scenario does not meet the data packet compensation condition; The method further comprises: If the packet loss scenario type does not satisfy the data packet compensation condition, the waveform of the first data packet is reconstructed using a packet loss hiding PLC method.
5. The data processing method according to any one of claims 1 to 4, characterized in that: The associated data packets are the M data packets adjacent to the first data packet, or the associated data packets are the N data packets adjacent to the first data packet, or the associated data packets are the M data packets adjacent to the first data packet and the N data packets adjacent to the first data packet, where M is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.
6. A data processing device, characterized in that: include: a determining unit, configured to, when detecting that a first data packet is lost, acquire associated data packets corresponding to the first data packet, wherein the associated data packets include data packets adjacent to the first data packet and / or data packets adjacent to the first data packet; The determining unit is further configured to input the encoding parameters of the associated data packet into a packet loss scenario judgment model, and determine a packet loss scenario type corresponding to the first data packet according to an output result of the packet loss scenario judgment model; a generating unit, configured to generate a second data packet according to a data packet adjacent to the first data packet if the packet loss scenario type satisfies a data packet compensation condition, wherein the packet loss scenario type is configured to generate the second data packet by determining a data packet from two data packets adjacent to the first data packet, wherein the two data packets adjacent to the first data packet include a data packet adjacent to a data packet before the first data packet and a data packet adjacent to a data packet after the first data packet; An adding unit is used to add the second data packet to a target position corresponding to the first data packet before it is lost.
7. The data processing device according to claim 6, characterized in that: The packet loss scenario type includes a first packet loss scenario, the data packet compensation condition includes a first compensation condition, and the first packet loss scenario satisfies the first compensation condition; The generating unit is specifically configured to generate the second data packet according to a data packet adjacent to the first data packet.
8. The data processing device according to claim 6, characterized in that: The packet loss scenario type includes a second packet loss scenario, the data packet compensation condition includes a second compensation condition, and the second packet loss scenario satisfies the second compensation condition; The generating unit is specifically configured to generate the second data packet according to a data packet adjacent to the first data packet.
9. The data processing device according to claim 6, characterized in that: The packet loss scenario type includes a third packet loss scenario, and the third packet loss scenario does not meet the data packet compensation condition; The device also includes: a reconstruction unit, which is used to reconstruct the waveform of the first data packet by using a PLC method if the packet loss scenario type does not meet the data packet compensation condition.
10. The data processing device according to any one of claims 6 to 9, characterized in that: The associated data packets are the M data packets adjacent to the first data packet, or the associated data packets are the N data packets adjacent to the first data packet, or the associated data packets are the M data packets adjacent to the first data packet and the N data packets adjacent to the first data packet, where M is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.
11. A base station, characterized in that: The method comprises a data processing device as claimed in any one of claims 6 to 10.
12. A core network device, characterized in that: The method comprises a data processing device as claimed in any one of claims 6 to 10.
13. A terminal, characterized in that: The method comprises a data processing device as claimed in any one of claims 6 to 10.
14. A communication system, characterized in that: It comprises the base station as claimed in claim 11, the core network device as claimed in claim 12 and the terminal as claimed in claim 13.
15. A computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to perform the method according to any one of claims 1 to 5.
16. A computer program product comprising instructions, characterized in that When the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 5.
17. A data processing device, comprising a processor and a memory, characterized in that: The processor is coupled to the memory, and is configured to read and execute instructions stored in the memory to implement the steps of any one of claims 1 to 5.
Citation Information
Patent Citations
Lost packet compensation method and device under user datagram protocol transmission mode
CN101800632A
Voice packet loss hiding method and system
CN104021792A
Voice packet loss compensation method and system thereof
CN106788876A
Packet loss retransmission method and device
CN107277423A
Forward error correction feedback information transmission method and device
CN107483144A
Cited By
Data processing method and related apparatus
WO2021073496A1
