Voice decoding method, voice coding method, related device, equipment and medium

Abstract

The invention discloses a voice decoding method, a voice coding method, a related device, equipment and a medium, and the voice decoding method comprises the steps: obtaining voice coding data of a voice transmitting end; sequentially reading packet contents in the continuous data packets meeting the decoding conditions and sequentially combining the packet contents to obtain contents to be decoded, and decoding based on the contents to be decoded to obtain a plurality of voice frames until the plurality of data packets are processed; wherein the decoding condition comprises that the flag bit represents hybrid coding, and coding algorithms represented by the coding type are the same; and restoring to obtain the original voice of the voice sending end based on the plurality of voice frames decoded all the time. According to the scheme, on the premise of realizing hybrid encoding / decoding, the use limitation can be reduced as much as possible, and the expandability is improved.

Description

Technical Field

[0001] The present application relates to the field of speech coding and decoding technology, and in particular to a speech decoding method, a speech coding method, and related devices, equipment, and media. Background Art

[0002] With the rapid development of digital communication technology, especially the popularization of the Internet and mobile communications, voice communication has become an indispensable part of daily life. In order to meet the growing demand for voice communication, how to transmit high-quality voice signals under limited bandwidth resources and complex and changing signal environments has become an important research topic. Hybrid encoding / decoding technology is a voice encoding / decoding technology that was born under this background.

[0003] At present, the existing hybrid encoding / decoding technology relies on the combination of SDP (Session Description Protocol, SDP) and RTP (Real-Time Transport Protocol, RTP) to jointly realize the transmission and quality control of voice data. However, this approach will restrict multi-protocol and multi-device communication on the one hand. For example, this dependence means that the function and performance of the SDP protocol may be restricted and affected by other protocols, and the RTP and other protocols need to be parsed before decoding, resulting in processing redundancy, etc. On the other hand, it will also cause insufficient scalability. For example, some terminals may not support the SDP protocol, and the SDP protocol may lag behind in supporting emerging encoding / decoding technologies. In view of this, how to reduce usage restrictions as much as possible and improve scalability under the premise of realizing hybrid encoding / decoding has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a speech decoding method, a speech encoding method and related devices, equipment and media, which can reduce usage restrictions as much as possible and improve scalability under the premise of realizing hybrid encoding / decoding.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a speech decoding method, including: obtaining speech coding data from a speech sending end; wherein the speech coding data includes a plurality of data packets, the data packets include a packet header and packet content, the packet header includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is a mixed coding, and the coding type is used to identify the coding algorithm used by the data packet; reading the packet contents in consecutive data packets that meet the decoding conditions in sequence and combining them in sequence to obtain content to be decoded, and decoding based on the content to be decoded to obtain a plurality of speech frames, until the plurality of data packets are processed; wherein the decoding conditions include that the mark bit represents the mixed coding and the coding algorithm represented by the coding type is the same; based on the plurality of speech frames decoded in succession, the original speech of the speech sending end is restored.

[0006] In order to solve the above technical problems, the second aspect of the present application provides a speech encoding method, comprising: encoding the speech frames in the original speech in sequence to obtain a data packet; wherein the content of the data packet contains speech frames encoded by the same encoding algorithm, and in the case of hybrid encoding, there are data packets encoded by different encoding algorithms, and the header of the data packet includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is hybrid encoding, and the coding type is used to identify the coding algorithm used by the data packet; based on the original speech encoding, each data packet is encoded in sequence, encapsulated to obtain speech encoding data, and the speech encoding data is sent to a speech receiving end; wherein the speech receiving end decodes the speech encoding data based on the speech decoding method in the above first aspect.

[0007] In order to solve the above technical problems, the third aspect of the present application provides a speech decoding device, including: a data acquisition module, a reading and decoding module and a speech restoration module, the data acquisition module is used to acquire the speech coding data of the speech sending end; wherein the speech coding data includes a plurality of data packets, the data packets contain a packet header and a packet content, the packet header includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is a mixed coding, and the coding type is used to identify the coding algorithm adopted by the data packet; the reading and decoding module is used to read the packet contents in consecutive data packets that meet the decoding conditions in sequence and combine them in sequence to obtain the content to be decoded, and decode based on the content to be decoded to obtain a plurality of speech frames until the plurality of data packets are processed; wherein the decoding conditions include that the mark bit represents the mixed coding and the coding algorithm represented by the coding type is the same; the speech restoration module is used to restore the original speech of the speech sending end based on a plurality of speech frames decoded previously.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a speech encoding device, including: a data encoding module and an encapsulation and sending module, the data encoding module is used to encode the speech frames in the original speech in sequence to obtain a data packet; wherein the content of the data packet contains speech frames encoded by the same encoding algorithm, and in the case of hybrid encoding, there are data packets encoded by different encoding algorithms, and the header of the data packet includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is hybrid encoding, and the coding type is used to identify the coding algorithm used by the data packet; the encapsulation and sending module is used to encode each data packet encoded in sequence based on the original speech encoding, encapsulate the speech encoding data, and send the speech encoding data to the speech receiving end; wherein the speech receiving end decodes the speech encoding data based on the speech decoding device in the above third aspect.

[0009] In order to solve the above technical problems, the fifth aspect of the present application provides an electronic device, which at least includes a memory and a processor coupled to each other, and the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the speech decoding method in the above first aspect, or to implement the speech encoding method in the above second aspect.

[0010] In order to solve the above technical problems, the sixth aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the speech decoding method of the first aspect above, or to implement the speech encoding method of the second aspect above.

[0011] The above scheme obtains the voice coding data of the voice sending end, and the voice coding data includes several data packets, the data packets include a packet header and packet content, the packet header includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is mixed coding, and the coding type is used to identify the coding algorithm used by the data packet, and then the packet contents in the continuous data packets that meet the decoding conditions are read in sequence and combined in sequence to obtain the content to be decoded, and decoded based on the content to be decoded to obtain several voice frames, until the several data packets are processed, and the decoding conditions include the mark bit representing the mixed coding and the coding type representing the same coding algorithm, so as to restore the original voice of the voice sending end based on the several voice frames decoded in succession. On the one hand, by indicating whether it is mixed coding and the coding type in the packet header of the voice coding data, the mixed decoding process can be guided accordingly, so that mixed coding / decoding can be realized. On the other hand, due to the guidance of the mark bit and the coding type in the decoding process, the packet contents in the continuous read data packets that all represent mixed coding and use the same coding algorithm are directly read and combined in sequence as the content to be decoded for decoding, without combining with the SDP protocol, and can be free from the restrictions of the SDP protocol on usage scenarios, scalability, etc. as much as possible. Therefore, under the premise of realizing hybrid encoding / decoding, it is possible to reduce usage restrictions as much as possible and improve scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flowchart of an embodiment of a speech decoding method of the present application; Figure 2a is a schematic diagram of an embodiment of speech coding data; Figure 2b This is a process diagram of an embodiment of a speech decoding method of the present application; Figure 3 It is a flowchart of an embodiment of a speech coding method of the present application; Figure 4 It is a schematic diagram of the framework of an embodiment of a speech decoding device of the present application; Figure 5 It is a schematic diagram of the framework of an embodiment of a speech encoding device of the present application; Figure 6 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application; Figure 7 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0013] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0014] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0015] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0016] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the speech decoding method of the present application. Specifically, it may include the following steps: Step S11: Acquire speech coding data from the speech transmitting end.

[0017] In the disclosed embodiment, the speech coding data includes a plurality of data packets, the data packets include a packet header and packet content, the packet header includes at least a flag bit and a coding type, the flag bit is used to identify whether the data packet is a mixed coding, and the coding type is used to identify the coding algorithm used by the data packet. It should be noted that the specific number of data packets in the speech coding data is not limited. Exemplarily, the speech coding data may include 2 data packets, 3 data packets, 4 data packets, etc., which are not limited here. In addition, the specific coding process of the speech coding data can refer to the relevant description in the following speech coding method embodiment, which will not be repeated here.

[0018] In an implementation scenario, please refer to Figure 2a , Figure 2a FIG. 1 is a schematic diagram of an embodiment of speech coding data. Figure 2a As shown, the packet header in the data packet can be located before the packet content, and the packet content contains the coded data of the voice frames in the original voice after being encoded. The voice coding data using hybrid coding contains data packets with multiple coding algorithms, such as, but not limited to: AMRNB (Adaptive Multi-rate Narrow Band) data packets, EVS (Enhanced Voice Service) data packets, etc., and the coding algorithms in hybrid coding are not limited here. Of course, in actual applications, voice coding data may not be limited to Figure 2a Each data packet shown uses a different coding algorithm. For example, take the case where the speech coding data includes 3 data packets, two of which can use the same coding algorithm, and another data packet can use another different coding algorithm. Other situations can be deduced by analogy, and no examples are given here one by one.

[0019] In one implementation scenario, as a possible example, the flag bit can be marked with "0" or "1", such as "1" can be used to mark the data packet as mixed coding, and "0" can be used to mark the data packet as non-mixed coding. Of course, the above example is only a possible example of the flag bit in the actual application process, and other possible setting methods are not given one by one here.

[0020] In one implementation scenario, as a possible example, taking a total of 4 preset coding algorithms (respectively recorded as coding algorithm A, coding algorithm B, coding algorithm C, and coding algorithm D) as an example, two binary digits may be used to represent the coding type, such as coding type "00" may be used to represent that the data packet adopts coding algorithm A, coding type "01" may be used to represent that the data packet adopts coding algorithm B, coding type "10" may be used to represent that the data packet adopts coding algorithm C, and coding type "11" may be used to represent that the data packet adopts coding algorithm D. Of course, the above example is only one possible example of the coding type, and other possible settings will not be given examples one by one here.

[0021] Step S12: sequentially reading the contents of consecutive data packets that meet the decoding conditions and sequentially combining them to obtain the content to be decoded, and performing decoding based on the content to be decoded to obtain a number of voice frames, until all the data packets are processed.

[0022] In the disclosed embodiment, the decoding condition includes that the mark bit represents the mixed coding and the coding type represents the same coding algorithm. For example, taking the speech coding data sequentially including data packet 1, data packet 2, ..., data packet N as an example, if the mark bits of both consecutive data packets 1 and 2 represent the mixed coding (e.g., the mark bits are all "1") and the coding type represents the same coding algorithm (e.g., the coding type is all "00"), but although the mark bit of data packet 3 is the same as it (e.g., the mark bit is also "1"), the coding type is different from it (e.g., the coding type is "01"), then it can be considered that data packet 1 and data packet 2 meet the decoding conditions, and the packet content in data packet 1 (e.g., recorded as packet content 1) and the packet content in data packet 2 (e.g., recorded as packet content 2) can be read in sequence and combined to obtain the content to be decoded (e.g., it can be recorded as "packet content 1-packet content 2"). Of course, the above example is only a simple example in the actual application process, and other possible situations are not given one by one here.

[0023] In one implementation scenario, each data packet in the speech coding data can be selected in turn as a target packet, and then the target packet is detected to see if it meets the following requirements: the mark bit indicates mixed coding and is the same as the coding type of the previous data packet, so that in response to the detection result indicating that the requirements are met, the packet content in the target packet is read and spliced ​​to the packet content of the previous data packet for caching, and the step of selecting each data packet in the speech coding data in turn as the target packet is returned until the detection result indicates that the requirements are not met, and the cached packet content is obtained as the content to be decoded. It should be noted that when the target packet is the first data packet in the speech coding data, or when the content to be decoded where the packet content of the previous data packet is located has been decoded, it is only necessary to detect whether its mark bit indicates mixed coding, without detecting whether the coding type is the same as that of the previous data packet (because it does not have a previous data packet). In the above method, when the relevant decoding requirements are met, the packet content of the data packet is directly spliced ​​to the packet content of the previous data packet until it is detected that a subsequent data packet does not meet the relevant decoding requirements, and the cached packet content can be used as the data to be decoded for decoding, so the IO operation time can be reduced and the decoding processing efficiency can be improved.

[0024] In a specific implementation scenario, for ease of understanding, the speech coding data still includes data packet 1, data packet 2, ..., data packet N as an example, assuming that the mark bit of data packet 1 represents mixed coding and the coding type is "00", the mark bit of data packet 2 represents mixed coding and the coding type is "00", the mark bit of data packet 3 represents mixed coding and the coding type is "01", and the mark bit of data packet 4 represents mixed coding and the coding type is "00", then when it is detected that the continuous data packets 1 and 2 meet the above requirements, the packet content of data packet 2 can be spliced ​​after the packet content of data packet 1 as the content to be decoded and cached, after which on the one hand the cached content to be decoded can be taken out for decoding, on the other hand, it can continue to return to perform similar detection on data packet 3 and its subsequent data packets, because the coding type of data packet 4 is different from that of data packet 3, only the packet content of data packet 3 can be used as the content to be decoded and cached, after which on the one hand the cached content to be decoded can be taken out for decoding, on the other hand, it can continue to return to perform similar detection on data packet 4 and its subsequent data packets. Of course, the above example is only a possible example in the actual application process, and other possible situations are not given one by one here.

[0025] In a specific implementation scenario, the packet header may also include a first packet length of the packet header and a second packet length of the packet content. Then, the first packet length in the target packet may be obtained as a reference packet length, and the second packet length in the target packet may be obtained as a verification packet length. Then, based on the reference packet length, the packet data after the packet header in the target packet may be selected as the data to be verified. Thus, in response to the data to be verified meeting the verification packet length (i.e., the data length of the data to be verified is the verification packet length), the data to be verified may be read and determined as the packet content in the target packet. It should be noted that, unlike the aforementioned situation, in response to the data to be verified not meeting the verification packet length, the target packet may be discarded, and the steps of selecting each data packet in the speech coding data in turn as the target packet may be returned. In the above manner, the packet content is extracted by using the packet length information in the packet header, which can ensure the integrity of the extracted packet content as much as possible, thereby helping to improve the integrity of the original speech restored by subsequent decoding.

[0026] In a specific implementation scenario, in response to the detection result including the target packet having a mark bit representing a mixed coding but having a different coding type from the previous data packet, the packet content in the target packet is read and cached, and the step of selecting each data packet in the speech coding data in turn as the target packet is returned. That is to say, in the case where the detection result includes the target packet having a mark bit representing a mixed coding but having a different coding type from the previous data packet, the packet content in the target packet is no longer spliced ​​after the packet content of the previous data packet (refer to "data packet 3" in the aforementioned example), and the cached data containing the packet content in the previous data packet can be decoded as the data to be decoded. In the above manner, in response to the detection result including the target packet having a mark bit representing a mixed coding but having a different coding type from the previous data packet, the packet content in the target packet is read and cached, and the step of selecting each data packet in the speech coding data in turn as the target packet is returned, can ensure that the packet contents of continuous data packets using the same coding algorithm are uniformly decoded as much as possible, which helps to improve the decoding efficiency, and the packet contents of data packets using different decoding algorithms will not be decoded together, which helps to improve the decoding accuracy. Therefore, it helps to improve the decoding accuracy and decoding efficiency at the same time.

[0027] In a specific implementation scenario, in response to the detection result including that the mark bit in the target packet does not represent the mixed coding, the target packet can be discarded, and the step of selecting each data packet in the speech coding data in turn as the target packet is returned. Still taking the speech coding data including data packet 1, data packet 2, ..., data packet N as an example, when data packet 5 is selected as the target packet, it is further assumed that if the mark bit of data packet 5 is "0", that is, data packet 5 is not mixed coding, data packet 5 can be discarded, and the step of selecting each data packet in the speech coding data in turn as the target packet is continued, that is, data packet 6 is continued to be selected as the target packet. Of course, the above example is only a possible example in the actual application process, and other possible situations are not limited thereto, and examples are not given one by one here. In addition, it should be noted that in this implementation scenario, discarding the target packet can mean that the target packet will not be decoded later, or it can mean that only the target packet and other target packets that meet the above requirements are not included in the process of decoding and restoring the original voice (i.e., the mixed decoding process), but outside this process, the target packet can be decoded separately, and the specific situation of discarding the target packet in this implementation scenario is not limited here. In the above method, in response to the detection result including that the mark bit in the target packet does not represent the mixed coding, the target packet is discarded, and the step of selecting each data packet of the speech coding data in turn as the target packet is returned. Therefore, the data packet whose mark bit does not represent the mixed coding can be excluded from the mixed decoding process, which helps to improve the orderliness of the mixed decoding and reduce the error probability of the mixed decoding.

[0028] In a specific implementation scenario, in response to the detection result including that a complete header or packet content is not detected in the target packet, the target packet can be discarded, and the step of selecting each data packet in the speech coding data in turn as the target packet is returned. Still taking the speech coding data including data packet 1, data packet 2, ..., data packet N as an example, when data packet 6 is selected as the target packet, it is further assumed that if the header or packet content in data packet 6 is incomplete, data packet 6 can be discarded, and the step of selecting each data packet in the speech coding data in turn as the target packet is continued, that is, data packet 7 is continued to be selected as the target packet. Of course, the above example is only a possible example in the actual application process, and other possible situations are not limited thereto, and examples are not given one by one here. In the above manner, when the detection result includes that a complete header or packet content is not detected in the target packet, the target packet is discarded, and each data packet in the speech coding data is selected in turn as the target packet, which can ensure the decoding integrity as much as possible.

[0029] In an implementation scenario, after obtaining the content to be decoded, the coding algorithm represented by the coding type in the data packet from which the content to be decoded originates can be specifically selected as the target algorithm, and the content to be decoded is decoded based on the decoding algorithm matching the target algorithm to obtain several voice frames. Still taking the example that the voice coded data includes data packet 1, data packet 2, ..., data packet N, as mentioned above, data packet 1 and data packet 2 meet the above requirements, and the packet content of data packet 2 is spliced ​​after the packet content of data packet 1 to form the content to be decoded. For this content to be decoded, the coding algorithm (i.e., coding algorithm A) represented by the coding type (i.e., "00") in the data packet (i.e., data packet 1 and data packet 2) from which it originates can be obtained as the target algorithm, and the content to be decoded is decoded based on the decoding algorithm (i.e., decoding algorithm A) matching the target algorithm to obtain several audio frames. Of course, the above example is only a possible example of decoding the content to be decoded in the actual application process, and does not limit other possible situations. Other possible situations are not given examples one by one here. The above method selects the coding algorithm represented by the coding type in the data packet from which the content to be decoded originates as the target algorithm, and decodes the content to be decoded based on a decoding algorithm matching the target algorithm to obtain several voice frames. This can achieve unified decoding of the content in consecutive data packets using the same coding algorithm, which helps to improve the efficiency of hybrid decoding.

[0030] In another implementation scenario, as another possible example, please refer to Figure 2b , Figure 2b 1 is a process diagram of an embodiment of the speech decoding method of the present application. Figure 2bAs shown, after receiving the voice coding data, the packet header of the first data packet can be read first. If the mark bit in the packet header indicates that it is not a mixed coding, the data packet can be discarded. If the mark bit in the packet header indicates that it is a mixed coding, the packet content can be read according to the packet length in the packet header (such as the first packet length and the second packet length mentioned above), and then the subsequent data packets can be read and it can be determined whether the encoding type in the packet header is the same as that of the previous data packet. If different, the stored packet content can be directly decoded according to its encoding format (that is, the decoding algorithm that matches the encoding algorithm represented by the encoding type in the packet header), and it can be determined whether the read data packet is the last data packet in the voice coding data. If so, the decoded data can be directly integrated and output, otherwise the packet header of the next data packet can be read, and the above-mentioned step of checking whether the mark bit indicates mixed coding can be returned. In addition, if the encoding type in the packet header of the subsequent data packet is judged to be the same as that of the previous data packet, the process can return to the step of checking whether the mark bit represents the mixed encoding, and if the verification is yes, its packet content can be spliced ​​after the packet content in the previous data packet, until a subsequent data packet is read whose encoding type is different from that of the previous data packet or the mark bit does not represent the mixed encoding, then the stored packet content can be decoded according to its encoding format, and if the read data packet is not the last data packet, continue to read the packet header of the next data packet, and return to the step of checking whether the mark bit represents the mixed encoding, and repeat this cycle until the last data packet is read.

[0031] Step S13: based on the decoded speech frames, the original speech of the speech transmitting end is restored.

[0032] Specifically, the original voice of the voice transmitter can be restored by combining several voice frames obtained by historical decoding in order according to the time sequence of previous decoding. As a possible example, the voice coding data still includes data packet 1, data packet 2, ..., data packet N as an example. As mentioned above, since data packet 1 and data packet 2 meet the decoding requirements that both of their mark bits represent mixed coding and the coding types are the same, but data packet 3 does not meet the above decoding requirements, the packet content in data packet 2 can be spliced ​​after the packet content of data packet 1 as the content to be decoded, and the decoding algorithm matching the coding algorithm represented by the coding types of data packets 1 and 2 is used to decode the content to be decoded, and obtain several voice frames decoded for the first time. Further, since the coding type of data packet 4 is different from the coding type of data packet 3, the packet content in data packet 3 can be directly decoded as the content to be decoded, and obtain several voice frames decoded for the second time. In the process of continuing to process subsequent data packets, it is found that the mark bit of data packet 5 indicates that it is not mixed coding, so data packet 5 can be discarded, and the packet content of data packet 4 can be directly decoded to obtain several voice frames decoded for the third time. When continuing to process data packet 6, if it is found that the header or packet content of data packet 6 is incomplete, data packet 6 can be discarded, and data packet 7 and subsequent data packets can be processed until data packet N is processed. The specific processing process can refer to the aforementioned possible situations, and no one is given here. On this basis, the first decoded several voice frames, the second decoded several voice frames, the third decoded several voice frames, and so on can be sequentially combined until the last decoded several voice frames, and the original voice of the voice transmitter can be restored. Of course, the above example is only a possible example in the actual application process, and does not limit other possible situations. Other possible situations will not be given one by one here.

[0033] The above scheme obtains the voice coding data of the voice sending end, and the voice coding data includes several data packets, the data packets include a packet header and packet content, the packet header includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is mixed coding, and the coding type is used to identify the coding algorithm used by the data packet, and then the packet contents in the continuous data packets that meet the decoding conditions are read in sequence and combined in sequence to obtain the content to be decoded, and decoded based on the content to be decoded to obtain several voice frames, until the several data packets are processed, and the decoding conditions include the mark bit representing the mixed coding and the coding type representing the same coding algorithm, so as to restore the original voice of the voice sending end based on the several voice frames decoded in succession. On the one hand, by indicating whether it is mixed coding and the coding type in the packet header of the voice coding data, the mixed decoding process can be guided accordingly, so that mixed coding / decoding can be realized. On the other hand, due to the guidance of the mark bit and the coding type in the decoding process, the packet contents in the continuous read data packets that all represent mixed coding and use the same coding algorithm are directly read and combined in sequence as the content to be decoded for decoding, without combining with the SDP protocol, and can be free from the restrictions of the SDP protocol on usage scenarios, scalability, etc. as much as possible. Therefore, under the premise of realizing hybrid encoding / decoding, it is possible to reduce usage restrictions as much as possible and improve scalability.

[0034] See also Figure 3 , Figure 3 This is a flow chart of an embodiment of the speech coding method of the present application. Specifically, it may include the following steps: Step S31: Encode the speech frames in the original speech in sequence to obtain data packets.

[0035] In the disclosed embodiment, the packet content in the data packet contains speech frames encoded by the same coding algorithm. In the case of hybrid coding, there are data packets encoded by different coding algorithms, and the header in the data packet includes at least a mark bit and a coding type. The mark bit is used to identify whether the data packet is hybrid coding, and the coding type is used to identify the coding algorithm used by the data packet. It should be noted that the specific connotation of the data packet can be referred to the above disclosed embodiment, which will not be repeated here. As a possible example, taking the original speech containing M speech frames as an example, the speech frames 1 to i can be encoded using the coding algorithm A to obtain the packet content, and assembled with the header (the mark bit in the header can be set to "1" to indicate that hybrid coding is used, and the coding type can be set to "00" to indicate that the coding algorithm A is used, and the first packet length of the header and the second packet length of the packet content are set) to form a data packet 1; for subsequent speech frames i+1 to speech frame M, the same can be applied, and data packets 2, data packets 3, ..., and data packets N can be encoded in sequence.

[0036] Step S32: Encapsulate each data packet encoded in sequence based on the original voice encoding to obtain voice encoding data, and send the voice encoding data to the voice receiving end.

[0037] In the disclosed embodiment, the voice receiving end decodes the voice coded data based on the above-mentioned voice decoding method embodiment. For details, please refer to the above-mentioned disclosed embodiment, which will not be repeated here. In addition, each data packet obtained by sequential encoding can be packaged and encapsulated into voice coded data in a format predetermined by the voice sending end and the voice receiving end, and the specific packaging format is not limited here.

[0038] In the above scheme, the speech frames in the original speech are sequentially encoded to obtain data packets, and the content of the data packets contains speech frames encoded by the same encoding algorithm. In the case of hybrid encoding, there are data packets encoded by different encoding algorithms. The header of the data packet at least includes a mark bit and a coding type. The mark bit is used to identify whether the data packet is hybrid encoding, and the coding type is used to identify the coding algorithm used by the data packet. Then, the data packets sequentially encoded based on the original speech encoding are encapsulated to obtain speech encoding data, and the speech encoding data is sent to the speech receiving end. The speech receiving end decodes the speech encoding data based on the above speech decoding method embodiment. On the one hand, by indicating whether it is hybrid encoding and the coding type in the header of the speech encoding data, the hybrid decoding process is guided accordingly, and hybrid encoding / decoding can be achieved. On the other hand, due to the guidance of the mark bit and the coding type in the decoding process, the contents of the packets in the continuously read data packets that all represent hybrid encoding and use the same coding algorithm are directly read and sequentially combined into the content to be decoded for decoding, without the need to combine the SDP protocol, and can be free from the restrictions of the SDP protocol on usage scenarios, scalability, etc. as much as possible. Therefore, under the premise of realizing hybrid encoding / decoding, the usage restrictions can be reduced as much as possible and the scalability can be improved.

[0039] See also Figure 4 , Figure 4It is a schematic diagram of the framework of an embodiment of the speech decoding device of the present application. The speech decoding device 40 includes: a data acquisition module 41, a read decoding module 42 and a speech restoration module 43, the data acquisition module 41 is used to acquire the speech coding data of the speech sending end; wherein, the speech coding data includes a plurality of data packets, the data packets include a packet header and a packet content, the packet header includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is a mixed coding, and the coding type is used to identify the coding algorithm used by the data packet; the read decoding module 42 is used to sequentially read the packet content in the continuous data packets that meet the decoding conditions and sequentially combine them to obtain the content to be decoded, and decode based on the content to be decoded to obtain a plurality of speech frames, until the plurality of data packets are processed; wherein, the decoding conditions include that the mark bit represents the mixed coding and the coding algorithm represented by the coding type is the same; the speech restoration module 43 is used to restore the original speech of the speech sending end based on the plurality of speech frames decoded in succession.

[0040] In the above scheme, the speech decoding device 40 obtains speech coding data from the speech sending end, and the speech coding data includes a plurality of data packets, the data packets include a packet header and packet content, the packet header includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is a mixed coding, and the coding type is used to identify the coding algorithm used by the data packet, and then the packet contents in the continuous data packets that meet the decoding conditions are read in sequence and combined in sequence to obtain the content to be decoded, and decoding is performed based on the content to be decoded to obtain a plurality of speech frames, until the plurality of data packets are processed, and the decoding conditions include that the mark bit represents the mixed coding and the coding type represents the same coding algorithm, so that the original speech of the speech sending end is restored based on the plurality of speech frames decoded in succession. On the one hand, by indicating whether it is a mixed coding and the coding type in the packet header of the speech coding data, the mixed decoding process is guided accordingly, so that mixed coding / decoding can be realized. On the other hand, because the decoding process is guided by the mark bit and the coding type, the packet contents in the continuous read data packets that all represent the mixed coding and use the same coding algorithm are directly read and sequentially combined into the content to be decoded for decoding, without combining with the SDP protocol, and can be free from the restrictions of the SDP protocol on the usage scenarios, scalability, etc. as much as possible. Therefore, under the premise of realizing hybrid encoding / decoding, it is possible to reduce usage restrictions as much as possible and improve scalability.

[0041] In some disclosed embodiments, the reading and decoding module 42 includes a selection submodule for sequentially selecting each data packet in the speech coding data as a target packet; the reading and decoding module 42 includes a detection submodule for detecting whether the target packet meets the following requirements: the mark bit represents the mixed coding and is the same as the coding type of the previous data packet; the reading and decoding module 42 includes a loop submodule for responding to the detection result indicating that the requirements are met, reading the packet content in the target packet and splicing it to the packet content of the previous data packet for caching, and returning to the step of sequentially selecting each data packet in the speech coding data as a target packet, until the detection result indicates that the requirements are not met, and obtaining the cached packet content as the content to be decoded.

[0042] In some disclosed embodiments, the packet header also includes a first packet length of the packet header and a second packet length of the packet content; the loop submodule includes a packet length acquisition unit, which is used to acquire the first packet length in the target packet as a reference packet length, and acquire the second packet length in the target packet as a verification packet length; the loop submodule includes a data acquisition unit, which is used to select packet data in the target packet that is located after the packet header based on the reference packet length as data to be verified; the loop submodule includes a content reading unit, which is used to read the data to be verified and determine it as the packet content in the target packet in response to the data to be verified being consistent with the verification packet length.

[0043] In some disclosed embodiments, the reading and decoding module 42 includes a first discarding submodule, which is used to respond to the detection result including that the mark bit in the target packet represents a mixed encoding but is different from the encoding type of the previous data packet, read the packet content in the target packet and cache it, and return to the step of selecting each data packet in the speech encoding data in turn as the target packet.

[0044] In some disclosed embodiments, the reading and decoding module 42 includes a second discarding submodule, which is used to discard the target packet in response to the detection result including that the mark bit in the target packet does not represent the mixed coding, and return to the step of selecting each data packet in the speech coding data in turn as the target packet.

[0045] In some disclosed embodiments, the reading and decoding module 42 includes a third discarding submodule, which is used to discard the target packet in response to the detection result including that a complete packet header or packet content is not detected in the target packet, and return to the step of sequentially selecting each data packet in the speech encoding data as the target packet.

[0046] In some disclosed embodiments, the reading and decoding module 42 includes an algorithm selection submodule, which is used to select a coding algorithm represented by a coding type in a data packet from which the content to be decoded originates, as a target algorithm; the reading and decoding module 42 includes a data decoding submodule, which is used to decode the content to be decoded based on a decoding algorithm matching the target algorithm to obtain a number of speech frames.

[0047] See also Figure 5, Figure 5 It is a schematic diagram of the framework of an embodiment of the speech coding device of the present application. The speech coding device 50 includes: a data coding module 51 and an encapsulation and sending module 52, wherein the data coding module 51 is used to sequentially encode speech frames in the original speech to obtain data packets; wherein the content of the data packets contains speech frames encoded by the same encoding algorithm, and in the case of hybrid encoding, there are data packets encoded by different encoding algorithms, and the header of the data packets at least includes a mark bit and a coding type, wherein the mark bit is used to identify whether the data packet is hybrid encoding, and the coding type is used to identify the coding algorithm used by the data packet; the encapsulation and sending module 52 is used to encapsulate each data packet sequentially encoded based on the original speech encoding to obtain speech coding data, and send the speech coding data to the speech receiving end; wherein the speech receiving end decodes the speech coding data based on the speech decoding device in the above-mentioned speech decoding device embodiment.

[0048] In the above scheme, the speech encoding device 50 sequentially encodes the speech frames in the original speech to obtain data packets, and the content of the data packets contains speech frames encoded by the same encoding algorithm. In the case of hybrid encoding, there are data packets encoded by different encoding algorithms. The header of the data packet at least includes a mark bit and a coding type. The mark bit is used to identify whether the data packet is hybrid encoding, and the coding type is used to identify the coding algorithm used by the data packet. Then, the data packets sequentially encoded based on the original speech encoding are encapsulated to obtain speech encoding data, and the speech encoding data is sent to the speech receiving end. The speech receiving end decodes the speech encoding data based on the above speech decoding method embodiment. On the one hand, by indicating whether it is hybrid encoding and the coding type in the header of the speech encoding data, the hybrid decoding process is guided accordingly, so that hybrid encoding / decoding can be achieved. On the other hand, due to the guidance of the mark bit and the coding type in the decoding process, the contents of the packets in the continuously read data packets that all represent hybrid encoding and use the same coding algorithm are directly read and sequentially combined into the content to be decoded for decoding without combining with the SDP protocol, which can be free from the restrictions of the SDP protocol on usage scenarios, scalability, etc. as much as possible. Therefore, under the premise of realizing hybrid encoding / decoding, it is possible to reduce usage restrictions as much as possible and improve scalability.

[0049] See also Figure 6 , Figure 6It is a schematic diagram of the framework of an embodiment of an electronic device of the present application. The electronic device 60 includes at least a memory 61 and a processor 62 coupled to each other, the memory 61 at least stores program instructions, and the processor 62 is used to execute the program instructions to implement the steps in any of the above-mentioned speech decoding method embodiments, or to implement the steps in any of the above-mentioned speech encoding method embodiments. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. As a possible example, the electronic device 60 may include but is not limited to a server, etc., and the specific type of the electronic device 60 is not limited here. It should be noted that when the processor 62 implements the steps in any of the above-mentioned speech decoding method embodiments, the electronic device 60 may be a voice receiving end, and when the processor 62 implements the steps in any of the above-mentioned speech encoding method embodiments, the electronic device 60 may be a voice sending end.

[0050] Specifically, the processor 62 is used to control itself and the memory 61 to implement the steps in any of the above-mentioned speech decoding method embodiments, or to implement the steps in any of the above-mentioned speech encoding method embodiments. The processor 62 can also be called a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip with signal processing capabilities. The processor 62 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 62 can be implemented by an integrated circuit chip.

[0051] In the above scheme, the electronic device 60 can realize hybrid encoding / decoding by indicating whether it is hybrid encoding and the encoding type in the packet header of the voice encoding data to guide the hybrid decoding process accordingly. On the other hand, because it is guided by the mark bit and the encoding type during the decoding process, it directly reads the contents of the packets in the continuously read data packets that all represent hybrid encoding and use the same encoding algorithm and sequentially combines them into the contents to be decoded for decoding without combining with the SDP protocol, which can be free from the restrictions of the SDP protocol on usage scenarios, scalability, etc. as much as possible. Therefore, it can reduce usage restrictions as much as possible and improve scalability under the premise of realizing hybrid encoding / decoding.

[0052] See also Figure 7 , Figure 7The computer-readable storage medium 70 of the present application is a schematic diagram of a framework of an embodiment of the computer-readable storage medium 70. The computer-readable storage medium 70 stores program instructions 71 that can be executed by a processor, and the program instructions 71 are used to implement the steps in any of the above-mentioned speech decoding method embodiments, or to implement the steps in any of the above-mentioned speech encoding method embodiments.

[0053] In the above scheme, the computer-readable storage medium 70 can realize hybrid encoding / decoding by indicating whether it is hybrid encoding and the encoding type in the packet header of the speech encoding data to guide the hybrid decoding process accordingly. On the other hand, because it is guided by the mark bit and the encoding type during the decoding process, the contents of the packets in the continuous reading data packets that all represent hybrid encoding and use the same encoding algorithm are directly read and sequentially combined into the contents to be decoded for decoding without combining with the SDP protocol, which can be free from the restrictions of the SDP protocol on usage scenarios, scalability, etc. as much as possible. Therefore, under the premise of realizing hybrid encoding / decoding, it is possible to reduce usage restrictions as much as possible and improve scalability.

[0054] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0055] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0056] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0057] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0058] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0059] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0060] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A speech decoding method, characterized in that: include: Acquire speech coding data from a speech transmitting end; wherein the speech coding data comprises a plurality of data packets, the data packets comprise a packet header and packet content, the packet header comprises at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is mixed coded, and the coding type is used to identify a coding algorithm adopted by the data packet; The packet contents in the continuous data packets meeting the decoding conditions are read in sequence and combined in sequence to obtain the content to be decoded, and decoding is performed based on the content to be decoded to obtain a plurality of voice frames until the plurality of data packets are processed; wherein the decoding conditions include that the mark bit represents the mixed coding and the coding algorithms represented by the coding types are the same; Based on several previously decoded speech frames, the original speech of the speech transmitting end is restored.

2. The method according to claim 1, characterized in that: The step of sequentially reading the packet contents in the continuous data packets that meet the preset decoding conditions and sequentially combining them to obtain the content to be decoded includes: Selecting each of the data packets in the speech coding data in turn as a target packet respectively; Detecting whether the target packet meets the following requirements: the flag bit represents the mixed coding and is the same as the coding type of the previous data packet; In response to the detection result representation satisfying the requirements, the packet content in the target packet is read and spliced ​​to the packet content of the previous data packet for caching, and then the step of sequentially selecting each of the data packets in the speech encoding data as the target packet is returned until the detection result representation does not satisfy the requirements, and the cached packet content is obtained as the content to be decoded.

3. The method according to claim 2, characterized in that The packet header also includes a first packet length of the packet header and a second packet length of the packet content, and the reading of the packet content in the target packet includes: Obtaining the first packet length in the target packet as a reference packet length, and obtaining the second packet length in the target packet as a verification packet length; Selecting, based on the reference packet length, packet data located after the packet header in the target packet as data to be verified; In response to the data to be verified being consistent with the verification packet length, the data to be verified is read and determined to be the packet content in the target packet.

4. The method according to claim 2, characterized in that: The method further comprises: In response to the detection result including that the mark bit in the target packet represents the mixed coding but is different from the coding type of the previous data packet, the packet content in the target packet is read and cached, and the step of selecting each of the data packets in the speech coding data in turn as the target packets is returned.

5. The method according to claim 2, characterized in that: The method further comprises: In response to the detection result including that the flag bit in the target packet does not represent the mixed coding, the target packet is discarded, and the step of sequentially selecting each of the data packets in the speech coding data as the target packets is returned.

6. The method according to claim 2, characterized in that The method further comprises: In response to the detection result including that no complete packet header or packet content is detected in the target packet, the target packet is discarded, and the step of sequentially selecting each of the data packets in the speech coding data as the target packets is returned.

7. The method according to claim 1, characterized in that The decoding is performed based on the content to be decoded to obtain a plurality of speech frames, including: Selecting the encoding algorithm represented by the encoding type in the data packet from which the content to be decoded originates as the target algorithm; The content to be decoded is decoded based on a decoding algorithm that matches the target algorithm to obtain the plurality of speech frames.

8. A speech coding method, characterized in that: include: Encode the speech frames in the original speech in sequence to obtain a data packet; wherein the content of the data packet contains speech frames encoded by the same encoding algorithm, and in the case of hybrid encoding, there are data packets encoded by different encoding algorithms, and the packet header in the data packet includes at least a mark bit and an encoding type, the mark bit is used to identify whether the data packet is hybrid encoding, and the encoding type is used to identify the encoding algorithm used by the data packet; Each of the data packets encoded in sequence based on the original voice encoding is encapsulated to obtain voice encoding data, and the voice encoding data is sent to a voice receiving end; wherein the voice receiving end decodes the voice encoding data based on the voice decoding method described in any one of claims 1 to 7.

9. A speech decoding device, characterized in that: include: A data acquisition module, used to acquire voice coding data from a voice sending end; wherein the voice coding data includes a plurality of data packets, the data packets include a packet header and packet content, the packet header includes at least a mark bit and a coding type, the mark bit is used to identify whether the data packet is mixed coding, and the coding type is used to identify the coding algorithm used by the data packet; A reading and decoding module, used for sequentially reading the packet contents in the continuous data packets that meet the decoding conditions and sequentially combining them to obtain the content to be decoded, and decoding based on the content to be decoded to obtain a plurality of voice frames until the plurality of data packets are processed; wherein the decoding conditions include that the mark bit represents the mixed coding and the coding algorithms represented by the coding type are the same; The voice restoration module is used to restore the original voice of the voice sending end based on a number of previously decoded voice frames.

10. A speech decoding device, characterized in that: include: A data encoding module, used for sequentially encoding speech frames in the original speech to obtain data packets; wherein the packet content in the data packet contains speech frames encoded by the same encoding algorithm, and in the case of hybrid encoding, there are data packets encoded by different encoding algorithms, and the packet header in the data packet at least includes a mark bit and an encoding type, the mark bit is used to identify whether the data packet is hybrid encoding, and the encoding type is used to identify the encoding algorithm used by the data packet; An encapsulation and sending module is used to encapsulate each of the data packets encoded in sequence based on the original voice encoding to obtain voice encoding data, and send the voice encoding data to a voice receiving end; wherein the voice receiving end decodes the voice encoding data based on the voice decoding device according to claim 10.

11. An electronic device, characterized in that: The method comprises at least a memory and a processor coupled to each other, wherein the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the speech decoding method according to any one of claims 1 to 7, or the speech encoding method according to claim 8.

12. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the speech decoding method described in any one of claims 1 to 7, or the speech encoding method described in claim 8.

Images