Data processing method, encoding end, decoding end, and communication system
By including metadata in the bitstream, the decoder uses inter-frame predictive compression technology to determine the decoded features of audio frames, solving the problem that the decoder cannot know the decoded features and improving decoding efficiency and accuracy.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- BEIJING XIAOMI MOBILE SOFTWARE CO LTD
- Filing Date
- 2025-01-22
- Publication Date
- 2026-07-30
AI Technical Summary
In existing technologies, the decoding end cannot effectively know how to obtain the decoded audio features from the bitstream, resulting in low decoding efficiency.
By including metadata in the bitstream, the decoder determines the decoded features of the audio frame based on the information and features in the metadata, and uses inter-frame predictive compression technology to improve decoding efficiency.
This enables the decoding end to accurately reproduce audio features, improving decoding efficiency and accuracy.
Smart Images
Figure CN2025074078_30072026_PF_FP_ABST
Abstract
Claims
1. A data processing method, characterized in that, The method is executed by the decoding end, and the method includes: Receive a bitstream, the bitstream including metadata of a first audio frame; The first feature is determined based on the first information in the metadata of the first audio frame; Based on the second information in the metadata and the first feature, the audio features of the first audio frame after decoding are determined.
2. The method according to claim 1, characterized in that, The audio features of the first audio frame are subjected to inter-frame prediction compression based on the first feature and the second information, and the metadata is generated by performing inter-frame prediction compression on the audio features of the first audio frame based on the first feature and the second information.
3. The method according to claim 1 or 2, characterized in that, Based on the second information in the metadata and the first feature, the audio features of the decoded first audio frame are determined, including: Based on the first feature and the scaling factor of the first feature, the audio features of the decoded first audio frame are determined, wherein the second information is the scaling factor of the first feature.
4. The method according to claim 3, characterized in that, The audio features of the first audio frame are subjected to inter-frame prediction compression based on the first feature and the scaling factor of the first feature.
5. The method according to any one of claims 1-4, characterized in that, The first feature is the candidate feature with the highest similarity to the audio features of the first audio frame; Each candidate feature is determined based on the audio features of at least one audio frame; The first information is the index of the second audio frame, and the candidate feature with the highest similarity is determined based on the features after decoding the second audio frame.
6. The method according to claim 5, characterized in that, The first information is the index of the second audio frame; the first feature is determined in the following way: Based on the index of the second audio frame, the features of the second audio frame after decoding are determined from the features of at least one audio frame after decoding. The first feature is determined based on the features of the second audio frame after decoding.
7. The method based on claim 6, characterized in that, The determination of the first feature based on the features decoded from the second audio frame includes any one of the following: The second audio frame consists of one element, and the decoded features of the second audio frame are used as the first feature. The number of second audio frames is multiple. The features of the decoded second audio frames are weighted and summed to determine the first feature.
8. The method based on claim 7, characterized in that, When there are multiple second audio frames, the weighting coefficients for each second audio frame are determined in the following way: A pre-configured mapping relationship, wherein the mapping relationship is the relationship between the time interval between the encoding times of the second audio frame and the first audio frame and the weighting coefficient of the second audio frame; or The metadata includes weighting coefficients for each of the second audio frames.
9. The method according to any one of claims 1-8, characterized in that, The audio features are used for machine hearing tasks.
10. The method according to claim 9, characterized in that, The machine hearing task includes at least one of the following: Automatic speech recognition; Automatic speaker verification; Emotion recognition, and Audio event classification.
11. A data processing method, characterized in that, The method is executed by the encoding end, and the method includes: Send a bitstream, the bitstream including metadata of the first audio frame; The metadata is used to obtain the audio features of the first audio frame after decoding.
12. The method according to claim 11, characterized in that, Inter-frame prediction compression is performed on the audio features of the first audio frame based on the first feature and the second information. The metadata is generated by inter-frame prediction compression of the audio features of the first audio frame based on a first feature and second information. The metadata includes the first information and the second information, and the first information is used to determine the first feature.
13. The method according to claim 12, characterized in that, The inter-frame prediction compression of the audio features of the first audio frame based on the first feature and the second information includes: Based on the first feature and the scaling factor of the first feature, inter-frame prediction compression is performed on the audio features of the first audio frame. The second information is the scaling factor of the first feature.
14. The method according to claim 12 or 13, characterized in that, The first feature is the candidate feature with the highest similarity to the audio feature; Each candidate feature is determined based on the audio features of at least one audio frame; The first information is the index of the second audio frame, and the candidate feature with the highest similarity is determined based on the features after decoding the second audio frame.
15. The method according to claim 14, characterized in that, When the candidate feature with the highest similarity is determined based on the audio features encoded from multiple second audio frames, the candidate feature with the highest similarity is determined by weighted summation of the audio features encoded from multiple second audio frames.
16. The method according to claim 15, characterized in that, The metadata includes weighting coefficients for the plurality of second audio frames.
17. The method according to any one of claims 14-16, characterized in that, Obtaining at least one candidate feature includes obtaining the at least one candidate feature from the storage area; The candidate features stored in the storage area are updated in the following manner: If the remaining storage space in the storage area is greater than or equal to a preset threshold, the encoded audio features of the latest acquired audio frame will be used as candidate features and stored in the storage area. If the remaining storage space in the storage area is less than a preset threshold, the candidate feature with the earliest storage time in the storage area is replaced with the audio feature encoded by the latest acquired audio frame.
18. A data processing method, characterized in that, include: The encoding end sends a bit stream, which includes metadata of the first audio frame; The decoding end receives the bit stream; The decoding end determines a first feature based on the first information in the metadata of the first audio frame, and determines the audio features of the decoded first audio frame based on the second information in the metadata and the first feature.
19. A decoding device, characterized in that, include: A transceiver module is used to receive a bit stream, the bit stream including metadata of a first audio frame; The processing module is configured to determine a first feature based on first information in the metadata of the first audio frame; Based on the second information in the metadata and the first feature, the audio features of the first audio frame after decoding are determined.
20. An encoding device, characterized in that, include: A transceiver module for sending a bit stream, the bit stream including metadata of a first audio frame; The metadata is used to obtain the audio features of the first audio frame after decoding.
21. A decoding terminal, characterized in that, The decoding end includes: One or more processors; The processor is used to execute the data processing method according to any one of claims 1 to 10.
22. An encoding terminal, characterized in that, The encoding end includes: One or more processors; The processor is used to execute the data processing method according to any one of claims 11 to 17.
23. A communication device, characterized in that, The communication device is used to perform the data processing method according to any one of claims 1 to 10, or to perform the data processing method according to any one of claims 11 to 17.
24. A communication system, characterized in that, Includes both the encoding and decoding ends; The decoding end is configured to implement the data processing method of any one of claims 1 to 10, and the encoding end is configured to implement the data processing method of any one of claims 11 to 17.
25. A storage medium storing instructions, characterized in that, When the instruction is executed on the communication device, the communication device performs the data processing method as described in any one of claims 1 to 10, or performs the data processing method as described in any one of claims 11 to 17.
26. A program product comprising at least one of a program and instructions, characterized in that, When at least one of the programs or instructions is executed by a communication device, it implements the data processing method as described in any one of claims 1 to 17.