Audio transcoding method and apparatus, electronic device, storage medium, and program product
By detecting and adjusting the audio content type and source stream volume of the audio stream, and using timestamp alignment and encapsulation insertion techniques, the volume is dynamically optimized, solving the problem of volume fluctuations in existing technologies and improving the user's audiovisual experience.
Patent Information
- Application Number
- PCT/CN2026/094154
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-07
- Filing Date
- 2026-04-30
- Publication Date
- 2026-11-12
AI Technical Summary
Existing audio transcoding technologies suffer from volume fluctuations when switching between different scenarios, resulting in an unstable audio-visual experience for users and failing to meet their needs for comfortable volume.
By detecting the audio content type and source stream volume of the original audio stream, and using timestamp alignment and encapsulation insertion techniques, the audio content type and source stream volume information are added to the original audio stream. Combined with reference volume settings and volume comparison, the target volume of the transcoded audio stream is dynamically adjusted to achieve adaptive volume optimization.
The volume is dynamically adjusted during transcoding to maintain consistency across different scenarios, thus enhancing the user's audiovisual experience.
Smart Images

Figure CN2026094154_12112026_PF_FP_ABST
Abstract
Description
Audio transcoding methods, apparatuses, electronic devices, storage media, and software products
[0001] Cross-references to related applications
[0002] This application claims priority to Chinese Patent Application No. 202510585894.3, filed on May 7, 2025, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This disclosure relates to the field of audio processing technology, and in particular to an audio transcoding method, an audio transcoding apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology
[0004] With the increasing prevalence of digital multimedia content dissemination, audio transcoding technology, as a core technology for achieving efficient transmission and adaptation of audio content, is widely used in many fields such as live streaming, video-on-demand, and online music.
[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] The purpose of this disclosure is to provide an audio transcoding method, audio transcoding apparatus, electronic device, and computer-readable storage medium, which can at least to some extent improve the problem in the related art that the user's audio-visual experience is affected by fluctuations in volume.
[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.
[0008] According to one aspect of this disclosure, an audio transcoding method is provided, comprising: in response to an acquired original audio stream, detecting an audio content type and a source stream volume of the original audio stream; adding the audio content type and the source stream volume to the original audio stream based on an alignment operation with a timestamp of the original audio stream to obtain an enhanced audio stream; determining a target volume of the transcoded audio stream based on a relationship between the source stream volume and a reference volume, wherein, when a switch in the audio content type is detected during the transcoding process, a second source stream volume after the switch is adjusted based on a first source stream volume before the switch, so as to determine the target volume based on a relationship between the adjusted second source stream volume and the reference volume.
[0009] In one embodiment of this disclosure, in response to the acquired raw audio stream, detecting the audio content type and source stream volume of the raw audio stream includes: dividing the raw audio stream based on a specified time window, collecting audio features at audio sampling points within each time window, inputting the audio features into a feature detection model, and having the feature detection model output the audio content type, noise detection result, and corresponding timestamp information, wherein the feature detection model is generated using deep learning; and inputting the raw audio stream into a volume detection model, using the corresponding output volume curve as the source stream volume.
[0010] In one embodiment of this disclosure, based on an alignment operation with the timestamp of the original audio stream, the audio content type and source stream volume are added to the original audio stream to obtain an enhanced audio stream. This includes: aligning the audio content type, noise detection result, and volume curve in units of time windows to obtain alignment information; encapsulating the alignment information into SEI data units based on the format of Supplemental Enhancement Information (SEI); and inserting the SEI data units into the original audio stream based on the corresponding timestamp information to obtain the enhanced audio stream.
[0011] In one embodiment of this disclosure, determining the target volume of a transcoded audio stream based on the relationship between the source stream volume and the reference volume includes: determining a reference time period corresponding to the transcoding process, and determining the reference volume based on the volume attributes of transcoded audio files belonging to the reference time period recorded in a big data module; calculating the ratio between the absolute difference between the source stream volume and the reference volume and the reference volume; if the ratio is less than or equal to the reference ratio, maintaining the target volume at the source stream volume; if the ratio is greater than the reference ratio, adjusting the target volume to the reference volume.
[0012] In one embodiment of this disclosure, when a change in the audio content type is detected during transcoding, adjusting the volume of the second source stream after the change based on the volume of the first source stream before the change includes: when a change in the audio content type is detected, determining the median value of the first source stream volume and the second source stream volume; and using the median value as the adjustment result of the first source stream volume.
[0013] In one embodiment of this disclosure, the method further includes: parsing the noise detection result in the SEI data unit; if it is determined based on the noise detection result that there is noise information in the original audio stream, performing a noise reduction function during the transcoding process.
[0014] In one embodiment of this disclosure, before detecting the audio content type and source stream volume of the acquired original audio stream, the method further includes: collecting audio data under different scenarios; labeling the audio data based on the scenario type, the content type, and the presence of noise to obtain labeled audio; converting the labeled audio into a Mel spectrogram, the Mel spectrogram representing the time-frequency domain features of the labeled audio; and training a deep learning model based on the Mel spectrogram to obtain the feature detection model.
[0015] In one embodiment of this disclosure, before detecting the audio content type and source stream volume of the acquired original audio stream, the method further includes: constructing the volume detection model based on a volume detection formula, wherein the volume detection formula is: L p =20*log 10 (Prms / Pref)dB, where Prams is the sound amplitude value at any moment in the original audio stream, and Pref is the maximum reference value of the sound amplitude.
[0016] In one embodiment of this disclosure, the method further includes: reporting the transcoded audio stream including the target volume to a big data module based on a timestamp dimension, so as to update the transcoded audio file in the big data module.
[0017] According to another aspect of this disclosure, an audio transcoding apparatus is provided, comprising: a detection module, configured to detect an audio content type and a source stream volume of the acquired original audio stream in response to the acquired original audio stream; an adding module, configured to add the audio content type and the source stream volume to the original audio stream based on an alignment operation with a timestamp of the original audio stream to obtain an enhanced audio stream; and a determining module, configured to determine a target volume of the transcoded audio stream based on a relationship between the source stream volume and a reference volume, wherein, when a switch in the audio content type is detected during the transcoding process, a second source stream volume is adjusted based on a first source stream volume before the switch, and the target volume is determined based on a relationship between the adjusted second source stream volume and the reference volume.
[0018] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the audio transcoding method of any one of the above by executing the executable instructions.
[0019] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the audio transcoding method described in any of the above.
[0020] The audio transcoding scheme provided in the embodiments of this disclosure obtains the audio content type and source stream volume by detecting the audio content type and source stream volume of the original audio stream. It adds the audio content type and source stream volume information to the original audio stream using timestamp alignment and encapsulation insertion technology to form an enhanced audio stream. Based on the reference volume setting, volume comparison, and volume adjustment strategy when switching audio content, it determines the target volume of the transcoded audio stream, thereby constructing an audio transcoding processing structure to dynamically and continuously adjust the volume during the transcoding process, unify the volume gain in a single and / or multiple audio streams, maintain the consistency of volume when switching between different scenes, and improve the user's audiovisual experience.
[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0023] Figure 1 shows a schematic diagram of an audio transcoding system structure according to an embodiment of the present disclosure;
[0024] Figure 2 shows a flowchart of an audio transcoding method according to an embodiment of the present disclosure;
[0025] Figure 3 shows a schematic diagram of another audio transcoding system structure in an embodiment of this disclosure;
[0026] Figure 4 shows a flowchart of another audio transcoding method in an embodiment of this disclosure;
[0027] Figure 5 shows a schematic diagram of an audio transcoding apparatus according to an embodiment of the present disclosure;
[0028] Figure 6 shows a schematic diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation
[0029] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0030] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0031] With the increasing prevalence of digital multimedia content dissemination, audio transcoding technology, as a core technology for achieving efficient audio content transmission and adaptation, is widely used in many fields such as live streaming, video-on-demand, and online music. Currently, mainstream audio transcoding technologies generally follow a fixed parameter setting pattern during operation. That is, at the start of the transcoding operation, technicians set key parameters such as audio sampling rate, bit depth, and channels once based on conditions such as the target playback device and network bandwidth, and these parameters remain unchanged throughout the entire transcoding process. Furthermore, the volume of the transcoded live stream or video-on-demand file depends entirely on the volume level of the source stream or file itself; the transcoding system lacks an active volume adjustment mechanism.
[0032] Taking the live streaming industry as an example, with the increasing diversity of live streaming content, it covers various types such as game streaming, talent shows, online education, and e-commerce sales. During a live stream, the same stream may experience multiple scene transitions, such as the alternation between live performances and promotional interludes in a live gala. The audio content types (voices, music, ambient sounds, etc.) and volume levels differ significantly in different scenes. Furthermore, in scenarios involving switching between multiple live streaming rooms, because each stream is independently transcoded without volume calibration, the sound in the live stream will jitter when users switch rooms, and the transcoded stream will also jitter, resulting in volume fluctuations. This existing audio transcoding technology, based on fixed parameter settings and lacking volume adjustment, struggles to meet users' needs for a stable and comfortable audio-visual experience and urgently requires improvement and innovation.
[0033] Figure 1 is a schematic diagram of the structure of a computer system provided in an exemplary embodiment of this application. The system includes: a plurality of terminals 120 and a server cluster 140.
[0034] Terminal 120 can be a mobile terminal such as a mobile phone, game console, tablet computer, e-book reader, smart glasses, MP4 (Moving Picture Experts Group Audio Layer IV) player, smart home device, AR (Augmented Reality) device, VR (Virtual Reality) device, etc. Alternatively, terminal 120 can also be a personal computer (PC), such as a laptop computer and a desktop computer, etc.
[0035] The terminal 120 may contain an application for providing audio transcoding.
[0036] Terminal 120 is connected to server cluster 140 via a communication network. In some embodiments, the communication network is a wired network or a wireless network.
[0037] Server cluster 140 is a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center. Server cluster 140 provides background services for applications that provide audio transcoding. In some embodiments, server cluster 140 undertakes the primary computing task, and terminal 120 undertakes the secondary computing task; or, server cluster 140 undertakes the secondary computing task, and terminal 120 undertakes the primary computing task; or, terminal 120 and server cluster 140 collaborate on computing using a distributed computing architecture.
[0038] In some embodiments, server cluster 140 is used to store audio transcoding program information.
[0039] In some embodiments, the application clients installed on different terminals 120 are the same, or the application clients installed on two terminals 120 are clients of the same type of application on different control system platforms. Depending on the terminal platform, the specific form of the application client can also differ; for example, the application client can be a mobile client, a PC client, or a World Wide Web (Web) client.
[0040] Those skilled in the art will understand that the number of terminals 120 described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.
[0041] In some embodiments, the system may further include a management device (not shown in Figure 1) connected to the server cluster 140 via a communication network. In some embodiments, the communication network is a wired network or a wireless network.
[0042] In some embodiments, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies may be used to replace or supplement the aforementioned data communication technologies.
[0043] The audio transcoding method in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and embodiments.
[0044] As shown in Figure 2, an audio transcoding method according to an embodiment of the present disclosure includes:
[0045] Step S202: In response to the acquired raw audio stream, detect the audio content type and source stream volume of the raw audio stream.
[0046] In some embodiments, the audio content type of the original audio stream can be detected based on a feature extraction and classification model, an audio fingerprint, or a rule and pattern matching method.
[0047] In some embodiments, the source stream volume is detected by a peak detection method, or by a root mean square (RMS) energy method, or by a VAD (volume detection) algorithm.
[0048] In some embodiments, taking a live broadcast gala as an example, when the gala starts broadcasting, after the system receives the original audio stream, it can use a pre-trained audio classification model to detect the type of audio content. For example, when the host starts speaking, the model identifies the audio content type as the host's explanation; when the singer starts singing, the model identifies it as a song performance.
[0049] In some embodiments, the detection of source stream volume can be determined by calculating the average energy of the audio signal over a period of time. For example, when the host is speaking normally, the source stream volume is detected to be 70dB; when the singer is singing a high note, the source stream volume reaches 85dB, etc.
[0050] Step S204: Based on the alignment operation with the timestamp of the original audio stream, add the audio content type and source stream volume to the original audio stream to obtain the enhanced audio stream.
[0051] In some embodiments, during a live broadcast, the original audio stream is continuously updated. The system records the timestamp of each audio segment. When the audio content type (such as the host's introduction or the singer's performance) and the source stream volume are detected, this information is aligned with the corresponding timestamp. According to the SEI (Supplemental Enhancement Information) format specification, this information is encapsulated into data units. Based on the timestamp, the SEI data units are inserted into the corresponding positions of the original audio stream to form an enhanced audio stream.
[0052] For example, when the host introduces the next program, the system adds the audio content type (host introduction) and source stream volume (72dB) to the corresponding timestamp of the audio stream, making it easier to analyze and process the audio later.
[0053] Step S206: Determine the target volume of the transcoded audio stream based on the relationship between the source stream volume and the reference volume. When a change in audio content type is detected during transcoding, the second source stream volume after the change is adjusted based on the first source stream volume before the change, so as to determine the target volume based on the relationship between the adjusted second source stream volume and the reference volume.
[0054] In some embodiments, a representative reference volume can be determined by analyzing the volume attributes (such as average volume, peak volume, etc.) of transcoded audio files within a reference time period using a big data module.
[0055] In some embodiments, during the transcoding process of a live gala, the reference volume can also be set to the average volume of audio from previous similar programs of the same type.
[0056] In some embodiments, the absolute difference ratio between the source stream volume and the reference volume can be calculated, and the source stream volume can be judged to meet the transcoding requirements based on the preset reference ratio, thereby adjusting the target volume.
[0057] In some embodiments, the ratio r between the source stream volume and the reference volume can also be calculated, and the target volume of the transcoded audio stream can be adjusted according to the calculated ratio r. If r > 1, it means that the source stream volume is greater than the reference volume, and the target volume can be set to the reference volume or reduced by a certain ratio. If r < 1, the target volume can be set to the reference volume or increased by a certain ratio.
[0058] In some embodiments, the source stream volume range and the reference volume range can be linearly mapped, and the target volume can be calculated using a linear interpolation formula for the source stream volume.
[0059] In some embodiments, the target volume can also be dynamically adjusted by taking into account the source stream volume, reference volume, and other audio characteristics.
[0060] In some embodiments, when switching audio content types, the median of the volume before and after the switch can be calculated and used as the adjusted volume, which can make the volume transition smooth.
[0061] In some embodiments, when switching audio content types, the volume can also be linearly transitioned from the first source stream volume to the second source stream volume.
[0062] In this embodiment, the audio content type and source stream volume of the original audio stream are detected to obtain the audio content type and source stream volume. The audio content type and source stream volume information are added to the original audio stream using timestamp alignment and encapsulation insertion technology to form an enhanced audio stream. The target volume of the transcoded audio stream is determined based on the reference volume setting, volume comparison, and volume adjustment strategy when switching audio content, thereby constructing an audio transcoding processing structure to dynamically and continuously adjust the volume during the transcoding process, unify the volume gain in a single and / or multiple audio streams, maintain the consistency of volume when switching between different scenes, and improve the user's audiovisual experience.
[0063] As shown in Figure 3, in some embodiments, the audio transcoding system 300 includes an upstream subsystem, a transcoding subsystem, and a downstream subsystem.
[0064] The upstream subsystem inputs the source file, which serves as the source audio stream for the transcoding system.
[0065] The transcoding subsystem includes an AI detection module 302, a big data module 304, and a transcoding module 306. The AI detection module 302 is used to detect the attributes of the source in real time, including sound attributes such as content type, volume, and noise, in order to detect the audio content type and source stream volume of the original audio stream.
[0066] The big data module 304 is used to record, analyze, and summarize the volume attributes of the source and transcoding stream / file in the transcoding system in real time to determine the reference volume.
[0067] The transcoding module 306 is used to dynamically configure the volume attribute value of the transcoding stream / file in real time based on AI detection and big data statistics, and to continuously transcode in real time.
[0068] The downstream subsystem is used to output transcoded audio streams.
[0069] In one embodiment of this disclosure, in response to the acquired original audio stream, detecting the audio content type and source stream volume of the original audio stream includes: dividing the original audio stream based on a specified time window, collecting audio features at audio sampling points within each time window, inputting the audio features into a feature detection model, and having the feature detection model output the audio content type, noise detection result, and corresponding timestamp information, wherein the feature detection model is generated by deep learning; and inputting the original audio stream into a volume detection model, using the corresponding output volume curve as the source stream volume.
[0070] In some embodiments, audio stream data is acquired in real time and divided into time windows, typically each containing several audio sampling points. Features are extracted from the audio data within each time window, and then these features are input into a trained AI model, i.e., a feature detection model. The model outputs a classification result for the current audio content type and a judgment on the presence of noise. For example, the output result might be "human voice, no noise" or "music, with noise." If the model has multi-category classification capabilities, it can further subdivide the type of human voice (such as male voice, female voice, children's voice, etc.) or the type of music (such as pop music, classical music, rock music, etc.) to output classification results and timestamps.
[0071] In some embodiments, the model built based on the VAD volume detection formula quantifies the volume of the original audio stream at each time point by calculating the logarithmic relationship between the root mean square amplitude of the audio signal and the reference amplitude, generating a continuous volume curve, and realizing comprehensive information detection of the original audio stream.
[0072] In this embodiment, a complete audio information detection structure is formed by combining time window segmentation and audio feature extraction techniques with a deep learning feature detection model and a volume detection formula. This structure obtains information on the audio content type, noise level, and volume changes of the original audio stream, providing a reliable data foundation for subsequent audio processing. It also improves the automation level and detection accuracy of audio processing, meeting the needs of comprehensive audio information detection in application scenarios such as audio transcoding.
[0073] In one embodiment of this disclosure, an enhanced audio stream is obtained by adding the audio content type and source stream volume to the original audio stream based on the alignment operation with the timestamp of the original audio stream. This includes: aligning the audio content type, noise detection result, and volume curve in units of time windows to obtain alignment information; encapsulating the alignment information into SEI data units based on the format of the supplementary enhancement information (SEI); and inserting the SEI data units into the original audio stream based on the corresponding timestamp information to obtain the enhanced audio stream.
[0074] In some embodiments, using time windows as indexes, the audio content type, noise detection results, and volume curves corresponding to each window are matched according to timestamps to establish a correspondence between information and the time position of the audio stream. According to the SEI format specification, the aligned information is encoded and encapsulated to form an SEI data unit containing header information (used to identify data type, length, etc.) and content data. Finally, based on the timestamp information, the SEI data unit is accurately inserted into a specific position in the original audio stream (such as between audio frames or in the metadata area), so that the original audio stream, while maintaining its original playback function, adds rich auxiliary information and provides more comprehensive data support for subsequent processing.
[0075] In this embodiment, a structure for enhancing the original audio stream information is constructed through time window alignment, SEI format encapsulation, and timestamp-based insertion techniques. Based on this structure, audio content type, noise level, and volume information can be added to the original audio stream without loss, ensuring the temporal consistency between the information and the audio stream. Without affecting the normal playback of the original audio stream, the data dimensions of the audio stream are expanded, providing richer information resources for subsequent operations such as audio transcoding and analysis, and improving the flexibility and functionality of audio processing.
[0076] In one embodiment of this disclosure, determining the target volume of a transcoded audio stream based on the relationship between the source stream volume and the reference volume includes: determining a reference time period corresponding to the transcoding process, and determining the reference volume based on the volume attributes of transcoded audio files belonging to the reference time period recorded in the big data module; calculating the absolute difference between the source stream volume and the reference volume, and the ratio between the absolute difference and the reference volume; if the ratio is less than or equal to the reference ratio, maintaining the target volume at the source stream volume; if the ratio is greater than the reference ratio, adjusting the target volume to the reference volume.
[0077] In some embodiments, a suitable reference time period (such as the last hour, the same audio transcoding time period within a day, etc.) can be determined according to the characteristics and requirements of the transcoding task.
[0078] In some embodiments, volume data (such as average volume, maximum volume, minimum volume, etc.) of transcoded audio files within the time period can be extracted from the big data module, and a representative reference volume can be determined through statistical analysis or machine learning algorithms.
[0079] In some embodiments, the absolute difference between the source stream volume and the reference volume is calculated and compared with the reference volume. This ratio is then compared with a preset reference ratio. When the ratio is within a reasonable range, the source stream volume is maintained as the target volume to preserve the characteristics of the audio source stream volume. When the ratio exceeds the range, the target volume is adjusted to the reference volume so that the transcoded audio volume conforms to big data statistical patterns and generally accepted standards, ensuring that the audio volume is appropriate in different playback scenarios.
[0080] In some embodiments, the source stream volume A in the SEI information is parsed in real time and compared with the overall audio volume value of the current transcoding system obtained from the big data module as a reference volume B, wherein...
[0081] If abs(A, B) / B <= 2%, the target volume remains the same as the source volume A.
[0082] abs(A, B) / B>2%, the transcoding volume is set to the reference volume B.
[0083] In this embodiment, a reference volume is determined through data analysis by the big data module. Combined with volume difference ratio calculation and comparison technology, a dynamic adjustment structure for the target volume of transcoded audio is constructed. This structure can automatically adjust the target volume of transcoded audio based on the relationship between the source stream volume and the big data statistical volume, thereby achieving adaptive volume optimization. This ensures that the volume of the transcoded audio conforms to the statistical law of the overall audio volume, improving the volume adaptability and listening experience of the transcoded audio in different playback environments.
[0084] As shown in Figure 4, in one embodiment of this disclosure, when a change in audio content type is detected during transcoding, the volume of the second source stream after the change is adjusted based on the volume of the first source stream before the change, including:
[0085] Step S402: When a switch in audio content type is detected, determine the median value of the first source stream volume and the second source stream volume.
[0086] Step S404: Use the median value as the adjustment result of the first source stream volume.
[0087] Step S406: Determine the reference volume based on the volume attributes of the transcoded audio files belonging to the reference time period recorded in the big data module.
[0088] Step S408: Calculate the absolute difference between the adjustment result and the reference volume, and the ratio between the absolute difference and the reference volume.
[0089] Step S410: If the ratio is less than or equal to the reference ratio, keep the target volume as the adjustment result.
[0090] Step S412: If the ratio is greater than the reference ratio, adjust the target volume to the reference volume.
[0091] In some embodiments, during the transcoding process, changes in the audio content type are monitored in real time. When a content type switch is detected (such as switching from voice to music), the source stream volume values before and after the switch (i.e., the first source stream volume and the second source stream volume) are obtained. The median value is obtained by calculating the average of the two volume values. This median value integrates the characteristics of the volume before and after the switch. This median value is used as the adjustment target for the volume after the switch, so that the volume can achieve a smooth transition at the moment of content switching.
[0092] In this embodiment, by real-time detection of audio content type and combined with a median volume calculation and adjustment strategy, a structure for smooth volume transition when switching audio content is constructed. This structure can quickly calculate and apply an appropriate intermediate volume value when the audio content type changes, effectively preventing sudden volume changes, improving the continuity of audio playback and user listening comfort, and providing a reliable solution for volume processing of different content types during audio transcoding.
[0093] In one embodiment of this disclosure, the method further includes: parsing the noise detection results in the SEI data unit; if it is determined based on the noise detection results that there is noise information in the original audio stream, performing a noise reduction function during the transcoding process.
[0094] In some embodiments, during transcoding, the SEI data units in the enhanced audio stream are first parsed to extract noise detection results. Based on preset judgment rules, such as whether it is marked as having noise or a noise intensity threshold, it is determined whether there is noise in the original audio stream. If noise is detected, the denoising algorithm integrated into the transcoding system, such as a spectrum analysis-based filtering algorithm or a deep learning denoising model, is called to process the audio stream. While preserving the effective audio signal, noise components are suppressed or eliminated, thereby improving the purity and quality of the transcoded audio.
[0095] In one embodiment of this disclosure, before detecting the audio content type and source volume of the original audio stream in response to the acquired original audio stream, the method further includes: collecting audio data under different scenarios; labeling the audio data based on scenario type, content type, and the presence of noise to obtain labeled audio; converting the labeled audio into a Mel spectrogram, wherein the Mel spectrogram represents the time-frequency domain features of the labeled audio; and training a deep learning model based on the Mel spectrogram to obtain a feature detection model.
[0096] In some embodiments, specific classification criteria are defined for scene type (e.g., indoor, outdoor, vehicle interior), content type (e.g., human voice, music, animal sounds, mechanical sounds), and the presence or absence of noise (with or without noise) so that labeling can be performed based on the above criteria.
[0097] In some embodiments, the audio signal can be segmented into multiple short frames by performing frame processing on the audio data. A Fast Fourier Transform (FFT) is performed on each frame of audio data to convert it from the time domain to the frequency domain, thus obtaining the spectrum. The spectrum is then filtered by a Mel filter bank to map it onto the Mel frequency scale, thus obtaining the Mel spectrum. The logarithm of the Mel spectrum is then taken to obtain the Mel spectrogram.
[0098] In some embodiments, deep learning model architectures include, but are not limited to, convolutional neural networks (CNN), recurrent neural networks (RNN) and their variants (such as LSTM, GRU), etc.
[0099] In some embodiments, a validation set is used to evaluate the model during training, monitoring its performance metrics (such as accuracy, recall, F1 score, etc.). Based on the evaluation results, the model is adjusted and optimized, such as adjusting the model architecture, increasing or decreasing training data, and adjusting training parameters. When the model's performance on the validation set reaches a good level and no longer improves significantly, the model is considered to have converged.
[0100] In this embodiment, a training structure for a feature detection model is constructed through multi-source audio data collection, precise annotation, Mel spectrogram transformation, and deep learning training. This structure can utilize a large amount of diverse audio data, and through annotation and feature transformation, combined with the learning capabilities of deep learning, enable the model to identify audio content types and noise. This provides a reliable model foundation for audio stream information detection and improves the intelligence and accuracy of audio processing.
[0101] In one embodiment of this disclosure, before detecting the audio content type and source stream volume of the original audio stream in response to the acquired original audio stream, the method further includes: constructing a volume detection model based on a volume detection formula, wherein the volume detection formula is: L p =20*log 10 (Prms / Pref)dB, where Prms is the sound amplitude value at any moment in the original audio stream, and Pref is the maximum reference value of the sound amplitude.
[0102] In this embodiment, based on the VAD volume detection formula, a volume detection model structure is constructed through model architecture design and formula calculation logic implementation. This structure can calculate the volume value of the original audio stream at different times and generate a continuous volume curve, providing intuitive and quantitative volume information for audio processing. This makes the audio volume processing process more standardized and accurate, meeting the volume detection needs of applications such as audio transcoding.
[0103] In one embodiment of this disclosure, the method further includes: reporting the transcoded audio stream, including the target volume, to the big data module based on the timestamp dimension to update the transcoded audio file in the big data module.
[0104] In this embodiment, by adding timestamps, reporting data, and updating the big data module, key information such as the target volume of the transcoded audio stream is promptly transmitted back to the big data module. This ensures that the audio data in the big data module is continuously updated, providing a data foundation for big data-based audio analysis, model optimization, and transcoding strategy adjustment. This forms a closed loop in audio processing from transcoding to data feedback optimization.
[0105] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.
[0106] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."
[0107] The audio transcoding apparatus 500 according to this disclosure will now be described with reference to FIG5. The audio transcoding apparatus 500 shown in FIG5 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.
[0108] The audio transcoding device 500 is manifested as a hardware module. Components of the audio transcoding device 500 may include, but are not limited to: a detection module 502, used to detect the audio content type and source stream volume of the acquired original audio stream in response to the acquired original audio stream; an adding module 504, used to add the audio content type and source stream volume to the original audio stream based on an alignment operation with the timestamp of the original audio stream, to obtain an enhanced audio stream; and a determining module 506, used to determine the target volume of the transcoded audio stream based on the relationship between the source stream volume and a reference volume, wherein, when a switch in audio content type is detected during transcoding, the second source stream volume after the switch is adjusted based on the first source stream volume before the switch, and the target volume is determined based on the relationship between the adjusted second source stream volume and the reference volume.
[0109] The electronic device 600 according to this embodiment of the present disclosure will now be described with reference to FIG6. The electronic device 600 shown in FIG6 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.
[0110] As shown in Figure 6, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including storage unit 620 and processing unit 610).
[0111] The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 610 can perform steps S202 and S208 as shown in FIG1, as well as other steps defined in the audio transcoding method of this disclosure.
[0112] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.
[0113] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0114] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0115] Electronic device 600 can also communicate with one or more external devices 660 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 640. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 650. As shown, network adapter 650 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0116] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.
[0117] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.
[0118] The program product for implementing the above-described method according to embodiments of the present disclosure may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.
[0119] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0120] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.
[0121] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.
[0122] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0123] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0124] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.
[0125] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.
[0126] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
Claims
1. An audio transcoding method, comprising: In response to the acquired raw audio stream, the audio content type and source stream volume of the raw audio stream are detected; Based on the alignment operation with the timestamp of the original audio stream, the audio content type and source stream volume are added to the original audio stream to obtain an enhanced audio stream; as well as The target volume of the transcoded audio stream is determined based on the relationship between the source stream volume and the reference volume. When a switch in the audio content type is detected during transcoding, the volume of the second source stream after the switch is adjusted based on the first source stream volume before the switch, so as to determine the target volume based on the relationship between the adjusted second source stream volume and the reference volume.
2. The audio transcoding method according to claim 1, wherein, In response to the acquired raw audio stream, the audio content type and source stream volume of the raw audio stream are detected, including: The original audio stream is divided into specified time windows. Audio features are collected at audio sampling points within each time window. These audio features are then input into a feature detection model, which outputs the audio content type, noise detection results, and corresponding timestamp information. The feature detection model is generated using deep learning. The original audio stream is input into the volume detection model, and the corresponding output volume curve is used as the source stream volume.
3. The audio transcoding method according to claim 2, wherein, Based on the alignment operation with the timestamp of the original audio stream, the audio content type and source stream volume are added to the original audio stream to obtain an enhanced audio stream, including: Using the time window as a unit, the audio content type, noise detection result, and volume curve are aligned to obtain alignment information; Based on the format of the Supplemental Enhancement Information (SEI), the aligned information is encapsulated into SEI data units; and Based on the corresponding timestamp information, the SEI data unit is inserted into the original audio stream to obtain the enhanced audio stream.
4. The audio transcoding method according to claim 1, wherein, Determining the target volume of the transcoded audio stream based on the relationship between the source stream volume and the reference volume includes: The reference time period corresponding to the transcoding process is determined, and the reference volume is determined based on the volume attributes of the transcoded audio files belonging to the reference time period recorded in the big data module. Calculate the ratio of the absolute difference between the source stream volume and the reference volume to the reference volume; If the ratio is less than or equal to the reference ratio, the target volume is maintained at the source stream volume; and If the ratio is greater than the reference ratio, the target volume is adjusted to the reference volume.
5. The audio transcoding method according to claim 2, wherein, When a switch in the audio content type is detected during transcoding, the volume of the second source stream after the switch is adjusted based on the volume of the first source stream before the switch, including: When a switch in the audio content type is detected, the median value of the first source stream volume and the second source stream volume is determined; and The median value is used as the adjustment result of the first source stream volume.
6. The audio transcoding method according to claim 3 further includes: Analyze the noise detection results in the SEI data unit; as well as If the noise detection results indicate that the original audio stream contains noise information, a noise reduction function is performed during the transcoding process.
7. The audio transcoding method according to claim 2, wherein, Before detecting the audio content type and source stream volume of the acquired raw audio stream, the method further includes: Collect audio data from different scenarios; The audio data is labeled based on the scene type, the content type, and the presence of noise to obtain labeled audio. The labeled audio is converted into a Mel spectrogram, which characterizes the time-frequency domain features of the labeled audio; and The deep learning model is trained based on the Mel spectrogram to obtain the feature detection model.
8. The audio transcoding method according to claim 2, wherein, Before detecting the audio content type and source stream volume of the acquired raw audio stream, the method further includes: The volume detection model is constructed based on the volume detection formula, which is: L p =20*log 10 (Prms / Pref)dB, where Prams is the sound amplitude value at any moment in the original audio stream, and Pref is the maximum reference value of the sound amplitude.
9. The audio transcoding method according to claim 1 further includes: The transcoded audio stream, including the target volume, is reported to the big data module based on the timestamp dimension to update the transcoded audio file in the big data module.
10. An audio transcoding apparatus, comprising: The detection module is used to detect the audio content type and source stream volume of the acquired raw audio stream in response to the acquired raw audio stream. An add module is used to add the audio content type and source stream volume to the original audio stream based on an alignment operation with the timestamp of the original audio stream, thereby obtaining an enhanced audio stream; The determination module is used to determine the target volume of the transcoded audio stream based on the relationship between the source stream volume and the reference volume. When a switch of the audio content type is detected during the transcoding process, the second source stream volume after the switch is adjusted based on the first source stream volume before the switch, so as to determine the target volume based on the relationship between the adjusted second source stream volume and the reference volume.
11. An electronic device, comprising: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the audio transcoding method of any one of claims 1 to 9 by executing the executable instructions.
12. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the audio transcoding method according to any one of claims 1 to 9.
13. A computer program product having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the audio transcoding method according to any one of claims 1 to 9.
