Audio reproduction method, apparatus, electronic device and storage medium

Abstract

The present disclosure discloses an audio playback method, device, electronic device and storage medium, the audio playback method including: decoding an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data; recording a reference decode start time and an actual decode processing time of the decoding process of each frame of the audio data of the at least one frame of audio data; obtaining a reference offset time and a desired playback time corresponding to the audio data; adjusting the reference decode start time for each frame of the audio data of the at least one frame of audio data based on the reference offset time, obtaining an adjusted decode start time; determining a reference playback time of the audio data based on the adjusted decode start time and the actual decode processing time; adjusting the reference playback time based on a time change; when the adjusted reference playback time reaches the desired playback time, the audio data is played according to the terminal device; and setting the audio playback time of the adjusted audio data to be equal to or shorter than the desired playback time so as to enable synchronous playback of the audio data.

Description

[Technical field]

[0001] The present application relates to the technical field of display control, and in particular to an audio playback method, device, electronic device and storage medium. [Background technology]

[0002] With the rapid development of audio playback technology, there are more and more kinds of audio players. Audio synchronization technology is an important basis for audio playback. In order to ensure audio synchronous playback, the audio server needs to calibrate the reference time of the system connected to the audio player before transmitting audio data.

[0003] However, since the audio player needs to perform back-end data processing such as decoding the audio data after receiving the audio data, it needs to reserve a decoding time for processing the audio data before playing it. Summary of the Invention [Problem to be solved by the invention]

[0004] It is difficult to standardize and identify the back-end decoding processing times for different audio data. Moreover, some audio data has a huge data volume, which requires a long decoding time. The decoding rate places high requirements on product hardware. After the hardware performance reaches a bottleneck, the desired audio playback time cannot be accurately synchronized with the reference time of the terminal where the audio player is located, which makes it easy for audio playback to cause phenomena such as audio-to-picture asynchronization, dropped frames, and playback delays. [Means for solving the problem]

[0005] SUMMARY OF THE DISCLOSURE The embodiments of the present invention provide an audio playback method, apparatus, electronic device and storage medium, which can increase the reserved time of back-end decoding processing and achieve synchronization of audio playback.

[0006] An audio reproducing method provided by an embodiment of the present invention includes: decoding an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data; said at least one frame of audio data; Recording a reference time for starting a decoding process and an actual decoding process time for each frame of audio data; obtaining a reference offset time and a desired playback time corresponding to the audio data; adjusting the decoding start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain an adjusted decoding start time; determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time; Update the reference duration based on time changes to get the adjusted reference duration To do, When the adjusted reference playback time reaches the desired playback time, the audio data is played back according to the terminal device.

[0007] Accordingly, an audio playback device provided by an embodiment of the present invention comprises: a decoding module for decoding an audio data set to be played on a terminal device, the audio data set including at least one frame of audio data; said at least one frame of audio data; a recording module for recording a reference time for starting a decoding process and an actual time for decoding the audio data of each frame; an acquisition module for acquiring a reference offset time and a desired playback time corresponding to the audio data; an adjustment module for adjusting the decoding start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain a decoding start adjustment time; a determining module for determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time; Update the reference duration based on time changes to get the adjusted reference duration a timing module for detecting a timing error; and a playback module for playing the audio data according to a terminal device when the adjusted reference playback time reaches a desired playback time.

[0008] Optionally, in some embodiments of the present application, the audio data includes current audio data, and the acquisition module: a sampling unit for sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data; said at least one frame of historical audio data; A first acquisition unit for respectively acquiring a historical reference playback time and a desired playback time after decoding processing for the historical audio data of each frame; Based on the history reference playback time and the desired playback time, said at least one frame of historical audio data; a first identifying unit for identifying an offset time of each frame of historical audio data; and a second identifying unit for identifying a reference offset time of the current audio data based on at least one of the offset times.

[0009] Optionally, in some embodiments of the present application, the second identification unit is a first specifying subunit for specifying an average value of at least one of the offset times to obtain an average offset time; and a second identifying subunit for identifying a reference offset time of the current audio data based on the average offset time.

[0010] Optionally, in some embodiments of the present application, the second identification unit is determining an average value of at least one of the offset times to obtain an average offset time; taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data; The average integer offset time is determined to be a reference offset time of the current audio data, and is further used.

[0011] Optionally, in some embodiments of the present application, the second identification unit is taking an integer number of the at least one offset time to obtain at least one integer offset time; determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on the at least one integer offset time; The integer average offset time is further used to identify the integer average offset time as a reference offset time of the current audio data.

[0012] Optionally, in some embodiments of the present application, the sampling unit comprises: obtaining historical audio data based on a predetermined frequency, the historical audio data including at least one frame of historical audio data; Sampling the at least one frame of historical audio data is further utilized.

[0013] Optionally, in some embodiments of the present application, the reference offset time includes an actual offset time, and the adjustment module: a second acquiring unit for acquiring a reservation processing time of the audio data; a third determination unit for determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; and a third obtaining unit for adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjustment time.

[0014] Optionally, in some embodiments of the present application, the second acquisition unit is The third specifying subunit specifies a reservation processing time for the audio data based on the reference time for starting decoding of the audio data and the desired playback time.

[0015] Optionally, in some embodiments of the present application, the second acquisition unit is obtaining a reservation processing time for the audio data; determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; determining an error offset time of the audio data based on the reference offset time and the actual offset time; The decoding start reference time is adjusted based on the reference offset time and the error offset time to obtain a decoding start adjustment time.

[0016] Optionally, in some embodiments of the present application, the recording module comprises: performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data; The method further includes determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time.

[0017] Optionally, in some embodiments of the present application, the specific module may include: obtaining a decoding start time and a decoding end time of a decoding process for the audio data; determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time; and determining a reference playback time of the audio data based on the base reference playback time and the decoding end time.

[0018] Optionally, in some embodiments of the present application, the acquisition module is It is used to directly extract the desired playback time of the audio data from the audio data.

[0019] Optionally, in some embodiments of the present application, the acquisition module is identifying an audio data type of the audio data; Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time; Obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type is further used. Effect of the Invention

[0020] In the embodiment of the present application, the reference playback time of the audio data is shortened, so that the reference playback time is changed with time, and the audio data is played back after the reference playback time reaches the desired playback time after the change, that is, the audio data can be played back synchronously after the reference playback time of the audio data meets the desired playback time. Among them, the reference playback time is shortened by shortening the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, shortening the reference playback time corresponding to the audio data provides a sufficient decoding time for the decoding processing of the audio data, and avoids phenomena such as audio-picture asynchronous, dropped frames, or playback delays in the playback of the audio data due to the decoding processing time being too long. [Brief description of the drawings]

[0021] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of the present application will be described in detail in conjunction with the accompanying drawings to make the technical solutions and advantages of the present invention clearer. [Figure 1] FIG. 2 is a scene schematic diagram of the audio playback method provided by the present invention; [Diagram 2] FIG. 2 is a flow diagram of an audio playback method provided by the present invention; [Diagram 3] FIG. 2 is another flow diagram of the audio playback method provided by the present invention; [Figure 4] FIG. 2 is another flow diagram of the audio playback method provided by the present invention; [Diagram 5] 1 is a schematic diagram of an audio playback device provided by the present invention; [Figure 6] 1 is a schematic diagram of an electronic device provided by the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The realization of the object, functional features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings. It should be understood that the specific examples described herein are merely for the purpose of interpreting the present application, and are not intended to limit the present application.

[0023] The present application provides an audio playback method, device, electronic device, and storage medium. Specifically, the audio playback method provided in the embodiments of the present invention can be executed by an electronic device including a terminal device or a server. The terminal device may be a television, a mobile phone, a notebook, a desktop, or a tablet. The server may be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, and may be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms, and may be directly or indirectly connected via wired or wireless communications.

[0024] Specifically, the embodiment of the present application takes as an example that the terminal device and the server jointly execute the audio playback method. Data transmission between the terminal device and the server can be performed in the form of a wired network, a wireless network, or a broadcast, and in addition to the terminal device and the server, other devices that auxiliary execute the audio playback method can also be added, and the type of the other devices is not limited here. Among them, the specific procedure for the terminal device and the server to jointly execute the audio playback method is as follows:

[0025] The terminal device receives an audio data set (audio data packet) to be played from the server, and then the processor in the terminal device decodes the received audio data set, and records the reference decode start time of each frame audio data and the actual decode processing time of each frame audio data. The terminal device also obtains a reference offset time and a desired playback time of each frame audio data from the audio data, and then adjusts the reference decode start time for each frame audio data based on the reference offset time to obtain an adjusted decode start time. Then, the terminal device determines a reference playback time for the end of the decode processing of the audio data based on the adjusted decode start time and the actual decode processing time. Finally, after the reference playback time changes with time, if the reference playback time is equal to the desired playback time, the terminal device plays the audio data. Here, if the terminal device is a TV, the audio data can be directly played without using an additional audio player.

[0026] Here, the reference time of the audio data (e.g., the reference time to start decoding) is synchronized with the reference clock of the terminal device. The reference clock is adjusted and synchronized to the correct time by the Network Time Protocol (NTP). The source of the correct time is Universal Time Coordinated (UTC). For example, when the reference clock of the terminal device indicates Beijing time, if the start time of the decoding of the audio data by the terminal device is 6 o'clock Beijing time, the reference time to start decoding is also 6 o'clock.

[0027] The reference playback time is the time that can be played back after the audio data is decoded, and in theory, the reference playback time is also synchronized with Beijing time. That is, the Beijing time after the audio data is decoded is the reference playback time of the audio data. However, the timing of the reference clock is obtained by updating after the data transmission, and the timing of the reference clock has a time error because the data transmission consumes time. Therefore, in the embodiment of the present application, the reference playback time is determined based on the reference time to start decoding and the actual decoding processing time of the decoding. Among them, the actual decoding processing time is timed by feedback of the crystal oscillation, so the time recording is very accurate, so that the actual reference playback time after the audio data is decoded can be obtained, and the time stamp of the obtained reference playback time is not synchronized with the time stamp of the reference clock, but the increase of the reference playback time increases with time (Beijing time). For example, if the reference playback time after the audio data is decoded is exactly 8 o'clock, the Beijing time is 8:01, that is, the reference playback time is not the same as the time stamp of the Beijing time.

[0028] Here, when the reference playback time of the audio data is equal to the desired playback time, the audio data can be played back; however, since it takes time to decode the audio data, the reference playback time after the audio data is decoded often exceeds the desired playback time corresponding to the audio data. Therefore, the reference playback time after the audio data is decoded is adjusted back so as to be synchronized with the desired playback time, or the reference playback time increases with time and then is synchronized with the desired playback time, and then the audio data is played back.

[0029] Here, the Network Time Protocol (NTP) is used for computer time synchronization, allowing computers to synchronize with servers or clock sources (e.g., quartz clocks, GPS, etc.) and providing highly accurate time correction (less than 1 millisecond difference from the standard on a LAN, tens of milliseconds on a WAN). Here, NTP may obtain UTC from acquisition time sources such as atomic clocks, observatories, satellites, or the Internet.

[0030] Here, the desired playback time is the desired playback time of the audio data, for example, a presentation time label (PTS, Presentation Time Stamp). When the desired playback time of the audio data is synchronized with the reference playback time of the terminal, the audio data is played back. That is, when the desired playback time of the audio data is synchronized with the international standard time and the desired playback time of the audio data is delayed from the reference playback time of the terminal, the audio data cannot be played back accurately, for example, phenomena such as audio-video asynchronization, dropped frames, playback delays, etc. occur.

[0031] In the embodiment of the present application, the reference playback time of the audio data is shortened, so that the reference playback time is changed with time, and the audio data is played back after the reference playback time reaches the desired playback time after the change, that is, the audio data can be played back synchronously after the reference playback time of the audio data meets the desired playback time. Among them, the reference playback time is shortened by shortening the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, shortening the reference playback time corresponding to the audio data provides a sufficient decoding time for the decoding processing of the audio data, and avoids phenomena such as audio-picture asynchronous, dropped frames, or playback delays in the playback of the audio data due to the decoding processing time being too long.

[0032] Each of the embodiments will be described in detail below. Note that the order of description of the embodiments below does not limit the priority of the embodiments. 2 is a flow diagram of the audio playback method provided by the present invention. The specific flow of the audio playback method is as follows.

[0033] Step 101: Decode an audio data set to be played on a terminal device, where the audio data set includes at least one frame of audio data.

[0034] Here, the audio data received and played by the terminal device is often coded and compressed data. For example, in digital television, the transmitted packet TLV is a packet of ISDB S3 standard, and the TLV packet corresponding to a 4K super high vision program generally has a very large amount of data of the program content, so in order to improve the transmission efficiency, the audio data is generally coded and compressed, and the audio data received by the terminal device is compressed data to be decoded, and the terminal device can play the audio data after decoding the audio data.

[0035] Step 102: said at least one frame of audio data;The reference time for starting the decoding process and the actual decoding process time for the audio data of each frame are recorded.

[0036] Here, the decode start reference time is synchronized with a reference clock of the terminal device, that is, when the decoding process of the audio data is started, the time of the reference clock is the decode start reference time of the audio data. Here, the actual audio data decoding processing time is the actual time required for the audio data decoding processing.

[0037] Here, the actual processing time of decoding the audio data can be accurately timed by a hardware timer, i.e., optionally, in some embodiments of the present application, said at least one frame of audio data; "Recording the actual decoding processing time for the decoding process of the audio data of each frame" performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data; The method further includes determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time.

[0038] For example, time can be measured by feedback of a quartz crystal. For example, an oscillator is used to measure time from 0:00:00 when the terminal device is powered on, and the time measurement process is a timing feedback by 27M quartz crystal, among which, measuring time by the oscillator is relatively accurate.

[0039] Here, time is measured at the start of decoding the audio data to obtain the decoding start time, and at the end of decoding the audio data to obtain the decoding end time of the audio data, and the actual decoding processing time for decoding the audio data can be obtained by the difference between the decoding end time and the decoding start time.

[0040] Step 103: Obtain a reference offset time and a desired playback time corresponding to the audio data.

[0041] Here, if the reference playback time of the audio data is longer than the desired playback time, the audio data cannot be accurately played back, and phenomena such as audio-image asynchronization, dropped frames, playback delay, etc. occur, so it is necessary to adjust the reference playback time of the audio data. That is, the adjustment of the audio data playback refers to the data of the reference playback time, and does not adjust the data corresponding to the reference clock, that is, the display time of the terminal is not adjusted.

[0042] Here, the reference offset time may be specified by the difference between a reference playback time and a desired playback time of the audio data, and the reference playback time is backed up to the difference so that the adjusted reference playback time is equal to or shorter than the desired playback time.

[0043] Here, the decoding time of audio data is at the millisecond level, so the adjustment of the reference time is also a difference of milliseconds that is difficult for the human body to perceive. For example, if the original reference playback time of audio data is 500ms at 7 o'clock, that is, the time displayed on the reference clock of the terminal device is exactly 7 o'clock, and the reference offset time is 200ms, so the adjusted reference playback time is 300ms at 7 o'clock. At this time, the time displayed on the reference clock is exactly 7 o'clock, and the adjustment of 200ms at the millisecond level is difficult for users to perceive, but the time of 200ms cannot be ignored for audio decoding, so if the original reference playback time exceeds the desired playback time, the audio data will not be played accurately. Therefore, the reference offset time is adjusted accordingly to adjust the reference playback time, so that the audio data can be played, and sufficient time is provided for decoding the audio data, which reduces the performance requirements of the hardware product, and the user is unlikely to perceive the difference in time.

[0044] In some embodiments of the present application, the reference offset time of the current audio data may be determined based on the offset status of the past history data. That is, optionally, in some embodiments, "obtaining the reference offset time corresponding to the audio data" specifically means: sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data; said at least one frame of historical audio data; Obtaining a historical reference playback time and a historical desired playback time after decoding processing for each frame of historical audio data; Based on the history reference playback time and history desired playback time, said at least one frame of historical audio data; determining an offset time of each frame of historical audio data; determining a reference offset time of the current audio data based on at least one of the offset times.

[0045] Here, based on obtaining the offset time of historical audio data of the same data type, a reference offset time to which the reference playback time of the current audio data should be adjusted is estimated, and since the historical audio data of the same data type as the current audio data is sampled, the obtaining of the reference offset time is accurate.

[0046] In order to ensure the accuracy of the reference offset time, the sampled historical audio data may be exchanged according to a certain frequency, i.e., in some embodiments, "sampling at least one frame of historical audio data" specifically means: obtaining historical audio data based on a predetermined frequency, the historical audio data including at least one frame of historical audio data; and sampling the at least one frame of historical audio data.

[0047] Here, the different or new historical audio data has a certain timeliness and accuracy as a reference for obtaining the reference offset time.

[0048] Here, determining a reference offset time of the current audio data by averaging offset times corresponding to the historical audio data; that is, optionally, in some embodiments, "determining a reference offset time of the current audio data based on at least one of the offset times" specifically means: determining an average value of at least one of the offset times to obtain an average offset time; determining a reference offset time of the current audio data based on the average offset time.

[0049] Here, the average value indicates the overall offset of the historical audio data, which is the same data type as the current audio data, and this overall offset can reflect the adjustment range of the reference playback time of the current audio data. Therefore, taking the average offset time of the historical audio data as the reference offset time of the current audio data has a certain degree of accuracy and reference value.

[0050] Here, since the decoding time is at the millisecond level, in order to improve calculation efficiency, a time of 1 millisecond or less can be ignored, and an offset value at an integer level can be adjusted for the offset time. That is, optionally, in some embodiments, "determining a reference offset time of the current audio data based on at least one of the offset times" specifically means: determining an average value of at least one of the offset times to obtain an average offset time; taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data; determining the average integer offset time as a reference offset time of the current audio data.

[0051] Here, the method of obtaining an integer may be a method of converting a decimal into an integer, for example, adjusting the decimal point to 1, and making the reference playback time after the adjustment shorter than the desired playback time, so that the audio data can be played back smoothly.

[0052] Here, the adjustment interval of the audio data reference playback time can be intuitively known from the integer reference offset time.

[0053] Here, since the decoding time is at the millisecond level, in order to improve the calculation efficiency, the time of 1 millisecond or less can be ignored, i.e., only the integer part of each offset time can be taken into consideration for selecting the reference offset time, and the integer of each offset time is taken first, and the average value of these offset times is calculated. That is, optionally, in some embodiments, "determining the reference offset time of the current audio data based on at least one of the offset times" specifically means: taking an integer number of the at least one offset time to obtain at least one integer offset time; determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on the at least one integer offset time; determining the integer average offset time as a reference offset time of the current audio data.

[0054] Here, to ensure that the adjusted reference playback time is equal to or shorter than the desired playback time, each offset time may be converted from a decimal to an integer. For example, the decimal part of each offset data is adjusted to 1, i.e., the integer part increases by 1 ms.

[0055] Here, since the reference offset time indicates the basic offset data or overall offset of the historical audio data having the same data type as the current audio data, the reference playback time can generally be adjusted to be less than or equal to the desired playback time by adjusting the reference offset time relative to the reference playback time.

[0056] Here, the reference offset time is obtained based on obtaining an average value, and the average value of each audio data offset time represents the offset of the entire audio data set, so the reference offset time obtained by the above method can meet the adjustment needs of the reference playback time to a certain extent.

[0057] However, for audio data requiring higher accuracy, in addition to the reference offset time, an error offset time that still exists can be identified based on the adjustment of the reference offset time of the audio data based on the actual offset time corresponding to the audio data, and the reference playback time of the audio data can be adjusted simultaneously based on the reference offset time and the error offset time. That is, optionally, in some embodiments, "adjusting the decoding start reference time based on the reference offset time to obtain the decoding start adjustment time" can be obtaining a reservation processing time for the audio data; determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; determining an error offset time of the audio data based on the reference offset time and the actual offset time; adjusting the decoding start reference time based on the reference offset time and the error offset time to obtain a decoding start adjusted time.

[0058] Here, since the reference offset time is calculated from the average value of the offset times of the historical audio data, there is an error even when the reference playback time is adjusted based on the reference offset time. Therefore, the error offset time is specified based on the actual offset time and the reference offset time, and after the reference playback time is adjusted based on the reference offset time, further adjustment is made based on the error offset time. This improves the accuracy of the adjustment of the reference playback time of the audio data, and ensures that the adjusted reference playback time will be equal to or shorter than the desired playback time.

[0059] For example, if the reference offset time is an integer, precision will be lost; therefore, in order to meet the precision requirement, actual audio data corresponding to the audio data of each frame is identified, an error offset time is identified based on the actual audio data and the reference offset time, and the reference playback time is adjusted based on the reference offset time, and then further adjusted based on the error offset time.

[0060] Here, for audio data requiring higher accuracy, the offset time is adjusted based on the actual offset time after the audio data is decoded, and the reference offset time further includes the actual offset time. That is, optionally, in some embodiments, "adjusting the decoding start reference time based on the reference offset time to obtain the decoding start adjustment time" can be obtaining a reservation processing time for the audio data; determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjusted time.

[0061] Here, after the audio data is decoded and before the audio data is output, the reference time for starting decoding of the audio data is adjusted based on the actual offset time of the audio data.

[0062] Here, the reserved processing time is a processing time reserved for decoding audio data, and the reserved processing time may be acquired according to the reference decoding start time and the desired playback time. That is, optionally, in some embodiments of the present application, "acquiring the reserved processing time of audio data" may be The method includes specifying a reserved processing time for the audio data based on a reference time for starting decoding of the audio data and the desired playback time.

[0063] Here, the reservation processing time of the audio data can be obtained based on the difference between the desired playback time and the reference decoding start time, and the actual offset time of the audio data decoding can be obtained based on a comparison between the reservation processing time and the actual decoding processing time.

[0064] Here, the desired playback time of the audio data can be directly extracted from the data information of the audio data. That is, optionally, in some embodiments, "obtaining the desired playback time corresponding to the audio data" specifically means: Extracting a desired playback duration of the audio data directly from the audio data.

[0065] Here, the reference offset time of the audio data may be determined based on the type of the audio data. That is, selectively, in some embodiments, "obtaining a reference offset time corresponding to the audio data" specifically means: identifying an audio data type of the audio data; Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time; and obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type.

[0066] Here, the mapping relationship set may be determined based on a conventional relationship between the type of historical audio data and the offset time, for example, a reference offset time corresponding to the data type of audio data is jointly determined based on a statistic of the type of historical audio data and the offset time.

[0067] Step 104: For each frame of audio data of the at least one frame of audio data, adjust the decoding start reference time according to the reference offset time to obtain a decoding start adjusted time.

[0068] Here, the reference time to start decoding before the audio data decoding process and the reference playback time after the decoding process are on the same timeline. That is, a change in the reference time to start decoding results in a change in the reference playback time, so that an adjustment of the reference time to start decoding when the decoding process is started corresponds to an adjustment of the reference playback time after the decoding process.

[0069] Here, reference is made to Fig. 3. Fig. 3 is a flow diagram of the reference playback time adjustment in the embodiment of the present invention, in which an adjustment can be made to the decode start reference time at the start of audio data processing, for example, after the decode start reference time is adjusted backward, a change adjustment is also made to the final reference playback time after audio data processing. That is, this corresponds to updating and adjusting the final reference playback time after adjusting the decode start reference time, and is specifically as follows. Step 111: Obtain an audio data set which is an audio data packet (TLV stream). Step 112: When starting to decode audio data in the audio data packet, record the real time of the reference clock of the terminal device at the start of the audio data decoding (ie, the reference time to start decoding the audio data) and the actual decoding processing time. Step 113: A desired playback time corresponding to the audio data is obtained. Step 114: By adjusting the decoding start reference time based on the reference offset time and the error offset time, the sum of the adjusted decoding start adjustment time and the decoding actual processing time (i.e., the reference playback time) becomes equal to or shorter than the desired playback time corresponding to the audio data. Step 115: The reference playback time after adjustment (ie, after the audio data is decoded) is updated with the continuing timing, the reference playback time is changed to the desired playback time corresponding to the audio data, and the audio data is played back at this time.

[0070] Here, by adjusting the reference decoding start time when starting to decode the audio data, the hardware device that reproduces the audio data can obtain a reference playback time that is equal to or shorter than the desired playback time. That is, after adjusting the reference decoding start time, the reference playback time obtained by the hardware device after decoding is the adjusted reference playback time.

[0071] Step 105: determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time, where the reference playback time is less than or equal to the desired playback time.

[0072] Here, the sum of the reference time to start decoding the audio data and the actual decoding processing time is the reference time after the audio data is decoded, and since the offset time of the historical audio data is referenced, the reference start time of the current audio data is adjusted so that the reference playback time after the audio data is decoded becomes equal to or shorter than the desired playback time of the audio data.

[0073] Here, the reference playback time of the audio data is determined based on the time information before the audio data is decoded and the time information after the audio data is decoded. That is, optionally, in some embodiments, "determining the reference playback time of the audio data based on the decode start adjustment time and the decode actual processing time" specifically means: obtaining a decoding start time and a decoding end time of a decoding process for the audio data; determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time; determining a reference playback time of the audio data based on the base reference playback time and the decoding end time.

[0074] Here, the base reference playback time of the audio data can be obtained by the difference between the decode start adjustment time and the decode start time, and the base reference playback time and the decode end time can be obtained by the sum of the base reference playback time and the decode end time. That is, the difference between the reference playback time and the decode start adjustment time is equal to the difference between the decode end time and the decode start time.

[0075] Here, by acquiring the reference playback time from the base reference playback time, the adjustment of the reference offset time can be adjusted at the start of decoding of the audio data, that is, the reference time for starting decoding of the audio data is adjusted.

[0076] Here, by adjusting the base reference reproduction time before the start of decoding of audio data, the decode start adjustment time after adjustment can be made to act in synchronization with the final reference reproduction time.

[0077] Step 106: Update the reference play time based on time changes to get the adjusted reference duration do.

[0078] Here, the reference playback time is a reference playback time for audio data, and when the reference playback time reaches a desired playback time, the audio data can be played back.

[0079] Here, since the reference offset time of the current audio data can be estimated from the offset time of the historical audio data, the reference playback time is adjusted back using the reference offset time so that the reference playback time is not synchronized with the time of the reference clock and is adjusted to be less than the desired playback time.

[0080] Step 107: When the adjusted reference playback time reaches the desired playback time, the audio data is played back according to the terminal device.

[0081] Here, as the reference playback time continues to be clocked, the audio data can be played back after the timestamp corresponding to the reference playback time becomes the same as the timestamp of the desired playback time. Since the decoding of the audio data is at the millisecond level, the time difference between the adjusted reference playback time and the desired playback time should also be at the millisecond level.

[0082] Here, since the difference in the timestamps corresponding to the desired playback times of each audio data in the audio data set is relatively constant, the difference in the reference playback times corresponding to each audio data after adjustment is also relatively constant, enabling continuous playback of audio.

[0083] Now, referring to Fig. 4, in the embodiment of the present application, a specific process of adjusting the reference playback time of the audio data according to the reference offset time is as follows.

[0084] Step 121: Obtain an audio data set, i.e., an audio data packet (TLV stream), obtain a timestamp of the reference time to start decoding the audio data from the audio data set, verify the timestamp of the reference time to start decoding, determine the validity of the timestamp data, and if it is valid, proceed to step 122; if it is invalid, continue to execute step 121.

[0085] Step 122: Obtain a time stamp of the desired playback time of each frame of audio data from the audio data set, verify the time stamp of the desired playback time, and determine the validity of the time stamp data. If it is valid, proceed to step 123; if it is invalid, continue to step 122.

[0086] Step 123: Record the decoding start time of the audio data decoding, and then verify the time stamp of the decoding start time to determine the validity of the time stamp data. If it is valid, proceed to step 124; if it is invalid, continue to step 123. Step 124: A base reference playback time is obtained. Step 125: The base reference playback time is adjusted based on the reference offset time (effectively, the decoding start reference time is adjusted). Step 126: The actual offset time is obtained. Step 127: Based on the reference offset time and the actual offset time, determine the error offset time of the reference playback time, adjust the reference offset time, and then make further adjustments to the base reference playback time based on the error offset time (effectively making adjustments to the decoding start reference time), obtain the base reference playback adjustment time, and verify the validity of the base reference playback adjustment time. If it is invalid, proceed to step 127; if it is valid, proceed to step 128. Step 128: Update the base reference playback adjustment time to the hardware device of audio playback, and determine the adjusted reference playback time according to the decoding actual processing time.

[0087] In the adjustment process, the reference decoding start time is adjusted based on the reference offset time first, and then the error offset time after adjustment is determined based on the actual decoding processing time. Then, the adjustment is continued based on the error offset time, thereby improving the accuracy of the adjustment of the reference decoding start time, that is, the accuracy of the final reference playback time of the audio data.

[0088] In the embodiment of the present application, the reference playback time of the audio data is shortened, so that the reference playback time is changed with time, and the audio data is played back after the reference playback time reaches the desired playback time after the change, that is, the audio data can be played back synchronously after the reference playback time of the audio data meets the desired playback time. Among them, the reference playback time is shortened by shortening the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, shortening the reference playback time corresponding to the audio data provides a sufficient decoding time for the decoding processing of the audio data, and avoids phenomena such as audio-picture asynchronous, dropped frames, or playback delays in the playback of the audio data due to the decoding processing time being too long.

[0089] In order to better implement the audio playback method described in the present disclosure, the present disclosure also provides an audio playback device according to the audio playback method, where the meanings of the nouns are the same as those in the above-mentioned audio playback method, and specific implementation details may refer to the description in the embodiments of the method.

[0090] Please refer to Figure 5. Figure 5 is a schematic block diagram of an audio playback device provided in the present application, which includes a decoding module 201, a recording module 202, an acquisition module 203, an adjustment module 204, an identification module 205, a timing module 206, and a playback module 207. Specifically, the following is included: The decoding module 201 is adapted to decode an audio data set to be played on a terminal device, said audio data set including at least one frame of audio data. Here, the audio data received and played by the terminal device is often coded and compressed data. For example, in digital television, the transmitted packet TLV is a packet of ISDB S3 standard, and the TLV packet corresponding to a 4K super high vision program generally has a very large amount of data of the program content, so in order to improve the transmission efficiency, the audio data is generally coded and compressed, and the audio data received by the terminal device is compressed data to be decoded, and the terminal device can play the audio data after decoding the audio data. The recording module 202 includes: said at least one frame of audio data; The reference time for starting the decoding process and the actual decoding process time for the audio data of each frame are recorded. Here, the decode start reference time is synchronized with a reference clock of the terminal device, that is, when the decoding process of the audio data is started, the time of the reference clock is the decode start reference time of the audio data. Here, the actual audio data decoding processing time is the actual time required for the audio data decoding processing.

[0091] Optionally, in some embodiments of the present application, the recording module 202 may: performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data; and determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time.

[0092] Here, the reference time is updated by receiving the time of the reference clock. However, since the process of data transmission and updating consumes time, it is inaccurate to obtain the actual decoding time of the audio data based on the reference time. Therefore, in the present application, the timing of the actual decoding time of the audio data is realized based on a quartz crystal oscillation timing device, and it is more accurate to use a timing process based on quartz crystal oscillation feedback.

[0093] The acquisition module 203 acquires a reference offset time and a desired playback time corresponding to the audio data.

[0094] Here, if the reference playback time of the audio data is longer than the desired playback time, the audio data cannot be accurately played back, and phenomena such as audio-image asynchronization, dropped frames, playback delay, etc. occur, so it is necessary to adjust the reference playback time of the audio data. That is, the adjustment of the audio data playback refers to the data of the reference playback time, and does not adjust the data corresponding to the reference clock, that is, the display time of the terminal is not adjusted.

[0095] Here, the reference offset time may be specified by the difference between a reference playback time and a desired playback time of the audio data, and the reference playback time is backed up to the difference so that the adjusted reference playback time is equal to or shorter than the desired playback time.

[0096] Here, the decoding time of audio data is at the millisecond level, so the adjustment of the reference time is also a difference of milliseconds that is difficult for the human body to perceive. For example, if the original reference playback time of audio data is 500ms at 7 o'clock, that is, the time displayed on the reference clock of the terminal device is exactly 7 o'clock, and the reference offset time is 200ms, so the adjusted reference playback time is 300ms at 7 o'clock. At this time, the time displayed on the reference clock is exactly 7 o'clock, and the adjustment of 200ms at the millisecond level is difficult for users to perceive, but the time of 200ms cannot be ignored for audio decoding, so if the original reference playback time exceeds the desired playback time, the audio data will not be played accurately. Therefore, the reference offset time is adjusted accordingly to adjust the reference playback time, so that the audio data can be played, and sufficient time is provided for decoding the audio data, which reduces the performance requirements of the hardware product, and the user is unlikely to perceive the difference in time.

[0097] Optionally, in some embodiments of the present application, the audio data includes current audio data, and the acquisition module 203 is further configured to: a sampling unit for sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data; said at least one frame of historical audio data; A first acquisition unit for respectively acquiring a historical reference playback time and a desired playback time after decoding processing for the historical audio data of each frame; Based on the history reference playback time and the desired playback time, said at least one frame of historical audio data; a first identifying unit for identifying an offset time of each frame of historical audio data; and a second identifying unit for identifying a reference offset time of the current audio data based on at least one of the offset times.

[0098] Here, based on obtaining the offset time of historical audio data of the same data type, a reference offset time to which the reference playback time of the current audio data should be adjusted is estimated, and since the historical audio data of the same data type as the current audio data is sampled, the obtaining of the reference offset time is accurate.

[0099] Optionally, in some embodiments of the present application, the second identification unit is a first specifying subunit for specifying an average value of at least one of the offset times to obtain an average offset time; and a second identifying subunit for identifying a reference offset time of the current audio data based on the average offset time.

[0100] Here, the average value indicates the overall offset of the historical audio data, which is the same data type as the current audio data, and this overall offset can reflect the adjustment range of the reference playback time of the current audio data. Therefore, taking the average offset time of the historical audio data as the reference offset time of the current audio data has a certain degree of accuracy and reference value.

[0101] Optionally, in some embodiments of the present application, the second identification unit is determining an average value of at least one of the offset times to obtain an average offset time; taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data; The average integer offset time is determined to be a reference offset time of the current audio data, and is further used.

[0102] Optionally, in some embodiments of the present application, the second identification unit is taking an integer number of the at least one offset time to obtain at least one integer offset time; determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on the at least one integer offset time; The integer average offset time is further used to identify the integer average offset time as a reference offset time of the current audio data.

[0103] Optionally, in some embodiments of the present application, the sampling unit comprises: obtaining historical audio data based on a predetermined frequency, the historical audio data including at least one frame of historical audio data; Sampling the at least one frame of historical audio data is further utilized.

[0104] Optionally, in some embodiments of the present application, the acquisition module 203 may further include: It is used to directly extract the desired playback time of the audio data from the audio data.

[0105] Optionally, in some embodiments of the present application, the acquisition module 203 may further include: identifying an audio data type of the audio data; Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time; Obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type is further used.

[0106] The adjustment module 204 adjusts the decoding start reference time for each frame of the at least one frame of audio data according to the reference offset time to obtain an adjusted decoding start time.

[0107] Here, the adjustment of the reference time to start decoding may be based on a reference offset time, or may be based on an actual offset time, or the error offset time may be adjusted based on the adjustment of the reference offset time, thereby ensuring the accuracy of the adjustment of the reference playback time.

[0108] Optionally, in some embodiments of the present application, the reference offset time includes an actual offset time, and the adjustment module 204: a second acquiring unit for acquiring a reservation processing time of the audio data; a third determination unit for determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; and a third obtaining unit for adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjustment time.

[0109] Optionally, in some embodiments of the present application, the second acquisition unit is The third specifying subunit specifies a reservation processing time for the audio data based on the reference time for starting decoding of the audio data and the desired playback time.

[0110] Optionally, in some embodiments of the present application, the second acquisition unit is obtaining a reservation processing time for the audio data; determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; determining an error offset time of the audio data based on the reference offset time and the actual offset time; The decoding start reference time is adjusted based on the reference offset time and the error offset time to obtain a decoding start adjustment time.

[0111] The determining module 205 determines a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time.

[0112] Optionally, in some embodiments of the present application, the identification module 205 may include: obtaining a decoding start time and a decoding end time of a decoding process for the audio data; determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time; and determining a reference playback time of the audio data based on the base reference playback time and the decoding end time.

[0113] The timing module 206 updates the reference play time based on the time change. to get the adjusted reference duration do.

[0114] The playback module 207 plays the audio data according to the terminal device when the adjusted reference playback time reaches the desired playback time.

[0115] In this embodiment, first, the decoding module 201 performs a decoding process on the audio data acquired by the terminal device, and at the same time, the recording module 202 records the decoding start reference playback time before the audio data decoding process and the decoding actual processing time of the decoding process; then, the acquisition module 203 extracts the desired playback time of each frame audio data and the reference offset time corresponding to each frame audio data from the audio data; then, the adjustment module 204 adjusts (adjusts to be smaller) the decoding start reference playback time according to the reference offset time, so that the decoding start adjustment time is not synchronized with the reference clock time; then, the identification module 204 determines the reference playback time after the audio data decoding process (i.e., the reference playback time is determined according to the decoding start adjustment time and the decoding actual processing time, and is not synchronized with the reference clock time) according to the decoding start adjustment time and the decoding actual processing time; at the same time, the timing module 206 retimes the adjusted reference playback time, and when the current time corresponding to the retimed reference playback time is equal to the desired playback time of the audio data, the playback module 207 plays the corresponding audio data. In the embodiment of the present application, the reference playback time of the audio data is shortened, so that the reference playback time is changed with time, and the audio data is played back after the reference playback time reaches the desired playback time after the change, that is, the audio data can be played back synchronously after the reference playback time of the audio data meets the desired playback time. Among them, the reference playback time is shortened by shortening the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, shortening the reference playback time corresponding to the audio data provides a sufficient decoding time for the decoding processing of the audio data, and avoids phenomena such as audio-picture asynchronous, dropped frames, or playback delays in the playback of the audio data due to the decoding processing time being too long.

[0116] The present application also provides an electronic device, and as shown in FIG. 6, a structural schematic diagram of the electronic device according to the present application is provided, which is specifically described as follows: The electronic device may include components such as a processor 401 of one or more processing cores, a memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that the electronic device structure shown in FIG. 6 does not constitute a limitation of the electronic device, which may include more or fewer components than shown, combine some components, or include a different arrangement of components, including: The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect each part of the entire electronic device, executes or executes software programs and / or modules stored in the memory 402, and performs various functions of the electronic device and processes data by calling data stored in the memory 402, thereby monitoring the entire electronic device. Optionally, the processor 401 may include one or more processing cores. Preferably, the processor 401 may integrate an application processor that mainly processes an operating system, a user interface, and application programs, and a modulation / demodulation processor that mainly processes wireless communication. Note that the above-mentioned modulation / demodulation processor does not have to be integrated into the processor 401. The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and decoding processes by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area that can store an operating system, an application program required for at least one function (e.g., a sound playback function, an image playback function, etc.), and a storage data area. The storage data area may store data created according to the use of the electronic device, etc. The memory 402 may also include a high-speed random access memory, and may further include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage device. Correspondingly, the memory 402 may further include a memory controller that provides access to the memory 402 by the processor 401. The electronic device includes a power source 403 that provides power to various components, and preferably the power source 403 is logically connected to the processor 401 via a power management system, thereby enabling functions such as charging, discharging, and managing power consumption via the power management system. The power source 403 may include any components such as one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.

[0117] The electronic device may include an input unit 404 that is used to receive input numeric or textual information and generate keyboard, mouse, joystick, optical or trackball signal inputs associated with user settings and function control.

[0118] Although not shown, the electronic device may also include a display unit, etc., which will not be described here. Specifically, in this embodiment, the processor 401 of the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402 to realize various functions as follows:

[0119] An audio data set to be played on a terminal device is decoded, the audio data set including at least one frame of audio data. said at least one frame of audio data; A reference decode start time and an actual decode processing time of the decoding process for each frame of audio data are recorded. A reference offset time and a desired playback time corresponding to the audio data are obtained. For each frame of audio data of the at least one frame of audio data, the reference decode start time is adjusted based on the reference offset time to obtain an adjusted decode start time. A reference playback time of the audio data is determined based on the adjusted decode start time and the actual decode processing time, and the reference playback time is equal to or less than the desired playback time. The reference playback time is updated based on a change in time. to get the adjusted reference duration When the adjusted reference playback time reaches the desired playback time, the audio data is played back according to the terminal device.

[0120] For specific implementations of the above operations, please refer to the above embodiments, and duplicated descriptions will not be given here. By reducing the reference playback time of the audio data, the reference playback time is changed with time, and after the change, the audio data is played back when the desired playback time is reached, i.e., after the reference playback time of the audio data meets the desired playback time, the audio data can be played back synchronously. Among these, the reference playback time is reduced by reducing the reference decoding start time, but the actual decoding processing time of the audio data is not affected or limited. Therefore, reducing the reference playback time corresponding to the audio data provides a sufficient decoding time for the decoding processing of the audio data, and prevents phenomena such as audio-picture asynchronous, dropped frames, or playback delays from occurring in the playback of the audio data due to the decoding processing time being too long.

[0121] Those skilled in the art will understand that all or part of the steps in the various methods of the above-described embodiments can be performed by instructions stored in a computer-readable storage medium and loaded and executed by a processor, or by instructions that control associated hardware.

[0122] The present application therefore provides a storage medium having stored thereon a number of instructions that can be loaded by a processor to perform the steps of any of the audio reproduction methods described herein. For example, the instructions can perform the steps of:

[0123] An audio data set to be played on a terminal device is decoded, the audio data set including at least one frame of audio data. said at least one frame of audio data;A reference decode start time and an actual decode processing time of the decoding process for each frame of audio data are recorded. A reference offset time and a desired playback time corresponding to the audio data are obtained. For each frame of audio data of the at least one frame of audio data, the reference decode start time is adjusted based on the reference offset time to obtain an adjusted decode start time. A reference playback time of the audio data is determined based on the adjusted decode start time and the actual decode processing time, and the reference playback time is equal to or less than the desired playback time. The reference playback time is updated based on a change in time. to get the adjusted reference duration When the adjusted reference playback time reaches the desired playback time, the audio data is played back according to the terminal device. For specific implementations of the above operations, please refer to the above embodiments, and duplicated descriptions will not be provided here.

[0124] Here, the storage medium may include a read only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or the like.

[0125] The instructions stored on the storage medium can perform steps in any of the audio playback methods provided in this specification, and can therefore achieve beneficial effects that can be achieved by any of the audio playback methods provided in this specification, which will not be described in detail herein with reference to the aforementioned embodiments. The audio playback method, device, electronic device, and storage medium provided in this specification have been described in detail above, and the specification has used specific examples to explain the principles and embodiments of the present invention. However, the description of the above embodiments is only used to help understand the method of the present invention and its core concept. At the same time, for those skilled in the art, there are changes in the specific embodiments and application scope based on the concept of the present invention, so the contents of this specification should not be understood as limitations of the present invention. The above is only a preferred embodiment of the present application, which does not limit the patent scope of the present application. Any equivalent structural or flow conversion made by utilizing the contents of the description and accompanying drawings of the present application, or operation directly or indirectly in other related technical fields, is also within the patent protection scope of the present application.

Claims

1. 1. A method for audio reproduction, comprising: decoding an audio data set to be played on a terminal device, the audio data set comprising at least one frame of audio data; recording a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data; obtaining a reference offset time and a desired playback time corresponding to the audio data; adjusting the decode start reference time for each frame of the audio data of the at least one frame of the audio data based on the reference offset time to obtain an adjusted decode start time; determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time, and determining whether the reference playback time is equal to or shorter than the desired playback time; updating the reference play time based on a time change to obtain an adjusted reference play time; When the adjusted reference playback time reaches a desired playback time, the audio data is played back according to the terminal device.

13. An audio playback method comprising:

2. The audio data includes current audio data, and obtaining a reference offset time corresponding to the audio data includes: sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data; obtaining a historical reference playback time and a historical desired playback time after decoding processing for each frame of the historical audio data of the at least one frame of the historical audio data; determining an offset time of each frame of the at least one frame of the historical audio data based on the historical reference playback time and the historical desired playback time; determining a reference offset time of the current audio data based on at least one of the offset times; 2. The audio reproducing method according to claim 1.

3. determining a reference offset time of the current audio data based on at least one of the offset times, determining an average value of at least one of the offset times to obtain an average offset time; determining a reference offset time of the current audio data based on the average offset time.

3. The audio reproducing method according to claim 2.

4. determining a reference offset time of the current audio data based on at least one of the offset times, determining an average value of at least one of the offset times to obtain an average offset time; taking an integer number of the average offset times to obtain an average integer offset time corresponding to the historical audio data; determining the average integer offset time to a reference offset time of the current audio data; 3. The audio reproducing method according to claim 2.

5. determining a reference offset time of the current audio data based on at least one of the offset times, taking an integer number of at least one of the offset times to obtain at least one integer offset time; determining an average integer offset time of each frame of audio data of the at least one frame of audio data based on at least one of the integer offset times; determining the integer average offset time as a reference offset time of the current audio data; 3. The audio reproducing method according to claim 2.

6. Sampling at least one frame of historical audio data obtaining historical audio data based on a predetermined frequency, the historical audio data being historical audio data of at least one frame; and sampling the at least one frame of historical audio data.

3. The audio reproducing method according to claim 2.

7. The reference offset time includes an actual offset time, and the decoding start reference time is adjusted based on the reference offset time to obtain a decoding start adjustment time. obtaining a reservation processing time for the audio data; determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; adjusting the decoding start reference time based on the actual offset time to obtain a decoding start adjustment time.

2. The audio reproducing method according to claim 1.

8. The acquiring of the reservation processing time of the audio data includes: specifying a reservation processing time for the audio data based on a reference time for starting decoding of the audio data and the desired playback time; 8. The audio reproducing method according to claim 7.

9. Adjusting the decoding start reference time based on the reference offset time to obtain a decoding start adjustment time, obtaining a reservation processing time for the audio data; determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; determining an error offset time of the audio data based on the reference offset time and the actual offset time; adjusting the decoding start reference time based on the reference offset time and the error offset time to obtain a decoding start adjustment time.

2. The audio reproducing method according to claim 1.

10. Recording an actual decoding processing time of the decoding process for each frame of the audio data of the at least one frame of the audio data includes: performing timing feedback based on a crystal oscillation to obtain a decoding start time and a decoding end time of the audio data; determining an actual processing time for decoding the audio data based on the decoding start time and the decoding end time; 2. The audio reproducing method according to claim 1.

11. determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time, obtaining a decoding start time and a decoding end time of a decoding process for the audio data; determining a base reference playback time of the audio data based on the decoding start adjustment time and the decoding start time; determining a reference playback time of the audio data based on the base reference playback time and the decoding end time, 2. The audio reproducing method according to claim 1.

12. Obtaining a desired playback time corresponding to the audio data includes: extracting a desired playback duration of said audio data directly from said audio data; 2. The audio reproducing method according to claim 1.

13. Obtaining a reference offset time corresponding to the audio data includes: identifying an audio data type of the audio data; Obtaining a mapping relationship set, the mapping relationship set including a mapping relationship between a predetermined audio data type and a predetermined reference offset time; and obtaining a reference offset time corresponding to the audio data based on the mapping relationship set and the audio data type.

2. The audio reproducing method according to claim 1.

14. 1. An audio playback device, comprising: a decoding module for decoding an audio data set to be played on a terminal device, said audio data set including at least one frame of audio data; a recording module for recording a reference time for starting a decoding process and an actual decoding process time of each frame of the audio data of the at least one frame of the audio data; an acquisition module for acquiring a reference offset time and a desired playback time corresponding to the audio data; an adjustment module for adjusting the decoding start reference time for each frame of the audio data of the at least one frame based on the reference offset time to obtain a decoding start adjustment time; a determining module for determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time; a timing module for updating the reference play time according to a time change to obtain an adjusted reference play time; a playback module for playing the audio data according to a terminal device when the adjusted reference playback time reaches a desired playback time; 1. An audio playback device comprising:

15. The audio data includes current audio data, and the acquisition module is further configured to: a sampling unit for sampling at least one frame of historical audio data, said historical audio data being of the same data type as said current audio data; a first acquiring unit for acquiring a historical reference playback time and a historical desired playback time after decoding processing for each frame of the historical audio data of the at least one frame; a first determination unit for respectively determining an offset time of each frame of the historical audio data of the at least one frame according to the historical reference playback time and the desired playback time; a second determining unit for determining a reference offset time of the current audio data based on at least one of the offset times; 15. An audio reproducing device according to claim 14.

16. The second identification unit is a first specifying subunit for specifying an average value of at least one of the offset times to obtain an average offset time; a second specifying subunit for specifying a reference offset time of the current audio data based on the average offset time; 16. An audio reproducing device according to claim 15.

17. the reference offset time includes an actual offset time, and the adjustment module a second acquiring unit for acquiring a reservation processing time of the audio data; a third determination unit for determining an actual offset time of the audio data based on the scheduled processing time and the actual decoding processing time; a third obtaining unit for adjusting the decoding start reference time according to the actual offset time to obtain a decoding start adjustment time; 15. An audio reproducing device according to claim 14.

18. The second acquisition unit is a third specifying subunit for specifying a reservation processing time of the audio data based on a reference time for starting decoding of the audio data and the desired playback time; 20. An audio reproducing device according to claim 17.

19. An electronic device including a memory, a processor, and a computer program stored on the memory and executable by the processor, the program, when executed by the processor, decoding an audio data set to be played on a terminal device, the audio data set comprising at least one frame of audio data; recording a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data; obtaining a reference offset time and a desired playback time corresponding to the audio data; adjusting the decode start reference time for each frame of the audio data of the at least one frame of the audio data based on the reference offset time to obtain an adjusted decode start time; determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time; updating the reference playing time based on the time change to obtain an adjusted reference playing time; When the adjusted reference playback time reaches a desired playback time, the audio data is played back according to the terminal device.

1. An electronic device comprising:

20. A computer-readable storage medium having a computer program stored thereon, the computer program being capable of, when executed by a processor, decoding an audio data set to be played on a terminal device, the audio data set comprising at least one frame of audio data; recording a reference time for starting decoding and an actual decoding process time for each frame of the audio data of the at least one frame of audio data; obtaining a reference offset time and a desired playback time corresponding to the audio data; adjusting the decode start reference time for each frame of the audio data of the at least one frame of the audio data based on the reference offset time to obtain an adjusted decode start time; determining a reference playback time of the audio data based on the decoding start adjustment time and the decoding actual processing time; updating the reference play time based on a time change to obtain an adjusted reference play time; When the adjusted reference playback time reaches a desired playback time, the audio data is played back according to the terminal device. A computer-readable storage medium comprising:

Images