Voice playback method, device, and computer-readable storage medium

Abstract

The embodiments of the present application disclose a method, device, and computer-readable storage medium for voice playback. The method can obtain a spectrum display period in a spectrogram display window and a refresh period of a spectrogram in the spectrogram display window. The method can determine an incremental spectrogram period and an overlapping spectrogram period between two adjacent spectrogram frames based on the refresh period and the spectrum display period. The method can calculate an incremental spectrogram segment corresponding to the incremental spectrogram period based on voice data. The method can obtain a historical spectrogram segment corresponding to the overlapping spectrogram period and fuse the historical spectrogram segment with the incremental spectrogram segment according to the voice playback sequence to obtain a target spectrogram. The method can then play the target spectrogram based on the target spectrogram and voice data. In this way, the spectrogram can be refreshed and generated in real time, improving the computational efficiency of the spectrogram, effectively avoiding jamming and incomplete spectrogram display during voice playback, and improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a voice playback method, device, and computer-readable storage medium. Background Art

[0002] With the development of information technology, voice information technology has been widely used in various industries, meeting the application needs of users, such as voice-to-text, text-to-speech, short voice broadcasts and other applications, bringing convenience to users. However, in order to enable users to understand the progress of voice playback during voice playback and improve the visual experience of the voice playback process, this can be achieved by refreshing and generating spectrograms. When refreshing and generating spectrograms, the relevant technology obtains the real-time time period during the playback of voice data and calculates the spectrogram of the time period to achieve refresh and generate spectrograms, so as to display the spectrogram corresponding to the voice data while the voice is playing, so that users can understand the playback progress of the voice data and improve the user experience.

[0003] However, when refreshing and generating a spectrogram, the relevant technology mainly calculates the spectrogram for the entire time period. The amount of calculation is large, and the calculation time is generally longer than the actual refresh cycle of the spectrogram. This may result in the calculation of the spectrogram not being completed within the refresh cycle, resulting in freezes during voice playback and incomplete display of the spectrogram, affecting the user experience. Summary of the Invention

[0004] The embodiments of the present application provide a speech playback method, device, and computer-readable storage medium that can refresh and generate a spectrogram in real time, improve the computational efficiency of the spectrogram, effectively avoid lags during speech playback and incomplete spectrogram display, and enhance the user experience.

[0005] The present invention provides a method for playing a voice message, including:

[0006] Obtaining a frequency spectrum display period in a spectrogram display window, and obtaining a refresh period of the spectrogram in the spectrogram display window;

[0007] Determining an incremental spectrogram period and an overlapping spectrogram period between two adjacent spectrogram frames according to the refresh cycle and the spectrum display cycle;

[0008] Calculating, based on the speech data, an incremental spectrogram segment corresponding to the incremental spectrogram period;

[0009] Obtaining historical spectrogram segments corresponding to the overlapping spectrogram time periods, and fusing the historical spectrogram segments with incremental spectrogram segments according to the speech playback sequence to obtain a target spectrogram;

[0010] Playing is performed according to the target spectrogram and speech data.

[0011] Accordingly, an embodiment of the present application provides a voice playback device, comprising:

[0012] an acquiring unit, configured to acquire a frequency spectrum display period in a spectrogram display window, and acquire a refresh period of the spectrogram in the spectrogram display window;

[0013] a determining unit, configured to determine an incremental spectrogram period and an overlapping spectrogram period between two adjacent spectrogram frames according to the refresh period and the spectrum display period;

[0014] a calculation unit, configured to calculate, based on the speech data, an incremental spectrogram segment corresponding to the incremental spectrogram period;

[0015] a fusion unit, configured to obtain historical spectrogram segments corresponding to the overlapping spectrogram time periods, and fuse the historical spectrogram segments with incremental spectrogram segments according to the speech playback sequence to obtain a target spectrogram;

[0016] A playback unit is used to play according to the target spectrogram and speech data.

[0017] In some embodiments, the determining unit is further configured to:

[0018] Get the current playback progress in the spectrogram display window;

[0019] Determining a period difference between the spectrum display period and the refresh period, and determining an overlapping spectrogram period between two adjacent spectrogram frames according to the period difference and the playback progress;

[0020] An incremental spectrogram period between two adjacent spectrogram frames is determined according to the refresh cycle and the playback progress.

[0021] In some embodiments, the computing unit is further configured to:

[0022] Performing pre-emphasis processing on the voice data to obtain a pre-emphasized voice signal;

[0023] Performing frame and window processing on the pre-emphasized speech signal according to a preset refresh rate to obtain a multi-frame spectrum signal;

[0024] A target spectrum signal corresponding to the incremental spectrogram period is selected from the multi-frame spectrum signals, and the incremental spectrogram period in the target spectrum signal is calculated and processed to obtain an incremental spectrogram segment.

[0025] In some embodiments, the computing unit is further configured to:

[0026] Performing short-time Fourier transform processing on the incremental spectrogram period in the target spectrum signal to obtain a target spectrum complex matrix;

[0027] performing a rotation process on the target spectrum complex matrix to obtain a rotated target spectrum complex matrix;

[0028] Mapping is performed on the amplitude values in the target spectrum complex matrix after the rotation process to obtain incremental spectrogram segments.

[0029] In some embodiments, the fusion unit is further configured to:

[0030] determining a spectrogram overlap amount between two adjacent spectrogram frames according to the overlapping spectrogram time period and the spectrum display period;

[0031] The historical spectrogram segment corresponding to the overlapping spectrogram period is determined according to the spectrogram overlap amount.

[0032] In some embodiments, the fusion unit is further configured to:

[0033] Obtaining the previous frame of historical spectrogram corresponding to the overlapping spectrogram period;

[0034] A historical spectrogram segment corresponding to the spectrogram overlap amount is determined from the previous frame of historical spectrogram.

[0035] In some embodiments, the fusion unit is further configured to:

[0036] Determining a temporal relationship of spectra in the previous frame of historical spectrogram;

[0037] Based on the time-series relationship of the spectrum, a historical spectrogram segment corresponding to the spectrogram overlap amount is selected from the subsequent direction of the historical spectrogram of the previous frame.

[0038] In some embodiments, the fusion unit is further configured to:

[0039] Determining the end time of the spectrum of the historical spectrogram segment in the previous spectrum display cycle according to the speech playback timing, and determining the start time of the spectrum of the incremental spectrogram segment;

[0040] Comparing the end time point of the spectrum with the start time point of the spectrum;

[0041] If it is detected that the end time point of the spectrum is consistent with the start time point of the spectrum, the historical spectrogram segment and the incremental spectrogram segment are spliced to obtain a target spectrogram.

[0042] In addition, an embodiment of the present application also provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the voice playback method provided in the embodiment of the present application.

[0043] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any one of the voice playback methods provided in the embodiment of the present application.

[0044] In addition, embodiments of the present application further provide a computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the voice playback methods provided in embodiments of the present application.

[0045] The embodiment of the present application can obtain the spectrum display period in the spectrogram display window and the refresh period of the spectrogram in the spectrogram display window; determine the incremental spectrogram period and the overlapping spectrogram period between two adjacent spectrogram frames based on the refresh period and the spectrum display period; calculate the incremental spectrogram segment corresponding to the incremental spectrogram period based on the speech data; obtain the historical spectrogram segment corresponding to the overlapping spectrogram period, and fuse the historical spectrogram segment with the incremental spectrogram segment according to the speech playback timing to obtain the target spectrogram; and play according to the target spectrogram and the speech data. It can be concluded that this solution can determine the incremental spectrogram period and overlapping spectrogram period between two adjacent frames of spectrogram according to the spectrum display period and refresh period of the spectrogram display window in the terminal interface when displaying the spectrogram, and then, only perform spectrogram calculation on the incremental spectrogram period of each refresh to reduce the amount of spectrogram calculation during refresh, and then, merge the calculated incremental spectrogram segment with the historical spectrogram segment corresponding to the overlapping spectrogram period to obtain the target spectrogram to be played after the refresh, and play it; in this way, the generated spectrogram can be refreshed in real time, the calculation efficiency of the spectrogram can be improved, and the phenomenon of jamming and incomplete spectrogram display during voice playback can be effectively avoided, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1This is a schematic diagram of a scenario of a voice playback system provided in an embodiment of the present application;

[0048] Figure 2 A flowchart of the steps of the voice playback method provided in an embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of the frequency spectrum distribution of the previous frame of the spectrogram in two adjacent frames of spectrogram provided in an embodiment of the present application;

[0050] Figure 4 This is a schematic diagram of the frequency spectrum distribution of the latter of two adjacent spectrograms provided in an embodiment of the present application;

[0051] Figure 5 Schematic diagram of the difference in spectral information distribution between two adjacent spectrogram frames provided in an embodiment of the present application;

[0052] Figure 6 Schematic diagram of the spectrum information distribution of the target spectrogram provided in the embodiment of the present application;

[0053] Figure 7 This is a structural diagram of a voice playback device provided in an embodiment of the present application;

[0054] Figure 8 It is a structural diagram of the computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0056] The present invention provides a method and apparatus for voice playback, and a computer-readable storage medium. The present invention will be described from the perspective of a voice playback apparatus, which can be integrated into a computer device, which can be a terminal device. The terminal device can be, but is not limited to, a television, a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smartwatch, a smart wearable device, or other similar device.

[0057] For example, see Figure 1 , is a schematic diagram of a scenario of a voice playback system provided in an embodiment of the present application. The scenario includes a terminal or a server.

[0058] The terminal or server can obtain the spectrum display period in the spectrogram display window and the refresh period of the spectrogram in the spectrogram display window; determine the incremental spectrogram period and the overlapping spectrogram period between two adjacent spectrogram frames based on the refresh period and the spectrum display period; calculate the incremental spectrogram segments corresponding to the incremental spectrogram period based on the voice data; obtain the historical spectrogram segments corresponding to the overlapping spectrogram period, and fuse the historical spectrogram segments with the incremental spectrogram segments according to the voice playback timing to obtain the target spectrogram; and play according to the target spectrogram and the voice data.

[0059] Among them, voice playback can include obtaining the spectrum display period and refresh period, determining the incremental spectrogram period and overlapping spectrogram period within each spectrum display period, calculating the incremental spectrogram segments, fusing the historical spectrogram segments with the incremental spectrogram segments, broadcasting the target spectrogram and other processing methods.

[0060] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0061] In the embodiment of the present application, the description will be made from the perspective of a voice playback device, which can be integrated into a computer device such as a terminal device or a server. Figure 2 , Figure 2 This is a flowchart of the steps of a voice playback method provided in an embodiment of the present application. Taking a terminal device as an example, when the processor on the terminal device executes the program corresponding to the voice playback method, the specific process of the voice playback method is as follows:

[0062] 101. Obtain a frequency spectrum display period in a spectrogram display window, and obtain a refresh period of a spectrogram in the spectrogram display window.

[0063] The spectrogram can be a speech spectrum graph corresponding to the speech data; the horizontal axis of the spectrogram is time, the vertical axis is frequency, and the coordinate point value is the speech data energy. It should be noted that the spectrogram uses a two-dimensional plane to express three-dimensional information, so the energy value is represented by color. Darker colors indicate stronger speech energy at that point.

[0064] The spectrogram display window may be an interface window for displaying a spectrogram, for displaying the spectrogram corresponding to the speech data, for example, a spectrogram window in an application (app). Specifically, when the speech data is played, the spectrogram corresponding to the speech data may be played simultaneously, so as to display the speech data in a visual manner, so that the user can understand the energy and playback progress of the speech data.

[0065] The spectrum display period may be the display period of the spectrogram display window when displaying the spectrogram. Specifically, the cross-sectional size of the spectrogram display window is limited, and therefore, the spectrograms that can be displayed within each unit display duration are also limited. For example, taking a frame as a unit display duration, the spectrum amount of the spectrogram that can be presented by the spectrogram display window when displaying each frame of the spectrogram is limited, and the duration between the start time scalar and the end time scalar on the horizontal axis of each frame of the spectrogram of the limited spectrum amount is regarded as the spectrum display period. For example, if the start time scalar in a certain frame of the spectrogram is 01.091 and the end time scalar is 02.092, then the spectrum display period can be determined to be 01.001, that is, the spectrum period that the spectrogram display window can display for each frame of the spectrogram is 01.001.

[0066] Among them, the refresh cycle can be the refresh time of the spectrogram display window when displaying each frame of the spectrogram, that is, when the spectrogram display window displays the spectrogram, the interval from refreshing the current frame of the spectrogram to displaying the next frame of the spectrogram, that is, the interval for refreshing the spectrogram.

[0067] In an embodiment of the present application, the speech playback process can be accompanied by the display of a spectrogram. Therefore, during the speech playback process, the spectrogram needs to be refreshed regularly to play and display the spectrogram in real time, so as to present the progress of the speech data playback process and the speech energy to the user and improve the user experience.

[0068] In order to be able to refresh and display the spectrogram in real time, the embodiment of the present application needs to first obtain the spectrum display period in the spectrogram display window. The specific process can be: obtaining the historical display record of the spectrogram display window, which can be a record of the spectrogram display test or a spectrogram playback record, selecting a frame of the target historical spectrogram from the historical playback record, and determining the spectrum display period in the spectrogram display window based on the spectrum start time and spectrum end time in the target historical spectrogram. In addition, the embodiment of the present application also needs to determine the refresh period of the spectrogram display window when displaying the spectrogram. The refresh period can be determined by the timer of the spectrogram display window, and can be specifically determined according to a preset refresh frequency, such as 24 Hz per second, that is, the preset refresh frequency determines its refresh period.

[0069] Through the above method, the spectrum display period and refresh period of the spectrogram display window when playing and displaying the spectrogram can be obtained, so that the spectrogram calculation can be performed based on the spectrum display period and refresh period to improve the calculation efficiency of the spectrogram.

[0070] 102. Determine an incremental spectrogram period and an overlapping spectrogram period between two adjacent spectrogram frames according to a refresh period and a spectrum display period.

[0071] In the embodiment of the present application, when playing voice data, the spectrogram display window will refresh and display the corresponding spectrogram, that is, the voice data corresponds to multiple spectrogram frames, and the multiple spectrogram frames have a time sequence relationship. In particular, due to the short refresh cycle, there will be some identical voice spectrum information and some different voice spectrum information between two adjacent spectrogram frames. In terms of the refresh display timing of the spectrogram, between the spectrogram of the latter frame and the spectrogram of the former frame in the two adjacent spectrogram frames, the time period occupied by the portion of the identical voice spectrum information in the spectrogram of the latter frame is the overlapping spectrogram period, and the time period occupied by the portion of the different voice spectrum information in the spectrogram of the latter frame is the incremental spectrogram period.

[0072] Among them, the overlapping spectrogram period can be the time period of the same spectrum information between two adjacent spectrogram frames displayed in the spectrogram display window. Specifically, when the spectrogram display window refreshes and displays the spectrogram, there is partially the same spectrum information between the two adjacent spectrogram frames, that is, there is overlapping voice spectrum information between the two adjacent spectrogram frames, wherein the overlapping voice spectrum information is in the latter time period of the spectrogram that is earlier in time sequence, and the overlapping voice spectrum information is in the former time period of the spectrogram that is later in time sequence; and in the embodiment of the present application, relative to the next spectrogram frame refreshed by calculation, the overlapping spectrogram period is the former time period of the spectrum display cycle in the refreshed calculated spectrogram. Based on the overlapping spectrogram period, the overlapping spectrogram segment between the next spectrogram frame and the current spectrogram frame can be determined.

[0073] Among them, the incremental spectrogram period can be the period in which there is a different portion of speech spectrum information between the current frame spectrogram displayed in the spectrogram display window and the previous frame spectrogram. Specifically, as the voice playback progresses, the spectrogram display window will refresh and calculate the next frame spectrogram according to a specific refresh cycle. The next frame spectrogram contains part of the same speech spectrum information of the current frame spectrogram and newly added different speech spectrum information. The period occupied by the newly added different speech spectrum information in the next frame spectrogram is the incremental spectrogram period. Based on the overlapping spectrogram period, the incremental spectrogram segment between the next frame spectrogram and the current frame spectrogram can be determined.

[0074] In order to improve the calculation efficiency of the spectrogram during voice playback, the embodiment of the present application can calculate the newly added voice spectrum information in each frame of the spectrogram to merge the calculated newly added voice spectrum information with some repeated voice spectrum information in the previous frame of the spectrogram; therefore, it is necessary to first determine the incremental spectrogram time period and the overlapping spectrogram time period in each refreshed spectrogram, so as to subsequently accurately calculate the voice spectrum information according to the determined time period.

[0075] In some embodiments, the incremental spectrogram period and the overlapping spectrogram period can be determined according to the playback progress of the voice data. Specifically, step 102 "determining the incremental spectrogram period and the overlapping spectrogram period between two adjacent spectrogram frames according to the refresh cycle and the spectrum display cycle" may include: obtaining the current playback progress in the spectrogram display window; determining the period difference between the spectrum display cycle and the refresh cycle, and determining the overlapping spectrogram period between two adjacent spectrogram frames according to the period difference and the playback progress; determining the incremental spectrogram period between two adjacent spectrogram frames according to the refresh cycle and the playback progress.

[0076] The playback progress can be the playback progress of a segment of voice data, which can reflect the playback progress of the voice data. Specifically, the playback progress of the voice data is represented by a certain playback moment in the playback sequence. For example, for a segment of 10 seconds of voice data, the playback time moments such as 0.005 seconds, 1.003 seconds, and 5.605 seconds in the playback sequence can all represent the voice playback progress.

[0077] Specifically, in order to determine the incremental spectrogram period and the overlapping spectrogram period when refreshing the spectrogram, the embodiment of the present application first obtains the current playback progress in the spectrogram display window to determine the current voice progress moment during the voice playback process based on the current playback progress, such as when the voice data playback progress is at 1.003 seconds, 5.605 seconds, etc. Then, the overlapping spectrogram period when refreshing the next frame of the spectrogram is calculated, specifically: the period difference between the spectrum display period and the refresh period is calculated, and the period difference is repeated between the voice spectrum information in the next frame of the spectrogram and the part of the voice spectrum information between the previous frame, and the period occupied by the repeated voice spectrum information in the next frame is determined based on the period difference and the current voice playback progress, i.e., the overlapping spectrogram period. In addition, the incremental spectrogram period when refreshing the next frame of the spectrogram is calculated. Since the difference in voice spectrum information between the latter frame and the previous frame in the adjacent two frames of the spectrogram is the newly added voice spectrum information, and the duration corresponding to the newly added voice spectrum information is the same as the refresh period, the period occupied by the newly added voice spectrum information between the adjacent two frames of the spectrogram can be determined based on the refresh period and the playback progress, i.e., the incremental spectrogram period.

[0078] In the above manner, the weight coefficient and volume of the corresponding acoustic event can be determined according to the obtained acoustic signal, so that the priority of the acoustic event can be determined subsequently according to the weight coefficient and volume.

[0079] 103. Based on the speech data, calculate the incremental spectrogram segment corresponding to the incremental spectrogram period.

[0080] Among them, the incremental spectrogram segment can be a partial spectrogram corresponding to the newly added speech spectrum information when refreshing the next frame of spectrogram, that is, the incremental spectrogram segment is only a spectrogram segment in the next frame of spectrogram to be played and displayed.

[0081] In order to improve the efficiency of spectrogram generation during speech playback, the embodiment of the present application may only calculate the newly added spectrogram portion of the next frame spectrogram relative to the previous frame spectrogram when refreshing and generating the next frame spectrogram. Specifically, after determining the incremental spectrogram time period of the next frame spectrogram, the speech sub-data / target speech data segment corresponding to the incremental spectrogram time period may be selected from the speech data, thereby generating a corresponding incremental spectrogram segment for the speech sub-data / target speech data segment. In this way, the incremental spectrogram segment can be subsequently fused and superimposed with the repeated speech spectrum portion of the current frame, without having to calculate all the speech spectrum information of the next frame spectrogram, thereby reducing the amount of calculation when refreshing and generating the next frame spectrogram, and improving the efficiency of refreshing and generating the next frame spectrogram.

[0082] In some implementations, step 103 of “calculating incremental spectrogram segments corresponding to incremental spectrogram time periods based on speech data” may include:

[0083] (103.1) Pre-emphasize the voice data to obtain a pre-emphasized voice signal.

[0084] The voice data may be in the form of a voice signal, such as a digital voice signal, an analog voice signal, etc., which is not limited here.

[0085] It's important to note that because the power spectrum of the speech signal corresponding to speech data decreases with increasing frequency, most of the speech energy is concentrated in the low-frequency portion, resulting in a very low signal-to-noise ratio in the high-frequency portion. Therefore, to improve the signal-to-noise ratio in the high-frequency portion, the speech signal can be pre-emphasized. For example, pre-emphasized speech signals can be processed using a high-pass filter to increase the signal-to-noise ratio in the high-frequency portion, thereby producing a pre-emphasized speech signal.

[0086] (103.2) The pre-emphasized speech signal is subjected to frame division and windowing processing according to a preset refresh rate to obtain a multi-frame spectrum signal.

[0087] In an embodiment of the present application, during speech playback, the pre-emphasized speech signal may be framed and windowed according to a preset refresh rate to obtain a plurality of consecutive frames of spectral signals, so that a spectrogram corresponding to each frame of the spectral signal can be subsequently generated. It should be noted that the refresh rate of the spectrogram may be 24 Hz, and the period corresponding to each frame may be 20 ms to 50 ms. Specifically, in an actual situation, the period of each frame is the same.

[0088] To ensure a smooth transition and continuity of matching information between frames, after a speech signal is framed, there will be overlapping spectral information between the spectrograms of adjacent frames. The framing and windowing process is as follows: For example, a speech signal x(t) is framed, and the signal data is converted from a one-dimensional signal to a two-dimensional signal, such as x(m, n), where m represents the frame length, representing the length of each frame, and n is the number of frames, representing the number of frames. Thus, after framing and windowing, the multi-frame spectral signal corresponding to the speech signal is obtained.

[0089] (103.3) Selecting a target spectrum signal corresponding to an incremental spectrogram period from the multi-frame spectrum signal, and performing computational processing on the incremental spectrogram period in the target spectrum signal to obtain an incremental spectrogram segment.

[0090] In an embodiment of the present application, after the pre-emphasized speech signal is framed and windowed, a specific frame of spectrum signals can be selected from multiple frames of spectrum signals to generate a spectrogram. In addition, a specific segment of spectrum signals in a specific time sequence segment of a frame of spectrum signals can be selected to generate a spectrogram. It is understood that speech signals have time sequence characteristics and can be understood as time sequence signals. Based on the time sequence relationship of the signals, the signals of the corresponding time period can be selected, such as selecting the target spectrum signal in the incremental spectrogram time period, to generate corresponding incremental spectrogram segments for all or part of the target spectrum signal.

[0091] In some embodiments, the step (103.3) of "calculating and processing the incremental spectrogram time periods in the target spectrum signal to obtain incremental spectrogram segments" may include: performing short-time Fourier transform processing on the incremental spectrogram time periods in the target spectrum signal to obtain a target spectrum complex matrix; rotating the target spectrum complex matrix to obtain a rotated target spectrum complex matrix; and mapping the amplitude values in the rotated target spectrum complex matrix to obtain incremental spectrogram segments.

[0092] For example, after determining the target spectrum signal of the incremental spectrogram period, the target spectrum signal segment corresponding to the incremental spectrogram period can be selected from the target spectrum signal to perform computational processing on the target spectrum signal segment to obtain the incremental spectrogram segment, wherein the computational processing process includes three parts: short-time Fourier transform, rotation, and mapping. Specifically, the short-time Fourier transform process is: performing a short-time Fourier transform (STFT) on the target spectrum signal segment so that the components with lower amplitudes are pulled higher relative to the components with higher amplitudes, so as to observe the periodic signal hidden in the low-amplitude noise. Thus, the target spectrum signal x(m, n) is subjected to a short-time Fourier transform to obtain X(m, n), wherein X(m, n) belongs to a complex matrix, i.e., a target spectrum complex matrix, which reflects the relationship between frequency and energy. The real part of the complex matrix represents the amplitude of the frequency, and the imaginary part of the complex matrix represents the phase of the frequency. The rotation process involves re-expressing the spectrum of the original speech frame using coordinates, with the horizontal axis representing frequency and the vertical axis representing amplitude. This coordinate axis is then rotated 90 degrees, with the horizontal axis representing amplitude and the vertical axis representing frequency, to obtain the rotated target spectrum complex matrix. Furthermore, the mapping process involves mapping the amplitude values and quantizing them to obtain the incremental spectrogram segments corresponding to the incremental spectrogram time periods.

[0093] Through the above method, after determining the incremental spectrogram time period in each refreshed spectrogram, the newly added speech spectrum information in each frame of the spectrogram can be accurately calculated according to the incremental spectrogram time period, thereby reducing the amount of calculation in the spectrogram generation process, improving the efficiency of refreshing and generating spectrograms, and being reliable.

[0094] 104. Obtain historical spectrogram segments corresponding to overlapping spectrogram periods, and fuse the historical spectrogram segments with the incremental spectrogram segments according to the speech playback sequence to obtain a target spectrogram.

[0095] Among them, the historical spectrogram segment can be an image segment containing the same speech spectrum information between two adjacent spectrogram frames. Specifically, in order to ensure a smooth transition of matching information between frames and maintain continuity, after a segment of speech signal is framed, there will be some identical spectrum information between the spectrograms of adjacent frames, that is, overlapping speech spectrum information, and the overlapping speech spectrum information corresponds to the partial image information in the spectrogram, that is, the spectrogram segment; and in the embodiment of the present application, during the speech playback process, the next frame of spectrogram needs to be refreshed and generated according to the preset refresh cycle / refresh rate, and the overlapping speech spectrum information in the next frame of spectrogram is the same partial speech spectrum information as the previous frame (current frame). Relative to the next frame of spectrogram, the same partial speech spectrum information is historical speech spectrum information, that is, the corresponding image can be regarded as a historical spectrogram segment.

[0096] In order to refresh and generate the target spectrogram of the next frame, the embodiment of the present application can obtain the historical spectrogram segment with the overlapping spectrogram period in the current frame after calculating and generating the incremental spectrogram segment corresponding to the incremental spectrogram period, so as to merge the calculated incremental spectrogram segment with the historical spectrogram segment of the current frame, thereby refreshing and generating the target spectrogram of the next frame for subsequent playback.

[0097] In some implementations, "obtaining historical spectrogram segments corresponding to overlapping spectrogram time periods" in step 104 may include:

[0098] (104.1) Determine the amount of spectrogram overlap between two adjacent spectrogram frames based on the overlapping spectrogram time period and the spectrum display period.

[0099] Among them, the spectrogram overlap amount is the amount of image information containing the same speech spectrum information between any two adjacent spectrogram frames, or the image width size of the speech spectrum information, or the spectrum information time period corresponding to the same speech spectrum information between any two adjacent spectrogram frames. For example, the speech spectrum information of the time period 01.151 to 02.152 in the previous frame spectrogram and the time period 01.151 to 02.152 in the next frame spectrogram are the same.

[0100] In order to determine the amount of spectrogram overlap for refreshing and generating the next frame of spectrogram, an embodiment of the present application can determine the amount of spectrogram overlap between the current frame and the next frame of spectrogram based on the overlapping spectrogram time period and the spectrum display period, so as to subsequently generate a historical spectrogram segment corresponding to the overlapping spectrogram time period based on the spectrogram overlap amount.

[0101] (104.2) Determine the historical spectrogram segment corresponding to the overlapping spectrogram period based on the spectrogram overlap amount.

[0102] Specifically, after determining the amount of spectrogram overlap, the historical spectrogram segment corresponding to the next frame can be determined. For example, step (104.2) "determining the historical spectrogram segment corresponding to the overlapping spectrogram period based on the amount of spectrogram overlap" may include:

[0103] (104.2.1) Obtain the previous frame of historical spectrogram corresponding to the overlapping spectrogram period.

[0104] Since the overlapping spectrogram period is the spectrum period corresponding to the same speech spectrum information between two adjacent spectrogram frames, the previous historical spectrogram frame can be the spectrogram that first completely occupies the overlapping spectrogram period. It should be noted that, relative to the refreshed next target spectrogram frame, the previous historical spectrogram frame can be understood as the current frame.

[0105] (104.2.2) Determine the historical spectrogram segment corresponding to the spectrogram overlap amount from the previous frame of the historical spectrogram.

[0106] After determining the previous frame of historical spectrogram corresponding to the overlapping spectrogram period, the historical spectrogram segment corresponding to the spectrogram overlap amount can be determined from the previous frame of historical spectrogram. Specifically, determining the historical spectrogram segment corresponding to the spectrogram overlap amount can include: determining the time-series relationship of the spectrograms in the previous frame of historical spectrogram; and selecting the historical spectrogram segment corresponding to the spectrogram overlap amount from the subsequent direction of the previous frame of historical spectrogram based on the time-series relationship of the spectrograms.

[0107] In some embodiments, after determining the historical spectrogram segments corresponding to the overlapping spectrogram time periods, a target spectrogram can be generated based on the historical spectrogram segments and the incremental spectrogram segments. For example, the "merging the historical spectrogram segments with the incremental spectrogram segments according to the speech playback timing to obtain the target spectrogram" in step 104 may include: determining the end time of the spectrum of the historical spectrogram segment in the previous spectrum display cycle and the start time of the spectrum of the incremental spectrogram segment according to the speech playback timing; comparing the end time of the spectrum with the start time of the spectrum; and if it is detected that the end time of the spectrum is consistent with the start time of the spectrum, then splicing the historical spectrogram segment with the incremental spectrogram segment to obtain the target spectrogram.

[0108] Specifically, the spectrum end time point of the historical spectrogram segment and the spectrum start time point of the incremental spectrogram segment are determined based on the speech playback timing, and the timing relationship between these two time points is used to splice the historical spectrogram segment and the incremental spectrogram segment, thereby completing the refresh and generating of the next frame of spectrogram, that is, the target spectrogram.

[0109] Through the above method, after calculating the incremental spectrogram segment, the incremental spectrogram segment can be spliced with the historical spectrogram segment of the previous frame to refresh and generate the target spectrogram of the next frame; in this way, when generating the target spectrogram, there is no need to additionally calculate the repeated speech spectrum information between adjacent frames, which reduces the amount of calculation when generating the spectrogram and improves the efficiency of generating the spectrogram.

[0110] 105. Play according to the target spectrogram and voice data.

[0111] Specifically, after obtaining the target spectrogram, the embodiment of the present application can play the target spectrogram and voice data, thereby realizing real-time playback of the target spectrogram, effectively avoiding freezes during voice playback and incomplete spectrogram display, and improving user experience.

[0112] In an embodiment of the present application, by determining the spectrum display period and refresh period of the spectrogram display window, and determining the incremental spectrogram period and overlapping spectrogram period of the next frame spectrogram relative to the current frame based on the spectrum display period and refresh period, the incremental spectrogram segment corresponding to the incremental spectrogram period in the next frame spectrogram is calculated, and the historical spectrogram segment in the previous frame spectrogram is obtained, and then the incremental spectrogram segment and the historical spectrogram segment are spliced to complete the refresh and generate the target spectrogram of the next frame. It should be noted that since the refresh period is greater than the incremental spectrogram period, that is, the refresh period can complete the calculation of the newly added part of the speech spectrum information in advance, so that during the speech playback process, each frame of the target spectrogram can be played and displayed on time, effectively avoiding the jamming phenomenon.

[0113] By implementing any one or combination of the embodiments of the present application, an application scenario of refreshing the spectrogram during speech playback can be realized. To facilitate understanding of the implementation application scenario, the following example illustrates the embodiment of the present application, and the specific scenario example is as follows:

[0114] See also Figure 3 and Figure 4 As shown, in the display page of the spectrogram, the width of the spectrogram represents a certain time period of the audio, recorded as t (seconds). During the speech playback process, time moves forward, and the spectrogram is refreshed by the timer. Assuming that the refresh period of the timer is t1 (seconds), the time interval between two adjacent frames of the spectrogram is also t1 (seconds). It can be seen that the overlapping time portion of the two adjacent frames of the spectrogram is (t-t1) / t, that is, the incremental spectrogram that needs to be calculated is also the part of t1, and the other parts use the spectrogram of the previous frame. It can be seen that the time t1 is the timer refresh period, which is generally 20-60 milliseconds. If the value of t is larger, the more overlapping parts, then the smaller the incremental time, and the fewer incremental spectrograms that need to be calculated.

[0115] For example, Figure 3 and Figure 4 As shown, the time period t represented by the current two-frame spectrogram is 1.119 (seconds). Assuming that the refresh period of the timer is 60ms, then Figure 3 and Figure 4 The time interval t1 is also 60ms. By calculating (t-t1) / t, we can know that the time interval between two adjacent frames is t1. Figure 3 and Figure 4 The overlapping time portion is (1.119-0.060) / 1.119=94.6%, that is, the incremental spectrogram that needs to be calculated is 5.4% of the original one-frame spectrogram calculation amount, which greatly reduces the amount of calculation. In addition, we can also use Figure 3 and Figure 4 The comparison between the two frames, i.e. Figure 5As shown in FIG, the speech spectrum information of part A in the previous frame (current frame) and the speech spectrum information of part A in the next frame are the same speech spectrum information, and the speech spectrum information of part B in the next frame is newly added speech spectrum information. There are many overlapping parts between the two adjacent frames. In order to reduce the amount of calculation when refreshing the next frame spectrogram, this part of the incremental spectrogram can be spliced on the old spectrogram, and the spectrogram segment of part A in the previous frame (current frame) is spliced with the incremental spectrogram of part B in the next frame, that is, A+B, so as to obtain the target spectrogram of the next frame (as shown in FIG). Figure 6 ), in this way, the refresh of the entire spectrogram can be completed. Compared with the complete calculation of the spectrogram, the amount of calculation is greatly reduced, achieving the purpose of smooth playback.

[0116] From the above, it can be seen that the embodiments of the present application can obtain the spectrum display period in the spectrogram display window, and obtain the refresh period of the spectrogram in the spectrogram display window; determine the incremental spectrogram period and the overlapping spectrogram period between two adjacent frames of spectrogram according to the refresh period and the spectrum display period; calculate the incremental spectrogram segment corresponding to the incremental spectrogram period based on the speech data; obtain the historical spectrogram segment corresponding to the overlapping spectrogram period, and fuse the historical spectrogram segment with the incremental spectrogram segment according to the speech playback timing to obtain the target spectrogram; and play according to the target spectrogram and speech data. It can be concluded that this solution can determine the incremental spectrogram period and overlapping spectrogram period between two adjacent frames of spectrogram according to the spectrum display period and refresh period of the spectrogram display window in the terminal interface when displaying the spectrogram, and then, only perform spectrogram calculation on the incremental spectrogram period of each refresh to reduce the amount of spectrogram calculation during refresh, and then, merge the calculated incremental spectrogram segment with the historical spectrogram segment corresponding to the overlapping spectrogram period to obtain the target spectrogram to be played after the refresh, and play it; in this way, the generated spectrogram can be refreshed in real time, the calculation efficiency of the spectrogram can be improved, and the phenomenon of jamming and incomplete spectrogram display during voice playback can be effectively avoided, thereby improving the user experience.

[0117] In order to better implement the above method, an embodiment of the present application further provides a voice playback device, which can be integrated into a computer device, such as a server or terminal.

[0118] For example, Figure 7 As shown, the voice playback device may include an acquisition unit 301 , a determination unit 302 , a calculation unit 303 , a fusion unit 304 and a playback unit 305 .

[0119] An acquiring unit 301 is configured to acquire a frequency spectrum display period in a spectrogram display window and a refresh period of a spectrogram in the spectrogram display window;

[0120] A determining unit 302 is configured to determine an incremental spectrogram period and an overlapping spectrogram period between two adjacent spectrogram frames according to a refresh period and a spectrum display period;

[0121] A calculation unit 303 is configured to calculate an incremental spectrogram segment corresponding to an incremental spectrogram period based on the speech data;

[0122] A fusion unit 304 is configured to obtain historical spectrogram segments corresponding to overlapping spectrogram time periods, and fuse the historical spectrogram segments with the incremental spectrogram segments according to the speech playback sequence to obtain a target spectrogram;

[0123] The playing unit 305 is used to play according to the target spectrogram and speech data.

[0124] In some embodiments, the determination unit 302 is further used to: obtain the current playback progress in the spectrogram display window; determine the period difference between the spectrum display period and the refresh period, and determine the overlapping spectrogram period between two adjacent spectrogram frames based on the period difference and the playback progress; determine the incremental spectrogram period between two adjacent spectrogram frames based on the refresh period and the playback progress.

[0125] In some embodiments, the computing unit 303 is further used to: perform pre-emphasis processing on the voice data to obtain a pre-emphasized voice signal; perform frame and window processing on the pre-emphasized voice signal according to a preset refresh rate to obtain a multi-frame spectrum signal; select a target spectrum signal corresponding to an incremental spectrogram period from the multi-frame spectrum signal, and perform computational processing on the incremental spectrogram period in the target spectrum signal to obtain an incremental spectrogram segment.

[0126] In some embodiments, the computing unit 303 is further used to: perform short-time Fourier transform processing on the incremental spectrogram period in the target spectrum signal to obtain a target spectrum complex matrix; perform rotation processing on the target spectrum complex matrix to obtain a rotated target spectrum complex matrix; perform mapping processing on the amplitude values in the rotated target spectrum complex matrix to obtain an incremental spectrogram segment.

[0127] In some embodiments, the fusion unit 304 is further used to: determine the amount of spectrogram overlap between two adjacent spectrogram frames based on the overlapping spectrogram time period and the spectrum display period; and determine the historical spectrogram segment corresponding to the overlapping spectrogram time period based on the spectrogram overlap amount.

[0128] In some embodiments, the fusion unit 304 is further configured to: obtain a previous frame of historical spectrogram corresponding to the overlapping spectrogram period; and determine a historical spectrogram segment corresponding to the spectrogram overlap amount from the previous frame of historical spectrogram.

[0129] In some embodiments, the fusion unit 304 is further used to: determine the spectrum time sequence relationship from the previous frame historical spectrogram; based on the spectrum time sequence relationship, select the historical spectrogram segment corresponding to the spectrogram overlap amount from the subsequent direction of the previous frame historical spectrogram.

[0130] In some embodiments, the fusion unit 304 is further used to: determine the spectrum end time point of the historical spectrogram segment in the previous spectrum display cycle according to the speech playback timing, and determine the spectrum start time point of the incremental spectrogram segment; compare the spectrum end time point with the spectrum start time point; if it is detected that the spectrum end time point is consistent with the spectrum start time point, then splice the historical spectrogram segment and the incremental spectrogram segment to obtain the target spectrogram.

[0131] As can be seen from the above, the embodiment of the present application can obtain the spectrum display period in the spectrogram display window and the refresh period of the spectrogram in the spectrogram display window through the acquisition unit 301; determine the incremental spectrogram period and the overlapping spectrogram period between two adjacent frames of spectrogram according to the refresh period and the spectrum display period through the determination unit 302; calculate the incremental spectrogram segment corresponding to the incremental spectrogram period based on the speech data through the calculation unit 303; obtain the historical spectrogram segment corresponding to the overlapping spectrogram period through the fusion unit 304, and fuse the historical spectrogram segment with the incremental spectrogram segment according to the speech playback timing to obtain the target spectrogram; and play according to the target spectrogram and the speech data through the playback unit 305. It can be concluded that this solution can determine the incremental spectrogram period and overlapping spectrogram period between two adjacent frames of spectrogram according to the spectrum display period and refresh period of the spectrogram display window in the terminal interface when displaying the spectrogram, and then, only perform spectrogram calculation on the incremental spectrogram period of each refresh to reduce the amount of spectrogram calculation during refresh, and then, merge the calculated incremental spectrogram segment with the historical spectrogram segment corresponding to the overlapping spectrogram period to obtain the target spectrogram to be played after the refresh, and play it; in this way, the generated spectrogram can be refreshed in real time, the calculation efficiency of the spectrogram can be improved, and the phenomenon of jamming and incomplete spectrogram display during voice playback can be effectively avoided, thereby improving the user experience.

[0132] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0133] The present application also provides a computer device, such as Figure 8 , which shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0134] The computer device may include one or more processing core processors 401, one or more computer readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will understand that Figure 8 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0135] Processor 401 is the control center of the computer device. It connects the various components of the entire computer device using various interfaces and lines. By running or executing software programs and / or modules stored in memory 402 and accessing data stored in memory 402, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, processor 401 may include one or more processing cores; preferably, processor 401 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 401.

[0136] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and voice playback by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0137] The computer device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 403 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0138] The computer device may further include an input unit 404, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0139] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device will load the executable files corresponding to one or more application processes into the memory 402 according to the following instructions, and the processor 401 will run the application stored in the memory 402 to implement various functions as follows:

[0140] Obtain the spectrum display period in the spectrogram display window, and obtain the refresh period of the spectrogram in the spectrogram display window; determine the incremental spectrogram period and the overlapping spectrogram period between two adjacent spectrogram frames according to the refresh period and the spectrum display period; calculate the incremental spectrogram segments corresponding to the incremental spectrogram period based on the speech data; obtain the historical spectrogram segments corresponding to the overlapping spectrogram period, and fuse the historical spectrogram segments with the incremental spectrogram segments according to the speech playback sequence to obtain the target spectrogram; and play according to the target spectrogram and the speech data.

[0141] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0142] From the above, it can be seen that the embodiment of the present application can determine the incremental spectrogram period and the overlapping spectrogram period between two adjacent frames of spectrogram according to the spectrum display period and refresh period of the spectrogram display window in the terminal interface when displaying the spectrogram, and then, only perform spectrogram calculation on the incremental spectrogram period of each refresh to reduce the amount of spectrogram calculation during refresh, and then, merge the calculated incremental spectrogram segment with the historical spectrogram segment corresponding to the overlapping spectrogram period to obtain the target spectrogram to be played after the refresh, and play it; in this way, the generated spectrogram can be refreshed in real time, the calculation efficiency of the spectrogram can be improved, and the phenomenon of jamming and incomplete spectrogram display during voice playback can be effectively avoided, thereby improving the user experience.

[0143] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0144] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the voice playback methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0145] Obtain the spectrum display period in the spectrogram display window, and obtain the refresh period of the spectrogram in the spectrogram display window; determine the incremental spectrogram period and the overlapping spectrogram period between two adjacent spectrogram frames according to the refresh period and the spectrum display period; calculate the incremental spectrogram segments corresponding to the incremental spectrogram period based on the speech data; obtain the historical spectrogram segments corresponding to the overlapping spectrogram period, and fuse the historical spectrogram segments with the incremental spectrogram segments according to the speech playback sequence to obtain the target spectrogram; and play according to the target spectrogram and the speech data.

[0146] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0147] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0148] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the voice playback method provided in various optional implementations of the above embodiments.

[0149] Since the instructions stored in the computer-readable storage medium can execute the steps in any one of the voice playback methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the voice playback methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0150] The above is a detailed introduction to a voice playback method, device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A voice playing method, characterized in that: include: Obtaining a frequency spectrum display period in a spectrogram display window, and obtaining a refresh period of the spectrogram in the spectrogram display window; According to the refresh cycle and the spectrum display cycle, the incremental spectrogram period and the overlapping spectrogram period between two adjacent spectrogram frames are determined: the period difference between the spectrum display period and the refresh cycle is calculated, the speech spectrum information in the next frame of the spectrogram is repeated with part of the speech spectrum information between the previous frame in the period difference, and the period occupied by the repeated speech spectrum information in the next frame is determined as the overlapping spectrogram period according to the period difference and the current speech playback progress; the period occupied by the newly added speech spectrum information between two adjacent spectrogram frames is determined as the incremental spectrogram period according to the refresh cycle and the current playback progress in the spectrogram display window; Calculating, based on the speech data, an incremental spectrogram segment corresponding to the incremental spectrogram period; Obtaining a historical spectrogram segment corresponding to the overlapping spectrogram period, and fusing the historical spectrogram segment with the incremental spectrogram segment according to the speech playback timing to obtain a target spectrogram: determining the spectrum end time point of the historical spectrogram segment in the previous spectrum display cycle and the spectrum start time point of the incremental spectrogram segment according to the speech playback timing; comparing the spectrum end time point with the spectrum start time point; if it is detected that the spectrum end time point is consistent with the spectrum start time point, splicing the historical spectrogram segment with the incremental spectrogram segment to obtain a target spectrogram; Playing is performed according to the target spectrogram and speech data.

2. The method according to claim 1, characterized in that The step of calculating the incremental spectrogram segment corresponding to the incremental spectrogram period based on the speech data includes: Performing pre-emphasis processing on the voice data to obtain a pre-emphasized voice signal; Performing frame and window processing on the pre-emphasized speech signal according to a preset refresh rate to obtain a multi-frame spectrum signal; A target spectrum signal corresponding to the incremental spectrogram period is selected from the multi-frame spectrum signals, and calculation processing is performed on the incremental spectrogram period in the target spectrum signal to obtain the incremental spectrogram segment.

3. The method according to claim 2, characterized in that The calculating and processing the incremental spectrogram period in the target spectrum signal to obtain the incremental spectrogram segment includes: Performing short-time Fourier transform processing on the incremental spectrogram period in the target spectrum signal to obtain a target spectrum complex matrix; performing a rotation process on the target spectrum complex matrix to obtain a rotated target spectrum complex matrix; Mapping is performed on the amplitude values in the target spectrum complex matrix after the rotation process to obtain the incremental spectrogram segment.

4. The method according to claim 1, wherein The obtaining of the historical spectrogram segments corresponding to the overlapping spectrogram time periods includes: determining a spectrogram overlap amount between two adjacent spectrogram frames according to the overlapping spectrogram time period and the spectrum display period; The historical spectrogram segment corresponding to the overlapping spectrogram period is determined according to the spectrogram overlap amount.

5. The method according to claim 4, characterized in that The determining of the historical spectrogram segment corresponding to the overlapping spectrogram period according to the spectrogram overlap amount includes: Obtaining the previous frame of historical spectrogram corresponding to the overlapping spectrogram period; A historical spectrogram segment corresponding to the spectrogram overlap amount is determined from the previous frame of historical spectrogram.

6. The method according to claim 5, characterized in that The determining of the historical spectrogram segment corresponding to the spectrogram overlap amount from the previous frame of historical spectrogram includes: Determining a temporal relationship of spectra in the previous frame of historical spectrogram; Based on the time-series relationship of the spectrum, a historical spectrogram segment corresponding to the spectrogram overlap amount is selected from the subsequent direction of the historical spectrogram of the previous frame.

7. A voice playback device, characterized in that: include: an acquiring unit, configured to acquire a frequency spectrum display period in a spectrogram display window, and acquire a refresh period of the spectrogram in the spectrogram display window; A determination unit is configured to determine, based on the refresh cycle and the spectrum display cycle, an incremental spectrogram period and an overlapping spectrogram period between two adjacent spectrogram frames: calculating a period difference between the spectrum display cycle and the refresh cycle, wherein the speech spectrum information in the next frame of the spectrogram is repeated with part of the speech spectrum information between the previous frame, and determining, based on the period difference and the current speech playback progress, a period occupied by the repeated speech spectrum information in the next frame as the overlapping spectrogram period; and determining, based on the refresh cycle and the current playback progress in the spectrogram display window, a period occupied by the newly added speech spectrum information between two adjacent spectrogram frames as the incremental spectrogram period; a calculation unit, configured to calculate, based on the speech data, an incremental spectrogram segment corresponding to the incremental spectrogram period; a fusion unit, configured to obtain a historical spectrogram segment corresponding to the overlapping spectrogram period, and fuse the historical spectrogram segment with the incremental spectrogram segment according to the speech playback timing to obtain a target spectrogram: determining the spectrum end time point of the historical spectrogram segment in the previous spectrum display cycle and the spectrum start time point of the incremental spectrogram segment according to the speech playback timing; comparing the spectrum end time point with the spectrum start time point; and if it is detected that the spectrum end time point is consistent with the spectrum start time point, splicing the historical spectrogram segment with the incremental spectrogram segment to obtain a target spectrogram; A playback unit is used to play according to the target spectrogram and speech data.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium is computer-readable and stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in the voice playback method according to any one of claims 1 to 6.

Images