Real-time dynamic time-frequency diagram expression method and system based on visual coding

Abstract

The invention discloses a real-time dynamic time-frequency diagram expression method and system based on visual coding, and the method comprises the steps: carrying out the complex frequency spectrum output conversion of a to-be-processed audio signal based on an approximate amplitude method, and obtaining an approximate amplitude audio signal; storing and dynamically refreshing the approximate amplitude audio signal through a preset cache arrangement mode to obtain real-time rolling time-frequency graph data; and converting the real-time rolling time-frequency graph data into RGB888 pixel signals based on a segmentation threshold mapping algorithm to obtain real-time dynamically displayed time-frequency graph data. According to the embodiment of the invention, the calculation complexity can be reduced, the problem of horizontal picture tearing caused by pointer refreshing is solved, and the contrast ratio of the image is increased. The method can be widely applied to the technical field of real-time visualization.

Description

Technical Field

[0001] This application relates to the field of real-time visualization technology, and particularly to a method and system for expressing real-time dynamic time-frequency diagrams based on visual encoding. Background Art

[0002] In modern audio processing devices and applications, time-frequency diagrams are important tools for analyzing and displaying audio signals. A time-frequency diagram graphically presents the time, frequency, and amplitude information of an audio signal. The horizontal axis represents the time process, the vertical axis represents the frequency components, and the data points reflect the amplitude intensity of the corresponding frequency at a specific moment. This graphical presentation not only gives users an intuitive feeling of the function of the audio system but also provides a tool for engineers and technicians to quickly evaluate the signal processing effect. However, the implementation solutions of related technologies rely on high-precision complex FFT operations and full-frame caching, and there are the following problems:

[0003] 1) High computational complexity: Precise modulus calculation requires squaring and square root operations, consuming a large amount of hardware resources.

[0004] 2) Large memory occupancy: Full-frame time-frequency diagram caching requires storing spectral data at multiple time points, which is difficult to implement in an embedded scenario.

[0005] 3) Insufficient real-time performance: The data refresh and display synchronization mechanism is imperfect, prone to screen delay or tearing.

[0006] In summary, the technical problems existing in related technologies need to be improved. Summary of the Invention

[0007] The main objective of the embodiments of this application is to propose a method and system for expressing real-time dynamic time-frequency diagrams based on visual encoding, which can reduce the computational complexity, solve the problem of horizontal screen tearing caused by pointer refreshing, and increase the contrast of the image.

[0008] To achieve the above objective, on the one hand, an embodiment of this application proposes a method for expressing real-time dynamic time-frequency diagrams based on visual encoding, and the method includes:

[0009] Performing complex spectrum output conversion on the audio signal to be processed based on an approximate amplitude method to obtain an approximate amplitude audio signal;

[0010] Storing the approximate amplitude audio signal through a preset cache arrangement method and performing dynamic refresh to obtain real-time scrolling time-frequency diagram data;

[0011] Based on a segmented threshold mapping algorithm, converting the real-time scrolling time-frequency diagram data into RGB888 pixel signals to obtain real-time dynamic display time-frequency diagram data.

[0012] In some embodiments, the conversion of the complex spectrum output of the audio signal to be processed by the approximate amplitude method to obtain an approximate amplitude audio signal includes:

[0013] Obtain the audio signal to be processed and perform Fourier transform processing to obtain an audio signal with a complex spectrum amplitude output;

[0014] Perform complex spectrum output conversion on the audio signal with the complex spectrum amplitude output by the approximate amplitude method to obtain the approximate amplitude audio signal.

[0015] In some embodiments, the conversion of the complex spectrum output of the audio signal with the complex spectrum amplitude output by the approximate amplitude method to obtain the approximate amplitude audio signal includes:

[0016] Based on the approximate amplitude method, perform parallel calculation of the absolute value, comparison of the maximum value, and comparison of the minimum value on the audio signal with the complex spectrum amplitude output, and output the approximate amplitude audio signal.

[0017] In some embodiments, the expression of the approximate amplitude method is specifically as follows:

[0018]

[0019] In the above formula, a + bi represents the audio signal with the complex spectrum amplitude output, f(·) represents the expression of the approximate amplitude method, a represents the real part, b represents the imaginary part, max(·) represents the calculation of taking the maximum value, and min(·) represents the calculation of taking the minimum value.

[0020] In some embodiments, the storage and dynamic refresh of the approximate amplitude audio signal through a preset cache arrangement method to obtain real-time rolling spectrum amplitude signal data includes:

[0021] Store the approximate amplitude audio signal according to the preset cache arrangement method to obtain the stored time-frequency map data;

[0022] Based on the pointer, perform time-frequency map rolling refresh design on the stored time-frequency map data to obtain the real-time rolling time-frequency map data.

[0023] In some embodiments, the storage of the approximate amplitude audio signal according to the preset cache arrangement method to obtain the stored time-frequency map data includes:

[0024] Stitch the continuous approximate amplitude audio signals row by row to obtain the stitched approximate amplitude audio signal;

[0025] Based on the spliced approximate amplitude audio signal, arrange it in a two-dimensional manner along the time axis and frequency axis, and calculate the base address of the data according to the current display line number for storage, to obtain the stored time-frequency map data.

[0026] In some embodiments, the "based on" refers to performing a time-frequency map scrolling and refreshing design on the stored time-frequency map data to obtain the real-time scrolling time-frequency map data, including:

[0027] Maintain a dynamic pointer to mark the latest data position of the stored time-frequency map data, synchronously read historical data according to the pointer offset, and update the display content in combination with the field synchronization signal to obtain the real-time scrolling time-frequency map data.

[0028] In some embodiments, the "based on" the segmented threshold mapping algorithm to convert the real-time scrolling time-frequency map data into an RGB888 pixel signal to obtain the real-time dynamic display time-frequency map data, including:

[0029] Based on the segmented threshold mapping algorithm, set the dynamic range according to the amplitude data of the 16-bit unsigned number;

[0030] Based on the dynamic range, perform bit-width truncation and saturation processing on the RGB channels of the real-time scrolling time-frequency map data respectively, and output the real-time dynamic display time-frequency map data.

[0031] In some embodiments, the conversion process expression for converting the real-time scrolling spectral amplitude signal data into an RGB888 pixel signal is specifically as follows:

[0032]

[0033] In the above formula, R, G, and B represent the brightness values of the three elements of red, green, and blue, and f represents the amplitude data of the 16-bit unsigned number.

[0034] To achieve the above object, on the other hand, an embodiment of the present application proposes a real-time dynamic time-frequency map expression system based on visual encoding, and the system includes:

[0035] The first module is used to perform complex spectrum output conversion on the audio signal to be processed based on the approximate amplitude method to obtain an approximate amplitude audio signal;

[0036] The second module is used to store and dynamically refresh the approximate amplitude audio signal through a preset cache arrangement method to obtain real-time scrolling time-frequency map data;

[0037] The third module is used to convert the real-time scrolling time-frequency map data into an RGB888 pixel signal based on the segmented threshold mapping algorithm to obtain the real-time dynamic display time-frequency map data.

[0038] The embodiments of the present application at least include the following beneficial effects: The present application provides a method and system for expressing real-time dynamic time-frequency diagrams based on visual coding. This solution performs complex spectrum output conversion on the audio signal to be processed through an approximate amplitude method, while avoiding the introduction of non-linear functions such as square roots, avoiding the bit-width expansion of square operations and square root operations, reducing the computational complexity and the consumption of hardware resources. Further, the approximate amplitude audio signal is stored through a preset cache arrangement method and dynamically refreshed, which can take into account the scanning method from top to bottom of the line scan. The continuous address supports the burst read mode of the line cache at high speed, and can avoid the problem of horizontal tearing of the picture caused by the refresh of the pointer. Finally, based on the segmented threshold mapping algorithm, the real-time scrolling time-frequency diagram data is converted into RGB888 pixel signals, which can increase the contrast of the image and make the comparison of numerical sizes more obvious. Description of the Drawings

[0039] Figure 1 is a flowchart of a method for expressing real-time dynamic time-frequency diagrams based on visual coding provided by an embodiment of the present application;

[0040] Figure 2 is a schematic structural diagram of a system for expressing real-time dynamic time-frequency diagrams based on visual coding provided by an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of single-frame spectrogram data provided by an embodiment of the present application;

[0042] Figure 4 is a schematic diagram of the time-frequency diagram data structure provided by an embodiment of the present application;

[0043] Figure 5 is a schematic diagram of the data storage arrangement provided by an embodiment of the present application;

[0044] Figure 6 is a schematic diagram of the function of the pointer provided by an embodiment of the present application;

[0045] Figure 7 is a schematic diagram of the audio processing system provided by an embodiment of the present application. Detailed Embodiments

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. When the following description involves the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the embodiments of this application. They are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0047] It can be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if", "when" as used herein may be interpreted as "when...", "while...", or "in response to determining".

[0048] The terms "at least one", "multiple", "each", "any one", etc. used in this application, "at least one" includes one, two, or more than two, "multiple" includes two or more than two, "each" refers to each of the corresponding multiple, and "any one" refers to any one of the multiple.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0050] Refer to Figure 1 , Figure 1 which is a flowchart of a real-time dynamic time-frequency graph expression method based on visual encoding provided by an embodiment of the present invention. Refer to Figure 1 , the method includes the following steps:

[0051] S100. Perform complex spectrum output conversion on the audio signal to be processed based on the approximate amplitude method to obtain an approximate amplitude audio signal;

[0052] It should be noted that in some embodiments, step S100 may include: S110. Obtain the audio signal to be processed and perform Fourier transform processing to obtain an audio signal with complex spectrum amplitude output; S120. Perform complex spectrum output conversion on the audio signal with complex spectrum amplitude output through the approximate amplitude method to obtain an approximate amplitude audio signal.

[0053] Specifically, based on the approximate amplitude method, instead of calculating the exact modulus value, the square and square root operations are avoided, the consumption of hardware resources is reduced, and the absolute value, maximum comparison, and minimum comparison of the audio signal output by the complex spectrum amplitude are calculated in parallel to output the approximate amplitude audio signal.

[0054] Among them, the expression of the approximate amplitude method is specifically as follows:

[0055]

[0056] In the above formula, a + bi represents the audio signal output by the complex spectrum amplitude, f(·) represents the expression of the approximate amplitude method, a represents the real part, b represents the imaginary part, max(·) represents the maximum value calculation, and min(·) represents the minimum value calculation.

[0057] In some specific embodiments, the audio data obtained after the fast Fourier transform is complex data, and only the amplitude data is required for the qualitative display of the time-frequency diagram. To complete the conversion of complex data to amplitude data and avoid introducing non-linear functions such as square root, the present invention uses an approximate amplitude calculation method, which describes the processing of the complex amplitude a + bi, where a is the real part and b is the imaginary part, and a real value is output after the function processing, which is the approximate amplitude. This approximate method maintains a good approximation effect with the accurate value of the modulus, and at the same time avoids the bit-width expansion of the square operation and the square root operation. There are only modulus value, comparison, and shift operations in the method. When implementing the algorithm, especially when implementing in hardware such as FPGA, less hardware resources can be consumed and better timing can be obtained.

[0058] S200. Store the approximate amplitude audio signal through a preset cache arrangement method and perform dynamic refresh to obtain real-time scrolling time-frequency diagram data;

[0059] It should be noted that in some embodiments, step S200 may include: S210. Store the approximate amplitude audio signal according to a preset cache arrangement method to obtain the stored time-frequency diagram data;

[0060] Among them, the continuous approximate amplitude audio signals are spliced by rows to obtain the spliced approximate amplitude audio signal; based on the spliced approximate amplitude audio signal, two-dimensional arrangement is performed along the time axis and the frequency axis, and the base address of the data is calculated according to the current display line number for storage to obtain the stored time-frequency diagram data.

[0061] In this embodiment, the result obtained by a single fast Fourier transform is a part of the time-frequency diagram. Considering the data content of the time-frequency diagram and the line scanning display method, at time m, if the fast Fourier transform outputs the results of N different points, as Figure 3 shown, the time-frequency diagram displays the content of M time points, asFigure 4 As shown, the value of a certain frequency component at a certain time point is f i,j (0 < i ≤ M, 0 < j ≤ N). In the continuous address space, the data arrangement order is as follows Figure 5 As shown, the data is concatenated by rows. This arrangement method conforms to the format where the horizontal axis of the time-frequency diagram is time and the vertical axis is frequency. At the same time, it takes into account the scanning method from top to bottom of the row scan. The continuous address supports the burst read mode of the row cache at high speed.

[0062] Among them, for a certain frequency component with a value of f i,j (0 < i ≤ M, 0 < j ≤ N), according to the content of M time points shown in the time-frequency diagram, it can be known that i refers to the i-th time point. According to the results of N different points output by the fast Fourier transform, it can be known that j refers to the j-th frequency component, and f i,j refers to the j-th frequency component output by the fast Fourier transform at the i-th time point.

[0063] S220. Design the time-frequency diagram to be scrolled and refreshed based on the pointer for the stored time-frequency diagram data, and obtain the real-time scrolled time-frequency diagram data.

[0064] Among them, by maintaining a dynamic pointer to mark the latest data position of the stored time-frequency diagram data, and synchronously reading historical data according to the pointer offset, and combining the vertical blanking interval signal to update the display content, the real-time scrolled time-frequency diagram data is obtained.

[0065] In some specific embodiments, when storing the time-frequency diagram data, the result output by each fast Fourier transform only needs to be stored once. Use a pointer P (0 < P ≤ M) to indicate the current time point, then only need to store the data to the corresponding position of this time point. For example, the data f P,N , f P,N-1 … f P,1 . In this way, refreshing each time the data can greatly reduce the storage bandwidth.

[0066] At the display module side, update the received pointer P after each vertical blanking interval ends. In this way, it is possible to avoid the problem of horizontal tearing of the picture caused by the refreshing of the pointer during display. Since the data pointed to by the pointer P is refreshed at any time, this part of the space is used as a buffer area and is not displayed. Then, map the data according to the time corresponding to the position of the pointer P. If the pointer P is incremented, then as Figure 6 shown, the data corresponding to P - 1 is the data of the previous moment, and the data corresponding to P - 2 is the data of the moment before the previous moment, and so on. If the correspondence between the time of the time-frequency diagram data and the x coordinate of the display is fixed, when the pointer P is updated as the audio processing progresses, the displayed spectrogram scrolls simultaneously, and the scrolling speed is consistent with the audio processing speed.

[0067] S300. Based on the segmented threshold mapping algorithm, convert the real-time scrolling time-frequency map data into RGB888 pixel signals to obtain the time-frequency map data for real-time dynamic display;

[0068] It should be noted that in some embodiments, step S300 may include: S310. Based on the segmented threshold mapping algorithm, set the dynamic range according to the amplitude data of 16-bit unsigned numbers; S320. Based on the dynamic range, perform bit-width truncation and saturation processing on the RGB channels of the real-time scrolling time-frequency map data respectively, and output the time-frequency map data for real-time dynamic display.

[0069] In some specific embodiments, since the original time-frequency map data is stored, a conversion from numerical data to pixel data is required. Considering reflecting the relative magnitude of the numerical values in an intuitive and visual way, for the amplitude data f of 16-bit unsigned numbers, the conversion method adopted in the embodiments of the present invention is as shown in the formula:

[0070]

[0071] In the above formula, R, G, and B represent the luminance values of the three elements of red, green, and blue, and f represents the amplitude data of 16-bit unsigned numbers.

[0072] Considering it as an 8-bit unsigned number type, when f < 2 2 , the pixel is not lit, which is used to shield weak noise. As the value of f increases, B, G, and R are gradually lit until the luminance reaches the maximum. When f > 2 14 , the pixel luminance reaches the highest and no longer changes. Here, it can be modified according to the dynamic range of f. Setting the boundary between full brightness and full darkness of the pixel near the boundary of the data dynamic range can increase the contrast of the image and make the comparison of numerical magnitudes more obvious.

[0073] Finally, it should also be noted that the embodiments of the present invention are implemented based on the Logos PGL50H FPGA platform. The implementation of the entire display system is mainly composed of a spectrum data transfer and storage module and a line buffer refresh module, and does not include a peripheral and driver module. The system structure is as Figure 7 shown. The modules involved in the present invention are the modulus approximation, transfer and storage module, line buffer module, and numerical conversion module in the figure. The above four modules complete the construction of a complete display output by receiving the output of the FFT module. In terms of system parameters, the FFT module processes 256-point audio data sampled at 16 kHz, outputs 128 effective frequency points, the time-frequency map is horizontally displayed for 255 frames (about 4 seconds in duration), and the data is stored in a 65 kB continuous address space of an external DDR3, supporting burst read and write modes.

[0074] The complex FFT amplitude approximation calculation module receives the 16-bit signed FFT result, calculates the absolute value in parallel, compares the maximum value with the minimum value, outputs the approximate amplitude, caches the calculation result in the FIFO with a required size of 128×16b. The time-frequency diagram data transfer module transfers the data in the FIFO to the DDR3 according to the storage strategy. The required continuous storage space size of the DDR3 is 256*16b. At the same time, it maintains a dynamic pointer P (range from 0 to 255), and increments P after each frame of FFT processing is completed to achieve circular coverage of the old data.

[0075] When triggered by the line synchronization signal, the line buffer refresh module calculates the base address of the data according to the current display line number, bursts and reads the data of this line (a total of 256*16b) from the DDR3 to the local buffer, and creates a data output stream according to the pointer P latched during the field synchronization. Then, in the pixel conversion module, the 16-bit amplitude f is mapped to the RGB888 format.

[0076] Through the above design, the present invention realizes efficient display of the time-frequency diagram. The system only consumes extremely low computing resources and storage resources, and can display the time-frequency diagram of the audio signal with high quality at the same time. This solution can be widely applied to resource-constrained embedded audio processing systems to meet the requirements of real-time performance and low resource consumption.

[0077] Please refer to Figure 2 , the embodiment of the present application also provides a real-time dynamic time-frequency diagram expression system based on visual coding, which can implement the above real-time dynamic time-frequency diagram expression method based on visual coding. The system includes:

[0078] The first module 201 is used to perform complex spectrum output conversion on the audio signal to be processed based on the approximate amplitude method to obtain an approximate amplitude audio signal;

[0079] The second module 202 is used to store and dynamically refresh the approximate amplitude audio signal through a preset cache arrangement method to obtain real-time scrolling time-frequency diagram data;

[0080] The third module 203 is used to convert the real-time scrolling time-frequency diagram data into RGB888 pixel signals based on the segmented threshold mapping algorithm to obtain real-time dynamic display time-frequency diagram data.

[0081] In summary, in the embodiment of the present invention, an audio signal is input into a Fourier transform module to obtain a complex spectrum amplitude output. Then, an approximate amplitude method is used to convert the complex spectrum output into an approximate amplitude signal. This conversion process consumes less resources, the data size is compressed after conversion, and the required storage space is correspondingly reduced. Further, according to the proposed cache arrangement method, the approximate amplitude data is stored in a large cache area. When storing data, a pointer pointing to the storage base address is output, and this pointer dynamically increments as the storage progresses. The display module synchronizes this pointer across clock domains. Finally, according to the line scanning principle, a row of data is fetched from the cache each time, and then the relative newness and oldness of the data in this row are decoded according to the pointer, and the data is output in the order from new to old. In this way, as the storage progresses -> the pointer is updated -> the display content moves, thus achieving a dynamic display effect. Through innovative data reuse and storage optimization strategies, the computing resources and memory occupancy are significantly reduced.

[0082] Therefore, the embodiment of the present invention has the following improvement points compared with the prior art:

[0083] 1) The approximate amplitude method is adopted to replace the accurate calculation of the complex FFT result, avoiding non-linear operations.

[0084] 2) A spectrum data cache structure based on line scanning is designed to support the burst read mode.

[0085] 3) The real-time scrolling update of the time-frequency diagram is realized based on the dynamic pointer synchronization mechanism.

[0086] 4) An efficient spectrum-pixel mapping algorithm is developed to generate an image in RGB888 format through logical operations.

[0087] It can be understood that the content in the above method embodiments is applicable to the system embodiments of the present invention. The functions specifically implemented by the system embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0088] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. However, this does not limit the scope of the rights of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A real-time dynamic time-frequency diagram expression method based on visual coding, characterized in that: The method comprises the following steps: Performing complex spectrum output conversion on the audio signal to be processed based on the approximate amplitude method to obtain an approximate amplitude audio signal; The approximate amplitude audio signal is stored and dynamically refreshed through a preset cache arrangement to obtain real-time scrolling time-frequency diagram data; Based on the segmented threshold mapping algorithm, the real-time scrolling time-frequency graph data is converted into RGB888 pixel signals to obtain real-time dynamically displayed time-frequency graph data.

2. The method according to claim 1, characterized in that The method of performing complex spectrum output conversion on the audio signal to be processed based on the approximate amplitude method to obtain the approximate amplitude audio signal includes: Acquire the audio signal to be processed and perform Fourier transform processing to obtain an audio signal with complex spectrum amplitude output; The complex spectrum output conversion is performed on the audio signal of the complex spectrum amplitude output by an approximate amplitude method to obtain the approximate amplitude audio signal.

3. The method according to claim 2, characterized in that The method of performing complex spectrum output conversion on the audio signal outputted by the complex spectrum amplitude by the approximate amplitude method to obtain the approximate amplitude audio signal comprises: Based on the approximate amplitude method, the absolute value, maximum value and minimum value of the audio signal output by the complex spectrum amplitude are calculated in parallel, and the approximate amplitude audio signal is output.

4. The method according to claim 2, characterized in that: The expression of the approximate amplitude method is specifically as follows: In the above formula, a+bi represents the audio signal output by the complex spectrum amplitude, f(·) represents the expression of the approximate amplitude method, a represents the real part, b represents the imaginary part, max(·) represents the maximum value calculation, and min(·) represents the minimum value calculation.

5. The method according to claim 1, characterized in that The method stores and dynamically refreshes the approximate amplitude audio signal in a preset cache arrangement to obtain real-time rolling spectrum amplitude signal data, including: The approximate amplitude audio signal is stored according to a preset cache arrangement to obtain stored time-frequency graph data; The stored time-frequency graph data is subjected to a time-frequency graph scrolling refresh design based on the pointer to obtain the real-time scrolling time-frequency graph data.

6. The method according to claim 5, characterized in that The storing of the approximate amplitude audio signal according to a preset cache arrangement to obtain the stored time-frequency graph data includes: splicing the continuous approximate amplitude audio signals row by row to obtain a spliced ​​approximate amplitude audio signal; Based on the spliced ​​approximate amplitude audio signal, the data is arranged in two dimensions along the time axis and the frequency axis, and the base address of the data is calculated according to the current display line number for storage, so as to obtain the stored time-frequency diagram data.

7. The method according to claim 5, characterized in that The step of performing a time-frequency graph scrolling refresh design on the stored time-frequency graph data based on the pointer to obtain the real-time scrolling time-frequency graph data includes: The latest data position of the stored time-frequency diagram data is marked by maintaining a dynamic pointer, and historical data is synchronously read according to the pointer offset, and the display content is updated in combination with the field synchronization signal to obtain the real-time scrolling time-frequency diagram data.

8. The method according to claim 1, characterized in that The real-time scrolling time-frequency graph data is converted into RGB888 pixel signals based on the segmented threshold mapping algorithm to obtain the real-time dynamic display time-frequency graph data, including: Based on the segmented threshold mapping algorithm, the dynamic range is set according to the amplitude data of the 16-bit unsigned number; Based on the dynamic range, bit width interception and saturation processing are performed on the RGB channels of the real-time scrolling time-frequency diagram data respectively, and the real-time dynamically displayed time-frequency diagram data is output.

9. The method according to claim 8, characterized in that The conversion process expression of the real-time rolling spectrum amplitude signal data into RGB888 pixel signal is specifically as follows: In the above formula, R, G, B represent the brightness values ​​of the three elements red, green, and blue, and f represents the amplitude data of a 16-bit unsigned number.

10. A real-time dynamic time-frequency diagram expression system based on visual coding, characterized in that: The system comprises: The first module is used to perform complex spectrum output conversion on the audio signal to be processed based on the approximate amplitude method to obtain an approximate amplitude audio signal; The second module is used to store and dynamically refresh the approximate amplitude audio signal through a preset cache arrangement to obtain real-time scrolling time-frequency diagram data; The third module is used to convert the real-time scrolling time-frequency graph data into RGB888 pixel signals based on a segmented threshold mapping algorithm to obtain real-time dynamically displayed time-frequency graph data.

Images