Terminal equipment and audio data analysis method

By visualizing the confidence results of the audio data in the terminal device, the problem of inaccurate audio data analysis results of the model analysis method is solved, and accurate analysis of the data performance of specific scenarios is achieved.

CN118942450AActive Publication Date: 2024-11-12HISENSE ELECTRONIC TECH (WUHAN) CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310532427.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-12
Estimated Expiration
2043-05-11

AI Technical Summary

Technical Problem

When analyzing audio data through model analysis methods, it is impossible to accurately analyze the data performance in a specific scenario, and the results are not intuitive and inaccurate.

Method used

By inputting test audio files into the preset model in the terminal device, the confidence result is obtained and stored as a confidence list, the sampling rate is calculated according to the reasoning step size of the preset model, the confidence list is saved as the target audio file, and the audio sound wave is extracted And import it into the audio track to generate a confidence waveform, output audio sound wave and confidence waveform in a visual form, and compare the two to generate waveform difference data to generate analysis results.

Benefits of technology

The problem of inaccurate data analysis results was solved by model analysis methods, and intuitive analysis results were provided, which improved the accuracy and processing efficiency of the analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942450A_ABST
    Figure CN118942450A_ABST
Patent Text Reader

Abstract

The invention provides terminal equipment and an audio data analysis method, and the method comprises the steps: inputting a test audio file into a preset model, obtaining a confidence coefficient result obtained by the reasoning of the preset model on the test audio file, and storing the confidence coefficient result as a confidence coefficient list; calculating the sampling rate of the test audio file according to the reasoning step length of the preset model, and storing the confidence coefficient list as a target audio file based on the sampling rate; extracting audio sound waves from the test audio file and importing the target audio file into the first audio track to obtain a confidence coefficient waveform of the target audio file; and outputting the audio sound wave and the confidence coefficient waveform in a visual form, comparing the audio sound wave and the confidence coefficient waveform to generate waveform difference data, and generating an analysis result of the test audio file according to the waveform difference data. And the terminal equipment stores the confidence coefficient list as a target audio file, and outputs an analysis result in a visual form, so as to solve the problem that the result is inaccurate when the data is analyzed through a model analysis method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of terminal devices, and in particular to a terminal device and an analysis method for audio data. Background Art

[0002] In order to understand the characteristics of a certain type of data, it can be achieved through model analysis. For example, the data of this type can be input into a classification model, and the characteristics of this type of data can be obtained through analysis of the classification model.

[0003] Taking the analysis of audio data as an example, the audio data can be input into an audio classification model, and a prediction result is generated through reasoning and prediction of the audio classification model. The prediction result is the data feature of the audio data.

[0004] However, when analyzing audio data through an audio classification model, only the overall performance of the audio data can be analyzed, that is, only the global performance of the audio data can be analyzed, and targeted analysis of specific scene data cannot be performed. For example, taking the voice wake-up in the voice wake-up model as an example, after the wake-up word is input into the voice wake-up model, the voice wake-up model can only analyze the overall performance of the wake-up word, and cannot analyze the performance of the data in a specific scene. For example, the data performance of users at fast and slow speech speeds, the data performance of specific timbres, etc. The voice wake-up model can only output abstract effects and cannot intuitively analyze the performance of the data. Therefore, there is a problem of inaccurate results when analyzing data through model analysis methods. Summary of the invention

[0005] Some embodiments of the present application provide a terminal device and an analysis method for audio data to solve the problem of inaccurate results when performing analysis on data using a model analysis method.

[0006] In a first aspect, some embodiments of the present application provide a terminal device, including:

[0007] A memory, wherein program instructions are stored in the memory;

[0008] A processor, wherein the processor is configured to:

[0009] Input the test audio file to the preset model;

[0010] Obtaining a confidence result obtained by performing inference on the test audio file using the preset model, and storing the confidence result as a confidence list;

[0011] Calculate the sampling rate of the test audio file according to the inference step size of the preset model;

[0012] Saving the confidence list as a target audio file based on the sampling rate;

[0013] Extracting audio sound waves from the test audio file, and importing the target audio file into a first audio track to obtain a confidence waveform of the target audio file;

[0014] Outputting the audio sound wave and the confidence waveform in a visual form;

[0015] The audio sound wave and the confidence waveform are compared to generate waveform difference data, so as to generate an analysis result of the test audio file according to the waveform difference data.

[0016] It can be seen from the above technical solution that some embodiments of the present application provide a terminal device, which aligns the test audio file with the analysis result of the preset model in the time domain through the sampling rate, saves the confidence list as the target audio file, and outputs the analysis result in a visual form, which can solve the problem of unintuitive and inaccurate results when performing analysis on data through model analysis methods.

[0017] In a second aspect, some embodiments of the present application provide an audio data analysis method, which can be applied to the terminal device of the first aspect, and the audio data analysis method includes:

[0018] Input the test audio file to the preset model;

[0019] Obtaining a confidence result obtained by performing inference on the test audio file using the preset model, and storing the confidence result as a confidence list;

[0020] Calculate the sampling rate of the test audio file according to the inference step size of the preset model;

[0021] Saving the confidence list as a target audio file based on the sampling rate;

[0022] Extracting audio sound waves from the test audio file, and importing the target audio file into a first audio track to obtain a confidence waveform of the target audio file;

[0023] Outputting the audio sound wave and the confidence waveform in a visual form;

[0024] The audio sound wave and the confidence waveform are compared to generate waveform difference data, so as to generate an analysis result of the test audio file according to the waveform difference data.

[0025] It can be seen from the above technical solution that some embodiments of the present application provide a method for analyzing audio data. The method aligns the test audio file with the analysis result of a preset model in the time domain through the sampling rate, saves the confidence list as the target audio file, and outputs the analysis result in a visual form, which can solve the problem of inaccurate results when performing analysis on data through model analysis methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate some embodiments of the present application or technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of an operation scenario between a terminal device and a control device provided in some embodiments of the present application;

[0028] Figure 2 A hardware configuration block diagram of a terminal device provided in some embodiments of the present application;

[0029] Figure 3 A hardware configuration block diagram of a control device provided in some embodiments of the present application;

[0030] Figure 4 A schematic diagram of software configuration in a terminal device provided in some embodiments of the present application;

[0031] Figure 5 A schematic diagram of the confidence effect of the audio classification model output provided in some embodiments of the present application;

[0032] Figure 6 A schematic flow chart of a method for analyzing audio data performed by a terminal device provided in some embodiments of the present application;

[0033] Figure 7 A schematic diagram of a process for a terminal device to obtain a confidence result provided in some embodiments of the present application;

[0034] Figure 8 A schematic diagram of a process for a terminal device provided in some embodiments of the present application to store a confidence result as a confidence list;

[0035] Fig. 9 A schematic diagram of a flow chart of a terminal device providing some embodiments of the present application calculating a sampling rate of a test audio file according to an inference step length of a preset model;

[0036] Fig.10 A schematic diagram of a process for a terminal device provided in some embodiments of the present application to save a confidence list as a target audio file;

[0037] Fig.11 A schematic diagram of a process for extracting audio sound waves from a test audio file by a terminal device provided in some embodiments of the present application;

[0038] Fig.12 A schematic diagram of the effect of a terminal device provided in some embodiments of the present application importing a target audio file into a first audio track;

[0039] Fig.13 A schematic diagram of a process for a terminal device provided in some embodiments of the present application to output audio sound waves and confidence waveforms in a visual form;

[0040] Fig.14 A schematic diagram of the effect of visually outputting audio sound waves and confidence waveforms in the same display channel provided by some embodiments of the present application;

[0041] Fig.15 A schematic diagram of a process for generating waveform difference data by comparing an audio sound wave with a confidence waveform provided by a terminal device in some embodiments of the present application;

[0042] Fig.16a A schematic diagram of a visualization effect of waveform difference data provided in some embodiments of the present application;

[0043] Fig.16b A schematic diagram of another visualization effect of waveform difference data provided in some embodiments of the present application;

[0044] Fig.17 A schematic diagram of a process of marking a total display position in a display channel of a terminal device provided in some embodiments of the present application;

[0045] Fig.18 A schematic diagram of the output effect of audio sound waves and confidence waveforms at a first resolution provided in some embodiments of the present application;

[0046] Fig.19 A schematic diagram of the output effect of an audio sound wave and a confidence waveform at a second resolution provided in some embodiments of the present application;

[0047] Fig. 20 A flowchart of an audio data analysis method provided in some embodiments of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of some embodiments of the present application clearer, the technical solutions of some embodiments of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0049] It should be noted that the brief description of terms in some embodiments of the present application is only for the convenience of understanding the implementation methods described below, and is not intended to limit the implementation methods of some embodiments of the present application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0050] In some embodiments of the present application, the terms "first", "second", "third", etc. in the specification and claims and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0051] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0052] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0053] Figure 1 This is a schematic diagram of an operation scenario between a terminal device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the terminal device 200 through the mobile terminal 300 and the control device 100 .

[0054] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the terminal device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., to control the terminal device 200 wirelessly or by other wired methods. The user may control the terminal device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc.

[0055] In some embodiments, the mobile terminal 300 can install software applications with the terminal device 200, and realize connection communication through the network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the terminal device 200 to realize the synchronous display function.

[0056] like Figure 1 It is also shown that the terminal device 200 also communicates data with the server 400 through various communication methods. The terminal device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks.

[0057] In addition to providing the broadcast receiving television function, the terminal device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0058] Figure 2 A hardware configuration block diagram of a terminal device provided for some embodiments of the present application.

[0059] In some embodiments, the terminal device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0060] In some embodiments, the detector 230 is used to collect signals from the external environment or from interaction with the external environment.

[0061] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0062] In some embodiments, the communicator 220 is a component for communicating with an external device or a server according to various communication protocol types.

[0063] In some embodiments, the controller 250 controls the operation of the terminal device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the terminal device 200.

[0064] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0065] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0066] Figure 3 This is a hardware configuration block diagram of a control device provided in some embodiments of the present application. Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0067] The control device 100 is configured to control the terminal device 200 , and can receive the user's input operation instructions, and convert the operation instructions into instructions that the terminal device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the terminal device 200 .

[0068] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the terminal device 200 according to user needs.

[0069] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the terminal device 200 .

[0070] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0071] The communication interface 130 implements communication of control signals and data signals with the terminal device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0072] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0073] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the terminal device 200.

[0074] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0075] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0076] Figure 4A schematic diagram of the software configuration in a terminal device provided for some embodiments of the present application. In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as the "application layer"), the application framework layer (Application Framework) layer (referred to as the "framework layer"), the Android runtime (Android runtime) and the system library layer (referred to as the "system runtime library layer"), and the kernel layer.

[0077] In some embodiments, at least one application is running in the application layer. These applications may be window programs, system settings programs, clock programs, camera applications, etc. that come with the operating system; or they may be applications developed by third-party developers.

[0078] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take.

[0079] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes managers (Managers), content providers (Content Provider) and a view system (View System) and the like.

[0080] In some embodiments, the activity manager is used to manage the life cycle of each application and the general navigation back function.

[0081] In some embodiments, a window manager is used to manage all window programs.

[0082] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is accessed, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0083] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, touch sensor, pressure sensor, etc.), etc.

[0084] In some embodiments, the core layer further includes a power driver module for performing power management.

[0085] In some embodiments, Figure 4 The software programs and / or modules corresponding to the software architecture in Figure 2 or Figure 3 in the first memory or the second memory shown.

[0086] Based on the above terminal device 200, media data can be played and identified through the terminal device 200. Different media data categories correspond to different data features. In order to understand the features of a certain type of data, a model analysis method can be used. For example, the data of the type to be understood can be input into a classification model, and the features of the data of the type can be obtained through analysis, reasoning and prediction of the classification model.

[0087] For example, taking audio data in media data as an example, in order to understand the characteristics of audio data, the audio data can be input into an audio classification model. The audio data can contain different timbres, different speech speeds, etc. Through the reasoning and prediction of the audio classification model, a prediction result of the audio data can be generated, and the prediction result is the data feature of the audio data.

[0088] Figure 5 A schematic diagram of the confidence effect of the audio classification model output provided in some embodiments of the present application, such as Figure 5 As shown, in some embodiments, when analyzing audio data through an audio classification model, only the overall performance of the audio data can be analyzed, that is, only the global performance of the audio data can be analyzed, and specific scene data cannot be analyzed in a targeted manner. In the process of performing iterative optimization on test data through an audio classification model, it is necessary to perform analysis of audio features and related sound scenes based on the performance of the audio classification model on the test data. However, Figure 5 As shown, the audio classification model is not very sensitive to the test data and features, and can only obtain the overall situation of time information and confidence. If you want to analyze the performance of the audio classification model for different features and sound scenes, you need to find the changing pattern of confidence.

[0089] For example, taking the voice wake-up in the voice wake-up model as an example, after the wake-up word is input into the voice wake-up model, the voice wake-up model can only analyze the overall performance of the wake-up word, but cannot analyze the performance of the data in a specific scenario. For example, the data performance of the user at fast and slow speech speeds, the data performance of a specific timbre, etc. The voice wake-up model can only output abstract effects, and cannot intuitively analyze the performance of the data for specific scenarios such as non-awakening or false awakening. Therefore, there is a problem of inaccurate results when analyzing data through model analysis methods.

[0090] In order to solve the problem of inaccurate results when performing analysis on data through a model analysis method, some embodiments of the present application provide a terminal device 200. The terminal device 200 may include a memory 190 and a processor 112. The memory 190 stores program instructions, and the processor 112 can run the program instructions in the memory 190. In order to facilitate the understanding of the technical solutions in some embodiments of the present application, each step is described in detail below in conjunction with some specific embodiments and drawings. Figure 6 A schematic flow chart of a method for analyzing audio data performed by a terminal device provided in some embodiments of the present application, such as Figure 6 As shown, when the terminal device 200 performs the analysis of the audio data, the following steps S1-S7 may be included, and the specific contents are as follows:

[0091] Step S1: The terminal device 200 inputs a test audio file into a preset model.

[0092] In some embodiments, the preset model may be an audio classification model. In order to understand the audio data features of the test audio file, the test audio file may be input into the audio classification model. After the input is completed, the audio classification model may perform reasoning and prediction on the test audio file to generate corresponding reasoning results. After step S1 is executed, the following step S2 may be executed.

[0093] Step S2: The terminal device 200 obtains the confidence result obtained by performing inference on the test audio file using the preset model, and stores the confidence result as a confidence list.

[0094] After the test audio file is input into the preset model, the preset model can perform inference on the test audio file to obtain a confidence result. Figure 7 A schematic diagram of a process for a terminal device to obtain a confidence result provided in some embodiments of the present application, such as Figure 7 As shown, in some embodiments, the terminal device 200 can obtain the confidence result obtained by performing reasoning on the test audio file using the preset model in the following manner. The terminal device 200 can first generate a request instruction for obtaining the confidence result, and in response to the request instruction, it will run the preset model and return the confidence result of the test audio file according to the request instruction. The terminal device 200 then obtains the output format of the confidence result based on the returned confidence result, and then outputs the confidence result according to the output format. Among them, the output format of the confidence result can be set according to actual needs, and this application does not make specific limitations on this.

[0095] In order to facilitate intuitive display of the confidence results of the test audio files at a later stage, in some embodiments, after obtaining the confidence results of the test audio files, the terminal device 200 may store the confidence results in the form of a confidence list. Figure 8A schematic diagram of a process for a terminal device provided in some embodiments of the present application to store a confidence result as a confidence list, such as Figure 8 As shown, when storing the confidence results as a confidence list, the terminal device 200 may first traverse the confidence results, then create a confidence list for storing the confidence results based on the output format of the confidence results, and then add the confidence results to the confidence list.

[0096] Exemplarily, when traversing the confidence results, the attribute information contained in the confidence results can be obtained. For example, the confidence results may contain multiple field information. Then, a confidence list for storing the confidence results is created by combining the multiple field information and the output format of the confidence results. After the confidence list is created, the confidence results are added to the confidence list. After step S2 is executed, the following step S3 can be executed.

[0097] Step S3: The terminal device 200 calculates the sampling rate of the test audio file according to the inference step of the preset model.

[0098] In order to ensure that the analysis results of the test audio file and the preset model are aligned in the time domain, in some embodiments, the terminal device 200 can calculate the sampling rate of the test audio file according to the inference step of the preset model to achieve the alignment of the analysis results of the preset model and the test audio file in the time domain. Fig. 9 A schematic diagram of a process for calculating the sampling rate of a test audio file according to the inference step length of a preset model by a terminal device provided in some embodiments of the present application, such as Fig. 9 As shown, when calculating the sampling rate of the test audio file, the terminal device 200 can first obtain the unit time for sampling the test audio file and obtain the inference step of the preset model, and then calculate the sampling rate of the test audio file based on the ratio of the unit time to the inference step. It should be noted that there is no fixed order between the unit time and the inference step of obtaining the preset model, that is, the unit time can be obtained first and then the inference step, or the inference step can be obtained first and then the unit time, or both can be obtained at the same time, and this application does not limit this.

[0099] For example, with a unit time of 1 second and a step length of 100 milliseconds, after obtaining a unit time of 1 second, i.e., 1000 milliseconds and a step length of 100 milliseconds, the sampling rate of the test audio file can be calculated by the ratio of the unit time to the inference step length, i.e., the ratio of 1000 milliseconds to 100 milliseconds. In other words:

[0100] The sampling rate of the test audio file = unit time / inference step = 1000 milliseconds / 100 milliseconds = 10

[0101] That is, the sampling rate needs to be set to 10, and 10 sampling points are sampled per second. In this way, by setting the sampling rate of the test audio file, it can be ensured that the tested test audio file and the analysis result of the preset model correspond to each other, that is, the alignment of the test audio file and the analysis result of the preset model in the time domain is ensured. After the sampling rate calculation in step S3 is completed, the following step S4 can be executed.

[0102] Step S4: The terminal device 200 saves the confidence list as a target audio file based on the sampling rate of the test audio file.

[0103] In order to intuitively display the confidence results of the test audio file, in some embodiments, after the sampling rate of the test audio file is calculated, the terminal device 200 can save the confidence list as a target audio file based on the sampling rate of the test audio file. Fig.10 A schematic diagram of a process for a terminal device according to some embodiments of the present application to save a confidence list as a target audio file, such as Fig.10 As shown, when the terminal device 200 saves the confidence list as a target audio file, it can first traverse the confidence list to obtain the initial format of the confidence list, then obtain the target audio format of the target audio file, and finally, convert the initial format into the target audio format to save the confidence list as a target audio file.

[0104] Exemplarily, the format of the target audio file is a standard digital audio file wav format, which is easier to preserve the quality of the original audio and is a lossless music format. After obtaining the initial format of the confidence list, the terminal device 200 can save the confidence list in wav format to ensure that the audio in the test audio file will not be damaged. In actual operation, the confidence list can be saved as a target audio file through an audio conversion tool. For example, the confidence list can be input into an audio conversion tool, and the audio conversion tool outputs the corresponding conversion result after the conversion operation. The confidence list can also be saved as a target audio file in other forms, and the specific method is not limited in this application.

[0105] It should be noted that when the terminal device 200 saves the confidence list as the target audio file, it is converted after the step of calculating the sampling rate of the test audio file according to the inference step of the preset model, that is, after the test audio file and the analysis result of the preset model are aligned in the time domain. In this way, the accuracy of the analysis result of the test audio file can be guaranteed, and the problem of inaccuracy in analyzing data by the model analysis method can be solved. After step S4 is executed, the following step S5 can be executed.

[0106] Step S5: the terminal device 200 extracts audio sound waves from the test audio file, and imports the target audio file into the first audio track to obtain a confidence waveform of the target audio file.

[0107] In order to obtain the audio features in the test audio file, the terminal device 200 may extract audio sound waves from the test audio file. Fig.11 A schematic diagram of a process for extracting audio sound waves from a test audio file by a terminal device provided in some embodiments of the present application, such as Fig.11 As shown, when the terminal device 200 extracts audio sound waves from a test audio file, it can first parse the test audio file to obtain the audio amplitude information of the test audio file, then traverse the audio amplitude information to obtain the vibration range of the audio amplitude information, and then generate the audio sound waves of the test audio file based on the vibration range.

[0108] Exemplarily, when extracting audio sound waves, the test audio file can be input into the audio analysis software, and the test audio file can be parsed by the audio analysis software. After analysis, the audio amplitude information can be generated from the test audio file. It can be understood that the audio amplitude information has a certain vibration range. A larger vibration range indicates that the audio fluctuation in the test audio file is larger, and a smaller vibration range indicates that the audio fluctuation in the test audio file is smaller. Based on the vibration range of the audio, the audio sound waves of the test audio file can be generated.

[0109] In order to obtain the audio features in the target audio file, the terminal device 200 may import the target audio file into the first audio track to obtain the confidence waveform of the target audio file. Fig.12 A schematic diagram of the effect of a terminal device importing a target audio file into a first audio track provided in some embodiments of the present application, such as Fig.12 As shown, since the target audio file is in a common audio format, its confidence waveform can be intuitively presented through the audio analysis application. In this way, the user can intuitively see the confidence waveform effect of the target audio file, without having to analyze and summarize the rules one by one from a large number of confidence results, thereby solving the problem of unintuitive results when the data is analyzed by the model analysis method, and can also improve the processing efficiency of the terminal device 200. After step S5 is executed, the following step S6 can be executed.

[0110] Step S6: The terminal device 200 outputs the audio sound wave and the confidence waveform in a visualized form.

[0111] After the terminal device 200 extracts the audio sound wave from the test audio file and imports the target audio file into the first audio track to obtain the confidence waveform of the target audio file, the terminal device 200 can output the audio sound wave and the confidence waveform in a visual form. Fig.13A schematic diagram of a process of outputting audio sound waves and confidence waveforms in a visual form by a terminal device provided in some embodiments of the present application, such as Fig.13 As shown, the terminal device 200 can first obtain the display channel where the confidence waveform is located, that is, the display channel of the first audio track, and then create a second audio track in the display channel, and import the audio sound wave into the second audio track. In this way, the first audio track where the confidence waveform of the target audio file is located and the second audio track where the audio sound wave in the test audio file is located will be displayed in the same display channel, and the audio sound wave and the confidence waveform can be displayed simultaneously in the same display channel and different audio tracks, that is, the audio sound wave and the confidence waveform are output simultaneously in a visual form.

[0112] For example, Fig.14 A schematic diagram of the effect of visually outputting audio sound waves and confidence waveforms in the same display channel provided by some embodiments of the present application, such as Fig.14 As shown, in the same display channel, the confidence waveform is displayed in the first audio track and the audio sound wave is displayed in the second audio track. In this way, the characteristics of the audio data can be intuitively understood through the confidence waveform of the target audio file and the audio sound wave in the test audio file. After step S6 is executed, the following step S7 can be executed.

[0113] Step S7: the terminal device 200 compares the audio sound wave with the confidence waveform to generate waveform difference data, so as to generate an analysis result of the test audio file according to the waveform difference data.

[0114] In order to obtain the difference between the audio sound wave and the confidence waveform to determine the problematic audio data, in some embodiments, after obtaining the confidence waveform of the target audio file and the audio sound wave in the test audio file in step S6, the terminal device 200 compares the audio sound wave and the confidence waveform to generate waveform difference data, so as to generate an analysis result of the test audio file based on the waveform difference data, that is, to determine the problematic audio data based on the waveform difference data.

[0115] Fig.15 A schematic diagram of a process for generating waveform difference data by comparing an audio sound wave and a confidence waveform provided by a terminal device in some embodiments of the present application, such as Fig.15 As shown, when generating waveform difference data, the terminal device 200 can first traverse the audio amplitude information of the test audio file, and then traverse the confidence waveform to obtain the waveform amplitude information of the confidence waveform, and finally compare the audio amplitude information and the waveform amplitude information to obtain the waveform difference data.

[0116] Exemplarily, when the terminal device 200 traverses the audio amplitude information of the test audio file, it can obtain the audio amplitude range of the test audio file. When traversing the confidence waveform, it can obtain the vibration range of the confidence waveform. Thereafter, the audio amplitude range of the test audio file can be compared with the vibration range of the confidence waveform. Where the ranges or vibration amplitudes of the two do not match, it can be determined that they are waveform difference data.

[0117] Fig.16a A schematic diagram of a visualization effect of waveform difference data provided in some embodiments of the present application, Fig.16b Another schematic diagram of the visualization effect of waveform difference data provided in some embodiments of the present application is as follows: Fig.16a , Fig.16b As shown, by comparing the confidence waveform of the target audio file in the first audio track and the audio sound wave of the test audio file in the second audio track in the same display channel, it can be intuitively seen that there is a mismatch between the confidence waveform and the audio sound wave amplitude, that is, the waveform difference data.

[0118] In order to enable the user to intuitively see the location of the waveform difference data, in some embodiments, the terminal device 200 may mark the range of the waveform difference data. Fig.17 A schematic diagram of a process of marking a total display position in a display channel of a terminal device provided in some embodiments of the present application, such as Fig.17 As shown, the terminal device 200 can first locate the waveform difference data, that is, obtain the first display position where the audio amplitude information of the test audio file is located and the second display position where the waveform amplitude information of the confidence waveform is located. Since the two are in the same display channel, the first display position and the second display position can be merged to generate the total display position of the waveform difference data, and the total display position is marked in the display channel, so that the total display position of the waveform difference data can be marked, as shown in FIG. Fig.16a , Fig.16b In this way, by comparing the audio sound wave and the confidence waveform to generate waveform difference data, it is possible to locate which specific part of the audio data in the test audio file has a problem, rather than just reflecting the overall characteristics of the audio as a whole. This can improve the accuracy of the test audio file analysis and solve the problem of inaccurate results when performing data analysis using a model analysis method.

[0119] In order to more accurately output the location of the waveform difference data, in some embodiments, the terminal device 200 can also provide output of waveform difference data with different resolutions. Fig.18 This is a schematic diagram of the output effect of audio sound waves and confidence waveforms at the first resolution provided in some embodiments of the present application. Fig.19A schematic diagram of the output effect of the audio sound wave and the confidence waveform under the second resolution provided in some embodiments of the present application, wherein the resolutions of the audio sound wave and the confidence waveform set by the two are different. Fig.18 At the resolution shown, the output of the audio sound wave and the confidence waveform is smoother. Fig.19 At the resolution shown, the output of the audio sound wave and the confidence waveform is more refined. In this way, users can set the resolution of the audio sound wave and the confidence waveform according to actual needs, so as to more accurately determine the waveform difference data, thereby further improving the accuracy of the test audio file analysis.

[0120] It can be seen from the above technical solution that the above embodiment provides a terminal device, which inputs a test audio file into a preset model, obtains the confidence result obtained by the preset model performing reasoning on the test audio file, and stores the confidence result as a confidence list; calculates the sampling rate of the test audio file according to the reasoning step of the preset model, and saves the confidence list as a target audio file based on the sampling rate; extracts audio sound waves from the test audio file, and imports the target audio file into the first audio track to obtain the confidence waveform of the target audio file; outputs the audio sound waves and the confidence waveform in a visual form, and compares the audio sound waves and the confidence waveform to generate waveform difference data, so as to generate the analysis result of the test audio file according to the waveform difference data. After the terminal device aligns the test audio file with the analysis result of the preset model in the time domain by the sampling rate, it saves the confidence list as the target audio file, and outputs the analysis result in a visual form, which can solve the problem of inaccurate results when the data is analyzed by the model analysis method.

[0121] Some embodiments of the present application also provide a method for analyzing audio data, which can be applied to the terminal device 200 in the above embodiments. Fig. 20 A flowchart of an audio data analysis method provided in some embodiments of the present application is shown in FIG. Fig. 20 As shown, in some embodiments, a method for analyzing audio data may include the following steps S1-S7, the specific contents of which are as follows:

[0122] Step S1: The terminal device 200 inputs a test audio file into a preset model.

[0123] In some embodiments, the preset model may be an audio classification model. In order to understand the audio data features of the test audio file, the test audio file may be input into the audio classification model. After the input is completed, the audio classification model may perform reasoning and prediction on the test audio file to generate corresponding reasoning results. After step S1 is executed, the following step S2 may be executed.

[0124] Step S2: The terminal device 200 obtains the confidence result obtained by performing inference on the test audio file using the preset model, and stores the confidence result as a confidence list.

[0125] After the test audio file is input into the preset model, the preset model can perform reasoning on the test audio file to obtain a confidence result. In some embodiments, the terminal device 200 can obtain the confidence result obtained by the preset model performing reasoning on the test audio file in the following manner. Exemplarily, the terminal device 200 can first generate a request instruction for obtaining a confidence result. In response to the request instruction, for the preset model end, the preset model will receive the request instruction and return the confidence result of the test audio file according to the request instruction. The terminal device 200 receives the confidence result returned by the preset model according to the request instruction, obtains the output format of the confidence result, and then outputs the confidence result according to the output format. Among them, the output format of the confidence result can be set according to actual needs, and this application does not make specific limitations on this.

[0126] In order to facilitate intuitive display of confidence results, in some embodiments, after obtaining the confidence results of the test audio file, the terminal device 200 can store the confidence results in the form of a confidence list. When storing the confidence results as a confidence list, the terminal device 200 can first traverse the confidence results, then create a confidence list for storing the confidence results based on the output format of the confidence results, and then add the confidence results to the confidence list. After step S2 is executed, the following step S3 can be executed.

[0127] Step S3: The terminal device 200 calculates the sampling rate of the test audio file according to the inference step of the preset model.

[0128] In order to ensure that the analysis results of the test audio file and the preset model are aligned in the time domain, in some embodiments, the terminal device 200 can calculate the sampling rate of the test audio file according to the inference step of the preset model to achieve the alignment of the analysis results of the preset model and the test audio file in the time domain. When calculating the sampling rate of the test audio file, the terminal device 200 can first obtain the unit time used to sample the test audio file and obtain the inference step of the preset model, and then calculate the sampling rate of the test audio file based on the ratio of the unit time to the inference step. It should be noted that there is no fixed order in the order of the unit time and the inference step of obtaining the preset model, that is, the unit time can be obtained first and then the inference step, or the inference step can be obtained first and then the unit time, or both can be obtained at the same time, and this application does not limit this. After the sampling rate calculation in step S3 is completed, the following step S4 can be executed.

[0129] Step S4: The terminal device 200 saves the confidence list as a target audio file based on the sampling rate of the test audio file.

[0130] In order to facilitate intuitive display of the confidence results of the test audio file, in some embodiments, after the sampling rate of the test audio file is calculated, the terminal device 200 can save the confidence list as a target audio file based on the sampling rate of the test audio file. When saving the confidence list as a target audio file, the terminal device 200 can first traverse the confidence list to obtain the initial format of the confidence list, then obtain the target audio format of the target audio file, and finally, convert the initial format into the target audio format to save the confidence list as the target audio file.

[0131] Exemplarily, taking the format of the target audio file as the standard digital audio file wav format as an example, the wav format is easier to preserve the quality of the original audio and is a lossless music format. After obtaining the initial format of the confidence list, the terminal device 200 can save the confidence list in wav format to ensure that the audio in the test audio file will not be damaged. In the actual operation process, the confidence list can be saved as a target audio file through an audio conversion tool. For example, the confidence list can be input into an audio conversion tool, and the audio conversion tool outputs the corresponding conversion result after the conversion operation. The confidence list can also be saved as a target audio file in other forms, and the specific method is not limited in this application. After step S4 is executed, the following step S5 can be executed.

[0132] Step S5: the terminal device 200 extracts audio sound waves from the test audio file, and imports the target audio file into the first audio track to obtain a confidence waveform of the target audio file.

[0133] In order to obtain the audio features in the test audio file, the terminal device 200 can extract the audio sound waves from the test audio file. When the terminal device 200 extracts the audio sound waves from the test audio file, it can first parse the test audio file to obtain the audio amplitude information of the test audio file, then traverse the audio amplitude information to obtain the vibration range of the audio amplitude information, and then generate the audio sound waves of the test audio file based on the vibration range.

[0134] In order to obtain the audio features in the target audio file, the terminal device 200 can import the target audio file into the first audio track to obtain the confidence waveform of the target audio file. Since the target audio file is in a common audio format, its confidence waveform can be intuitively presented through the audio analysis software. In this way, the user can intuitively see the confidence waveform effect of the target audio file, without having to analyze and summarize the rules one by one from a large number of confidence results, thereby solving the problem of unintuitive results when the data is analyzed by the model analysis method, and can also improve the processing efficiency of the terminal device 200. After step S5 is executed, the following step S6 can be executed.

[0135] Step S6: The terminal device 200 outputs the audio sound wave and the confidence waveform in a visualized form.

[0136] After the terminal device 200 extracts the audio sound wave from the test audio file and imports the target audio file into the first track to obtain the confidence waveform of the target audio file, the terminal device 200 can output the audio sound wave and the confidence waveform in a visual form. The terminal device 200 can first obtain the display channel where the confidence waveform is located, that is, the display channel of the first track, and then create a second track in the display channel, and import the audio sound wave into the second track. In this way, the first track where the confidence waveform of the target audio file is located and the second track where the audio sound wave in the test audio file is located will be displayed in the same display channel, and the audio sound wave and the confidence waveform can be displayed simultaneously in the same display channel and different tracks, that is, the audio sound wave and the confidence waveform are output simultaneously in a visual form. In this way, in the same display channel, the confidence waveform is displayed in the first track, and the audio sound wave is displayed in the second track, and the characteristics of the audio data can be intuitively understood through the confidence waveform of the target audio file and the audio sound wave in the test audio file. After step S6 is executed, the following step S7 can be executed.

[0137] Step S7: the terminal device 200 compares the audio sound wave with the confidence waveform to generate waveform difference data, so as to generate an analysis result of the test audio file according to the waveform difference data.

[0138] In order to obtain the difference between the audio sound wave and the confidence waveform to determine the problematic audio data, in some embodiments, after obtaining the confidence waveform of the target audio file and the audio sound wave in the test audio file in step S6, the terminal device 200 compares the audio sound wave and the confidence waveform to generate waveform difference data, so as to generate an analysis result of the test audio file based on the waveform difference data, that is, to determine the problematic audio data based on the waveform difference data.

[0139] When generating waveform difference data, the terminal device 200 can first traverse the audio amplitude information of the test audio file, and then traverse the confidence waveform to obtain the waveform amplitude information of the confidence waveform, and finally compare the audio amplitude information and the waveform amplitude information to obtain the waveform difference data. Exemplarily, when the terminal device 200 traverses the audio amplitude information of the test audio file, it can obtain the audio amplitude range of the test audio file, and when traversing the confidence waveform, it can obtain the vibration range of the confidence waveform, and then compare the audio amplitude range of the test audio file with the vibration range of the confidence waveform. Where the ranges or vibration amplitudes of the two do not match, it can be determined to be waveform difference data.

[0140] In order to allow the user to intuitively see the location of the waveform difference data, in some embodiments, the terminal device 200 can mark the range of the waveform difference data. The terminal device 200 can first locate the waveform difference data, that is, obtain the first display position where the audio amplitude information of the test audio file is located and the second display position where the waveform amplitude information of the confidence waveform is located. Since the two are in the same display channel, the first display position and the second display position can be merged to generate the total display position of the waveform difference data, and the total display position is marked in the display channel, so as to realize the total display position of the marked waveform difference data. In this way, by comparing the audio sound wave and the confidence waveform to generate waveform difference data, it is possible to locate which specific part of the audio data in the test audio file has a problem, rather than just reflecting the overall characteristics of the audio as a whole, thereby improving the accuracy of the test audio file analysis and solving the problem of inaccurate results when performing analysis on the data through the model analysis method.

[0141] It can be seen from the above technical solution that the above embodiment provides an analysis method for audio data. The method aligns the test audio file with the analysis result of the preset model in the time domain through the sampling rate, saves the confidence list as the target audio file, and outputs the analysis result in a visual form. This can solve the problem of unintuitive and inaccurate results when performing analysis on data through the model analysis method.

[0142] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0143] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0145] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A terminal device, It is characterized in that include: A memory, wherein program instructions are stored in the memory; A processor, wherein the processor is configured to: Input the test audio file to the preset model; Obtaining a confidence result obtained by performing inference on the test audio file using the preset model, and storing the confidence result as a confidence list; Calculate the sampling rate of the test audio file according to the inference step size of the preset model; Saving the confidence list as a target audio file based on the sampling rate; Extracting audio sound waves from the test audio file, and importing the target audio file into a first audio track to obtain a confidence waveform of the target audio file; Outputting the audio sound wave and the confidence waveform in a visual form; The audio sound wave and the confidence waveform are compared to generate waveform difference data, so as to generate an analysis result of the test audio file according to the waveform difference data.

2. The terminal device according to claim 1, It is characterized in that The processor executes the step of obtaining a confidence result obtained by the preset model performing inference on the test audio file, and is further configured to: generating a request instruction for obtaining a confidence result; In response to the request instruction, receiving a confidence result returned by the preset model according to the request instruction; Obtaining an output format of the confidence result; The confidence result is output according to the output format.

3. The terminal device according to claim 2, It is characterized in that The processor performs the step of storing the confidence results as a confidence list, and is further configured to: Traversing the confidence results; Creating a confidence list for storing the confidence results based on the output format; The confidence result is added to the confidence list.

4. The terminal device according to claim 1, It is characterized in that The processor performs the step of calculating the sampling rate of the test audio file according to the inference step length of the preset model, and is further configured to: Obtaining a unit time for sampling the test audio file; Obtaining the inference step length of the preset model; Based on the ratio of the unit time to the inference step length, a sampling rate of the test audio file is calculated.

5. The terminal device according to claim 1, It is characterized in that The processor performs the step of saving the confidence list as a target audio file based on the sampling rate, and is further configured to: Traversing the confidence list to obtain an initial format of the confidence list; Acquire a target audio format of the target audio file; The initial format is converted into the target audio format to save the confidence list as a target audio file.

6. The terminal device according to claim 1, It is characterized in that The processor performs the step of extracting audio sound waves from the test audio file, and is further configured to: Parsing the test audio file to obtain audio amplitude information of the test audio file; Traversing the audio amplitude information to obtain the vibration range of the audio amplitude information; An audio sound wave of the test audio file is generated based on the vibration range.

7. The terminal device according to claim 6, It is characterized in that The processor performs the step of outputting the audio sound wave and the confidence waveform in a visual form, and is further configured to: Obtaining the display channel where the confidence waveform is located; creating a second audio track in the display channel; The audio sound wave is imported into the second audio track, so that the audio sound wave and the confidence waveform are simultaneously displayed in the same display channel and different audio tracks.

8. The terminal device according to claim 7, It is characterized in that The processor performs the step of comparing the audio sound wave with the confidence waveform to generate waveform difference data, and is further configured to: Traversing the audio amplitude information of the test audio file; Traversing the confidence waveform to obtain waveform amplitude information of the confidence waveform; The audio amplitude information and the waveform amplitude information are compared to obtain waveform difference data.

9. The terminal device according to claim 8, It is characterized in that The processor is further configured to: Positioning the waveform difference data to obtain a first display position where the audio amplitude information is located and a second display position where the waveform amplitude information is located; combining the first display position and the second display position to generate a total display position of the waveform difference data; The total display position is marked in the display channel.

10. A method for analyzing audio data, applied to a terminal device, It is characterized in that include: Input the test audio file to the preset model; Obtaining a confidence result obtained by performing inference on the test audio file using the preset model, and storing the confidence result as a confidence list; Calculate the sampling rate of the test audio file according to the inference step size of the preset model; Saving the confidence list as a target audio file based on the sampling rate; Extracting audio sound waves from the test audio file, and importing the target audio file into a first audio track to obtain a confidence waveform of the target audio file; Outputting the audio sound wave and the confidence waveform in a visual form; The audio sound wave and the confidence waveform are compared to generate waveform difference data, so as to generate an analysis result of the test audio file according to the waveform difference data.

Citation Information

Patent Citations

  • Data processing method and equipment

    CN113506584A

  • Classification model training method and device, computer equipment and storage medium

    CN115240659A

  • Audio feature detection technology

    CN115700877A

  • Method for detecting frequency in an audio signal

    US20030135377A1

  • Method for Ranking Fault-Test Pairs Based on Waveform Statistics in a Mutation-Based Test Program Evaluation System

    US20140136900A1