An audio visualisation method, system and glasses

Abstract

The application provides an audio visualization method, system and glasses, and is particularly applied to the field of human-computer interaction, and comprises the following steps: collecting audio data, wherein the audio data comprises first audio data and second audio data; extracting feature information in the first audio data and the second audio data to obtain a first audio sequence and a second audio sequence; marking redundant data of the first audio sequence and the second audio sequence; calculating the sequence length of the first audio sequence and the second audio sequence; calculating the similarity between the first audio sequence and the second audio sequence by using a dynamic time warping algorithm, an edit distance algorithm or a FASTA algorithm according to the redundant data and the sequence length; and rendering and presenting the first audio sequence and the second audio sequence. In this way, the audio similarity calculation accuracy is improved, and the intuitive feeling of a user for audio is further optimized.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction, and more specifically, to an audio visualization method, system, and glasses. Background Art

[0002] Currently, there is a demand for sound visualization in many fields both domestically and internationally. For example, military sonar detection systems include both passive and active sonar. Passive sonar systems analyze sound characteristics by frequency band and lock onto the sound to distinguish it from other sounds. They use the sound's attenuation and phase to determine the distance and direction of the sound source. Typically, there is no image display, only a target reading. Active sonar actively emits radiating ultrasonic waves in either pulsed or continuous form. By analyzing the duration and intensity of the reflected sound, the shape, size, and location of the target in front of it can be determined. Active sonar is now widely used in the military and fisheries sectors. Ultrasonic imaging in the medical field is also based on the principle of active sonar.

[0003] According to an analysis report issued by TNS, about 80% of audio visualization products transmit sound files directly into audio visualization components through audio cables. This method naturally eliminates the interference of external noise on the visualization components.

[0004] However, in the existing technology, if a capacitive sensor component is used to collect external sounds, due to the influence of various environmental noises such as redundant data, the visual data cannot accurately reflect the content expressed by the audio, and the user experience of audio visualization products is poor. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide an audio visualization method, system, and glasses. Different similarity calculation methods are used according to different audio characteristics to improve calculation accuracy, and the calculation results are visualized to enhance the user's intuitive experience of audio. The glasses also integrate a sound pressure level test indicator light and a built-in speaker to achieve real-time interaction between audio products and users.

[0006] In a first aspect, an embodiment of the present application provides an audio visualization method, comprising: S110, collecting audio data, the audio data comprising first audio data and second audio data; S120, extracting feature information in the first audio data and the second audio data to obtain a first audio sequence and a second audio sequence; S130, marking redundant data of the first audio sequence and the second audio sequence; S140, calculating sequence lengths of the first audio sequence and the second audio sequence; S150, calculating similarity between the first audio sequence and the second audio sequence by using a dynamic time warping algorithm, an edit distance algorithm or a FASTA algorithm according to the redundant data and the sequence lengths; and S160, rendering and presenting the first audio sequence and the second audio sequence.

[0007] Optionally, the redundant data comprises 0 reads, non-frequency interference reads, sudden high reads and sudden low reads.

[0008] Optionally, the S150 comprises: S151, determining whether the sequence lengths are less than a preset length threshold, if yes, performing S152; otherwise, performing S155; S152, determining whether the redundant data is less than a preset noise threshold, if yes, performing S153; otherwise, performing S154; S153, calculating the similarity between the first audio sequence and the second audio sequence by using the dynamic time warping algorithm; S154, calculating the similarity between the first audio sequence and the second audio sequence by using the edit distance algorithm; and S155, calculating the similarity between the first audio sequence and the second audio sequence by using the FASTA algorithm.

[0009] Optionally, the S153 comprises: calculating distances between elements in the first audio sequence and the second audio sequence to construct a distance matrix; calculating a shortest warping path by using a dynamic programming method based on the distance matrix; and taking a length of the shortest warping path as the similarity between the first audio sequence and the second audio sequence.

[0010] Optionally, the S154 comprises: constructing an initial matrix based on the sequence lengths of the first audio sequence and the second audio sequence; calculating an edit distance by using a dynamic programming method based on the initial matrix; and taking the edit distance as the similarity between the first audio sequence and the second audio sequence.

[0011] Optionally, the S155, the similarity between the first audio sequence and the second audio sequence is calculated by using the FASTA algorithm, comprising: obtaining an element that is completely matched between the first audio sequence and the second audio sequence, calculating the length of the element; constructing an initial region based on the element and the length of the element; scoring the initial region by using a substitution matrix, calculating an optimal scoring sub-region; aligning the initial region by using a dynamic programming method, and determining the similarity between the first audio sequence and the second audio sequence.

[0012] Optionally, the S160, the first audio sequence and the second audio sequence are rendered and presented, comprising: if the similarity is greater than a preset similarity threshold, similar elements in the first audio sequence and the second audio sequence are rendered and presented; if the similarity is less than or equal to the preset similarity threshold, dissimilar elements in the first audio sequence and the second audio sequence are rendered and presented.

[0013] In a second aspect, an audio visualization system is provided, comprising a sound collecting unit, a master control unit and a visualization unit; the sound collecting unit is configured to collect audio data, wherein the audio data comprises first audio data and second audio data; the master control unit is configured to extract feature information in the first audio data and the second audio data, obtain a first audio sequence and a second audio sequence, mark redundant data of the first audio sequence and the second audio sequence, calculate sequence lengths of the first audio sequence and the second audio sequence, and calculate a similarity between the first audio sequence and the second audio sequence by using a dynamic time warping algorithm, an edit distance algorithm or a FASTA algorithm according to the redundant data and the sequence lengths; and the visualization unit is configured to render and present the first audio sequence and the second audio sequence.

[0014] Optionally, the redundant data comprises 0 readings, non-frequency interference readings, sudden high readings and sudden low readings.

[0015] Optionally, the master control unit is specifically configured to perform the following steps: S151, determining whether the sequence length is less than a preset length threshold, if yes, performing step S152; otherwise, performing step S155; S152, determining whether the redundant data is less than a preset noise threshold, if yes, performing step S153; otherwise, performing step S154; S153, calculating the similarity between the first audio sequence and the second audio sequence by using the dynamic time warping algorithm; S154, calculating the similarity between the first audio sequence and the second audio sequence by using the edit distance algorithm; and S155, calculating the similarity between the first audio sequence and the second audio sequence by using the FASTA algorithm.

[0016] Optionally, the S153, the similarity between the first audio sequence and the second audio sequence is calculated by using a dynamic time warping algorithm, comprising: calculating the distance between each element in the first audio sequence and the second audio sequence, and constructing a distance matrix; based on the distance matrix, the shortest warping path is calculated by using a dynamic programming method; the length of the shortest warping path is taken as the similarity between the first audio sequence and the second audio sequence.

[0017] Optionally, the S154, the similarity between the first audio sequence and the second audio sequence is calculated by using an edit distance algorithm, comprising: based on the sequence length of the first audio sequence and the second audio sequence, an initial matrix is constructed; based on the initial matrix, the edit distance is calculated by using a dynamic programming method; the edit distance is taken as the similarity between the first audio sequence and the second audio sequence.

[0018] Optionally, the S155, the similarity between the first audio sequence and the second audio sequence is calculated by using a FASTA algorithm, comprising: obtaining the elements that are completely matched between the first audio sequence and the second audio sequence, and calculating the length of the elements; based on the elements and the length of the elements, an initial region is constructed; the initial region is scored by using a substitution matrix, and the optimal scoring sub-region is calculated; the initial region is aligned by using a dynamic programming method, and the similarity between the first audio sequence and the second audio sequence is determined.

[0019] Optionally, the visualization unit is specifically configured to: if the similarity is greater than a preset similarity threshold, render and present the similar elements in the first audio sequence and the second audio sequence; if the similarity is less than or equal to the preset similarity threshold, render and present the dissimilar elements in the first audio sequence and the second audio sequence.

[0020] In a third aspect, an audio visualization glasses is provided, comprising: a sound pressure level test indicator light and a built-in sound box; the built-in sound box comprises a box body, a microphone, a full-frequency speaker, a low-frequency speaker, a power switch, a wireless transmission module, a memory module and a processor module; the microphone is configured to collect audio signals; the full-frequency speaker is configured to output full-frequency signals; the low-frequency speaker is configured to output low-frequency signals; the power switch is configured to start the built-in sound box; the wireless transmission module is configured to transmit wireless audio; the sound pressure level test indicator light is configured to output corresponding light according to the full-frequency signals and the low-frequency signals output by the full-frequency speaker and the low-frequency speaker; the memory module is used for storing program instructions, and the program instructions are executed by the processor module, so that the glasses execute the method in the first aspect.

[0021] Advantages:

[0022] (1) According to the redundant data and sequence length of the audio sequence, the similarity calculation method between two audio sequences is determined, including dynamic time warping algorithm, edit distance algorithm or FASTA algorithm, which improves the calculation accuracy and visualizes the calculation results, and improves the user's intuitive feeling of audio.

[0023] (2) By the way of two frequency division, the audio similarity visualization system not only has high quality sound quality, but also has a shocking bass listening feeling, which improves the sound quality restoration degree of the user's listening feeling.

[0024] (3) Provide an audio visualization glasses, including decorative sound pressure level test indicator light and built-in sound box, convenient to carry, can be used for home entertainment, indoor monitoring, outdoor multi-scene use; At the same time, the indicator light is combined with different sound frequencies to light up, which greatly enhances the interactivity between the sound box and the user, so that we can have a wonderful audio-visual feast while listening to music. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0026] Figure 1 is a flowchart of an audio visualization method provided by an embodiment of the present application;

[0027] Figure 2a is a schematic diagram of a first audio sequence provided by an embodiment of the present application;

[0028] Figure 2b is a schematic diagram of a second audio sequence provided by an embodiment of the present application;

[0029] Figure 3 is a schematic diagram of a warping path between audio sequences provided by an embodiment of the present application;

[0030] Figure 4 is a three-dimensional space coordinate diagram of audio tonality provided by an embodiment of the present application;

[0031] Figure 5 is a structural diagram of an audio visualization system provided by an embodiment of the present application;

[0032] Figure 6 is a structural diagram of an audio visualization glasses provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings of the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0034] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0035] The embodiments of the present application provide an audio visualization method, system and glasses, including collecting audio data, the audio data including first audio data and second audio data; extracting feature information in the first audio data and the second audio data to obtain first audio sequence and second audio sequence; marking redundant data of the first audio sequence and the second audio sequence; calculating sequence length of the first audio sequence and the second audio sequence; calculating similarity between the first audio sequence and the second audio sequence by using dynamic time warping algorithm, edit distance algorithm or FASTA algorithm according to the redundant data and the sequence length; rendering and presenting the first audio sequence and the second audio sequence. In this way, the accuracy of audio similarity calculation is improved, and the intuitive feeling of users for audio is further optimized.

[0036] The audio visualization method can be integrated in an electronic device, which can be a terminal, a server or the like. The terminal can be a light field camera, a vehicle-mounted camera, a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer or a personal computer (PC) or the like. The server can be a single server or a server cluster composed of multiple servers.

[0037] It can be understood that the audio visualization method and system of the embodiments can be executed on a terminal, on a server, or jointly executed by a terminal and a server. The above examples should not be understood as limiting the present application.

[0038] Figure 1 A flowchart of an audio visualization method provided by the embodiments of the present application is shown, please refer to Figure 1 , which specifically includes the following steps:

[0039] S110, collect audio data, the audio data comprising first audio data and second audio data.

[0040] S120, extract feature information in the first audio data and the second audio data, to obtain a first audio sequence and a second audio sequence.

[0041] S130, mark redundant data of the first audio sequence and the second audio sequence.

[0042] The redundant data can be 0 reading, non-frequency interference reading, sudden high reading and sudden low reading.

[0043] Optionally, after marking the redundant data, the redundant data can be removed to obtain clearer audio sequences for similarity calculation.

[0044] S140, calculate the sequence length of the first audio sequence and the second audio sequence.

[0045] Optionally, the sequence length can be after removing the redundant data.

[0046] S150, according to the redundant data and the sequence length, using dynamic time warping algorithm, edit distance algorithm or FASTA algorithm, calculate the similarity between the first audio sequence and the second audio sequence.

[0047] In an embodiment, step S150 specifically comprises the following steps:

[0048] S151, judge whether the sequence length is less than a preset length threshold, if yes, execute step S152; otherwise, execute step S155.

[0049] S152, judge whether the redundant data is less than a preset noise threshold, if yes, execute step S153; otherwise, execute step S154.

[0050] S153, use dynamic time warping algorithm to calculate the similarity between the first audio sequence and the second audio sequence.

[0051] The main method of dynamic time warping is to stretch or shorten the time position of each data point in the source audio sequence until each data point in the target audio sequence is one-to-one corresponding in time position, in this process, the time axis of the source audio sequence will be unevenly twisted, bent or linearly drifted, dynamic time warping algorithm can be used to judge whether two audio sequences match, if they match, the length of the warping path can be calculated, the shorter the path, the higher the similarity.

[0052] Specifically, a matrix diagram filtering method is used to extract the pitch values of the two music segments; a relative pitch representation method is used to represent the contour of the music melody by an audio sequence, and the audio sequence is displayed in a two-dimensional diagram of three horizontal lines, the first audio sequence M1 may be as shown in FIG. 2B. Figure 2a may be as shown in FIG. 2C. may be as shown in FIG. 2D. Figure 2b

[0053] The distance between each element in the first audio sequence M1 and the second audio sequence M2 is calculated to construct a distance matrix.

[0054] Based on the distance matrix, a dynamic programming method is used to calculate the shortest warping path.

[0055] The calculation formula is as follows:

[0056]

[0057] In the formula, D(·) can be the Euclidean distance or other calculation formula capable of measuring the distance size; W(M1, M2) represents the warping path between M1 and M2, K represents the total number of paths in the warping path, w(k) i represents the kth path of the ith element in the first audio sequence M1, w(k) j represents the kth path of the jth element in the second audio sequence M2.

[0058] Optionally, the warping path W(M1, M2) between the first audio sequence M1 and the second audio sequence M2 may be as shown in FIG. 3. Figure 3

[0059] The length of the shortest warping path is taken as the similarity between the first audio sequence and the second audio sequence.

[0060] S154, an edit distance algorithm is used to calculate the similarity between the first audio sequence and the second audio sequence.

[0061] Although the warping path can correct the error of the data sequence caused by the scaling of the time axis, and its distance size can describe the error of the data itself, when the data position is exchanged or individual noise data occurs, the use of the dynamic time warping algorithm to describe the similarity has a large deviation. Therefore, when the redundant data is greater than or equal to a preset noise threshold, an edit distance algorithm is used to calculate the similarity between the sequences.

[0062] ​​The edit distance, also known as Levenshtein distance, is used to compare the similarity between two audio sequences. The edit distance refers to the minimum number of editing operations required to convert a source audio sequence into a target audio sequence, and the allowed editing operations include insertion, deletion and modification. The smaller the edit distance, the higher the similarity between the two audio sequences.

[0063] Specifically, based on the sequence lengths of the first audio sequence and the second audio sequence, an initial matrix M of size (m+1) x (n+1) is constructed.

[0064] Based on the initial matrix, the edit distance is calculated using a dynamic programming method.

[0065] The edit distance is used as the similarity between the first audio sequence and the second audio sequence.

[0066] S155, the similarity between the first audio sequence and the second audio sequence is calculated using the FASTA algorithm.

[0067] Although the edit distance can well describe the difference between two strings, since the audio sequence describing the music melody is usually long, using the dynamic programming method to solve it will cause problems such as excessive memory overhead and long computation time. Therefore, when the sequence length is greater than or equal to a preset length threshold, the FASTA algorithm is used to calculate the similarity between the sequences.

[0068] Specifically, any three of the four deoxynucleotides of DNA, adenine deoxynucleotide A, thymine deoxynucleotide T, cytosine deoxynucleotide C, and guanine deoxynucleotide G, correspond to pitch up pitch unchanged→ and pitch down For example, the deoxynucleotide adenine deoxynucleotide A corresponds to pitch up thymine deoxynucleotide T corresponds to pitch unchanged→, cytosine deoxynucleotide C corresponds to pitch down

[0069] Obtain the elements that are completely matched between the first audio sequence and the second audio sequence, and calculate the length of the elements.

[0070] Based on the elements and the length of the elements, an initial region is constructed.

[0071] The initial region is scored using a substitution matrix to calculate the optimal scoring sub-region.

[0072] The initial region is aligned using a dynamic programming method to determine the similarity between the first audio sequence and the second audio sequence.

[0073] Further, the feature information of the audio sequence can be converted into a geometric space using a music theory model, and the distance between the first audio sequence and the second audio sequence is calculated as the similarity.

[0074] Optionally, the music theory model can be a spiral array model, representing the perception of the pitch, chord and tonality of the audio in the same geometric space by human beings.

[0075] Specifically, after arranging the height of the audio sequence on a three-dimensional spiral line, the constitution of the tonic, the tonic chord, the dominant chord and the subdominant chord is analyzed, so as to map the audio tonality to a specific three-dimensional space coordinate point, and the distance between the audio tonality coordinate points of the first audio sequence and the second audio sequence is calculated as the similarity, as shown in the following formula: Figure 4 A three-dimensional space coordinate diagram of audio tonality is shown.

[0076] S160, rendering and presenting the first audio sequence and the second audio sequence.

[0077] If the similarity is greater than a preset similarity threshold, the similar elements in the first audio sequence and the second audio sequence are rendered and presented, thereby forming a visual sense of empathy.

[0078] If the similarity is less than or equal to the preset similarity threshold, the dissimilar elements in the first audio sequence and the second audio sequence are rendered and presented, thereby guiding the user to explore the difference of the data features.

[0079] The method improves the calculation accuracy, visualizes the calculation result, and improves the intuitive feeling of the user to the audio.

[0080] Figure 5 A structure diagram of an audio visualization system provided by an embodiment of the present application is shown. Please refer to Figure 5 The system comprises:

[0081] A sound collecting unit 510 is configured to collect audio data, wherein the audio data comprises first audio data and second audio data.

[0082] The sound collecting unit can be one or more sound sensors.

[0083] Since the sampling frequency of the LM358 is much higher than that of other chips, the LM358 can collect nearly simultaneous sound separately, so the LM358 chip can be used to collect sound comprehensively in the embodiment.

[0084] A master control unit 520 is configured to extract feature information from the first audio data and the second audio data, and obtain a first audio sequence and a second audio sequence.

[0085] marking redundant data of the first audio sequence and the second audio sequence.

[0086] Wherein, due to environmental noise during the sound collecting process of the sound collecting unit, the main control unit processes these environmental noises through a redundant data processing algorithm.

[0087] calculating sequence lengths of the first audio sequence and the second audio sequence.

[0088] According to the redundant data and the sequence lengths, a dynamic time warping algorithm, an edit distance algorithm or a FASTA algorithm is used to calculate the similarity between the first audio sequence and the second audio sequence.

[0089] A visualization unit 530 is configured to render and present the first audio sequence and the second audio sequence.

[0090] Optionally, the redundant data includes 0 reads, non-frequency interference reads, sudden high reads and sudden low reads.

[0091] Optionally, the main control unit 520 is specifically configured to perform the following steps: S151, determining whether the sequence length is less than a preset length threshold, if yes, performing step S152; otherwise, performing step S155; S152, determining whether the redundant data is less than a preset noise threshold, if yes, performing step S153; otherwise, performing step S154; S153, using a dynamic time warping algorithm to calculate the similarity between the first audio sequence and the second audio sequence; S154, using an edit distance algorithm to calculate the similarity between the first audio sequence and the second audio sequence; S155, using a FASTA algorithm to calculate the similarity between the first audio sequence and the second audio sequence.

[0092] Optionally, the S153, using a dynamic time warping algorithm to calculate the similarity between the first audio sequence and the second audio sequence, includes: calculating the distance between each element in the first audio sequence and the second audio sequence to construct a distance matrix; based on the distance matrix, using a dynamic programming method to calculate the shortest warping path; taking the length of the shortest warping path as the similarity between the first audio sequence and the second audio sequence.

[0093] Optionally, the S154, using an edit distance algorithm to calculate the similarity between the first audio sequence and the second audio sequence, includes: based on the sequence lengths of the first audio sequence and the second audio sequence, constructing an initial matrix; based on the initial matrix, using a dynamic programming method to calculate the edit distance; taking the edit distance as the similarity between the first audio sequence and the second audio sequence.

[0094] Optionally, the S155, the similarity between the first audio sequence and the second audio sequence is calculated by using the FASTA algorithm, comprising: obtaining an element that is completely matched between the first audio sequence and the second audio sequence, calculating the length of the element; constructing an initial region based on the element and the length of the element; scoring the initial region by using a substitution matrix, calculating an optimal scoring sub-region; aligning the initial region by using a dynamic programming method, and determining the similarity between the first audio sequence and the second audio sequence.

[0095] Optionally, the visualization unit 530 is specifically configured to: if the similarity is greater than a preset similarity threshold, render and present similar elements in the first audio sequence and the second audio sequence; and if the similarity is less than or equal to the preset similarity threshold, render and present dissimilar elements in the first audio sequence and the second audio sequence.

[0096] In this way, the redundant readings are processed while ensuring the same frequency response to the audio.

[0097] Figure 6 A structure schematic diagram of the audio visualization glasses provided by the embodiment of the application is shown. Please refer to Figure 6 The glasses 600 comprise:

[0098] A sound pressure level test indicator light 610, a built-in sound box 620.

[0099] The built-in sound box comprises a box body, a microphone 621, a full-frequency loudspeaker 622, a low-frequency loudspeaker 623, a power switch 624, a wireless transmission module 625, a memory module 626, and a processor module 627.

[0100] The box body is made of sla resin material, which not only has a fine appearance but also greatly reduces the weight of the box body, so that it has a relatively light weight in sound boxes of the same volume and can guarantee the taste of HIFI sound quality.

[0101] The microphone 621 is configured to collect audio signals. For example, when the glasses 600 are in an operation mode, such as a wearing mode, the microphone 621 receives external audio signals, and the received audio signals can be further stored in the memory module 626.

[0102] The full-frequency loudspeaker 622 is configured to output full-frequency signals.

[0103] The built-in sound box can comprise at least two full-frequency loudspeakers 622, which are placed in parallel on the upper half of the front side of the box body.

[0104] The low-frequency loudspeaker 623 is configured to output low-frequency signals.

[0105] The low-frequency speaker 623 is placed at the lower half of the back side of the box and opens upward.

[0106] The full-frequency speaker 622 and the low-frequency speaker 623 are used to accurately find the frequency division point through a large number of experiments in the sigmastudio programming software, and the crossover and Butterworth filter electronic frequency divider are used to realize two frequency division, so that the bass speaker unit can effectively work in the low-frequency range without mixing other frequency interference, and the volume of the box is effectively reduced.

[0107] The power switch 624 is configured to start the built-in sound box 620 and is placed at the top of the box.

[0108] The wireless transmission module 625 is configured to transmit wireless audio, specifically using aptX TM HD Bluetooth lossless technology, which can support 576Kbits / 48KHz decoding and 24bits / 48KHz audio encoding.

[0109] The sound pressure level test indicator light 610 is configured to output corresponding light according to the full-frequency signal and the low-frequency signal output by the full-frequency speaker 622 and the low-frequency speaker 623, and specifically can be an LED light.

[0110] Further, according to the algorithm, the LED light can be divided into nine groups, and different groups of LED lights will be lit in different sound frequencies.

[0111] The memory module 626 is configured to store program instructions, and the program instructions are executed by the processor module 627, so that the glasses execute the above-mentioned method embodiment.

[0112] Thus, by using two frequency division technology, the indicator light and the audio device are ingeniously combined to realize an audio visual glasses, which has higher sound quality restoration degree in human voice hearing, full mid-frequency, clear and transparent high-frequency, good and powerful low-frequency.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the modules / units / sub-units / components in the above-described device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0114] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely exemplary, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, and electrical, mechanical or other forms.

[0115] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0116] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0117] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0118] It should be noted that: similar reference numerals and letters in the following drawings represent similar items, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third" and the like are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0119] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any skilled person in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical range disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of audio visualization, characterized by, The method comprises the following steps: S110, collecting audio data, wherein the audio data comprises first audio data and second audio data; S120, extracting feature information in the first audio data and the second audio data to obtain a first audio sequence and a second audio sequence; S130, marking redundant data of the first audio sequence and the second audio sequence; S140, calculating sequence lengths of the first audio sequence and the second audio sequence; S150, calculating similarity between the first audio sequence and the second audio sequence by using a dynamic time warping algorithm, an edit distance algorithm or a FASTA algorithm according to the redundant data and the sequence lengths, wherein the step S150 comprises the following steps: S151, judging whether the sequence lengths are less than a preset length threshold, if yes, executing step S152, otherwise, executing step S155; S152, judging whether the redundant data is less than a preset noise threshold, if yes, executing step S153, otherwise, executing step S154; S153, calculating the similarity between the first audio sequence and the second audio sequence by using the dynamic time warping algorithm; S154, calculating the similarity between the first audio sequence and the second audio sequence by using the edit distance algorithm; S155, calculating the similarity between the first audio sequence and the second audio sequence by using the FASTA algorithm; S160, rendering and presenting the first audio sequence and the second audio sequence.

2. The method of claim 1, wherein, The redundant data comprises 0 reading, non-frequency interference reading, sudden high reading and sudden low reading.

3. The method of claim 1, wherein, The step S153 comprises the following steps: calculating distances between elements in the first audio sequence and the second audio sequence to construct a distance matrix; calculating a shortest warping path by using a dynamic programming method based on the distance matrix; and taking a length of the shortest warping path as the similarity between the first audio sequence and the second audio sequence.

4. The method of claim 1, wherein, The step S154 comprises the following steps: constructing an initial matrix based on sequence lengths of the first audio sequence and the second audio sequence; calculating an edit distance by using a dynamic programming method based on the initial matrix; and taking the edit distance as the similarity between the first audio sequence and the second audio sequence.

5. The method of claim 1, wherein, The step S155 comprises the following steps: obtaining elements that are completely matched between the first audio sequence and the second audio sequence, and calculating lengths of the elements; constructing an initial region based on the elements and the lengths of the elements; scoring the initial region by using a substitution matrix to obtain an optimal scoring sub-region; and aligning the initial region by using a dynamic programming method to determine the similarity between the first audio sequence and the second audio sequence.

6. The method of claim 1, wherein, The step S160 comprises the following steps: if the similarity is greater than a preset similarity threshold, rendering and presenting similar elements in the first audio sequence and the second audio sequence; if the similarity is less than or equal to the preset similarity threshold, rendering and presenting dissimilar elements in the first audio sequence and the second audio sequence.

7. An audio visualisation system characterised by comprising a sound collection unit, a main control unit and a visualization unit; the sound collection unit is configured to collect audio data, the audio data comprising first audio data and second audio data; the main control unit is configured to extract feature information from the first audio data and the second audio data, to obtain a first audio sequence and a second audio sequence; labeling redundant data of the first audio sequence and the second audio sequence; calculating sequence lengths of the first audio sequence and the second audio sequence; according to the redundant data and the sequence lengths, using a dynamic time warping algorithm, an edit distance algorithm or a FASTA algorithm to calculate the similarity between the first audio sequence and the second audio sequence, and further configured to perform: S151, judging whether the sequence lengths are less than a preset length threshold, if yes, performing step S152; otherwise, performing step S155; S152, judging whether the redundant data is less than a preset noise threshold, if yes, performing step S153; otherwise, performing step S154; S153, using the dynamic time warping algorithm to calculate the similarity between the first audio sequence and the second audio sequence; S154, using the edit distance algorithm to calculate the similarity between the first audio sequence and the second audio sequence; S155, using the FASTA algorithm to calculate the similarity between the first audio sequence and the second audio sequence; the visualization unit is configured to render and present the first audio sequence and the second audio sequence.

8. The system of claim 7, wherein, the visualization unit is specifically configured to: if the similarity is greater than a preset similarity threshold, rendering and presenting similar elements in the first audio sequence and the second audio sequence; if the similarity is less than or equal to the preset similarity threshold, rendering and presenting dissimilar elements in the first audio sequence and the second audio sequence.

9. Audio visualizing glasses, characterized in that comprising: a sound pressure level test indicator and a built-in sound box; the built-in sound box comprises a box body, a microphone, a full-frequency speaker, a low-frequency speaker, a power switch, a wireless transmission module, a memory module and a processor module; the microphone is configured to collect audio signals; the full-frequency speaker is configured to output full-frequency signals; the low-frequency speaker is configured to output low-frequency signals; the power switch is configured to start the built-in sound box; the wireless transmission module is configured to transmit wireless audio; the sound pressure level test indicator is configured to output corresponding light according to the full-frequency signals and the low-frequency signals output by the full-frequency speaker and the low-frequency speaker; the memory module is configured to store program instructions, the program instructions being executed by the processor module, so that the glasses perform the method of any one of claims 1-6.

Images