Audio recognition method, computer device and computer program product

Abstract

The application relates to an audio recognition method, a computer device, a storage medium and a computer program product. By determining a to-be-associated audio set matched with an acquired index audio feature in an audio library, a preset audio corresponding to the to-be-associated audio set is determined, an audio identifier of an audio in the to-be-associated audio set is associated with an audio identifier of the preset audio, and an audio relationship library is obtained. In recognition, the first audio identifier of the matching audio corresponding to the acquired audio feature and the associated second audio identifier are recognized by using the audio relationship library, and the corresponding preset audio is taken as an audio recognition result. Compared with a traditional mode of directly taking the recognized most similar audio segment as a recognition result, the present application takes the audio relationship library containing the association relationship between each matching audio and the preset audio corresponding to the matching audio as the basis of audio recognition, so that the audio meeting the user search intention can be taken as the recognition result, and the audio recognition accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio recognition method, computer device, storage medium, and computer program product. Background Technology

[0002] With the development of internet technology, users can now listen to songs and other audio content via mobile devices such as smartphones. However, when users hear a song but don't know its name, they need to identify the song's audio using their mobile devices. Currently, the common method for identifying song audio is to directly push similar audio clips to the user. However, directly searching for similar audio clips based on the received audio can lead to insufficient accuracy in audio search.

[0003] Therefore, current database-based audio recognition methods suffer from low recognition accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide an audio recognition method, computer device, computer-readable storage medium, and computer program product that can improve the accuracy of recognition in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides an audio recognition method, the method comprising:

[0006] Obtain the audio features of the audio to be identified;

[0007] Based on the audio features, search the audio library for matching audio that matches the audio to be identified;

[0008] Based on the first audio identifier of the matched audio, a second audio identifier associated with the first audio identifier is queried in the audio relationship database; the audio relationship database is used to record the association relationship between the audio identifier of at least one audio in the same audio set and the audio identifier of a preset audio, wherein the first audio identifier is one of the audio identifiers of the at least one audio, and the second audio identifier is the audio identifier of the preset audio;

[0009] Based on the preset audio corresponding to the second audio identifier, the audio recognition result of the audio to be recognized is obtained.

[0010] In one embodiment, obtaining the audio recognition result of the audio to be recognized based on the preset audio corresponding to the second audio identifier includes:

[0011] Obtain the preset audio corresponding to the second audio identifier in the audio library;

[0012] Obtain the audio tag corresponding to the preset audio;

[0013] The matched audio and the preset audio with the added audio tag are used as the audio recognition result of the audio to be recognized.

[0014] In one embodiment, it further includes:

[0015] The audio recognition results are displayed through the audio recognition result display interface.

[0016] In one embodiment, the method further includes: a step of constructing an audio relation library; wherein the step of constructing the audio relation library specifically includes:

[0017] Based on the indexed audio features, determine the set of audio files in the audio library that match the indexed audio features;

[0018] Determine the preset audio corresponding to the audio set;

[0019] The audio identifiers of the audio in the audio set are associated with the audio identifiers of the preset audio to obtain an audio relationship library.

[0020] In one embodiment, determining the set of audio files in the audio library that match the indexed audio features includes:

[0021] Based on the indexed audio features, determine the matching degree between each audio file in the audio library and the indexed audio features;

[0022] Audio files in the audio library with a matching degree greater than or equal to a preset matching degree threshold are combined into an audio set.

[0023] In one embodiment, associating the audio identifiers of audios in the audio set with the audio identifiers of preset audios to obtain an audio relationship library includes:

[0024] Generate key-value pairs based on the audio identifier of the preset audio and the audio identifiers of the audio in the audio set;

[0025] The key-value pairs are stored to obtain an audio relationship library.

[0026] In one embodiment, after storing the key-value pairs to obtain the audio relation library, the method further includes:

[0027] According to a preset period, the key-value pairs in the audio relation library are synchronized to the cache;

[0028] Detect changes to key-value pairs in the audio relational database within a preset time interval, and update the cache with the changes.

[0029] In one embodiment, determining the preset audio corresponding to the audio set includes:

[0030] If all audio files in the audio set belong to the preset restriction category, then the preset prompt category audio files will be used as the preset audio files of the audio set.

[0031] Secondly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0032] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0033] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0034] The aforementioned audio recognition method, computer device, storage medium, and computer program product, after acquiring indexed audio features, determine the set of audio samples in an audio library that match the indexed audio features, identify the preset audio corresponding to the set of audio samples to be associated, and associate the audio identifiers of the audio samples in the set of audio samples with the audio identifiers of the preset audio samples to obtain an audio relationship library. During recognition, the acquired audio features can be queried in the audio library to identify the matching audio samples and their first audio identifiers, and the second audio identifier associated with the first audio identifier can be queried using the audio relationship library. Based on the preset audio corresponding to the second audio identifier in the audio library, the audio recognition result of the audio to be recognized is obtained. Compared to the traditional method of directly using the most similar audio segment as the recognition result, this solution uses an audio relationship library containing the association relationships between each matching audio sample and the preset audio corresponding to these matching audio samples as the basis for audio recognition, thereby enabling the use of audio samples that match the user's search intent as the recognition result and improving the accuracy of audio recognition. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an audio recognition method in one embodiment;

[0036] Figure 2 This is a flowchart illustrating the audio relationship database identification step in one embodiment;

[0037] Figure 3 This is a flowchart illustrating the steps involved in building an audio relationship database in one embodiment.

[0038] Figure 4 This is a schematic diagram of the interface of an audio recognition method in one embodiment;

[0039] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] In one embodiment, such as Figure 1 As shown, an audio recognition method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. This embodiment includes the following steps:

[0042] Step S202: Obtain the audio features of the audio to be identified.

[0043] The audio to be identified can be any audio that requires audio recognition. This audio can be formed by sounds emitted from the outside world or internally by the terminal after the user triggers the audio recognition command. The terminal can obtain the audio features corresponding to the audio to be identified. These audio features can be audio fingerprints; that is, the terminal can use the audio fingerprint of the audio to be identified as the corresponding audio feature.

[0044] Step S204: Query the audio library for matching audio that matches the audio to be identified based on the audio features; the matching audio is the audio in the audio library that matches the audio features.

[0045] The audio feature can be the audio feature of the audio to be identified. The terminal can query an audio library using this audio feature to identify matching audio in the audio library that matches the audio feature, such as the audio with the highest similarity to the audio feature in the audio library. This matching audio can have a corresponding first audio identifier. Taking a song as an example, the terminal can perform song recognition based on the audio fingerprint of the song to be identified using an audio library, obtain the corresponding matching song from the audio library, and acquire the data identifier of the matching song as the aforementioned first audio identifier.

[0046] Step S206: Based on the first audio identifier of the matched audio, query the audio relationship database for the second audio identifier associated with the first audio identifier; the audio relationship database is used to record the association relationship between the audio identifier of at least one audio in the same audio set and the audio identifier of a preset audio, the first audio identifier is one of the audio identifiers of at least one audio, and the second audio identifier is the audio identifier of the preset audio.

[0047] The audio relationship database can be a database containing associations between audio and preset audio. These associations can be between audio identifiers. Each audio has a first audio identifier, and the preset audio has a second audio identifier. The terminal can use the first audio identifier to query the audio relationship database to obtain the second audio identifier associated with it. This second audio identifier is the audio identifier of the preset audio corresponding to the matched audio. Specifically, taking a song as an example, the terminal can query the audio relationship database using the first audio identifier corresponding to the matched song obtained from the song recognition method. This database includes pre-set associations, which are updated according to a preset time. It includes the first and second audio identifiers and their corresponding relationship lists. The first and second audio identifiers have a corresponding relationship based on the relationship list. These relationships can be in key-value format, where the first audio identifier is the key and the second audio identifier is the value. Therefore, during the query, the terminal can retrieve the second audio identifier associated with the first audio identifier.

[0048] Step S208: Obtain the audio recognition result of the audio to be recognized based on the preset audio corresponding to the second audio identifier in the audio library.

[0049] The second audio identifier can be an identifier of a preset audio corresponding to the matched audio. The terminal can query the audio library using the second audio identifier obtained from the above association relationship to obtain the preset audio corresponding to the second audio identifier, and then use the preset audio as the audio recognition result of the audio to be recognized. During the query process of the first audio identifier, if the terminal finds a first audio identifier in the audio relationship library, the terminal can replace the first audio identifier with the second audio identifier according to the association relationship of the first audio identifier; if the terminal finds no association relationship corresponding to the first audio identifier in the audio relationship library, the terminal can replace the first audio identifier with a third audio identifier. The third audio identifier can be a pre-written identifier corresponding to an audio, such as the audio identifier corresponding to the aforementioned prompt-type audio.

[0050] In the aforementioned audio recognition method, after obtaining indexed audio features, a set of audio samples matching the indexed audio features in an audio library is determined. Preset audio samples corresponding to the set of audio samples to be associated are then identified, and the audio identifiers of the audio samples in the set of audio samples to be associated are associated with the audio identifiers of the preset audio samples, thus obtaining an audio relationship library. During recognition, the obtained audio features can be queried in the audio library to identify the matching audio samples and their first audio identifiers. The audio relationship library is then used to query the second audio identifier associated with the first audio identifier. Based on the preset audio samples corresponding to the second audio identifiers in the audio library, the audio recognition result of the audio to be recognized is obtained. Compared to the traditional method of directly using the most similar audio segment as the recognition result, this solution uses an audio relationship library containing the association relationships between each matching audio sample and the preset audio samples corresponding to these matching audio samples as the basis for audio recognition. This allows audio samples that match the user's search intent to be used as the recognition result, improving the accuracy of audio recognition.

[0051] In one embodiment, obtaining the audio recognition result of the audio to be recognized based on the preset audio corresponding to the second audio identifier in the audio library includes: obtaining the preset audio corresponding to the second audio identifier in the audio library; obtaining the audio tag corresponding to the preset audio; and using the matching audio and the preset audio with added audio tags as the audio recognition result of the audio to be recognized.

[0052] In this embodiment, the terminal can obtain the audio recognition result of the audio to be recognized based on the preset audio corresponding to the second audio identifier. Specifically, the terminal can use the matching audio that best matches the audio features and the preset audio together as the audio recognition result. The terminal can obtain the preset audio corresponding to the second audio identifier from the audio library, obtain the audio tag corresponding to the preset audio, and display the matching audio and the preset audio with added audio tags as the audio recognition result. The audio tag can be a label indicating the authority of the preset audio. For example, if the audio is a song, the preset audio can be the original version of the matching song. The terminal can add a tag to the original audio of the original version, and simultaneously display the recognized matching song and the original audio on the song recognition interface, with the original audio having the corresponding tag, thus reinforcing the user's impression of the authentic song.

[0053] In some embodiments, terminal 102 may also display the audio recognition result of the audio to be recognized through an audio recognition result display interface. For example, terminal 102 may display the audio recognition result of the audio to be recognized through its display screen.

[0054] Through this embodiment, the terminal can display the matching audio that best matches the audio features and the preset audio corresponding to the matching audio together as the recognition result, and add a mark to the preset audio so that the user can clearly know that the preset audio is the authoritative version of the audio, thereby improving the accuracy of audio recognition.

[0055] In one embodiment, the audio recognition result of the audio to be recognized is obtained based on the preset audio corresponding to the second audio identifier in the audio library, including: obtaining the preset audio corresponding to the second audio identifier in the audio library; if the matched audio is a preset restricted audio, the preset audio is displayed as the audio recognition result.

[0056] In this embodiment, the matching audio identified based on the audio features of the audio to be identified can be a preset restricted audio category, such as illegal audio. When the terminal identifies that the matching audio is a preset restricted audio category, the terminal can obtain the preset audio corresponding to the second audio identifier in the audio library and display the preset audio corresponding to the preset restricted audio category as the audio result. The preset audio corresponding to the preset restricted audio category can be a type of prompt audio. For example, for some illegal songs, the terminal can replace its identification result with the identifiers of various promotional audios and output the promotional audio as the audio identification result.

[0057] Through this embodiment, when the terminal recognizes that the matched audio is a restricted type audio, it can use a preset prompt type audio as the audio recognition result, so that the user can know the recognition status of the audio to be recognized in a timely manner, thereby improving the efficiency of audio recognition.

[0058] In one embodiment, such as Figure 3 As shown, a method for constructing an audio relationship library is provided. The method is illustrated using a terminal as an example. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. This implementation includes the following steps:

[0059] Step S302: Based on the indexed audio features, determine the set of audio files in the audio library that match the indexed audio features to be associated.

[0060] The indexed audio feature can be an audio feature used to build an audio relational database. The terminal can first obtain the indexed audio feature as the basis for building the audio relational database. The indexed audio feature can be an audio feature found in the audio library. For example, the indexed audio feature could be a sample audio feature, and the audio library could be a song library; then the indexed audio feature could be a segment or an entire segment of audio from a song in the song library. Specifically, the audio feature can be an audio fingerprint. An audio fingerprint is a unique digital feature extracted from an audio segment using a specific algorithm, used to identify massive amounts of sound samples or to track and locate the position of samples in a database. The terminal can use the audio fingerprint to identify matching audio in the audio library.

[0061] The audio library can contain various audio formats, each with its own unique audio characteristics. After obtaining these indexed audio characteristics, the terminal can search the audio library for these characteristics to identify multiple audio files that match the indexed audio characteristics. Based on these matching audio files, the terminal can then obtain a set of audio files to be associated with the indexed audio. For example, if the audio library is a music library, the terminal can identify multiple songs in the music library that match the indexed audio characteristics and use these matching songs as the set of audio files to be associated.

[0062] In some embodiments, the audio that matches the indexed audio features can be determined based on the matching degree. For example, in some embodiments, determining the set of audio to be associated in an audio library that matches the indexed audio features includes: determining the matching degree between each audio in the audio library and the indexed audio features based on the indexed audio features; and taking audio in the audio library with a matching degree greater than or equal to a preset matching degree threshold as audio to be associated, thus obtaining the set of audio to be associated. In this embodiment, when determining the audio to be associated that matches the indexed audio features, the terminal can query the audio library based on the indexed audio features, determine the matching degree between each audio in the audio library and the indexed audio features, and combine audio in the audio library with a matching degree greater than or equal to a preset matching degree threshold as audio to be associated, thus obtaining the set of audio to be associated. The set of audio to be associated can consist of at least one audio to be associated. In some embodiments, the matching degree can be a similarity of an audio feature. For example, taking songs as an example, the aforementioned audio library could be a music library containing multiple songs, each with different versions. These versions might share similar audio features. The terminal can then use these indexed audio features to search the music library for songs with a similarity greater than a preset similarity threshold, allowing the terminal to create a set of audio tracks to be associated. It should be noted that there can be various types of audio, and each type can form a set of audio tracks to be associated. For instance, if the audio library contains multiple songs, each with multiple versions, then these multiple versions can form a set of audio tracks to be associated.

[0063] Step S304: Determine the preset audio corresponding to the audio set to be associated.

[0064] The audio set to be associated can be a set containing at least one audio file to be associated. These audio files can be audio files in the audio library whose matching degree with the indexed audio features is greater than a preset matching degree threshold. That is, there is a certain similarity between the audio files in the audio set to be associated. For example, if the audio files to be associated are songs, then the songs to be associated in the audio set to be associated can be different versions of the same song, including but not limited to the original version, cover versions, and song excerpts. The terminal can determine the preset audio corresponding to the audio set to be associated. The preset audio can be a pre-set audio to be returned to the user. The preset audio can be an authoritative version of each audio file to be associated in the audio set to be associated, that is, the preset audio can be an audio that matches the user's intention to identify the audio. For example, if the audio is a song, and the song exists in the audio library and can be legally played, then the terminal can determine the original version of the audio from the audio files to be associated in the audio set to be associated, and use it as the preset audio corresponding to the audio set to be associated. If the song does not exist in the audio library or cannot be legally played, the terminal can use a pre-set prompt audio as the preset audio corresponding to the audio set to be associated.

[0065] Step S306: Associate the audio identifiers of the audios in the audio set to be associated with the audio identifiers of the preset audios to obtain the audio relationship library.

[0066] The audio set to be associated can contain at least one audio to be associated, and if multiple audio sets exist, they are similar to each other. Each audio set to be associated can have a corresponding audio identifier, and this audio identifier can be unique for each audio set. The preset audio can be the original recording of the audio set to be associated or a specific audio. The terminal can associate the audio identifiers of the audio sets to be associated with the audio identifiers of the preset audio sets to obtain an audio relationship database. For example, for each audio set to be associated, the terminal associates the audio identifier of each audio set with the identifier of the preset audio set to be associated, thereby obtaining at least one association relationship for each audio set to be associated. There can be multiple audio sets to be associated, and the preset audio for each audio set can be different. The terminal can obtain multiple sets of association relationships based on multiple audio sets to be associated, and can store these association relationships to obtain the audio relationship database. The aforementioned audio relationship database can be a persistent database that stores a series of mapping relationships between audio identifiers and preset audio identifiers. These mapping relationships can be formally described as (A, B). Taking the audio as a song and the audio feature as an audio fingerprint as an example, this relationship means that when the audio fingerprint algorithm identifies song A, the result should be replaced with song B. This audio relationship database can be generated by an algorithm or manually edited and maintained, for example, by manually specifying the preset audio corresponding to each set of audio to be associated.

[0067] In the aforementioned method for constructing an audio relationship database, after obtaining indexed audio features, a set of audio samples matching the indexed audio features in the audio database is determined. Preset audio samples corresponding to the set of audio samples to be associated are then identified, and the audio identifiers of the audio samples in the set of audio samples to be associated are associated with the audio identifiers of the preset audio samples, thus obtaining the audio relationship database. During recognition, the obtained audio features can be queried in the audio database to identify the matching audio samples and their first audio identifiers. The audio relationship database is then used to query the second audio identifier associated with the first audio identifier. Based on the preset audio samples corresponding to the second audio identifiers in the audio database, the audio recognition result of the audio to be recognized is obtained. Compared to the traditional method of directly using the most similar audio segment as the recognition result, this solution uses an audio relationship database containing the associations between each matching audio sample and the preset audio samples corresponding to these matching audio samples as the basis for audio recognition. This allows audio samples that match the user's search intent to be used as the recognition result, improving the accuracy of audio recognition.

[0068] In one embodiment, associating the audio identifiers of audios in the audio set to be associated with the audio identifiers of preset audios to obtain an audio relationship library includes: generating key-value pairs based on the audio identifiers of preset audios and the audio identifiers of audios in the audio set to be associated; and storing the key-value pairs to obtain the audio relationship library.

[0069] In this embodiment, each audio in the aforementioned audio set to be associated can be associated with a preset audio corresponding to that audio set. This association can be based on audio identifiers. The terminal can associate each audio in the audio set to be associated with a preset audio. For each audio in the audio set to be associated, the terminal can generate a key-value pair based on the audio identifier of the preset audio corresponding to the audio set to be associated and the audio identifier of the audio in the audio set to be associated. For example, the terminal can use the audio identifier of the audio in the audio set to be associated as the key and the audio identifier of the corresponding preset audio as the value, thereby generating a key-value pair based on the key and value. The aforementioned audio set to be associated may include at least one audio, so the terminal can generate at least one key-value pair, and the terminal can store the key-value pair to obtain the aforementioned audio relational database. Specifically, the aforementioned audio relational database can be an instance of a MySQL database, which is a mapping relational database. MySQL is a relational database management system. Relational databases store data in different tables instead of putting all data in one large warehouse, thus increasing speed and improving flexibility. The aforementioned audio relational database mainly has two fields: track_id and mapping_id. Here, `track_id` represents the ID used for song recognition during audio identification, specifically the ID of the closest audio source obtained through audio feature recognition. `mapping_id` can be a business-defined lookup ID, representing the ID of the preset audio corresponding to the identified audio within its association relationship. Specifically, taking a song as an example, `track_id` and `mapping_id` represent the identifier of the song to be replaced and the identifier of the target song to be replaced, respectively, and each identifier may include a series of auxiliary fields. The terminal can construct and store key-value pairs based on `track_id` and `mapping_id` to obtain the aforementioned audio relationship database. This audio relationship database is used to determine the target audio to be output to the user during audio recognition.

[0070] In this embodiment, the terminal constructs key-value pairs between the audio identifiers of each audio in the set of audio to be associated and the corresponding preset audio identifiers to form an audio relationship library. The terminal can then perform audio recognition based on this audio relationship library, thereby improving the accuracy of audio recognition.

[0071] In one embodiment, after storing the key-value pairs to obtain the audio relational database, the method further includes: synchronizing the key-value pairs in the audio relational database to a cache according to a preset period, so as to query the key-value pairs based on the cache.

[0072] In this embodiment, after establishing and storing key-value pairs between the audio identifiers of the audio in the set to be associated and the audio identifiers of preset audio in the persistent audio database, the terminal can also synchronize the mapping relationships in the persistent database to the cache. For example... Figure 2 As shown, Figure 2 This is a flowchart illustrating the audio relational database identification steps in one embodiment. During synchronization, the terminal can synchronize the key-value pairs in the aforementioned audio relational database to a cache at a preset period for querying key-value pairs based on the cache. The cache can be a memory-based key-value (kv) database. A kv database is a type of database that stores data in key-value pairs. To meet high-speed access requirements, the terminal can synchronize the content of the persistent audio relational database to the cache at certain intervals using a synchronization program. The cache can be a load-balanced CKV+ database, which can be a memory-based database with the key being `track_id` and the value being `mapping_id`. Due to the identical storage format, the terminal can synchronize the associations in the aforementioned audio database to the cache. During the identification process, the terminal can use a service program to, after obtaining the audio fingerprint algorithm's identification result for the audio (e.g., obtaining an audio identifier), query the cache based on the audio identifier to determine if mapping is needed, and output the mapping result as the final result. Since the storage space in the cache is limited, the cache may only contain a portion of the data from the audio relational database, and the data in the cache will be replenished at preset intervals. During recognition, the terminal can quickly retrieve the data to be mapped from the cache according to preset rules. Furthermore, during a specific query, the terminal can use a mapping query program to check if the song's identifier exists in the cache after receiving the song identified by the audio fingerprint algorithm. If it exists, it replaces the identifier with the corresponding mapping_id; if it does not exist, no processing is performed, and the final result is returned to the caller.

[0073] Furthermore, the relationships within the aforementioned audio relational database can be modified, and the terminal can synchronize these changes to the corresponding locations in the cache. For example, in one embodiment, the method further includes detecting changes to key-value pairs in the audio relational database within a preset time interval and updating the cache with these changes. In this embodiment, the key-value pairs stored in the audio relational database may change, such as being added, deleted, or modified. The terminal can detect these changes within a preset time interval and update the cache with these changes. Specifically, if a key-value pair in the audio relational database changes within a preset time interval, the terminal can synchronize this change information to the corresponding key-value pair in the cache, i.e., perform the same change on the corresponding key-value pair in the cache. The terminal can implement this change information synchronization process using a database synchronization program. For example, the terminal can use a database synchronization program to select data added, deleted, and modified in the last two minutes from the MySQL database every two minutes, record each change, and update the corresponding data in the cache with the corresponding modifications, thereby achieving change information synchronization.

[0074] Through the above embodiments, when the association relationship in the audio relationship database changes, the terminal can promptly synchronize to the corresponding position in the cache, ensuring the timeliness of the data in the cache. Furthermore, the query of audio association relationships based on the cache has a higher concurrency and improves the efficiency of audio recognition.

[0075] In one embodiment, determining the preset audio corresponding to the audio set to be associated includes: if each audio in the audio set to be associated belongs to a preset restriction category, then the preset prompt category audio is used as the preset audio of the audio set.

[0076] In this embodiment, the audio in the aforementioned set of audio to be associated can be a restricted type of audio, such as audio involving violations or audio not found in the audio library. Furthermore, these unplayable audios can exist in multiple versions, forming corresponding sets of audio to be associated. If the terminal detects that all audio in the aforementioned set of audio to be associated belongs to a preset restricted type, the terminal can use a preset prompt-type audio as the preset audio corresponding to each audio in these sets of audio to be associated. For example, for violating audio, the terminal can use a preset prompt indicating that the audio is violating as the preset audio; for audio not found in the audio library, the terminal can use an audio indicating that the audio is copyright-free as the preset audio. It is clear that for other types of unplayable restricted audio, the terminal can determine the corresponding prompt-type audio as the preset audio.

[0077] Through this embodiment, when the terminal finds that the audio corresponding to the audio feature is a restricted audio, it can use the preset prompt audio as the preset audio, thereby improving the recognition efficiency during audio recognition.

[0078] In one embodiment, an application example of an audio recognition method is provided. In this embodiment, the audio can be a song, and the audio features can be audio fingerprints. Figure 4 As shown, Figure 4 This is a schematic diagram of the interface of an audio recognition method in one embodiment. Taking the audio to be recognized as "Daoxiang" as an example, different versions of "Daoxiang" have unique identifiers in the audio library. These different versions of the song are quite similar in content, so the fingerprints extracted by the terminal through the audio fingerprint algorithm also have high similarity. When a user identifies a currently playing "Daoxiang," the audio fingerprint algorithm converts the received sound signal into a fingerprint representation and compares it with the fingerprints in the entire audio library, returning all song identifiers with high similarity. The terminal can then sort them according to the similarity. These identified songs may include covers and / or pirated versions; in poor sound reception conditions, covers / pirated versions may even be ranked higher. Figure 4 As shown, assuming a version sung by another artist is identified as 20000, while the original version of "Rice Fragrance" is identified as 10000, the terminal can record their association in the audio relationship database, setting `track_id` to 20000 and `mapping_id` to 10000. This allows the terminal to ultimately return the original song during audio recognition. Furthermore, in some embodiments, the `mapping_id` in the aforementioned association can also be set to the identifier of the latest version in the latest album corresponding to a song, thereby increasing the exposure of the new album.

[0079] Through the above embodiments, the terminal can improve the accuracy and precision of audio recognition by using a pre-built audio relationship database, especially when the entire network is filled with a large number of music fragments and the music library is not well organized.

[0080] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0081] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for constructing an audio relational database and an audio recognition method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0082] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0083] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method for constructing an audio relational library and the audio recognition method.

[0084] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the above-described method for constructing an audio relational library and the audio recognition method.

[0085] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method for constructing an audio relational library and the audio recognition method.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0087] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0088] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An audio recognition method, characterized in that, The method includes: Obtain the audio features of the audio to be identified; Based on the audio features, search the audio library for matching audio that matches the audio to be identified; Based on the first audio identifier of the matched audio, a second audio identifier associated with the first audio identifier is queried in the audio relationship database. The audio relationship database includes pre-set association relationships, including the first audio identifier, the second audio identifier, and the corresponding relationship list. The audio relationship database is used to record the association relationship between the audio identifier of at least one audio in the same audio set and the audio identifier of a preset audio. The first audio identifier is one of the audio identifiers of the at least one audio, and the second audio identifier is the audio identifier of the preset audio. The preset audio is the original version of the audio. The construction steps of the audio relationship database include: determining the audio set in the audio database that matches the indexed audio features based on the indexed audio features; determining the preset audio corresponding to the audio set; using the audio identifier of the audio in the audio set to be associated as the key and the audio identifier of the corresponding preset audio as the value to obtain key-value pairs; storing the key-value pairs to obtain the audio relationship database; the indexed audio features represent the audio features used to construct the audio relationship database. Based on the preset audio corresponding to the second audio identifier, the audio recognition result of the audio to be identified is obtained, including: displaying the matching audio and the original version of the audio, and the original version of the audio has an audio tag; the audio tag represents that the original version of the audio has an authoritative tag.

2. The method according to claim 1, characterized in that, The step of obtaining the audio recognition result of the audio to be recognized based on the preset audio corresponding to the second audio identifier includes: Obtain the preset audio corresponding to the second audio identifier in the audio library; Obtain the audio tag corresponding to the preset audio; The matched audio and the preset audio with the added audio tag are used as the audio recognition result of the audio to be recognized.

3. The method according to claim 1 or 2, characterized in that, Also includes: The audio recognition results are displayed through the audio recognition result display interface.

4. The method according to claim 1, characterized in that, The process of determining the set of audio files in the audio library that match the indexed audio features includes: Based on the indexed audio features, determine the matching degree between each audio file in the audio library and the indexed audio features; Audio files in the audio library with a matching degree greater than or equal to a preset matching degree threshold are combined into an audio set.

5. The method according to claim 1, characterized in that, After storing the key-value pairs to obtain the audio relation library, the process further includes: According to a preset period, the key-value pairs in the audio relation library are synchronized to the cache; Detect changes to key-value pairs in the audio relational database within a preset time interval, and update the cache with the changes.

6. The method according to claim 1, characterized in that, The step of determining the preset audio corresponding to the audio set includes: If all audio files in the audio set belong to the preset restriction category, then the preset prompt category audio files will be used as the preset audio files of the audio set.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images