Marine mammal audio retrieval method and system based on biohashing ciphertext

Abstract

This application discloses a method and system for marine mammal audio retrieval based on biohash ciphertext, relating to the field of audio processing. The method comprises: extracting sub-band CQT energy-entropy ratio features from the original audio, constructing a key-address index table, and constructing a biohash sequence based on the key-address index table; segmenting the original audio using a dual-threshold segmentation method based on short-time energy and spectral distribution variance to obtain short-time audio segments, mapping the short-time audio segments to the key-address index table, and reconstructing the biohash sequence; performing an improved AES encryption on the original audio to obtain ciphertext audio; constructing a hash index table based on the mapping relationship between the reconstructed biohash sequence, key, and ciphertext audio, and uploading the ciphertext audio and hash index table to the cloud for retrieval. This application uses an audio segmentation algorithm to reconstruct the biohash sequence, improving retrieval efficiency, while also using an improved AES encryption algorithm to ensure data security.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a method and system for marine mammal audio retrieval based on bio-hash ciphertext. Background Art

[0002] With the increasing number and depth of human activities in the ocean, the marine ecosystem has been significantly impacted and altered. Therefore, monitoring and studying marine mammal populations is becoming increasingly important. Using sound signals as a medium is the most effective way to identify the daily biological activities of marine mammals in the ocean. However, due to the complexity of the marine environment, it is difficult to accurately and efficiently retrieve information about marine mammal audio from massive amounts of multimedia data.

[0003] The audio hash retrieval algorithm converts the biometric features of the audio into a hash code and stores it with the corresponding audio in a hash table, and retrieves the audio from the cloud. This algorithm has become one of the research topics in the field of audio retrieval.

[0004] However, since the cloud is an untrusted third party, the audio data and biometrics need to be protected during the hash retrieval process. Therefore, how to achieve efficient and secure audio retrieval is a key issue that needs to be solved urgently. Summary of the Invention

[0005] Based on this, it is necessary to address the above technical issues and provide a marine mammal audio retrieval method and system based on bio-hash ciphertext that can improve the security of cloud-based audio retrieval.

[0006] In a first aspect, the present application provides a method for marine mammal audio retrieval based on biohash ciphertext, which is applied to a server. The method comprises:

[0007] Extract sub-band CQT energy-entropy ratio features from the original audio, construct a key-address index table containing the mapping relationship between the original audio, sub-band CQT energy-entropy ratio features, and the key based on the sub-band CQT energy-entropy ratio features, and construct a bio-hash sequence based on the key-address index table;

[0008] A dual-threshold segmentation method based on short-time energy and spectrum distribution variance is used to segment the original audio to obtain short-time audio segments. The short-time audio segments are mapped to a key-address index table to reconstruct the bio-hash sequence.

[0009] Encrypt the original audio to obtain the ciphertext audio;

[0010] A hash index table is constructed based on the mapping relationship between the reconstructed biological hash sequence, key and ciphertext audio, and the ciphertext audio and hash index table are uploaded to the cloud for retrieval.

[0011] In one embodiment, extracting a sub-band CQT energy entropy ratio feature from the original audio includes:

[0012] Perform pre-emphasis processing, framing and windowing processing on the original audio in sequence to obtain a pre-processed signal;

[0013] The preprocessed signal is subjected to CQT variation and the sub-band CQT energy is calculated. The sub-band CQT entropy spectrum is calculated according to the sub-band CQT energy. The sub-band CQT energy-entropy ratio feature is obtained according to the sub-band CQT energy and the sub-band CQT entropy spectrum.

[0014] In one embodiment, constructing a key-address index table containing mapping relationships between original audio, sub-band CQT energy entropy ratio features, and keys based on the sub-band CQT energy entropy ratio features includes:

[0015] Classify the sub-band CQT energy entropy ratio features and obtain several feature labels;

[0016] A key corresponding to each feature label is constructed, and the original audio is classified according to each feature label. A key-address index table is constructed based on a single mapping relationship between the key, the sub-band CQT energy-entropy ratio features of different categories, and the original audio of different categories.

[0017] In one embodiment, a dual threshold segmentation method of short-time energy and spectrum distribution variance is used to segment the original audio to obtain short-time audio segments, and the short-time audio segments are mapped to a key-address index table. Reconstructing the biometric hash sequence includes:

[0018] Obtain short-time energy feature sequence and spectrum distribution variance feature sequence based on original audio;

[0019] Constructing the corresponding histograms of the short-time energy feature sequence and the spectral distribution variance feature sequence respectively, and extracting the corresponding thresholds of the short-time energy feature sequence and the spectral distribution variance feature sequence respectively from the histograms;

[0020] Compare the short-time energy feature sequence and the spectrum distribution variance feature sequence with the corresponding thresholds respectively, and construct the threshold sequence according to the comparison results;

[0021] Map the thresholded sequence to the original audio to obtain a short audio segment;

[0022] According to the position of the short audio segment in the original audio, the short audio segment is mapped to the key-address index table to determine the position index of the biometric hash sequence to obtain the hash segment;

[0023] Summarize the hash segments corresponding to each short audio segment and reconstruct the biological hash sequence.

[0024] In one embodiment, extracting the corresponding thresholds of the short-time energy feature sequence and the spectrum distribution variance feature sequence from the histogram includes:

[0025] Several local maxima are found from the histogram, several target maxima are selected from each local maximum according to their size, and control parameters are introduced to balance the several target maxima to obtain the threshold.

[0026] In one embodiment, encrypting the original audio to obtain the ciphertext audio includes: encrypting the original audio using an improved AES encryption algorithm;

[0027] Among them, the improved AES encryption algorithm includes:

[0028] Generate the target sequence through the Logistic system, and obtain a set of initial vectors and several groups of group keys by grouping the target sequence;

[0029] The plaintext sequence is grouped to obtain several groups of grouped plaintexts, each group key is processed in ascending order to obtain a position index, and the grouped plaintexts are scrambled and encrypted according to the position index to obtain a grouped ciphertext sequence;

[0030] After performing inner product processing on the first group ciphertext sequence and the initial vector, the first group ciphertext sequence and the first group key are AES encrypted to obtain the first group ciphertext; after performing inner product processing on each group ciphertext sequence and the previous group ciphertext, the first group ciphertext sequence and the corresponding group key are AES encrypted to obtain the corresponding group ciphertext;

[0031] The ciphertext of each group is concatenated to obtain the ciphertext.

[0032] In a second aspect, the present application provides a marine mammal audio retrieval method based on bio-hash ciphertext, which is applied to a mobile terminal and includes:

[0033] Extract sub-band CQT energy-entropy ratio features from the query audio, classify the query audio based on the sub-band CQT energy-entropy ratio features to obtain a label for the query audio, query a key-address index table based on the label to obtain a key, and construct a biometric hash sequence corresponding to the query audio based on the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, sub-band CQT energy-entropy ratio features of different categories, and original audio of different categories;

[0034] A dual-threshold segmentation method based on short-time energy and spectrum distribution variance is used to segment the query audio, obtain short-time audio segments, map the short-time audio segments to a key-address index table, reconstruct the bio-hash sequence, and upload the bio-hash sequence to the cloud.

[0035] Obtain the ciphertext audio that successfully matches the similarity between the biological hash sequence and the hash index table in the cloud, decrypt the ciphertext audio, and obtain the corresponding result of the query audio; among which, the hash index table is constructed based on the mapping relationship between the biological hash sequence, key and ciphertext audio.

[0036] In a third aspect, the present application provides a server comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method for marine mammal audio retrieval based on biohash ciphertext applied to the server are performed.

[0037] In a fourth aspect, the present application provides a mobile terminal, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned marine mammal audio retrieval method based on biohash ciphertext applied to the mobile terminal are implemented.

[0038] In the fifth aspect, the present application provides a marine mammal audio retrieval system based on bio-hash ciphertext, including the above-mentioned server and mobile terminal.

[0039] The above-mentioned marine mammal audio retrieval method and system based on bio-hash ciphertext extracts sub-band CQT energy entropy ratio features from the original audio, constructs a key-address index table containing the mapping relationship between the original audio, sub-band CQT energy entropy ratio features and keys based on the sub-band CQT energy entropy ratio features, and constructs a bio-hash sequence according to the key-address index table; uses a dual threshold segmentation method of short-time energy and spectrum distribution variance to segment the original audio to obtain short-time audio segments, maps the short-time audio segments to the key-address index table, and reconstructs the bio-hash sequence; encrypts the original audio to obtain ciphertext audio; constructs a hash index table based on the mapping relationship between the reconstructed bio-hash sequence, key and ciphertext audio, and uploads the ciphertext audio and hash index table to the cloud for retrieval. The sub-band CQT energy entropy ratio feature in this application combines the comprehensive characteristics of non-uniform frequency resolution and energy information, so that it has good resolution in both time and frequency, is suitable for analyzing complex audio signals, and exhibits strong robustness in noisy environments. In addition, the use of audio segmentation algorithm to reconstruct the biological hash sequence can not only effectively reduce redundant data, thereby improving retrieval efficiency, but also to a certain extent improve the abstract level of the original audio, thereby improving data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the process of extracting sub-band CQT energy entropy ratio features in one embodiment;

[0041] Figure 2 A diagram showing a key-address index table model in one embodiment;

[0042] Figure 3 A schematic diagram of a process for constructing a biohash sequence in one embodiment;

[0043] Figure 4 The figure is a flowchart of an improved AES encryption in one embodiment;

[0044] Figure 5 A diagram of a hash index table model in one embodiment;

[0045] Figure 6 Schematic diagram of a flow chart of a marine mammal audio retrieval method based on bio-hash ciphertext in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] The embodiment of the present application provides a marine mammal audio retrieval method based on bio-hash ciphertext, such as Figure 1 As shown, the following steps are included:

[0048] S101. Extract sub-band CQT energy entropy ratio features from the original audio, construct a key-address index table containing the mapping relationship between the original audio, sub-band CQT energy entropy ratio features and keys based on the sub-band CQT energy entropy ratio features, and construct a bio-hash sequence according to the key-address index table.

[0049] Firstly, the sub-band CQT energy-entropy ratio features are extracted from the original audio. Then, the features are classified by SVM and a key-address index table is constructed according to the classification results. Finally, a feature security template with a single mapping key is established based on the key-address index table and a bio-hash sequence is quantified to construct the feature security template.

[0050] S102. Use a dual threshold segmentation method of short-time energy and spectrum distribution variance to segment the original audio, obtain short-time audio segments, map the short-time audio segments to a key-address index table, and reconstruct the bio-hash sequence.

[0051] The original audio is segmented using a dual-threshold audio segmentation method based on short-time energy and spectrum distribution variance to obtain non-redundant short-time audio clips, and the bio-hash sequence is reconstructed.

[0052] S103: Encrypt the original audio to obtain ciphertext audio.

[0053] Use symmetric encryption algorithms (such as AES, DES, etc.), asymmetric encryption algorithms (RSA, ECC, etc.), hybrid encryption algorithms, etc. to encrypt the original audio, obtain the ciphertext audio, and upload it to the cloud to build a ciphertext audio library.

[0054] S104. Construct a hash index table based on the mapping relationship between the reconstructed biometric hash sequence, the key, and the ciphertext audio, and upload the ciphertext audio and the hash index table to the cloud for retrieval.

[0055] The hash index table is also uploaded to the cloud, and the retrieval service is implemented through the cloud's ciphertext audio library and hash index table. The user uploads the processed audio to be retrieved to the cloud, where it is matched with the biometric hash sequence in the hash index table. A successful biometric hash sequence can be retrieved by querying the hash index table to obtain the corresponding ciphertext audio. The ciphertext is then decrypted to obtain the retrieval result of the audio to be retrieved.

[0056] In one embodiment, extracting sub-band CQT energy-entropy ratio features from the original audio in S101 includes: performing pre-emphasis processing, framing, and windowing processing on the original audio in sequence to obtain a preprocessed signal; performing CQT changes on the preprocessed signal and calculating the sub-band CQT energy, calculating the sub-band CQT entropy spectrum based on the sub-band CQT energy, and obtaining the sub-band CQT energy-entropy ratio features based on the sub-band CQT energy and the sub-band CQT entropy spectrum.

[0057] Among them, CQT (Constant-Q Transform) is a transformation technology that can be used to capture energy characteristics at different frequencies in audio signals.

[0058] Specifically, if Figure 1 As shown, the original ocean audio is pre-emphasized to improve the high frequency part of the signal, and then the pre-processed signal S is obtained by framing and windowing. m (i)|i=1,2,…,N;m=1,2,…,L}, where the window function is a Hamming window, L represents the frame length, and N represents the number of frames.

[0059] The preprocessed signal x(j) undergoes CQT change according to formula (1) and the sub-band CQT energy E is calculated according to formula (2). b , and then calculate the sub-band CQT entropy spectrum G according to formulas (3) and (4) b Finally, formula (6) is used to calculate the sub-band CQT entropy ratio Γ = {Γ(i)|i = 1, 2, ..., N}.

[0060]

[0061] Where k = 1, 2, ..., K represents the frequency domain bins with constant Q variation, The value represented by f is rounded down. k is the center frequency of the kth bin, f s is the sampling frequency, ω(t) is the continuous window function, and the window With f k Inversely proportional real numbers.

[0062] In one embodiment, in S101, a key-address index table containing the mapping relationship between the original audio, the sub-band CQT energy entropy ratio feature and the key is constructed based on the sub-band CQT energy entropy ratio feature, including: classifying the sub-band CQT energy entropy ratio feature to obtain a number of feature labels; constructing a key corresponding to each feature label, and classifying the original audio according to each feature label, and constructing a key-address index table based on a single mapping relationship between the key, different categories of sub-band CQT energy entropy ratio features and different categories of original audio.

[0063] First, the biometric database is annotated using the K-modes clustering algorithm and classified using the SVM (Support Vector Machine) classifier. A key-address index table is then created using the labels, the classified feature dataset, the logical address of the audio, and the key. Finally, a feature security template with a single mapping key is created based on the parameters of the key-address index table and quantized to construct a biometric hash sequence.

[0064] Specifically, if Figure 3 As shown, the audio biometric dataset is used as a training set for the K-modes clustering algorithm to obtain the label λ of the dataset, and then the SVM classifier is used to train the classification model on the labeled feature set and the dataset is assigned to M categories, namely Γ λ ={Γ λ (i)|i=1,2,…,N; λ=1,2,…,M}, where Γ λ Indicates that the biometric feature belongs to the feature dataset of category λ.

[0065] Construct the corresponding key according to different feature labels λ λ , and then classify the corresponding original audio according to the feature label, and finally use the key key λ and different category feature datasets Γ λ , and different categories of original audio S λ The relationship of a single mapping is constructed as Figure 2 The key-address index table shown has a mapping relationship as shown in formula (6):

[0066]

[0067] Then build the feature security template. λ Excite the Henon chaotic system to generate a pseudo-random matrix Φ of M dimensions and length N λ ={Φ λ (i)|i=1,2,…,N;λ=1,2,…,M}, then Schmidt orthogonalize the pseudo-random matrix to construct the orthogonal matrix Θ λ ={Θ λ(i)|i=1,2,…,N;λ=1,2,…,M}. Finally, the M-dimensional vector Θ of the orthogonal matrix λ Perform inner product with one-to-one corresponding M-type feature data sets to construct security template V λ ={V λ (i)|i=1,2,…,N; λ=1,2,…,M}.

[0068] According to formula (7), the security template Ψ λ Quantify the bio-hash sequence H λ ={H λ (i)|i=1,2,…,N}, where the biohash H(1) is set to 0.

[0069]

[0070] In one embodiment, S102 includes: obtaining a short-time energy feature sequence and a spectral distribution variance feature sequence based on the original audio; constructing corresponding histograms of the short-time energy feature sequence and the spectral distribution variance feature sequence respectively, and extracting the corresponding thresholds of the short-time energy feature sequence and the spectral distribution variance feature sequence from the histograms respectively; comparing the short-time energy feature sequence and the spectral distribution variance feature sequence with the corresponding thresholds respectively, and constructing a thresholded sequence according to the comparison results; mapping the thresholded sequence to the original audio to obtain a short-time audio segment; mapping the short-time audio segment to a key-address index table according to the position of the short-time audio segment in the original audio to determine the position index of the biological hash sequence and obtain a hash segment; summarizing the hash segments corresponding to each short-time audio segment and reconstructing the biological hash sequence.

[0071] The original audio x(j) is segmented using the double threshold audio segmentation method of short-time energy and spectrum distribution variance to obtain a non-redundant short-time audio segment x k (n), where k represents the number of short audio segments.

[0072] Specifically, the input signal x(j) is framed and windowed to obtain the preprocessed signal x i (n), where i and n represent the number of audio frames and the frame length respectively.

[0073] According to formulas (8) and (9), the short-time energy characteristic sequence E(i) and the characteristic sequence C(i) of the spectral distribution variance of the preprocessed signal of each frame are calculated respectively. The formulas are as follows:

[0074]

[0075] Where DFT stands for discrete Fourier transform, f k represents the center frequency, and μ1 represents the spectral centroid.

[0076] Next, the short-time energy and spectrum distribution variance feature sequences of the original signal are used to construct a histogram and smooth it. Then, the first and second local maximum positions of the histogram corresponding to the short-time energy and spectrum spread are found. Finally, the thresholds of the short-time energy and spectrum distribution variance are calculated according to formula (10).

[0077]

[0078] Where δ is the threshold, ω1 and ω2 represent the first and second largest local maxima, i.e., the target maximum value, and weight = 5 is a control parameter, which can be set according to the actual situation.

[0079] Then, the short-time energy and spectral distribution variance are compared with their respective thresholds δ E and δ C Compare them and perform AND processing on the comparison results to construct the threshold sequence of the speech segment.

[0080] Map the thresholded sequence into the original signal to obtain each non-silent short-term speech segment x k (n) Obtain short audio segments at the start and end positions in the original audio.

[0081] Match the non-redundant short-time audio segment with the position index of the original audio, and then map the index to the position index of the corresponding biological hash sequence to obtain the hash segment of the non-redundant short-time audio segment Finally, the hash fragments are sequentially reconstructed into a biological hash sequence

[0082]

[0083] Where ξ is the hash segment length of the non-redundant short audio segment, ω represents the number of short audio segments, and λ is the category to which the reconstructed hash belongs.

[0084] In one embodiment, S103 includes: encrypting the original audio using an improved AES encryption algorithm. The original audio is encrypted using the improved AES encryption step to construct a ciphertext voice library, and then a classified ciphertext audio library C is constructed according to the audio label λ. λ And upload to the cloud.

[0085] Among them, the improved AES encryption algorithm includes: generating a target sequence through a logistic system, grouping the target sequence to obtain a group of initial vectors and several groups of group keys; grouping the plaintext sequence to obtain several groups of grouped plaintexts, performing ascending processing on each group of group keys to obtain a position index, scrambling and encrypting the grouped plaintexts through the position index to obtain a grouped ciphertext sequence; performing inner product processing on the first group of grouped ciphertext sequence and the initial vector, and then performing AES encryption with the first group of group keys to obtain the first group of grouped ciphertexts; performing inner product processing on each group of grouped ciphertext sequence and the previous group of grouped ciphertexts, and then performing AES encryption with the corresponding group key to obtain the corresponding grouped ciphertext; and splicing the groups of grouped ciphertexts to obtain ciphertexts.

[0086] Specifically, if Figure 4 As shown, the specific process of improved AES encryption is as follows:

[0087] S301, generate a length of Sequence and divide it into The data of the first group is used as the initial vector ρ0, and the vectors of the remaining groups are used as the group keys {ρ n |n=1,2,…,l}, where ρ n The length of is θ bits.

[0088] S302: Group the plaintext sequence to obtain l groups of grouped plaintext The group keys of each group are processed in ascending order to obtain the position index, and then the plaintext of each group is scrambled and encrypted by the position index to obtain the group ciphertext sequence {ξ n |n=1,2,…,l}.

[0089] S303, the first group of group ciphertext sequence ξ n Perform inner product processing with the initial vector θ0, and then use the group key ρ0 to perform AES encryption to obtain the first group of group ciphertext C1. Group ciphertext ξ n Perform inner product processing with the previous group ciphertext, and then execute the group key ρ n Perform AES encryption to obtain block ciphertext C n , the formula is as follows:

[0090]

[0091] S304, concatenate the grouped ciphertext data to obtain the entire plaintext data C = {C1, C2, ..., C n}.

[0092] In S104, different types of ciphertext audio C λ and rebuild hash indexes and key λ According to formula (13), a one-to-one hash index table is established. The biological hash index table model is as follows: Figure 5 shown.

[0093]

[0094] The server extracts audio features from the raw audio, then uses support vector machines (SVMs) to learn these features to construct diverse biometric templates and quantify them into a biometric hash sequence. Finally, an audio segmentation algorithm is used to reconstruct the hash sequence into a hash index. The server also performs a modified AES encryption on the raw audio data to construct a ciphertext audio library in the cloud. Finally, the ciphertext audio library and hash index are uploaded to the cloud to construct the biometric hash index table.

[0095] The improved AES encryption algorithm used in this embodiment can effectively ensure the security of audio data in the cloud and enhance the anti-attack capability of marine mammal audio.

[0096] In one embodiment, a method for retrieving marine mammal audio based on bio-hash ciphertext is applied to a mobile terminal, and the method includes: extracting sub-band CQT energy-entropy ratio features of the query audio, classifying the sub-band CQT energy-entropy ratio features to obtain a label of the query audio, querying a key-address index table according to the label to obtain a key, and constructing a bio-hash sequence corresponding to the query audio according to the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, sub-band CQT energy-entropy ratio features of different categories and original audio of different categories; using a dual threshold segmentation method of short-time energy and spectrum distribution variance to segment the query audio, obtain short-time audio segments, map the short-time audio segments to the key-address index table, reconstruct the bio-hash sequence, and upload the bio-hash sequence to the cloud; obtaining ciphertext audio that successfully matches the similarity between the bio-hash sequence and the hash index table in the cloud, decrypting the ciphertext audio, and obtaining the corresponding result of the query audio; wherein the hash index table is constructed based on the mapping relationship between the bio-hash sequence, the key and the ciphertext audio.

[0097] Specifically, the method comprises the following steps:

[0098] S501 : Extract sub-band CQT energy entropy ratio features S′(i) from the query audio using the same method as S101 .

[0099] S502. The sub-band CQT energy-entropy ratio feature of the query audio is used as a test set and a pre-trained SVM classification model is used to obtain the label of the query audio. Then, the key-address index table is queried based on the label to obtain the key key. Finally, the corresponding feature security template is stimulated based on the key and quantized to construct the biological hash sequence H′(i).

[0100] S503: Segment the query audio x′(n) into non-redundant short-duration audio segments x′ using an audio segmentation method. k (n), and then the bio-hash sequence H′(i) is hashed and reconstructed to obtain the bio-hash sequence And upload the reconstructed biohash sequence to the cloud.

[0101] S504, bio-hash sequence on the cloud mobile terminal Calculate the similarity distance Similary(:,:) with the hash index table corresponding to the kth category in the cloud, and take the biological hash sequence with the similarity distance value closest to 1 as the matching result. The formula is as follows:

[0102]

[0103] in is the edit distance, and They represent the biological hash sequence of the query audio and the hash index table of the kth category respectively. The value range of the similarity distance is [0,1]. The closer the similarity distance is to 1, the higher the sequence similarity between the two.

[0104] S505: Return the ciphertext audio corresponding to the cloud hash sequence to the mobile terminal and decrypt the ciphertext audio. Decryption and encryption are inverse processes. The decrypted voice segment is the query voice.

[0105] like Figure 6 The figure shows a complete flow chart of the marine mammal audio retrieval method implemented on the server and mobile sides.

[0106] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0107] Based on the same inventive concept, embodiments of the present application also provide a server, mobile terminal, and biohash-based marine mammal audio retrieval system for implementing the aforementioned biohash-based marine mammal audio retrieval method. The implementation solutions provided by the server, mobile terminal, and system are similar to those described in the aforementioned method. Therefore, the specific limitations in one or more of the following embodiments can be found in the aforementioned limitations on the biohash-based marine mammal audio retrieval method and will not be further elaborated here.

[0108] In one embodiment, a server is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor includes:

[0109] A biometric template construction module is used to extract sub-band CQT energy-entropy ratio features from the original audio, construct a key-address index table containing the mapping relationship between the original audio, the sub-band CQT energy-entropy ratio features, and the key based on the sub-band CQT energy-entropy ratio features, and construct a biometric hash sequence based on the key-address index table;

[0110] An audio segmentation module is used to segment the original audio using a dual-threshold segmentation method based on short-time energy and spectrum distribution variance, obtain short-time audio segments, map the short-time audio segments to a key-address index table, and reconstruct the biometric hash sequence;

[0111] The ciphertext audio library construction module is used to encrypt the original audio and obtain the ciphertext audio;

[0112] The end hash index table construction module is used to build a hash index table based on the mapping relationship between the reconstructed biological hash sequence, key and ciphertext audio, and upload the ciphertext audio and hash index table to the cloud for retrieval.

[0113] In one embodiment, a mobile terminal is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor includes:

[0114] A biometric template construction module is used to extract sub-band CQT energy-entropy ratio features from the query audio, classify the query audio based on the sub-band CQT energy-entropy ratio features to obtain a label for the query audio, query a key-address index table based on the label to obtain a key, and construct a biometric hash sequence corresponding to the query audio based on the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, sub-band CQT energy-entropy ratio features of different categories, and original audio of different categories;

[0115] An audio segmentation module is used to segment the query audio using a dual-threshold segmentation method based on short-time energy and spectral distribution variance, obtain short-time audio segments, map the short-time audio segments to a key-address index table, reconstruct the bio-hash sequence, and upload the bio-hash sequence to the cloud;

[0116] The audio retrieval module is used to obtain the ciphertext audio that successfully matches the similarity between the biological hash sequence and the hash index table in the cloud, decrypt the ciphertext audio, and obtain the corresponding result of the query audio; among them, the hash index table is constructed based on the mapping relationship between the biological hash sequence, key and ciphertext audio.

[0117] In one embodiment, a marine mammal audio retrieval system based on bio-hash ciphertext is provided, including a server side and a mobile side, which respectively execute all the steps in the above method embodiments.

[0118] Each module in the aforementioned server, mobile terminal, and biohash-based marine mammal audio retrieval system may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in hardware form, or may be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0119] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to these.

[0120] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A marine mammal audio retrieval method based on bio-hash ciphertext, applied to the server, characterized in that: The method comprises: Extracting sub-band CQT energy-entropy ratio features from the original audio, constructing a key-address index table containing a mapping relationship between the original audio, the sub-band CQT energy-entropy ratio features, and a key based on the sub-band CQT energy-entropy ratio features, and constructing a biometric hash sequence based on the key-address index table; wherein CQT is a constant-Q transform used to capture energy characteristics at different frequencies in the original audio; Segmenting the original audio using a dual-threshold segmentation method of short-time energy and spectrum distribution variance to obtain short-time audio segments, mapping the short-time audio segments to the key-address index table, and reconstructing the biometric hash sequence; Encrypting the original audio to obtain ciphertext audio; Constructing a hash index table based on the mapping relationship between the reconstructed biometric hash sequence, the key, and the ciphertext audio, and uploading the ciphertext audio and the hash index table to the cloud for retrieval; Wherein, reconstructing the bio-hash sequence includes: Acquire a short-time energy feature sequence and a spectral distribution variance feature sequence according to the original audio; Constructing corresponding histograms of the short-time energy feature sequence and the spectrum distribution variance feature sequence respectively, and extracting corresponding thresholds of the short-time energy feature sequence and the spectrum distribution variance feature sequence respectively from the histograms; Comparing the short-time energy characteristic sequence and the spectral distribution variance characteristic sequence with the corresponding thresholds respectively, and constructing a thresholding sequence according to the comparison results; Mapping the thresholded sequence to the original audio to obtain the short-time audio segment; According to the position of the short audio segment in the original audio, the short audio segment is mapped to the key-address index table to determine the position index of the biometric hash sequence to obtain a hash segment; The short audio segments corresponding to the hash segments are summarized to reconstruct the biometric hash sequence.

2. The method according to claim 1, characterized in that The sub-band CQT energy entropy ratio feature extracted from the original audio includes: Performing pre-emphasis processing, framing and windowing processing on the original audio in sequence to obtain a pre-processed signal; Performing a CQT change on the preprocessed signal and calculating a sub-band CQT energy, calculating a sub-band CQT entropy spectrum according to the sub-band CQT energy, and acquiring the sub-band CQT energy-entropy ratio feature according to the sub-band CQT energy and the sub-band CQT entropy spectrum.

3. The method according to claim 1, characterized in that The constructing of a key-address index table including a mapping relationship between the original audio, the sub-band CQT energy entropy ratio feature, and a key based on the sub-band CQT energy entropy ratio feature includes: Classifying the sub-band CQT energy entropy ratio features to obtain a number of feature labels; Construct a key corresponding to each feature tag, classify the original audio according to each feature tag, and construct the key-address index table based on a single mapping relationship between the key, the sub-band CQT energy-entropy ratio features of different categories, and the original audio of different categories.

4. The method according to claim 1, wherein The extracting the corresponding thresholds of the short-time energy feature sequence and the spectrum distribution variance feature sequence from the histogram respectively includes: Several local maxima are found from the histogram respectively, several target maxima are selected from the local maxima according to their sizes, a control parameter is introduced to balance the several target maxima, and the threshold is obtained.

5. The method according to claim 1, wherein The encrypting the original audio to obtain the ciphertext audio includes: encrypting the original audio using an improved AES encryption algorithm; The improved AES encryption algorithm includes: Generate a target sequence through a Logistic system, and obtain a set of initial vectors and several groups of group keys by grouping the target sequence; The plaintext sequence is grouped to obtain a plurality of grouped plaintexts, the group keys of each group are processed in ascending order to obtain a position index, the grouped plaintexts are scrambled and encrypted according to the position index to obtain a grouped ciphertext sequence; After performing an inner product process on the first group of group ciphertext sequences and the initial vector, the sequences are encrypted with the first group of group keys using AES to obtain a first group of group ciphertexts; after performing an inner product process on each group of group ciphertext sequences and the previous group of group ciphertexts, the sequences are encrypted with the corresponding group keys using AES to obtain the corresponding group ciphertexts; The grouped ciphertexts of each group are concatenated to obtain ciphertext.

6. A method for marine mammal audio retrieval based on bio-hash ciphertext, applied to a mobile terminal, comprising: Extract sub-band CQT energy-entropy ratio features from the query audio, classify the query audio based on the sub-band CQT energy-entropy ratio features to obtain a label for the query audio, query a key-address index table based on the label to obtain a key, and construct a biometric hash sequence corresponding to the query audio based on the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, the sub-band CQT energy-entropy ratio features of different categories, and the original audio of different categories; wherein CQT is a constant-Q transform, which is used to capture the energy characteristics at different frequencies in the query audio; Segment the query audio using a dual-threshold segmentation method of short-time energy and spectrum distribution variance to obtain short-time audio segments, map the short-time audio segments to the key-address index table, reconstruct the biometric hash sequence, and upload the biometric hash sequence to the cloud; Obtaining the ciphertext audio that successfully matches the similarity between the biometric hash sequence and the hash index table in the cloud, decrypting the ciphertext audio, and obtaining the corresponding result of the query audio; wherein the hash index table is constructed based on the mapping relationship between the biometric hash sequence, the key, and the ciphertext audio; Wherein, reconstructing the bio-hash sequence includes: Acquire a short-time energy feature sequence and a spectral distribution variance feature sequence according to the query audio; Constructing corresponding histograms of the short-time energy feature sequence and the spectrum distribution variance feature sequence respectively, and extracting corresponding thresholds of the short-time energy feature sequence and the spectrum distribution variance feature sequence respectively from the histograms; Comparing the short-time energy characteristic sequence and the spectral distribution variance characteristic sequence with the corresponding thresholds respectively, and constructing a thresholding sequence according to the comparison results; Mapping the thresholded sequence to the query audio to obtain the short audio segment; According to the position of the short audio segment in the query audio, the short audio segment is mapped to the key-address index table to determine the position index of the biometric hash sequence to obtain a hash segment; The short audio segments corresponding to the hash segments are summarized to reconstruct the biometric hash sequence.

7. A server comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A mobile terminal comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to claim 6 are implemented.

9. A marine mammal audio retrieval system based on bio-hash ciphertext, comprising a server and a mobile terminal, wherein the server performs the steps of the method according to any one of claims 1 to 5, and the mobile terminal performs the steps of the method according to claim 6.

Images