Marine mammal audio retrieval method and system based on biological hash ciphertext

Abstract

The invention discloses a marine mammal audio retrieval method and system based on a biological hash ciphertext, and relates to the field of audio processing. The method comprises the following steps: extracting sub-band CQT energy-entropy ratio characteristics from an original audio, constructing a key-address index table, and constructing a biological hash sequence according to the key-address index table; the method comprises the following steps: segmenting an original audio by adopting a short-time energy and spectrum distribution variance dual-threshold segmentation method to obtain short-time audio segments, mapping the short-time audio segments to a key-address index table, and reconstructing a biological hash sequence; carrying out improved AES encryption on the original audio to obtain a ciphertext audio; and constructing a hash index table according to the mapping relationship among the reconstructed biological hash sequence, the key and the ciphertext audio, and uploading the ciphertext audio and the hash index table to a cloud for retrieval. According to the method, the biological hash sequence is reconstructed by adopting the audio segmentation algorithm, so that the retrieval efficiency is improved, and meanwhile, the data security is improved by adopting the improved AES encryption algorithm.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a marine mammal audio retrieval method and system based on bio-hash ciphertext. Background Art

[0002] With the increasing number of human activities in the ocean and their in-depth involvement, the marine ecological environment has been affected and changed in a way that cannot be ignored. Therefore, the monitoring and research of marine mammal populations is becoming increasingly important. Using sound signals as a medium is the most effective means to identify the daily biological activities of marine mammals in the ocean. Due to the complexity of the marine environment, it is difficult to accurately and efficiently retrieve information about marine mammal audio from massive multimedia data.

[0003] The audio hash retrieval algorithm converts the biometric features of the audio into a hash code and stores it with the corresponding audio in a hash table, and retrieves the audio from the cloud. This algorithm has become one of the research topics in the field of audio retrieval.

[0004] However, since the cloud is an untrusted third party, the audio data and biometrics need to be protected during the hash retrieval process. Therefore, how to achieve efficient and secure audio retrieval is a key issue that needs to be solved urgently. Summary of the invention

[0005] Based on this, it is necessary to provide a marine mammal audio retrieval method and system based on bio-hash ciphertext that can improve the security of cloud-based audio retrieval in response to the above technical problems.

[0006] In a first aspect, the present application provides a marine mammal audio retrieval method based on bio-hash ciphertext, which is applied to a server. The method comprises:

[0007] Extract sub-band CQT entropy ratio features from the original audio, construct a key-address index table containing the mapping relationship between the original audio, the sub-band CQT entropy ratio features and the key based on the sub-band CQT entropy ratio features, and construct a bio-hash sequence according to the key-address index table;

[0008] The original audio is segmented using a dual threshold segmentation method of short-time energy and spectrum distribution variance to obtain short-time audio segments, which are then mapped to a key-address index table to reconstruct a bio-hash sequence.

[0009] Encrypt the original audio to obtain the ciphertext audio;

[0010] A hash index table is constructed based on the mapping relationship between the reconstructed biological hash sequence, key and ciphertext audio, and the ciphertext audio and hash index table are uploaded to the cloud for retrieval.

[0011] In one embodiment, extracting the sub-band CQT energy entropy ratio feature from the original audio includes:

[0012] The original audio is sequentially subjected to pre-emphasis processing, frame division and windowing processing to obtain a pre-processed signal;

[0013] The preprocessed signal is subjected to CQT change and the sub-band CQT energy is calculated. The sub-band CQT entropy spectrum is calculated according to the sub-band CQT energy. The sub-band CQT energy-entropy ratio feature is obtained according to the sub-band CQT energy and the sub-band CQT entropy spectrum.

[0014] In one embodiment, constructing a key-address index table including original audio, sub-band CQT energy entropy ratio feature and mapping relationship between keys based on the sub-band CQT energy entropy ratio feature includes:

[0015] Classify the sub-band CQT energy entropy ratio features to obtain several feature labels;

[0016] A key corresponding to each feature label is constructed, and the original audio is classified according to each feature label. A key-address index table is constructed based on a single mapping relationship between the key, sub-band CQT energy-entropy ratio features of different categories and original audio of different categories.

[0017] In one embodiment, a dual threshold segmentation method of short-time energy and spectrum distribution variance is used to segment the original audio, obtain short-time audio segments, map the short-time audio segments to a key-address index table, and reconstruct the biometric hash sequence, including:

[0018] Obtain short-time energy feature sequence and spectrum distribution variance feature sequence according to the original audio;

[0019] Constructing the corresponding histograms of the short-time energy feature sequence and the spectral distribution variance feature sequence respectively, and extracting the corresponding thresholds of the short-time energy feature sequence and the spectral distribution variance feature sequence respectively from the histograms;

[0020] The short-time energy characteristic sequence and the spectrum distribution variance characteristic sequence are respectively compared with the corresponding thresholds, and the thresholded sequence is constructed according to the comparison results;

[0021] Map the thresholded sequence to the original audio to obtain a short-term audio segment;

[0022] According to the position of the short-time audio segment in the original audio, the short-time audio segment is mapped to the key-address index table to determine the position index of the bio-hash sequence, and the hash segment is obtained;

[0023] Summarize the hash fragments corresponding to each short-term audio clip and reconstruct the biological hash sequence.

[0024] In one embodiment, extracting the short-time energy feature sequence and the spectral distribution variance feature sequence corresponding thresholds from the histogram includes:

[0025] Find several local maxima from the histogram respectively, select several target maxima from each local maximum according to their size, introduce control parameters to balance several target maxima, and obtain the threshold.

[0026] In one embodiment, encrypting the original audio to obtain the ciphertext audio includes: encrypting the original audio using an improved AES encryption algorithm;

[0027] Among them, the improved AES encryption algorithm includes:

[0028] Generate a target sequence through the Logistic system, and obtain a set of initial vectors and several groups of grouping keys by grouping the target sequence;

[0029] The plaintext sequence is grouped to obtain several groups of grouped plaintexts, each group of group keys is processed in ascending order to obtain a position index, and the grouped plaintexts are scrambled and encrypted according to the position index to obtain a grouped ciphertext sequence;

[0030] After the first group of group ciphertext sequences are processed with the initial vector by inner product, they are encrypted with the first group key by AES to obtain the first group ciphertext; after each group ciphertext sequence is processed with the previous group ciphertext by inner product, they are encrypted with the corresponding group key by AES to obtain the corresponding group ciphertext;

[0031] The ciphertext of each group is concatenated to obtain the ciphertext.

[0032] In a second aspect, the present application provides a marine mammal audio retrieval method based on bio-hash ciphertext, which is applied to a mobile terminal, and the method includes:

[0033] Extract sub-band CQT entropy ratio features from the query audio, classify based on the sub-band CQT entropy ratio features to obtain labels of the query audio, query the key-address index table according to the label to obtain the key, and construct the bio-hash sequence corresponding to the query audio according to the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, sub-band CQT entropy ratio features of different categories, and original audio of different categories;

[0034] A dual threshold segmentation method of short-time energy and spectrum distribution variance is used to segment the query audio, obtain short-time audio segments, map the short-time audio segments to a key-address index table, reconstruct the bio-hash sequence, and upload the bio-hash sequence to the cloud;

[0035] Obtain the ciphertext audio that successfully matches the similarity between the biological hash sequence and the hash index table in the cloud, decrypt the ciphertext audio, and obtain the corresponding result of the query audio; wherein the hash index table is constructed based on the mapping relationship between the biological hash sequence, the key and the ciphertext audio.

[0036] In a third aspect, the present application provides a server, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned marine mammal audio retrieval method based on biological hash ciphertext applied to the server are performed.

[0037] In a fourth aspect, the present application provides a mobile terminal, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned marine mammal audio retrieval method based on bio-hash ciphertext applied to the mobile terminal are implemented.

[0038] In a fifth aspect, the present application provides a marine mammal audio retrieval system based on bio-hash ciphertext, including the above-mentioned server and mobile terminal.

[0039] The above-mentioned marine mammal audio retrieval method and system based on bio-hash ciphertext extracts sub-band CQT entropy ratio features from the original audio, constructs a key-address index table containing the original audio, sub-band CQT entropy ratio features and the mapping relationship between keys based on the sub-band CQT entropy ratio features, and constructs a bio-hash sequence according to the key-address index table; the original audio is segmented using a dual threshold segmentation method of short-time energy and spectrum distribution variance to obtain short-time audio segments, and the short-time audio segments are mapped to the key-address index table to reconstruct the bio-hash sequence; the original audio is encrypted to obtain ciphertext audio; a hash index table is constructed based on the mapping relationship between the reconstructed bio-hash sequence, key and ciphertext audio, and the ciphertext audio and hash index table are uploaded to the cloud for retrieval. The sub-band CQT entropy ratio feature in this application combines the comprehensive characteristics of non-uniform frequency resolution and energy information, so that it has good resolution in both time and frequency, is suitable for analyzing complex audio signals, and exhibits strong robustness in noisy environments. In addition, the use of audio segmentation algorithm to reconstruct the biological hash sequence can not only effectively reduce redundant data, thereby improving retrieval efficiency, but also to a certain extent improve the abstract level of the original audio, thereby improving data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a process for extracting sub-band CQT entropy ratio features in an embodiment;

[0041] Figure 2 A diagram showing a key-address index table model in one embodiment;

[0042] Figure 3 A schematic diagram of a process for constructing a biohash sequence in one embodiment;

[0043] Figure 4 An improved AES encryption flow chart in one embodiment;

[0044] Figure 5 A hash index table model diagram in one embodiment;

[0045] Figure 6 Schematic diagram of the process of a marine mammal audio retrieval method based on bio-hash ciphertext in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] The present application embodiment provides a marine mammal audio retrieval method based on bio-hash ciphertext, such as Figure 1 As shown, the following steps are included:

[0048] S101. Extract sub-band CQT energy entropy ratio features from the original audio, construct a key-address index table containing the mapping relationship between the original audio, sub-band CQT energy entropy ratio features and keys based on the sub-band CQT energy entropy ratio features, and construct a bio-hash sequence according to the key-address index table.

[0049] Firstly, the sub-band CQT energy-entropy ratio features are extracted from the original audio. Then, the features are classified by SVM and a key-address index table is constructed according to the classification results. Finally, a feature security template with a single mapping key is established according to the key-address index table and a bio-hash sequence is quantified and constructed.

[0050] S102. Use a dual threshold segmentation method of short-time energy and spectrum distribution variance to segment the original audio, obtain short-time audio segments, map the short-time audio segments to a key-address index table, and reconstruct the bio-hash sequence.

[0051] The original audio is segmented using a double-threshold audio segmentation method based on short-time energy and spectral distribution variance to obtain non-redundant short-time audio clips, and the bio-hash sequence is reconstructed.

[0052] S103: Encrypt the original audio to obtain the ciphertext audio.

[0053] Use symmetric encryption algorithms (such as AES, DES, etc.), asymmetric encryption algorithms (RSA, ECC, etc.), hybrid encryption algorithms, etc. to encrypt the original audio, obtain the ciphertext audio, and upload it to the cloud to build a ciphertext audio library.

[0054] S104. Construct a hash index table according to the mapping relationship between the reconstructed biological hash sequence, the key and the ciphertext audio, and upload the ciphertext audio and the hash index table to the cloud for retrieval.

[0055] The hash index table is also uploaded to the cloud, and the retrieval service is realized through the ciphertext audio library and hash index table in the cloud. The user uploads the audio to be retrieved to the cloud after processing, and matches and retrieves it with the biological hash sequence in the hash index table. The successfully matched biological hash sequence can obtain the corresponding ciphertext audio by querying the hash index table, and the ciphertext can be decrypted to obtain the retrieval result of the audio to be retrieved.

[0056] In one embodiment, extracting sub-band CQT energy-entropy ratio features from original audio in S101 includes: performing pre-emphasis processing, framing and windowing processing on the original audio in sequence to obtain a preprocessed signal; performing CQT transformation on the preprocessed signal and calculating sub-band CQT energy, calculating the sub-band CQT entropy spectrum based on the sub-band CQT energy, and obtaining sub-band CQT energy-entropy ratio features based on the sub-band CQT energy and the sub-band CQT entropy spectrum.

[0057] Among them, CQT (Constant-Q Transform) is a transformation technology that can be used to capture energy characteristics at different frequencies in audio signals.

[0058] Specifically, Figure 1 As shown, the collected original ocean audio is pre-emphasized to increase the high frequency part of the signal, and then the pre-processed signal S = {S m (i)|i=1,2,…,N;m=1,2,…,L}, where the window function is a Hamming window, L represents the frame length, and N represents the number of frames.

[0059] The preprocessed signal x(j) undergoes CQT change according to formula (1) and the sub-band CQT energy E is calculated according to formula (2): b , and then calculate the sub-band CQT entropy spectrum G according to formulas (3) and (4): b Finally, formula (6) is used to calculate the sub-band CQT entropy ratio Γ = {Γ(i)|i = 1, 2, …, N}.

[0060]

[0061] Where k = 1, 2, ..., K represents the frequency domain bins with constant Q variation, The value represented by f is rounded down. k is the center frequency of the kth bin, f s is the sampling frequency, ω(t) is the continuous window function, and the window With f k Inversely proportional real numbers.

[0062] In one embodiment, constructing a key-address index table containing the mapping relationship between the original audio, the sub-band CQT energy entropy ratio feature and the key based on the sub-band CQT energy entropy ratio feature in S101 includes: classifying the sub-band CQT energy entropy ratio feature to obtain a number of feature labels; constructing a key corresponding to each feature label, and classifying the original audio according to each feature label, and constructing a key-address index table based on a single mapping relationship between the key, different categories of sub-band CQT energy entropy ratio features and different categories of original audio.

[0063] First, the biometric database is annotated using the K-modes clustering algorithm and classified using the SVM (Support Vector Machine) classifier. Then, the labels, classification feature datasets, logical addresses of the audio, and keys are used to establish a key-address index table. Finally, a feature security template with a single mapping key is established based on the parameters of the key-address index table and quantized to construct a biometric hash sequence.

[0064] Specifically, Figure 3 As shown in the figure, the audio biometric dataset is used as the training set of the K-modes clustering algorithm to obtain the label λ of the dataset, and then the SVM classifier is used to train the classification model on the labeled feature set and the dataset is assigned to M categories, that is, Γ λ ={Γ λ (i)|i=1,2,…,N; λ=1,2,…,M}, where Γ λ Represents the feature dataset that the biometric feature belongs to the λth category.

[0065] Construct the corresponding key according to different feature labels λ λ , and then classify the corresponding original audio according to the feature label, and finally use the key key λ and different category feature datasets Γ λ , and different categories of original audio S λ The relationship of a single mapping is constructed as Figure 2 The key-address index table shown has a mapping relationship as shown in formula (6):

[0066]

[0067] Then build a feature security template. λ Exciting the Henon chaotic system to generate a pseudo-random matrix Φ of M dimension and length N λ ={Φ λ (i)|i=1,2,…,N;λ=1,2,…,M}, then Schmidt orthogonalize the pseudo-random matrix to construct the orthogonal matrix Θ λ ={Θ λ(i)|i=1,2,…,N;λ=1,2,…,M}. Finally, the M-dimensional vector Θ of the orthogonal matrix λ Perform inner product with the one-to-one corresponding M-type feature data set to construct the security template V λ = {V λ (i)|i=1,2,…,N; λ=1,2,…,M}.

[0068] According to formula (7), the security template Ψ λ Quantify and obtain the bio-hash sequence H λ ={H λ (i)|i=1,2,…,N}, where the biohash H(1) is set to 0.

[0069]

[0070] In one embodiment, S102 includes: obtaining a short-time energy feature sequence and a spectral distribution variance feature sequence based on the original audio; constructing corresponding histograms of the short-time energy feature sequence and the spectral distribution variance feature sequence, respectively, and extracting the corresponding thresholds of the short-time energy feature sequence and the spectral distribution variance feature sequence from the histograms; comparing the short-time energy feature sequence and the spectral distribution variance feature sequence with the corresponding thresholds, respectively, and constructing a thresholded sequence based on the comparison results; mapping the thresholded sequence to the original audio to obtain a short-time audio segment; according to the position of the short-time audio segment in the original audio, mapping the short-time audio segment to the key-address index table to determine the position index of the biological hash sequence, and obtaining a hash segment; summarizing the hash segments corresponding to each short-time audio segment, and reconstructing the biological hash sequence.

[0071] The original audio x(j) is segmented using the double threshold audio segmentation method of short-time energy and spectrum distribution variance to obtain a non-redundant short-time audio segment x k (n), where k represents the number of short audio segments.

[0072] Specifically, the input signal x(j) is framed and windowed to obtain the preprocessed signal x i (n), where i and n represent the number of audio frames and the frame length respectively.

[0073] According to formulas (8) and (9), the short-time energy characteristic sequence E(i) and the characteristic sequence C(i) of the spectral distribution variance of the preprocessed signal of each frame are calculated respectively, and the formulas are as follows:

[0074]

[0075] Where DFT stands for discrete Fourier transform, f k represents the center frequency, and μ1 represents the spectral centroid.

[0076] Next, the short-time energy and spectrum distribution variance feature sequences of the original signal are used to construct a histogram and smooth it. Then, the first and second local maximum positions of the histogram corresponding to the short-time energy and spectrum spread are found. Finally, the thresholds of the short-time energy and spectrum distribution variance are calculated according to formula (10).

[0077]

[0078] Wherein δ is the threshold, ω1 and ω2 represent the first and second largest local maxima, i.e., the target maximum value, and weight=5 is the control parameter, which can be determined according to the actual situation.

[0079] Then, the short-time energy and spectral distribution variance are compared with their respective thresholds δ E and δ C The comparison is performed and the results of the comparison are ANDed to construct a threshold sequence of the speech segment.

[0080] Map the thresholded sequence into the original signal to obtain each non-silent short-term speech segment x k (n) Obtain short audio segments at the start and end positions in the original audio.

[0081] Match the non-redundant short-time audio segment with the position index of the original audio, and then map the index to the position index of the corresponding biological hash sequence to obtain the hash segment of the non-redundant short-time audio segment Finally, the hash fragments are reconstructed into a biological hash sequence one by one

[0082]

[0083] Where ξ is the hash segment length of the non-redundant short audio segment, ω represents the number of short audio segments, and λ is the category to which the reconstructed hash belongs.

[0084] In one embodiment, S103 includes: encrypting the original audio by using an improved AES encryption algorithm. Using the improved AES encryption step to construct a ciphertext voice library, and then constructing a classified ciphertext audio library C according to the audio label λ λ And upload to the cloud.

[0085] The improved AES encryption algorithm includes: generating a target sequence through a Logistic system, obtaining a set of initial vectors and several groups of group keys by grouping the target sequence; obtaining several groups of group plaintexts by grouping the plaintext sequence, performing ascending processing on each group key to obtain a position index, scrambling and encrypting the group plaintext through the position index to obtain a group ciphertext sequence; performing inner product processing on the first group ciphertext sequence and the initial vector, and then performing AES encryption with the first group key to obtain the first group ciphertext; performing inner product processing on each group ciphertext sequence and the previous group ciphertext, and then performing AES encryption with the corresponding group key to obtain the corresponding group ciphertext; and concatenating the groups of group ciphertexts to obtain the ciphertext.

[0086] Specifically, Figure 4 As shown, the specific process of improved AES encryption is as follows:

[0087] S301, generate a length of Sequence and divide it into The data of the first group is used as the initial vector ρ0, and the vectors of the remaining groups are used as the group keys {ρ n |n=1,2,…,l}, where ρ n The length of is θ bits.

[0088] S302: Group the plaintext sequence to obtain l groups of grouped plaintext The group keys of each group are processed in ascending order to obtain the position index, and then each group of plaintext is scrambled and encrypted by the position index to obtain the group ciphertext sequence {ξ n |n=1,2,…,l}.

[0089] S303, the first group of group ciphertext sequence ξ n The inner product is processed with the initial vector θ0, and then the group key ρ0 is used for AES encryption to obtain the first group of group ciphertext C1. Group ciphertext ξ n Perform inner product processing with the group ciphertext of the previous group, and then execute the group key ρ n Perform AES encryption to obtain block ciphertext C n , the formula is as follows:

[0090]

[0091] S304, concatenate the grouped ciphertext data to obtain the entire plaintext data C = {C1, C2, ..., C n}.

[0092] In S104, different types of ciphertext audio C λ and rebuild hash index and key λ According to formula (13), a one-to-one hash index table is established. The biological hash index table model is as follows: Figure 5 shown.

[0093]

[0094] The server extracts audio features from the original audio, then uses SVM to learn the features to construct a diverse biometric template and quantify the biometric hash sequence. Finally, the audio segmentation algorithm is used to reconstruct the hash sequence to obtain a hash index. At the same time, the server performs improved AES encryption on the original audio data to construct a ciphertext audio library in the cloud. Finally, the ciphertext audio library and hash index are uploaded to the cloud to construct the biometric hash index table.

[0095] The improved AES encryption algorithm adopted in this embodiment can effectively ensure the security of audio data in the cloud and enhance the anti-attack capability of marine mammal audio.

[0096] In one embodiment, a method for retrieving marine mammal audio based on bio-hash ciphertext is applied to a mobile terminal, the method comprising: extracting sub-band CQT energy-entropy ratio features of the query audio, classifying the sub-band CQT energy-entropy ratio features to obtain a label of the query audio, querying a key-address index table according to the label to obtain a key, and constructing a bio-hash sequence corresponding to the query audio according to the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, sub-band CQT energy-entropy ratio features of different categories, and original audio of different categories; segmenting the query audio using a dual threshold segmentation method of short-time energy and spectrum distribution variance to obtain short-time audio segments, mapping the short-time audio segments to the key-address index table, reconstructing the bio-hash sequence, and uploading the bio-hash sequence to the cloud; obtaining ciphertext audio that successfully matches the similarity between the bio-hash sequence and the hash index table in the cloud, decrypting the ciphertext audio, and obtaining the corresponding result of the query audio; wherein the hash index table is constructed based on the mapping relationship between the bio-hash sequence, the key, and the ciphertext audio.

[0097] Specifically, the method comprises the following steps:

[0098] S501. Extract sub-band CQT energy entropy ratio features S′(i) from the query audio using the same method as S101.

[0099] S502. The sub-band CQT energy-entropy ratio feature of the query audio is used as a test set and a pre-trained SVM classification model is used to obtain the label of the query audio. Then, the key-address index table is queried based on the label to obtain the key key. Finally, the corresponding feature security template is stimulated based on the key and quantized to construct the bio-hash sequence H′(i).

[0100] S503: Segment the query audio x′(n) into non-redundant short-duration audio segments x′ using an audio segmentation method. k (n), and then the bio-hash sequence H′(i) is hashed and reconstructed to obtain the bio-hash sequence And upload the reconstructed bio-hash sequence to the cloud.

[0101] S504, Bio-hash sequence on the cloud mobile terminal Calculate the similarity distance Similary(:,:) with the hash index table of the corresponding k-th category in the cloud, and take the biological hash sequence with the similarity distance value closest to 1 as the matching result. The formula is as follows:

[0102]

[0103] in is the edit distance, and They represent the biological hash sequence of the query audio and the hash index table of the kth category respectively. The value range of the similarity distance is [0,1]. The closer the similarity distance is to 1, the higher the sequence similarity between the two.

[0104] S505: Return the ciphertext audio corresponding to the cloud hash sequence to the mobile terminal and decrypt the ciphertext audio, wherein decryption and encryption are inverse processes, and the decrypted voice segment is the query voice.

[0105] like Figure 6 The figure shows a complete flow chart of the marine mammal audio retrieval method implemented on the server and mobile terminals.

[0106] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0107] Based on the same inventive concept, the embodiments of the present application also provide a server, a mobile terminal, and a marine mammal audio retrieval system based on bio-hash ciphertext for implementing the above-mentioned marine mammal audio retrieval method based on bio-hash ciphertext. The implementation solutions provided by the server, the mobile terminal, and the system for solving the problem are similar to the implementation solutions recorded in the above-mentioned method, so the specific limitations in one or more embodiments provided below can refer to the limitations of the marine mammal audio retrieval method based on bio-hash ciphertext above, and will not be repeated here.

[0108] In one embodiment, a server is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor includes:

[0109] A biometric template construction module is used to extract sub-band CQT energy entropy ratio features from the original audio, construct a key-address index table containing the mapping relationship between the original audio, the sub-band CQT energy entropy ratio features and the key based on the sub-band CQT energy entropy ratio features, and construct a biometric hash sequence according to the key-address index table;

[0110] An audio segmentation module is used to segment the original audio using a dual threshold segmentation method of short-time energy and spectrum distribution variance, obtain short-time audio segments, map the short-time audio segments to a key-address index table, and reconstruct a bio-hash sequence;

[0111] The ciphertext audio library building module is used to encrypt the original audio and obtain the ciphertext audio;

[0112] The end hash index table construction module is used to build a hash index table according to the mapping relationship between the reconstructed biological hash sequence, key and ciphertext audio, and upload the ciphertext audio and hash index table to the cloud for retrieval.

[0113] In one embodiment, a mobile terminal is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor includes:

[0114] A biometric template construction module is used to extract sub-band CQT energy entropy ratio features from the query audio, classify the query audio based on the sub-band CQT energy entropy ratio features to obtain a label of the query audio, query the key-address index table according to the label to obtain the key, and construct a biometric hash sequence corresponding to the query audio according to the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, sub-band CQT energy entropy ratio features of different categories, and original audio of different categories;

[0115] An audio segmentation module is used to segment the query audio using a dual threshold segmentation method of short-time energy and spectrum distribution variance, obtain short-time audio segments, map the short-time audio segments to a key-address index table, reconstruct the bio-hash sequence, and upload the bio-hash sequence to the cloud;

[0116] The audio retrieval module is used to obtain the ciphertext audio that successfully matches the similarity between the biological hash sequence and the hash index table in the cloud, decrypt the ciphertext audio, and obtain the corresponding result of the query audio; among which, the hash index table is constructed based on the mapping relationship between the biological hash sequence, key and ciphertext audio.

[0117] In one embodiment, a marine mammal audio retrieval system based on bio-hash ciphertext is provided, including a server side and a mobile side, which respectively execute all the steps in the above method embodiments.

[0118] Each module in the above-mentioned server, mobile terminal and marine mammal audio retrieval system based on bio-hash ciphertext can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processors involved in each embodiment provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., but are not limited thereto.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A marine mammal audio retrieval method based on bio-hash ciphertext, applied to the server, characterized in that: The method comprises: Extracting a sub-band CQT energy entropy ratio feature from the original audio, constructing a key-address index table including a mapping relationship between the original audio, the sub-band CQT energy entropy ratio feature and a key based on the sub-band CQT energy entropy ratio feature, and constructing a bio-hash sequence according to the key-address index table; The original audio is segmented using a dual threshold segmentation method of short-time energy and spectrum distribution variance to obtain short-time audio segments, the short-time audio segments are mapped to the key-address index table, and the bio-hash sequence is reconstructed; Encrypting the original audio to obtain ciphertext audio; A hash index table is constructed according to the mapping relationship between the reconstructed biological hash sequence, the key and the ciphertext audio, and the ciphertext audio and the hash index table are uploaded to the cloud for retrieval.

2. The method according to claim 1, characterized in that The sub-band CQT energy entropy ratio feature extracted from the original audio includes: Performing pre-emphasis processing, framing and windowing processing on the original audio in sequence to obtain a pre-processed signal; Perform CQT change on the preprocessed signal and calculate sub-band CQT energy, calculate sub-band CQT entropy spectrum according to the sub-band CQT energy, and obtain the sub-band CQT energy-entropy ratio feature according to the sub-band CQT energy and the sub-band CQT entropy spectrum.

3. The method according to claim 1, characterized in that: The constructing of a key-address index table including the original audio, the sub-band CQT energy entropy ratio feature and a mapping relationship between keys based on the sub-band CQT energy entropy ratio feature comprises: Classifying the sub-band CQT energy entropy ratio features to obtain a number of feature labels; Construct a key corresponding to each feature tag, classify the original audio according to each feature tag, and construct the key-address index table based on a single mapping relationship between the key, the sub-band CQT energy-entropy ratio features of different categories and the original audio of different categories.

4. The method according to claim 1, characterized in that: The dual threshold segmentation method of short-time energy and spectrum distribution variance is used to segment the original audio, obtain short-time audio segments, map the short-time audio segments to the key-address index table, and reconstruct the biometric hash sequence, including: Acquire a short-time energy feature sequence and a spectral distribution variance feature sequence according to the original audio; Constructing corresponding histograms of the short-time energy feature sequence and the spectral distribution variance feature sequence respectively, and extracting corresponding thresholds of the short-time energy feature sequence and the spectral distribution variance feature sequence respectively from the histograms; Comparing the short-time energy feature sequence and the spectral distribution variance feature sequence with the corresponding thresholds respectively, and constructing a thresholding sequence according to the comparison results; Mapping the thresholded sequence to the original audio to obtain the short-time audio segment; According to the position of the short-time audio segment in the original audio, the short-time audio segment is mapped to the key-address index table to determine the position index of the biometric hash sequence to obtain a hash segment; The short-term audio segments corresponding to the hash segments are summarized to reconstruct the biological hash sequence.

5. The method according to claim 4, characterized in that The extracting the short-time energy feature sequence and the spectral distribution variance feature sequence corresponding thresholds from the histogram respectively comprises: Several local maxima are found respectively from the histogram, several target maxima are selected from the local maxima according to their sizes, a control parameter is introduced to balance the several target maxima, and the threshold is obtained.

6. The method according to claim 1, characterized in that The encrypting the original audio to obtain the ciphertext audio comprises: encrypting the original audio by using an improved AES encryption algorithm; Wherein, the improved AES encryption algorithm includes: Generate a target sequence through a Logistic system, and obtain a set of initial vectors and several sets of grouping keys by grouping the target sequence; The plaintext sequence is grouped to obtain a plurality of groups of grouped plaintexts, each group of group keys is processed in ascending order to obtain a position index, and the grouped plaintexts are scrambled and encrypted according to the position index to obtain a grouped ciphertext sequence; After performing inner product processing on the first group of group ciphertext sequences and the initial vector, AES encryption is performed with the first group of group keys to obtain the first group of group ciphertexts; after performing inner product processing on each group of group ciphertext sequences and the previous group of group ciphertexts, AES encryption is performed with the corresponding group key to obtain the corresponding group ciphertexts; The grouped ciphertexts of each group are concatenated to obtain the ciphertext.

7. A marine mammal audio retrieval method based on bio-hash ciphertext, applied to a mobile terminal, the method comprising: Extract sub-band CQT energy entropy ratio features from the query audio, classify based on the sub-band CQT energy entropy ratio features to obtain a label of the query audio, query the key-address index table according to the label to obtain a key, and construct a bio-hash sequence corresponding to the query audio according to the key; wherein the key-address index table is constructed based on a single mapping relationship between the key, the sub-band CQT energy entropy ratio features of different categories, and the original audio of different categories; The query audio is segmented using a dual threshold segmentation method of short-time energy and spectrum distribution variance to obtain short-time audio segments, the short-time audio segments are mapped to the key-address index table, the bio-hash sequence is reconstructed, and the bio-hash sequence is uploaded to the cloud; Obtain the ciphertext audio that successfully matches the similarity between the biological hash sequence and the hash index table in the cloud, decrypt the ciphertext audio, and obtain the corresponding result of the query audio; wherein the hash index table is constructed based on the mapping relationship between the biological hash sequence, the key and the ciphertext audio.

8. A server, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A mobile terminal, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to claim 7 are implemented.

10. A marine mammal audio retrieval system based on bio-hash ciphertext, comprising a server and a mobile terminal, wherein the server executes the steps of the method described in any one of claims 1 to 6, and the mobile terminal executes the steps of the method described in claim 7.

Images