Method of authenticating an audio signal

Abstract

Therefore, a method of authenticating an audio signal is provided. The method comprising the steps of analysing received audio signals to determine audio blocks and to extract embedded authentication access information from the received audio blocks, analysing the extracted authentication access information to retrieve authentication information containing audio features of an original audio signal, analysing the audio blocks of the received audio signal to determine whether irregular audio blocks are present for which no authentication access information is available, comparing the irregular audio blocks with a corresponding set of audio features retrievable from a memory and notifying a user which parts of the irregular audio blocks are successfully authenticated. Thus, the method also allows an authentication of irregular audio blocks.

Description

[0001] Method of authenticating an audio signal

[0002] The present invention relates to a method of authenticating an audio signal.

[0003] In view of modern capabilities of creating deepfake video and deepfake audio, the need has risen to be able to verify or authenticate an audio signal like a speech of a politician etc.

[0004] Until now it has been very difficult to authenticate an audio signal of a person. Typically, a lot of manual research must be done in order to verify or authenticate such an audio signal and to verify the source of the audio signal.

[0005] It is therefore an object of the invention to provide a method of authenticating an audio signal which enable an easy and robust way to authenticate a variety of different audio signals.

[0006] This object is solved by a method of authenticating an audio signal according to claim 1 .

[0007] Therefore, a method of authenticating an audio signal is provided. The method comprising the steps of analysing received audio signals to determine audio blocks and to extract embedded authentication access information from the received audio blocks, analysing the extracted authentication access information to retrieve authentication information containing audio features of an original audio signal, analysing the audio blocks of the received audio signal to determine whether irregular audio blocks are present for which no authentication access information is available, comparing the irregular audio blocks with a corresponding set of audio features retrievable from a memory and notifying a user which parts of the irregular audio blocks are successfully authenticated. Thus, the method also allows an authentication of irregular audio blocks.

[0008] According to an aspect, if the received audio signal is composed of parts of at least two audio signals and if the irregular audio block contains a change from a first audio signal to a second audio signal, performing the steps: determining a last complete audio block from the first signal, retrieving the authentication information corresponding to an audio block which is subsequent to the last complete audio block from the memory, and section-wise or pairwise comparing one after the other, going forward in time, the audio features contained within the retrieved authentication information to the received irregular audio block until the change from the first audio signal to the second audio signal is detected.

[0009] According to an aspect, if the received audio signal is composed of parts of at least two audio signals and if the irregular audio block contains a change from a first audio signal to a second audio signal performing the steps: determining a first complete audio block from the second signal, retrieving the authentication information corresponding to an audio block which is preceding the first complete audio block from the memory, and section-wise or pairwise comparing one after the other, going backwards in time, the audio features contained within the retrieved authentication information to the received irregular audio block until the change from the second audio signal to the first audio signal is detected.

[0010] According to an aspect, the comparison of the irregular audio block with the audio features within the authentication information is accomplished by selecting a suitable type of audio features forthe comparison, extracting features of the selected type from the irregular audio signal block, if the type of the audio features within the authentication information does not correspond to the selected type, transforming these audio features into audio features of the selected type, and performing the comparison based on audio features of the selected type.

[0011] According to an aspect, a method of authenticating an audio signal is provided. Received audio signals are analysed to determine audio blocks and to extract information for authenticating an audio signal, like embedded authentication access information, from the received audio blocks. The authentication access information is analysed to determine the information to retrieve authenticating information from an external memory. The audio blocks of the received audio signal are analysed to determine whether audio blocks are present for which no authenticating information is available. The part of the received audio signal with no authenticating information is compared with a corresponding part of the original audio signal stored on a server. A user is notified if no successful authentication can be performed.

[0012] The authentication procedure can be as described above or the authentication can be performed in any other way. For the invention is only necessary to receive the information that for some audio blocks no authentication can be performed. Thereafter, the method deals with those audio blocks where no authentication information is available. The authentication information required to authenticate the received audio signal can be extracted directly from the audio block (as they are embedded into the audio blocks e.g., via a watermark or an appropriate audio container) or indirectly as only authentication access information are embedded into the audio blocks (also possible e.g., via a watermark or an appropriate audio container). The authentication access information include information on where and / or how the authentication information can be retrieved.

[0013] There are several ways to enable an authentication of an audio signal. Authentication information is provided together with the audio signal. The authentication information can be embedded e.g. with a watermark into the audio signal or authentication information can be provided externally while information required to access the authentication information can be embedded into the audio signal. The audio signal is divided into audio blocks and the audio blocks form the basis for the further authentication processing. The length of the audio block can be encoded into a block header.

[0014] A received or stored audio signal that is to be authenticated can be a signal from a single source or can be an audio signal composed from parts of audio signals from a plurality of different sources (e.g. music recordings, podcasts, radio features, speeches etc) or from different recordings of the same audio signal (e.g. multiple microphones). Hence, the authentication procedure must be able to authenticate a composite audio signal. To enable this procedure, authentication information must be provided for each audio block. However, if the audio signal is composed of different clippings from different audio signals, a clipping can start and end at places which do not correspond to the block boundaries of the audio blocks used for authentication. Thus, the authentication information for an entire audio block or for parts of the audio block may not be present. E.g. it might not be possible to decode the watermark with the authentication access information since the block carrying it is only partially part of the audio composition.

[0015] In order to deal with such scenarios according to an example those parts of the audio signal, for which no authentication information is available, are determined.

[0016] According to an aspect an authenticator is provided, which can carry out the steps of the above described method of authentication.

[0017] According to an aspect a computer program product for authenticating an audio signal is provided, wherein the computer program product comprises program code means causing the audio signal authenticator to carry out the method of authentication. According to an example, the authentication of the audio signal can be performed by detecting the block boundaries from special information in the watermark or audio container, and extracting the authentication access information (e.g. a link (e.g., consisting of UUID and user based unique block number,)) to the block specific authentication information (e.g. audio features) either from the audio watermark or directly from the audio container. The public cryptographic key of the user (e.g. identified by the UUID from link) can be used to verify the authenticity of the authentication information (e.g. audio features). The audio signal block is compared against the audio features to verify its authenticity.

[0018] With the above described method of authenticating an audio signal it becomes possible to detect malicious modifications of the audio signal by comparing an audio block with the authentication information associated to the audio block. It becomes also possible to detect a modification of the audio signal to be authenticated and / or a modification of the authentication information externally stored by checking the signature of the authentication information. If an audio block of an audio signal is modified and the authentication access information is changed as well, this will be noted by the authentication process as the signature of the audio signal and / or of the authentication information is that of another person or device.

[0019] According to an aspect of the invention, the authentication information, (e.g. the metadata) can comprise a microphone identification, position / location information of the microphone, time and data of the recording and / or a microphone model, etc. The metadata can also comprise e.g., audio features, time of recording, sequence number, location of recording). Optionally, metadata of the audio block can be the user of the microphone, the date, the time, a GPS position, etc. The authentication information (e.g. the metadata) can comprise audio features representing the audio signal or audio blocks of the audio signal based on human perception or the original audio signal in a transparently coded version to reduce the storage space required by the server. Thus, the probability is improved that the result of the authentication is reliable.

[0020] The authenticity check of the received audio signal can be performed by a decoder which can, for example, be implemented at a cloud service, a computer, a tablet or a smart device.

[0021] Optionally, metadata of the audio block can be the user of the microphone, the date, the time, a GPS position, etc. Thus, the probability is improved that the result of the authentication is reliable. Audio features relate to an audio representation of an audio signal trying to capture relevant aspects for human auditory perception of the audio signal. The audio features should be selected to reduce the required data rate (for the transmission of the audio features) and hence storage space required on a server. Although the audio features could possibly be the original audio signal itself, preferably the audio features can be a transparently coded version of the original signal requiring much less storage space. Another example for audio features could be the output of any model mimicking the human auditory perception. The audio features can be a Mel-frequency power spectrum of an audio signal with Mel Frequency Cepstral Coefficients (MFCCs).

[0022] According to an aspect, the authentication information stored externally can be used to manually authenticate the audio signal orthe audio block if the authentication process does not give a clear indication of the authenticity of the audio signal or the audio blocks.

[0023] According to an example, the actual audio features extracted from the audio blocks and optional additional metadata are stored in a signed way, e.g. on a server. A unique (unambiguous) link to the metadata stored e.g. on the server is embedded into a watermark or into a suitable audio container. The link can comprise a unique user identifier (UUID) in combination with a user based unique block number of the respective audio block. Audio features relate to an audio representation of an audio signal trying to capture the relevant aspects for human auditory perception and at the same time preferably reduces the required data rate and hence storage space required on the server. Although it could possibly be the original audio signal itself, preferably it can be a transparently coded version of the original signal requiring much less storage space. Another example for audio features could be the output of any model mimicking the human auditory perception.

[0024] In order to authenticate the audio signal of an audio block, the block boundaries can be detected from special information in the watermark or audio container. Then the link (e.g., consisting of UUID and user based unique block number,) to the block specific signed authentication information, consisting of the audio features, optional metadata and the signature, is extracted either from the audio watermark or directly from the audio container. By means of a public cryptographic key of the user (e.g., identified by the UUID from link,) the combination of the authentication information and the signature is verified. Accordingly, it is verified whether the authentication information has been signed by the claimed source of the audio signal. The audio signal block is compared against the audio features to verify perceptual similarity, and thus the authenticity of the audio signal block. For the perceptual comparison, audio features can be determined from the received audio signal block, which can be directly compared to the audio features from the signed authentication information stored e.g. on a server.

[0025] The proposed authentication method is able to prevent any malicious modification of the actual audio signal comparing the audio signal block against the signed authentication information. The authentication might fail in case the comparison allows for some limited amount of modification and the allowed modification threshold is exceeded. Moreover, if together with a malicious modification of the audio signal block the authentication information on the server is altered accordingly, this can be detected by a mismatch of the authentication information and the signature associated to the authentication information. Furthermore, if together with a malicious modification of the audio signal block the link (within the watermark or audio container) is changed to refer to different (but suitable) signed authentication information on the server, the verification of the signature and the subsequent comparison may be successful indeed, but the signature will be that of a different user, assuming that the private key of the original user for the cryptographic signing is inaccessible to the attacker.

[0026] Assuming that the (automated) comparison between the audio signal to be authenticated and the authentication information is carried out in a way to allow some minor modifications (e.g. by perceptual coding) and the result of this comparison is not clear (i.e., no unambiguous statement about the authenticity of the audio signal can be made), the link to the authentication information (in case the contained audio features would correspond to the original audio file or a perceptually coded version of it) could be used for a “manual” perceptual comparison, in order to increase the trust in the method.

[0027] By providing an audio block specific link (e.g., by the UUID and user based unique block number,) allows the advantage (e.g., in the watermarking scenario) that the audio file can be cut at any time instant without the cutting tool being aware of the authentication process and the embedded links. Alternatively, in case an audio container is used for the joint transport of the audio with the link, a file-based link might be once embedded into the contained header to reduce the data rate. It might be composed of the UUID and a position within the meta data (e.g., a sample index of the original file) to indicate how the audio file in question is aligned with the reference on the server.

[0028] These and other aspects of the invention are described in more detail with reference to the following figures. Fig. 1 shows a block diagram of an audio signal signing module,

[0029] Fig. 2 shows a diagram of an overall audio signing and authentication workflow,

[0030] Fig. 3. shows a block diagram of an audio signal authenticator,

[0031] Fig. 4 shows a block diagram of an audio signal signing module, and

[0032] Fig. 5 shows a block diagram of an audio signal authenticator.

[0033] Fig. 6 shows a block diagram of an audio signal signing module,

[0034] Fig. 7 shows a block diagram of an audio signal authenticator, and

[0035] Fig. 8 shows a graph of an exemplary composed audio file to be authenticated.

[0036] According to an example, an authentication is performed for an audio signal. The audio signal is authenticated based on information embedded in the audio blocks of the audio signal. The information embedded in the audio blocks can be authentication information directly embedded into the audio block e.g. be means of watermarks or authentication access information. The authentication access information contains information on where and how stored authentication information relating to an audio block can be retrieved from a memory, like a server. In this case the authentication information is not directly embedded in the audio blocks. The Figs. 1 to 5 relate to the example of using authentication information and the Figs. 6 to 8 relate to the example of using the authentication access information.

[0037] According to an example, the authentication of an audio block is performed. Only if not all audio blocks were authenticated then those not authenticated audio block can be analysed further as described below in more details.

[0038] In order to better understand the authentication of an audio signal first of all the generation of the authenticable audio signal is described.

[0039] Fig. 1 shows a diagram of an audio signal signing module. The audio signal signing module 500 receives an input audio signal 121 via an audio input 502 from a microphone 100 or another audio source such as an audio recorder and outputs an output audio signal 501 via an audio output 190. The output audio signal 501 can be based on the input audio signal 121. The microphone 100 can comprise at least one microphone capsule 110 and an ana- log-to-digital AD converter 120. The at least one microphone capsule 1 10 can capture an audio signal and can output a captured analogue audio signal 1 11. The captured analogue audio signal 111 can be forwarded to the analog-to-digital AD converter 120 which can digitize the audio signal 11 1 and can output a digital audio signal 121. The audio signal signing module 500 can be implemented as a device separate from the microphone 100 or can be comprised in the microphone 100.

[0040] The audio signal signing module 500 can receive the audio signal 121 from the microphone 100 or from another source, which is indicated in Fig. 1 by the selector 101 . The digital audio signal 121 is inputted to a block divider 130 which divides the audio signal 121 into audio blocks 131 and outputs a sequence of a plurality of audio blocks 131. Optionally, if the audio signal 121 is already divided into audio blocks 131 , the block divider 130 can be omitted. The sequence of audio blocks 131 are forwarded to an audio feature extractor 150 which generates audio features 151 for each audio block 131 in the sequence of audio blocks. A private key signature unit 140 receives authentication information comprising audio features 151 of each audio block 131 togetherwith optional further information 1 13 (e.g. metadata) like location or time and signs the authentication information with the private key 142, providing signed authentication information 141 , which can be stored on a memory 600 e.g. located on an external server. The signed authentication information 141 can comprise the authentication information 152 (see Fig. 2) comprising audio features 151 and optional further information 1 13 and a signature 144 (Fig. 2) generated by the private key signature unit 140. Alternatively, the authentication information 141 can be stored in an internal memory or any other memory, as long as it can be accessed by an authenticator used to authenticate a received audio signal.

[0041] The signed authentication information 141 itself comprises the authentication information, namely (a) audio features 151 , optionally (b) the further information 113 and (c) the signature 144. The individual audio blocks 131 are e.g. in parallel fed to an information embedder 160 for embedding authentication access information 162 e.g. into each audio block 131. The authentication access information 162 comprises information on where and / or how the authentication information 152 (or more precisely the signed authentication information 141), for that particular audio block 131 , can be accessed at the memory 600 e.g. on a server. The embedding can for instance be accomplished by means of watermarking.

[0042] Hence, the information embedder 160 can be implemented as a watermark generator which generates a watermark 163. Another option is to embed the authentication access information 162 into an appropriate audio container next to the actual audio blocks 131. Accordingly, the information embedder 160 outputs audio blocks 161 with embedded authentication access information 162. The output of the embedder 160 corresponds to the output audio signal 501 of the module 500 at the audio output terminal 190. The output audio signal 501 can be broadcasted or stored e.g. via a network 300 (Fig. 2). In particular, it can be even further modified to a little extent, including modification types like sample rate conversion, perceptual compression and cutting.

[0043] In the case that the information embedder 160 is a watermark generator its output 161 could alternatively be input to the audio feature extractor 150 instead of the original audio blocks 131 , which is indicated in Fig. 2 by the selector 102. This alternative offers the advantage that the audio features are extracted based on the final signal 161 that will be output by the audio signal signing module 500 rather than an intermediate signal 131 that is not outputted. As the watermark generator is designed to perceptually alter an audio signal as little as possible, both input options (131 or 161) for the audio feature extractor 150 are reasonable.

[0044] Audio features relate to an audio representation trying to capture the relevant aspects for human auditory perception and at the same time preferably reduces the required data rate and hence storage space required on the server or memory 600. Although it could possibly be the original audio signal itself, preferably it can be a transparently coded version of the original signal requiring much less storage space. Another example for audio features could be the output of any model mimicking the human auditory perception.

[0045] The audio signal 501 (with the embedded watermark or authentication access information included in an audio container) at the output of the audio signal module 500 can be stored or transmitted. As a watermark can be embedded into e. g. substantially each audio block 131 , each audio block 131 can be individually authenticated. The watermark 103 can also be embedded only in some of the audio blocks. The watermark 163 can comprise apart from the authentication access information further information like the block boundaries. This information can be used when authenticating the audio signal to determine a block length.

[0046] An audio watermark 163 can be a distinct identification which is embedded in an audio signal and which is for example previously used for identifying a copyright information of the audio signal. Preferably, the watermark is embedded into the audio signal such that it becomes very difficult to remove or destroy the watermark. Preferably, if the audio signal with the embedded watermark is copied, stored or transmitted, the embedded watermark will not change. The same reasoning applies to authentication access information included in an audio container.

[0047] Fig. 2 shows a diagram of an overall audio signing and authentication workflow. An audio signal is captured by a microphone 100 outputting a digital audio signal 121 . From this audio signal 121 audio features 151 are extracted and signed with the private key 142 of e.g., audio signal signing module 500. The audio features 151 together with the digital signature 144 generated by the signature unit 140 form the signed authentication information 141. The signed authentication information 141 can be forwarded to a server or memory 600, where the signed authentication information 141 is stored for later retrieval. The information on where and / or how the signed authentication information 141 can be retrieved, namely the authentication access information 162 is provided for the audio signal signing module 500. The audio signal signing module 500 uses this authentication access information 162 and embeds this information into the audio blocks 131 of the audio signal. Hence, the output signal 501 of the audio signal signing module 500 comprises the audio blocks 131 with the embedded authentication access information. The audio blocks 131 of the audio signal with the embedded authentication access information 162 are outputted by the audio signal signing module 500 and can be stored or distributed e.g. through a network 300.

[0048] This distribution can involve a modification or manipulation of either the authentication information 152 and / or the audio signal 121 . The signed authentication information 141 is validated using the public key 143. This validation indicates a potential modification of the audio features. By extracting audio features from the received signal and comparing them with the features contained in the signed authentication information 141 the integrity of the audio signal can be verified. A final authentication result 261 is determined from both checks.

[0049] An audio signal signing module 500 e.g. implemented as part of a microphone 100 is used to capture an audio signal. Alternatively, an audio signal signing module 500 can receive the audio signal. The audio signal can also be an audio signal captured beforehand by other means. As described with reference to Fig. 1 , the captured audio signal is divided into a plurality of audio blocks and the authentication information of at least one audio block is signed based on the private key 142 of the audio signal signing module 500. The signed authentication information 141 is outputted by the audio signal signing module 500 and can be stored on a server or memory 600. Authentication access information 162, regarding information where the authentication information can be retrieved from the server or memory 600 can be provided to the audio signal signing module 500. The authentication access information 162 is embedded into the audio block received or captured by the audio signal signing module 500. Thus, the audio signal signing module 500 outputs a) an output audio signal 501 , which comprises audio blocks with embedded authentication access information to a network 300 for storing or forwarding and b) signed authentication information 141 to the server or memory 600 where this information ca be stored.

[0050] Optionally, the authentication access information can be embedded into the audio blocks of the audio signal by means of a watermark. The watermark can relate to authentication access information relating to a location of a memory 600 where authentication information is stored. This authentication information can be generated by an audio feature extractor 150 as described above with reference to Fig. 1 . The audio blocks of the audio signal with the embedded authentication access information can be transmitted via a network 300 or stored. An authenticator 200 can receive the audio signal 501 with the embedded watermark and extract the authentication access information. The authentication information is retrieved from the server 600 based on the authentication access information. The signature of the authentication information as stored on the server or memory 600 can be validated based on a public key 143 associated to the audio signal signing module 500 which can be stored, for example, on a server 400 or a distributed ledger.

[0051] Fig. 3 shows a block diagram of an audio signal authenticator. The audio signal authenticator 200 receives an audio signal 171 with embedded authentication access information. This audio signal 171 ca be the output audio signal 501 from the audio signal signing module 500 of Fig. 1. This audio signal 171 is input into a block bounds detector and block divider 205, which determines block boundaries and outputs a plurality of audio blocks 201 with embedded authentication access information 212. If the authentication access information 212 has been embedded by means of watermarking, it is assumed that the block boundaries have been previously coded into the watermark, from which they can be detected at this stage. In case the audio signal is transported by an audio container that contains the authentication access information 212 as metadata, it is assumed that the authentication access information is available per block with the block boundaries being part of the metadata.

[0052] The audio blocks 201 with embedded authentication access information are input into the access information extractor 210, which extracts the authentication access information 212 (corresponding to the authentication access information 162 from the signing modules if no manipulation has taken place) from the audio blocks 201 . The authentication access information interpreter 230 determines access information from the authentication access information 212 namely, where to find the signed authentication information 141 belonging to the current audio block 201 on a server or memory 600. The authentication information 141 retrieved from the server or memory 600 comprise firstly audio features 151 representing the audio block 201 , secondly the signature 221 and thirdly optional further information 113 like location and / or time where and / or when the signing has taken place. The signature 221 corresponds to the signature 144 generated by the signature unit 140 if no manipulation has taken place.

[0053] In a perceptual similarity analyzer 240 the audio blocks 201 and the audio features 151 are perceptually compared to each other, providing a similarity statement 241. In the simplest case this can be a binary decision about whether the compared quantities are perceptually sufficiently equal or not. Alternatively, the similarity statement 241 can be more detailed, proving for instance a similarity score.

[0054] In the similarity analyzer the audio blocks can be compared to the audio features of the original audio signal. The audio features can be a true copy of the original audio signal. This will however require large storage space to store the original audio signal. If the audio features relate to other characteristics of the original audio signal, the similarity analyzer should compare the received audio features with the audio features of the audio signal to be authenticated. Hence, the required audio features can be extracted from the audio signal to be authenticated. This can be performed e.g. in the similarity analyzer. In other words, the similarity analyzer should compare the audio features (as received via the authentication access information, i.e. the audio features of the original audio signal) and corresponding audio features of the audio signal to be authenticated.

[0055] An authentication access information interpreter 230 further determines from the authentication access information 212 which owner of the private key 142 has originally signed the audio signal 171 , e.g. represented by an ID. The owner of the private key 142 can be a natural person, an organization, a registered microphone to name only a few possibilities. Using this ID, a public key 143 and the identity 270 of the owner of the private key can be retrieved from an external database. The public key 143 is subsequently employed to verify the combination of the signature 221 and the authentication information 152 (the audio features 151 and the further information 113) within a signature verifier 250, proving a signature verification result 251. The perceptual similarity statement 241 and the verification result 251 are finally combined in a combiner 260 to a final authentication result 261 . Note that the processing that is here proposed for audio content can be similarly applied to any time dependent data. In particular, it can be also applied to video content, if the following modifications are applied, i.e. the microphone 100 is replaced by a video camera, the audio feature extractor 150 is replaced by a video feature extractor, the authentication access information embedding unit 160 uses a visual watermark instead an audio watermark, or uses a video container instead of an audio container and the auditory perceptual comparison 240 is replaced by a visual perceptual comparison.

[0056] Fig. 4 shows a schematic block diagram of an audio signal signing module. The audio signal signing module 500 receives a digital audio signal 121 at its audio input 502, which is inputted to the block divider 130 which outputs a sequence of plurality of audio blocks 131 . The audio blocks 131 are forwarded to an audio feature extractor 150 which generates audio features 151 for each audio block 131 . In a private key signature unit 140, authentication information comprising audio features 151 of each audio block 131 together with optional further information 113 like location or time is signed with the private key 142 of a user or entity, providing signed authentication information 141. The signed authentication information 141 itself consists of the audio features 151 , the generated signature 144 and the optional further information 113. Both components, the individual audio blocks 131 and the signed authentication information 141 are required during the verification of the authentication method shown in Fig. 5. They can be in principle embedded into the same suitable audio container, but can also be distributed over different physical channels. Optionally, a user or entity identifier 145 can be added to both components, which can for instance enable the retrieval of the suitable public key for an automated verification.

[0057] Fig. 5 shows a schematic block diagram of an authenticator 200. The authenticator 200 receives an audio signal e.g. in form of audio blocks 201 which are to be verified as well as signed authentication information 141 . The signed authentication information 141 comprises audio features 151 , further information 113 and the signature 221 . The audio features 151 contained in the authentication information 141 are perceptually compared to the audio blocks 201 within the perceptual similarity analyzer 240, providing a similarity statement 241 . In the simplest case this can be a binary decision about whether the compared quantities are perceptually sufficiently equal or not. Alternatively, the similarity statement 241 can be more detailed, providing for instance a similarity score. Further, with knowledge about the user or entity that created the signature 221 , the corresponding public key 143 and optionally the user identity 270 are obtained. Optionally, this knowledge can be derived from a user or entity identifier 145 provided together with the authentication information 141 and the audio blocks 201 . The public key 143 is subsequently employed to verify the combination of the signature 221 and the authentication information 141 within the signature verification block 250, proving a signature verification result 251 . The perceptual similarity statement 241 and the verification result 251 are finally combined in combiner 260 to a final authentication result 261.

[0058] In the following Fig. 6 to 8 the generation of an authenticable audio signal based on embedding authentication information into the audio blocks as well an authentication of the audio signal based on the embedded authentication information is described. This authentication can be used in the authentication of an audio signal as described in the following.

[0059] Fig. 6 shows a diagram of an audio signal signing module. The audio signing module of Fig. 6 is merely an example of an audio signing module. The audio signal signing module 500 receives an input audio signal 121 via an audio input 502 from a microphone 100 or another audio source such as an audio recorder and outputs an output audio signal 501 via an audio output 190. The output audio signal 501 can be based on the input audio signal 121. The microphone 100 can comprise at least one microphone capsule 110 and an ana- log-to-digital AD converter 120. The at least one microphone capsule 1 10 can capture an audio signal and can output a captured analogue audio signal 1 11. The captured analogue audio signal 111 can be forwarded to the analog-to-digital AD converter 120 which can digitize the audio signal 111 and can output a digital audio signal 121 .The audio signal signing module 500 can be implemented as a device separate from the microphone 100 or can be comprised in the microphone 100.

[0060] The audio signal signing module 500 can receive the audio signal 121 from the microphone 100 or from another source, which is indicated in Fig. 6 by the selector 101 . The digital audio signal 121 is inputted to a block divider 130 which divides the audio signal 121 into audio blocks 131 each having a block length and outputs a sequence of a plurality of audio blocks 131 . The block divider 130 can divide the audio signal 121 into a plurality of audio blocks. Optionally, the audio blocks each receive a block number, which can be a subsequent block number. Optionally, if the audio signal 121 is already divided into audio blocks 131 , the block divider 130 can be omitted. The sequence of audio blocks 131 are forwarded to an audio feature extractor 150 which generates audio features 151 for each audio block 131 in the sequence of audio blocks 131 . A private key signature unit 140 receives authentication information comprising audio features 151 of each audio block 131 together with optional further information 113 (e.g. metadata) like location or time and signs the authentication information with the private key 142, providing signed authentication information 141. The signed authentication information 141 can comprise the authentication information 152 comprising audio features 151 and optional further information 113 and a signature 144 generated by the private key signature unit 140.

[0061] The signed authentication information 141 itself comprises the authentication information, namely (a) audio features 151 , optionally (b) the further information 113 and (c) the signature 144. The individual audio blocks 131 are e.g. in parallel fed to an information embedder 160 for embedding the signed authentication information 141 e.g. into each audio block 131. The embedding can for instance be accomplished by means of watermarking.

[0062] Hence, the information embedder 160 can be implemented as a watermark generator which generates a watermark 163. Another option is to embed the signed authentication information 141 into an appropriate audio container next to the actual audio blocks 131. Accordingly, the information embedder 160 outputs audio blocks 161 with embedded signed authentication information 141 . The output of the embedder 160 corresponds to the output audio signal 501 of the module 500 at the audio output terminal 190. The output audio signal 501 can be broadcasted or stored e.g. via a network 300 (Fig. 7). In particular, it can be even further modified to a little extent, including modification types like sample rate conversion, perceptual compression and cutting.

[0063] In the case that the information embedder 160 is a watermark generator its output 161 could alternatively be input to the audio feature extractor 150 instead of the original audio blocks 131 by the selector 102. This alternative offers the advantage that the audio features are extracted based on the final signal 161 that will be output by the audio signal signing module 500 rather than an intermediate signal 131 that is not outputted. As the watermark generator is designed to perceptually alter an audio signal as little as possible, both input options (131 or 161) for the audio feature extractor 150 are reasonable.

[0064] The audio signal 501 (with the embedded watermark or authentication access information included in an audio container) at the output of the audio signal module 500 can be stored or transmitted. As a watermark can be embedded into e. g. substantially each audio block 131 , each audio block 131 can be individually authenticated. The watermark 103 can also be embedded only in some of the audio blocks. The watermark 163 can comprise apart from the authentication access information further information like the block boundaries. This information can be used when authenticating the audio signal to determine a block length. An audio watermark 163 can be a distinct identification which is embedded in an audio signal and which is for example previously used for identifying a copyright information of the audio signal. Preferably, the watermark is embedded into the audio signal such that it becomes very difficult to remove or destroy the watermark. Preferably, if the audio signal with the embedded watermark is copied, stored or transmitted, the embedded watermark will not change. The same reasoning applies to authentication access information included in an audio container.

[0065] Fig. 7 shows a block diagram of an audio signal authenticator corresponding to the audio signal authenticator of Fig. 6. The audio signal authenticator 200 receives an audio signal 171 with embedded authentication access information. This audio signal 171 can be the output audio signal 501 from the audio signal signing module 500 of Fig. 6. This audio signal 171 is input into a block bounds detector and block divider 205, which determines block boundaries and outputs a plurality of audio blocks 201 with embedded authentication information.

[0066] The block bounds detector and block divider 205 detects the block boundaries e.g. by means of information in the block headers. As the block lengths of the audio blocks can vary the block lengths should be determined for each audio block or for a group of audio block having the same block length.

[0067] The audio blocks 201 are inputted to an authentication information extractor 206 which extracts signed authentication information 141 embedded in the audio blocks, which can comprise audio features 151 , a signature 144 and thirdly optional further information 1 13 like location and / or time where and / or when the signing has taken place. The signature 221 corresponds to the signature 144 generated by the signature unit 140 if no manipulation has taken place.

[0068] In a perceptual similarity analyzer 240 the audio blocks 201 and the audio features 151 are perceptually compared to each other, providing a similarity statement 241. In the simplest case this can be a binary decision about whether the compared quantities are perceptually sufficiently equal or not. Alternatively, the similarity statement 241 can be more detailed, proving for instance a similarity score.

[0069] In the similarity analyzer the audio blocks can be compared to the audio features of the original audio signal. The audio features can be a true copy of the original audio signal. This will however require large storage space to store the original audio signal. If the audio features relate to other characteristics of the original audio signal, the similarity analyzer should compare the received audio features with the audio features of the audio signal to be authenticated. Hence, the required audio features can be extracted from the audio signal to be authenticated. This can be performed e.g. in the similarity analyzer. In other words, the similarity analyzer should compare the audio features (as received via the authentication access information, i.e. the audio features of the original audio signal) and corresponding audio features of the audio signal to be authenticated.

[0070] An authentication information extractor 206 further determines from the authentication information 141 which owner of the private key 142 has originally signed the audio signal 171 , e.g. represented by an ID. The owner of the private key 142 can be a natural person, an organization, a registered microphone to name only a few possibilities. Using this ID, a public key 143 and the identity 270 of the owner of the private key can be retrieved from an external database. The public key 143 is subsequently employed to verify the combination of the signature 144 and the authentication information 152 (the audio features 151 and the further information 113) within a signature verifier 250, proving a signature verification result 251 . The perceptual similarity statement 241 and the verification result 251 are finally combined in a combiner 260 to a final authentication result 261 .

[0071] Note that the processing that is here proposed for audio content can be similarly applied to any time dependent data. In particular, it can be also applied to video content, if the following modifications are applied, i.e. the microphone 100 is replaced by a video camera, the audio feature extractor 150 is replaced by a video feature extractor, the authentication access information embedding unit 160 uses a visual watermark instead an audio watermark, or uses a video container instead of an audio container and the auditory perceptual comparison 240 is replaced by a visual perceptual comparison.

[0072] Figs. 8 shows a graph of an exemplary received audio file FA, which is composed of parts of two different audio signals A and B. The audio signal B can optionally be an excerpt of the signal A. In particular, FA comprises audio blocks A1-A3 from a first audio signal A as well as audio blocks B1 and B2 from a second audio signal B. As the audio blocks A1 , A2 and B2 have been completely received together with the corresponding watermark, the authenticator 300 is able to determine the validity and authenticity of the audio blocks A1 , A2 and B2. However, the part C of the received audio signal FA after the end of the audio block A2 and before the beginning of the audio block B2 does not include any valid extractable watermark and therefore no authentication information is available for this section of the audio file FA. The problem with the authenticating process arises if a received audio signal comprises clippings from different audio signal sources, where the clippings originate or correspond or do not correspond to the audio blocks from the first or second audio signal source A, B. In such a case, no valid watermark WM1 , WM2, WM3 is present and thus no watermark can be extracted.

[0073] A problem when trying to validate block C of the composed audio signal FA, namely the part which starts after the end of the audio block A2 and ends before the beginning of the audio block B2, is that it is not clear where the audio block A3 ends and the audio block B1 starts. To determine this cutting point, the reference audio features contained within the authentication information for blocks A3 and B1 are retrieved from the server 600 and compared to the audio signal block C. Block A3 is hereby the successor of block A2 and thus the corresponding authentication information can be found on the server 600. Similarly, block B1 is hereby the predecessor of block B2 and thus the corresponding authentication information can also be found on the server 600.

[0074] For the comparison a suitable type of audio features can be selected. Subsequently, audio features of the selected type are extracted from the audio signal block C. Further, the audio features contained within the authentication information for blocks A3 and B1 are retrieved from the server 600 and are transformed into audio features of the selected type. In case that the type of audio features contained within the authentication information for any of the blocks A3 or B1 already corresponds to the selected audio feature type, the corresponding transformation can be omitted. Typically, audio features are computed for blocks of a relatively short duration compared to the duration of the individual blocks A3 and B1 , e.g. of 20ms for MFCCs compared to e.g. 1s for the duration of A3 and B1. Hence, for the analysed audio signal block C a feature sequence is obtained, where each feature corresponds to a subblock.

[0075] Consequently, the comparison starts by pairwise comparing one after the other, going forward in time, the audio features from the beginning of the audio signal block C to the potentially transformed reference audio features from the beginning of block A3. Let us assume that the pairwise comparison is successful for all subblocks up to the N-th subblock, where the audio features differ too largely from each other. In a next step, the audio features for the signal block C are pairwise compared one after the other with the reference audio features for block B1 , however, this time starting from the end of blocks C and B1 and going backwards in time. If the comparison fails for any subblock succeeding the N-th subblock of the audio block C, it is a hint for a manipulation within the audio block C. Otherwise, if for all compared subblocks succeeding the N-th subblock of the audio block C the comparison is successful, the statement can be made that the original audio signal block A3 has been cut within the N-th subblock and that possibly there has been an additional manipulation only within the N-th subblock of block C. Apart from that it means that all subblocks of the audio block C except for the N-th have been successfully verified. Since the duration of subblocks is relatively short, a manipulation might be considered irrelevant to conclude that the whole audio block C has been successfully verified. If the cutting position is being reported, it can be combined with the hint for a potential manipulation within the related subblock.

[0076] According to an example, an authenticator comprises an input for receiving a received audio signal, a block information extractor, an authentication information extractor, a validation unit and a determination unit which determines parts of the received audio signal, where no authentication information is available. The block information extractor can comprise a block boundary extractor and a watermark extractor.

[0077] The block information extractor analyses the received audio signal to determine block boundaries and a watermark extractor determines whether a watermark is present in the received audio signal. The block boundaries are important as typically a watermark is associated to each audio block. A watermark extraction can only be performed if the correct block boundaries have been determined. Once the correct block boundaries have been determined, the watermark extractor can extract a watermark. The watermark can for example comprise authentication access information which can be used to extract authentication information from the authentication information server. This authentication information can be used to authenticate or validate the received audio signal. Preferably, authentication information is available for each audio block.

[0078] The authentication information extractor can extract the authentication information from the authentication information server based on access information extracted from the watermark. The validation unit can validate the audio blocks based on the associated authentication information. The determination unit can be used to determine whether any audio blocks are present in the received audio signal, where no authentication information is present. One reason, why no authentication information is present, can be the situation as described with reference to Fig. 8. In such case, the authenticator 300 can comprise an iteration unit 360 which performs the procedure described with reference to Fig. 8

[0079] Accordingly, with the above described authentication procedure it is possible to authenticate also those parts in the received audio signal where no direct authentication information is present. This, however, necessitates that the original audio source signal A and the original audio source signal B are stored in the authentication information server, namely in the audio signal memory 620. However, it can be expected that the amount of parts in the received audio signal, where no direct authentication information is present, should be small.

[0080] The above described authentication procedure is advantageous as it does not require a large bandwidth, as only those audio blocks from the audio signal sources in question are extracted where no actual authentication information is present. This authentication procedure only requires that the authentication information (e.g. audio features or the original audio signals) are stored in a server or stored at a location where they can be extracted.

[0081] It is not required that the entire reference signal is transmitted.

[0082] Thus, the authentication procedure as described above is a robust and easily implemented authentication procedure.

[0083] In a further example, those parts in the received audio signal without authentication information can originate from an audio source or audio signal where no authentication information is provided. If no authentication information can be extracted by the authenticator 300, the authenticator may output this information to the user. For example, the authenticator can output a representation of these parts of the received audio signal in a different colour, for example, in red. Accordingly, the authenticator can signal to the user that this part of the received audio signal does not have sufficient authentication information and the authentication procedure cannot validly be performed.

[0084] As no authentication information is available, it is possible that the audio blocks in question are audio blocks from a valid audio signal or from a malicious audio signal.

[0085] In the similarity analyzer the audio blocks can be compared to the audio features of the original audio signal. The audio features can be a true copy of the original audio signal. This will however require large storage space to store the original audio signal. If the audio features relate to other characteristics of the original audio signal, the similarity analyzer should compare the received audio features with the audio features of the audio signal to be authenticated. Hence, the required audio features can be extracted from the audio signal to be authenticated. This can be performed e.g. in the similarity analyzer. In other words, the similarity analyzer should compare the audio features (as received via the authentication access information, i.e. the audio features of the original audio signal) and corresponding audio features of the audio signal to be authenticated.

[0086] Further examples of the audio features are described by Torsten Dau, Dirk Pueschel, and Armin Kohlrausch. A quantitative model of the "effective" signal processing in the auditory system. I. model structure. The Journal of the Acoustical Society of America, 99(6):3615- 3622, 1996, in the following referred to as the perception model (PEMO). This paper describes a quantitative model aimed at describing how the auditory system processes acoustic signals. The focus is on developing a model that mimics the functional processing of auditory stimuli as perceived by humans, particularly in the context of complex sounds like speech or music.

[0087] Key components of the model can include peripheral processing, envelope extraction, modulation filtering and decision stage. The peripheral processing part of the model captures the initial stages of auditory processing, including the transformation of acoustic signals into neural representations through mechanisms such as the outer and middle ear filtering, as well as nonlinearities in the cochlea. The envelope extraction part relates to the extraction of the temporal envelope of sound, which is crucial for understanding amplitude modulation and speech processing. The modulation filtering part relates to a system that filters amplitude modulations, mimicking how the auditory system is sensitive to different modulation frequencies. At the decision stage the processed auditory signal is used to make decisions about the nature of the sound, representing higher-level auditory perception.

[0088] The model was designed to be consistent with experimental psychoacoustic data, particularly in its ability to simulate the auditory system's response to amplitude-modulated sounds. This approach provides a framework for understanding the effective signal processing in human auditory perception, especially for tasks involving the detection and discrimination of complex auditory patterns. Carrying out the comparison of the received audio blocks and the extracted or retrieved authentication information in a transformed domain (namely regarding the human auditory perception audio features) ensures that the identified differences are perceptually meaningful. In principle any similarity measure, e.g., the empirical cross-correlation coefficient or the relative error with respect to the reference, may be used to map the result of the comparison onto a numerical value.

[0089] When applying further transformations to the Mel-frequency power spectrum of an audio signal, Mel Frequency Cepstral Coefficients (MFCCs) are. MFCCs use the Mel scale, which is a perceptual scale of pitches which serves to approximate the way humans perceive sound. The Mel scale is nonlinear, such that it spaces out lower frequencies more finely and higher frequencies more coarsely, aligning more closely with human auditory perception.

[0090] Mel Frequency Cepstral Coefficients use the cepstrum as a transformation that converts a signal from the frequency domain back into a domain where the signal's rate of change can be analyzed. The idea is to capture the spectral properties of a signal in a way that emphasizes the information-carrying components.

[0091] The process to compute MFCCs from an audio signal can involve several steps: Pre-em- phasis: Boosting the high-frequency components of the signal to balance the spectrum, usually by applying a high-pass filter, Framing: The audio signal is divided into short overlapping frames, typically 20-40 milliseconds long. This is because speech signals are non- stationary, but they can be considered quasi-stationary within these short frames, and Windowing: Each frame is multiplied by a window function, such as a Hamming window, to reduce edge effects and smooth the signal. Fourier Transform: A Fast Fourier Transform (FFT) is applied to each windowed frame to convert the time-domain signal into the frequency domain. Mel Filter Bank: The resulting spectrum is passed through a series of triangular bandpass filters spaced according to the Mel scale. This step approximates how humans perceive sound frequencies. Logarithm: The logarithm of the power of each Mel- filtered spectrum is computed. This simulates the human ear's response to loudness, Discrete Cosine Transform (DCT): Finally, the logarithmic Mel spectrum is transformed using the DCT. The result is a set of coefficients called MFCCs. Typically, only the first 12-13 coefficients are kept, as these contain the most important information.

[0092] MFCCs can capture the broad spectral shape of the audio signal in a compact form, making them highly useful for machine learning algorithms in speech and audio processing. They help in distinguishing different phonemes, speaker identities, and other audio characteristics by providing a representation that is robust to variations in pitch, volume, and other factors.

[0093] The audio feature can also comprise:

[0094] 1 . Linear Predictive Coding (LPC) which is a method that represents the spectral envelope of a digital signal of speech in compressed form, using the information from a linear predictive model. It estimates the parameters of a filter that can be used to recreate the signal. LPCs are highly effective for modeling the formants (resonant frequencies) of speech sounds.

[0095] 2. Perceptual Linear Predictive (PLP) Coefficients which is similarto Linear Predictive Coding but incorporates aspects of human auditory perception, such as the critical-band spectral resolution, equal-loudness curve, and intensity-loudness power law. Perceptual Linear Predictive (PLP) Coefficients aims to mimic the non-linear perception of loudness and frequency by the human ear.

[0096] 3. Gammatone Filterbank Features relate to filterbanks which simulate the filtering that occurs in the human cochlea. They are similar to Mel filterbanks but use Gammatone filters instead of triangular filters. The Gammatone Filterbank Features are useful for capturing the detailed frequency structure of audio, particularly in tasks involving environmental sound classification or hearing aid design.

[0097] 4. Chroma Features, which represent the 12 different pitch classes (C, C#, D, etc.) of a musical octave. They capture harmonic and melodic characteristics of music. These are particularly useful in music information retrieval, key detection, and chord recognition tasks.

[0098] 5. Mel Spectrogram: While similar to MFCCs, a Mel spectrogram is the result of directly applying the Mel filterbank to a power spectrogram without further transformation (like DCT). The Mel spectrogram retains more detailed frequency information and is often used as input to deep learning models and is used in conjunction with Convolutional Neural Networks (CNNs) in tasks like audio event detection, speech recognition, and music genre classification.

[0099] 6. Constant-Q Transform (CQT), which is a time-frequency representation that provides a logarithmic frequency scale, similarto the Mel scale, but with a variable time resolution that matches the frequency resolution. It is particularly useful for musical applications because it provides a better representation of musical pitch than the linear FFT or even the Mel scale.

[0100] 7. Deep Learning-Based Features: Learned features from deep learning models, such as embeddings from a trained neural network, can also be used as an alternative to traditional hand-crafted features like MFCCs. These features are often more robust and can capture complex patterns that are not easily modeled by traditional methods.

[0101] 8. Spectral Subband Centroids (SSC) which captures the centroid of energy distribution in different subbands of the signal, which can be interpreted as the "center of mass" of the spectrum within each subband. It provides information about the distribution of energy across frequency bands and is sometimes used as a complement to MFCCs.

[0102] 9. RelAtive SpecTrAI (RASTA) Features which involves filtering the log energy of the speech signal to emphasize modulation frequencies important for speech recognition. RASTA-PLP is a combination of RASTA filtering and PLP analysis, providing robust features against noise and channel variations.

[0103] Audio features relate to an audio representation of an audio signal trying to capture relevant aspects for human auditory perception of the audio signal. The audio features should be selected to reduce the required data rate (for the transmission of the audio features) and hence storage space required on a server. Although the audio features could possibly be the original audio signal itself, preferably the audio features can be a transparently coded version of the original signal requiring much less storage space. Another example for audio features could be the output of any model mimicking the human auditory perception. The audio features can be a Mel-frequency power spectrum of an audio signal with Mel Frequency Cepstral Coefficients (MFCCs).

[0104] In particular the audio features can be acoustical characteristics, spectral characteristics, and statistical characteristics of the audio signal.

[0105] Acoustical features can include pitch (Fundamental Frequency); prosody (Rhythm, Stress, Intonation) and / or duration and silence patterns (inconsistencies in pauses, speech timing, or breathing sounds can be used as audio features. Spectral Features can include formant frequencies (like resonant frequencies in speech); Mel-Frequency Cepstral Coefficients (MFCCs) (the short-term power spectrum of sound and can reveal artifacts introduced during synthesis or manipulation); spectrogram analysis (a manipulation may exhibit unusual or smeared energy distribution in the time-frequency representation); and / or phase information.

[0106] Acoustical features can include statistical and signal processing features, like noise and residuals (like subtle background noises or artifacts introduced during synthesis may not match natural recordings); high-frequency content; and / or phase distortions (some manipulations may introduce phase anomalies, which can be detected with signal processing techniques).

[0107] Acoustical features can include temporal features, like jitter and shimmer (like frequency and amplitude variations); temporal coherence (like sudden transitions or changes in speech characteristics, like unnatural breaks in the signal).

[0108] Acoustical features can include behavioral or semantic Inconsistencies, like content coherence (logical inconsistencies or unnatural flow in the spoken content might be indicative of manipulation); emotion and naturalness (The emotional tone may not align with the content, or the voice may lack the natural nuances of human emotion).

[0109] The audio features can be any one of the above mentioned audio characteristics of the audio signal or a combination of them.

[0110] By combining these features with machine learning or signal processing techniques, models can be trained to detect artifacts and inconsistencies indicative of deepfake audio.

[0111] In an example, a watermark may comprise a chronological sequence of a pilot sequence (indicating a beginning of a watermark, a user ID and a block ID). If the watermark is detected and can only be partially decoded (e.g. only pilot sequence, but no meaningful user ID or block ID), it is nevertheless reasonable to notify the user accordingly. An advance user (for example a news agency) may deduce from the information that the content was signed at a certain point in time but seems to have been altered afterwards. The partial presence of the watermark could further be used to refute the claims that the file has never been changed. In a further example, if authentication information (providence information) is not bound to the content by a watermark but e.g. via a time-dependent metadata format, there is a possibility that the digital audio workstation references all relevant providence information of all source files with accurate time stamps. In this case, the above described authentication procedure is not necessary since all information is already available.

[0112] According to an example, each audio signal which is captured by the microphone 100 and which is outputted by the microphone can comprise a watermark by means of which it is possible to determine the actual microphone which has captured the audio signal. If the microphone is used by several users, several user IDs can be associated to the microphone. If the microphone is registered, then it is possible to identify the respective microphone. The user IDs can also be registered. The user IDs can be dedicated IDs or sub IDs of the microphone ID. The microphone ID and / or the user IDs can be part of the watermark. By means of the sequence numbers of the audio blocks, it can be determined when part of the audio signal has been removed.

[0113] Optionally, the audio signal could be part of a video file.

[0114] Optionally, the signed authentication information can be included in an audio file (e.g., similar to ADM file format) instead of including it into a watermark. In this case the audio signal may not be modified.

[0115] Optionally, the process of generating a signed audio signal could be performed in a software solution based on a pre-existing audio recording or an audio stream. The private key can be entered to the software e.g., by dongle, text input, and authorized e.g., by biometric signal like fingerprint or face detection.

[0116] The authentication process can calculate a similarity score between the audio features of the signed audio signal and the analysed signal. This could allow to determine a likelihood that the signal is still authentic even if minor signal processing like e.g., gain has happened.

[0117] For authentication the audio signal as well as the signed authentication information is needed. The signed authentication information might be provided via a side channel. In case watermarking is used the authentication access information must be read out from the audio signal and the authentication information must be retrieved accordingly. In case the watermarking readout needs the block boundaries one approach to achieve this is to try out different offsets of block boundaries until a successful watermark readout is possible. Another approach could be to embed some synchronization signal with a watermark into the audio signal.

[0118] To enhance the security of the system a process is provided that allows to withdraw keys e.g., in case they are stolen. The public keys for authentication can be provided in various ways. One option is to store all public keys in one centralized database to make it easy to find the needed key. To tackle potential trust issues in this central instance the database with all public keys could be provided on a distributed ledger technology. Another option is that the organization / person that uses the technology provides the public key on their own website.

[0119] List of reference signs

[0120] 100 microphone

[0121] 101 selector

[0122] 102 selector 110 microphone capsule

[0123] 111 microphone analog audio signal

[0124] 113 metadata

[0125] 120 AD converter

[0126] 121 digital audio signal 130 block divider

[0127] 131 audio blocks

[0128] 140 private key signature unit

[0129] 141 signed authentication information

[0130] 142 private key 143 public key

[0131] 144 signature

[0132] 145 identifier

[0133] 150 audio feature extractor

[0134] 151 audio features 152 authentication information

[0135] 160 information embedder / watermark generator

[0136] 161 audio blocks with embedded information

[0137] 162 authentication access information 163 watermark

[0138] 165 size information

[0139] 171 audio signal

[0140] 190 audio output

[0141] 200 audio signal authenticator 205 block bounds detector and block divider

[0142] 206 authentication information extractor

[0143] 210 access information extractor

[0144] 212 authentication access information

[0145] 221 signature 230 authentication access information interpreter

[0146] 240 perceptual similarity analyzer

[0147] 241 similarity result

[0148] 250 signature verifier

[0149] 251 signature verification result 260 combiner

[0150] 261 authentication result

[0151] 270 identity

[0152] 300 network 400 server

[0153] 500 audio signal signing module

[0154] 501 authenticable audio signal

[0155] 502 audio input

[0156] 600 server or memory

Claims

Claims1 . Method of authenticating an audio signal, comprising the steps of analysing received audio signals to determine audio blocks and to extract embedded authentication access information from the received audio blocks, analysing the extracted authentication access information to retrieve authentication information containing audio features of an original audio signal, analysing the audio blocks of the received audio signal to determine whether irregular audio blocks (C) are present for which no authentication access information is available, comparing the irregular audio blocks (C) with a corresponding set of audio features retrievable from a memory (600), in particular based on the authentication access information, and notifying a user which parts of the irregular audio blocks (C) are successfully authenticated.

2. Method according to claim 1 , wherein if the received audio signal is composed of parts of at least two audio signals (A,B) and if the irregular audio block (C) contains a change from a first audio signal (A) to a second audio signal (B), performing the steps: determining a last complete audio block (A2) from the first signal, retrieving the authentication information corresponding to an audio block (A3) which is subsequent to the last complete audio block (A2) from the memory (600), and section-wise or pairwise comparing one after the other, going forward in time, the audio features contained within the retrieved authentication information to the received irregular audio block (C) until the change from the first audio signal to the second audio signal is detected.

3. Method according to claims 1 or 2, wherein if the received audio signal is composed of parts of at least two audio signals (A,B) and if the irregular audio block (C) contains a change from a first audio signal (A) to a second audio signal (B), performing the steps: determining a first complete audio block (B2) from the second signal, retrieving the authentication information corresponding to an audio block (B1) which is preceding the first complete audio block (B2) from the memory (600), andsection-wise or pairwise comparing one after the other, going backwards in time, the audio features contained within the retrieved authentication information to the received irregular audio block (C) until the change from the second audio signal to the first audio signal is detected.

4. Method according to any one of the claims 1 , 2 or 3, wherein the comparison of the irregular audio block (C) with the audio features within the authentication information is accomplished by- selecting a suitable type of audio features for the comparison,- extracting features of the selected type from the irregular audio signal block (C),- if the type of the audio features within the authentication information does not correspond to the selected type, transforming these audio features into audio features of the selected type, and- and performing the comparison based on audio features of the selected type.

5. Method of authenticating an audio signal (200) according to any one of the clams 1 to 4, wherein the authentication information comprises a signature (144, 221) and audio features (151) belonging to the current audio block (201).

6. Audio signal authenticator (200) configured to perform a method according to one of claims 1 to 5.

7. Computer program product for authenticating an audio signal, wherein the computer program product comprises program code means causing the audio signal authenticator according to claim 6 to carry out the method of according to any one of the claims 1 to 5.

Images