A vocal challenge in the form of a sentence to understand.

Abstract

The purpose of this disclosure is to describe a computer-based process for verifying a human operator's use of a human-machine interface to access a service. The process comprises: - upon receiving a service access request from a computer system, sending a challenge message based on a sentence containing altered phonemes; - analyzing the response to this challenge message from the computer system to grant or deny access to the service based on the response. (See Figure 5 in the abstract.)

Description

Title of the invention: Method and device for verifying the use of a human / machine interface by a human operator Technical field

[0001] The present description relates to computer security applications, in particular for securing access to a computer service.

[0002] More particularly, it relates to the verification that a human is indeed the originator of a request in an interaction with a computer system, this verification being of the so-called “CAPTCHA” type. Prior art

[0003] Currently in computer interactions (electronic transactions, internet browsing) via a human / machine interface, visual CAPTCHAs are used to verify that it is indeed a human using the aforementioned human / machine interface.

[0004] More particularly, when it comes to voice interactions (for example an interaction on a telephone line with a voice server or more generally with any speech recognition module), an audio CAPTCHA technique is implemented (developed in particular for reasons of accessibility for the visually impaired).

[0005] While visual CAPTCHAs are widely used, the use of audio CAPTCHAs is less developed in proportion, because the proposed solutions are not very reliable.

[0006] An approach based on simple questions or "logic puzzle" or mathematical questions or on sentences requiring a minimum of understanding exists in the form of a service offered to website publishers via an API (for "application program interface").

[0007] However, the effectiveness of such an approach is neither certain nor sustainable in view of the rapid development of machines equipped with artificial intelligence, capable of having a sufficiently high level of semantic understanding to answer such questions.

[0008] It is of course possible to further complicate access to a service by means other than an audio CAPTCHA, but such a measure may discourage users and it remains desirable to preserve the use of the audio CAPTCHA for its simplicity. Summary

[0009] The present description improves the situation.

[0010] A method implemented by computer means is proposed for verifying the use of a human / machine interface by a human operator to access a service, the method comprising: - upon request for access to a service received from a computer entity, send a challenge message based on an altered sentence, said altered sentence comprising phonemes of which at least one phoneme is modified, - analyze a response to said challenge message, received from the IT entity, to grant access or not to the service depending on the response.

[0011] The aforementioned computer entity is normally a human / machine interface controlled by a human, and not a robot (except in the case of non-compliant use of the aforementioned service). Thus, for example, the altered sentence comprising one or more modified phonemes can be chosen from sentences frequently used in everyday life, and can be entered, even if degraded, easily by a human, but with greater difficulty by a robot.

[0012] Thus, this sentence with modified phonemes can contain spaces, for example: "Good..., I am ... happy to make your......sance" (for the sentence without alteration "Hello, I am delighted to make your acquaintance"). A robot-type machine (equipped with a speech detector to try to recognize a sentence spoken orally) can interpret the sentence above, for example, as follows: "Good! I am (in?) life to make your senses", whereas a human naturally interprets it correctly.

[0013] In one embodiment, the method may comprise prior steps: - for each sentence of a plurality of sentences, divide the sentence into phonemes and modify at least one of the phonemes of the sentence, - test the sentence thus altered to estimate a probability score of finding the sentence without alteration from the altered sentence, and - if the score is lower than a threshold, record the altered sentence in a memory.

[0014] Thus, in a current step, the method can include: - upon request for access to a service received by said computer entity, access said memory to send a challenge message based on an altered sentence previously recorded in said memory.

[0015] Thus, the implementation of such an embodiment makes it possible to ensure that a sentence that is simple to understand for a human can be degraded sufficiently for a machine so as not to be identified, by directly testing this degraded sentence on a search engine type machine (examples of such engines being given later).

[0016] Thus, for example, for the implementation of the method according to this embodiment, the aforementioned computer means may comprise at least: - a module for detecting keywords in an altered sentence (for example, a speech detection module for a sentence pronounced acoustically but with at least one alteration), - a search engine of the aforementioned type to try to find in a database or on the Internet for example the complete sentence based on these keywords, and - a memory for storing the altered sentence with modified phonemes (along with other such sentences stored in this memory) in case the original complete sentence cannot be found by the search engine (or at least is not at the top of the search engine's suggestions, as presented later).

[0017] Thus, in such an embodiment, the aforementioned test of the sentence thus altered may comprise: - propose the altered sentence to a search engine, the probability score then being a function of a relevance score of a search result produced by the search engine.

[0018] For example, if the initially intended true sentence (unaltered sentence) is only proposed by the search engine in nth position after other false sentences (not corresponding to the unaltered sentence), then the probability score can be 1 / n and if the threshold is for example 1 / 5, with n>5, then this altered sentence can be stored in memory. On the other hand, if n<5 (i.e. the sentence is proposed between the first and fifth position), then the altered sentence may not be a good candidate to be stored in memory, and can then be: - simply abandoned, or - be processed by the aforementioned computer means to modify more phonemes, or simply, - changing the phonemes that are modified to determine whether the score decreases, as proposed in the embodiment below.

[0019] Indeed, if the probability score is higher than the threshold (for example l / n>l / 5), the method can also include: - modify at least one additional phoneme in the altered sentence, - test the altered sentence to estimate the said probability score, and - repeat the modification of an additional phoneme and the test as long as the probability score is above the threshold.

[0020] In addition or as a variant, it may also be provided to: - change some of the phonemes that are modified in the sentence, - test the sentence including the changed part to estimate said probability score, and - repeat the change of part and the said test as long as the probability score is above the threshold.

[0021] For example, ambiguous phonemes such as "champ" / "chant", or "sens" / "sance" in the word "knowledge", can be preserved without degradation in the altered sentence to be stored in memory, while the less ambiguous phonemes of the sentence can be further degraded.

[0022] As indicated previously, the aforementioned challenge message may include an audio signal comprising data from the altered sentence, this audio signal then being intended to be played by a human / machine interface available to a user (for example on loudspeakers or on an audio headset connected to a smartphone or a computer).

[0023] Such an implementation makes it possible to increase the ambiguity of the sentence for a robot, but less so for a human, as for example with phonemes such as "champ" / "chant", or "sens" / "sance" (in "knowledge").

[0024] Advantageously, the altered sentence can in particular be pronounced in the audio signal with an intonation that is natural for a human.

[0025] For example, the sentence may be spoken with a chosen modulation of the fundamental frequency of the speech signal. Thus, a sentence spoken with a questioning intonation ("what uh--t-he?") may be more easily grasped by a human, but not by a robot (or an emphasis in the sentence "Well..., I'm... vi de faire vo......sance").

[0026] For the realization of the degradations of the phonemes in a sentence, several embodiments can be provided.

[0027] For example, phonemes may be altered in that they may be swapped with each other in the altered phrase, and / or in that a phoneme is replaced by noise over a segment of the same duration as the altered phoneme, and / or replaced by silence of the same duration as the altered phoneme.

[0028] Furthermore, in one embodiment, the aforementioned challenge may consist of guessing the altered sentence. Alternatively, it is possible to provide a challenge in which the user responds to a riddle presented in the altered sentence, or simply to a question (for example: "Can you answer this ambiguous question?": "what time is it?" or "what weather is it today?").

[0029] Furthermore, in one embodiment, in the event of an incorrect response being received to the challenge message, another altered sentence is proposed in a new challenge message (for example by being randomly drawn from the aforementioned memory to be proposed to the aforementioned computer entity, or on the contrary according to a predefined presentation order to the same entity to which the altered sentences are presented). Such an embodiment makes it possible to avoid replaying the same sentence and to ensure that a machine, executing an artificial intelligence for example, cannot attempt n responses if the aforementioned relevance score for this sentence was 1 / n.

[0030] According to another aspect, there is provided a computer program comprising instructions for implementing a method as defined above when this program is executed by a processor. According to another aspect, there is provided a non-transitory recording medium, readable by a computer, on which such a program is recorded.

[0031] According to another aspect, a computer device is provided, comprising a processing circuit configured for implementing the above method. Brief description of the drawings

[0032] Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the attached drawings, in which: Fig.l

[0033] [Fig.l] shows an example of a device for implementing the above method. Fig. 2

[0034] [Fig.2] shows an example of a method comprising a prior operation of constructing altered sentences according to a particular embodiment. Fig. 3

[0035] [Fig.3] shows an example of a method comprising a common operation of proposing a challenge based on one of the altered sentences according to a particular embodiment. Fig. 4

[0036] [Fig.4] shows an example of an initial sentence to be altered. Fig. 5

[0037] [Fig.5] shows a possible alteration of the sentence in [Fig.4]. Description of the embodiments

[0038] Reference is first made to [Fig.l] illustrating an exemplary embodiment of a DIS device for implementing the above method. The DIS device comprises, for example, a microphone MIC for acquiring, via an interface INT, the sound signal of an operator pronouncing a chosen sentence with natural intonation. This sound signal is processed by a speech detection and processing module DET to separate phonemes in the spoken sentence. The DIS device further comprises a processor PROC capable of cooperating with a memory MEM storing the instruction codes of a computer program for implementing the above method, in order to identify, for example, ambiguous phonemes and leave them in the original sentence (acquired via the interface INT), and to replace less ambiguous phonemes in the sentence with noise, for example, white noise or silence.The PROC processor is further configured to, once the aforementioned operation has been carried out, provide the resulting sound data of this altered sentence to a search engine. WORD, for example, in the form of text character data separated by spaces occupying the place of phonemes replaced by noise. The WORD search engine can interpret each sequence of characters before a space as a keyword. The set of keywords in the altered sentence is then subjected to a search on a wide area network NET such as the Internet to identify possible occurrences of the original sentence. More specifically, the search engine assigns a relevance score to the results of this search, this score being reflected in the order in which the true original (unaltered) sentence appears in the proposals returned by the WORD search engine, as described below.If the search score is below a chosen relevance threshold, then this sentence with the altered phonemes can be stored in the MEM memory, with a view to being proposed later as part of a challenge such as recognizing the original sentence or answering a question related to this sentence.

[0039] Alternatively to using an Internet search engine, the keywords identified by the speech detection module DET can be searched in a database of sentences and if none of the sentences in the database are candidates, then the probability score is low (minimum) and the sentence thus altered can be stored in the memory MEM.

[0040] The MEM memory also stores the data of the correct responses to each challenge associated with a sentence in order to be able to compare a response from a computer entity to the correct response to be given to the challenge.

[0041] Indeed, within the meaning of the present disclosure, such an altered sentence (with modified phonemes) is communicated to a computer entity (normally the human / machine interface HMI of a human user) to propose a challenge based on this sentence, and give it access to a service if the response to the challenge is correct (in the manner of a classic CAPTCHA).

[0042] In an exemplary embodiment, a communication interface COM of the DIS device transmits this altered sentence in the form of a sound signal SIG (preferably whose original intonation has been preserved), via a communication network RES, and to a human / machine interface IHM equipped with one or more loudspeakers HP to play the sound signal SIG. If the user of the IHM interface, by listening to the sound signal SIG, recognizes the original sentence, he can respond to the challenge by transmitting a response signal REP to the DIS device which thus acts here as a voice server giving access to a service if the response to the challenge REP is correct.

[0043] It is described below with reference now to [Fig.2] how the altered sentences can be constructed in an exemplary embodiment. During a first step SI, a sentence is selected. This sentence can be chosen by a user, for its frequent nature in everyday life and thus easily recognizable by a human, even altered. In step S2, this sentence is detected and cut by the speech detection module DET. In step S3, the processor PROC determines possibly ambiguous phonemes (such as "fields" / "chant") in the sentence in order to retain them. On the other hand, at least some of the unambiguous phonemes are preferentially noised in step S4. In one embodiment, the sentence thus modified is submitted to the search engine MOT to evaluate a relevance score and if this score is lower than the threshold THR, in step S5, then the sentence can be retained in the memory MEM, in step S6. On the other hand, as long as the relevance score is higher than the threshold THR, more unambiguous phonemes of the sentence are noised (KO arrow at the output of test S5 and noise step S4).

[0044] Once the sentences thus altered are designed and stored in the memory MEM, they can then be used to submit challenges (as a voice CAPTCHA for example). Thus, with reference to [Fig. 3], a first sentence PHRAi can be taken from the memory MEM in step S7 to transmit the corresponding sound signal SIGi via the communication interface COM of the device, in step S8. This sound signal is then provided, with a challenge whose response is known to the device DIS, to a remote computing entity requesting access to a service. Thus, the response to the challenge given by the remote computing entity is received and analyzed in step S9. If the response to the challenge is valid at test S10, access is given to the required service at step S11. Otherwise, another sentence PHRAi+1 is taken from the memory MEM after incrementing the index i (at step S12), to propose a new challenge on the basis of the new sentence PHRAi+1.This process can be repeated a certain number of times (test S13), until communication with this computer entity is cut off at step S14, if the answers given to the successive challenges are still false (and this in order to avoid, for example, a replay by a machine until a search occurrence corresponding to one of the proposed sentences is found).

[0045] Details of the embodiment are presented below.

[0046] In a particular embodiment, an approach is proposed for constructing a voice message to be transmitted to an interface requesting access to a service and which one wishes to submit to a challenge. This message is constructed with altered semantics to make it ambiguous to such a degree that it may be easy for a human to resolve this ambiguity, but not for a machine. It is thus a message comprising “semantic noise” defined below. The proposed solution involves the production of comprehension challenges in a fairly simple voice form, so that a human can respond to the challenge without difficulty, but nevertheless without a machine (however developed it may be) being able to respond to it.

[0047] More particularly, a simple way of introducing a ambiguity in understanding a sentence, of "intellectual" origin so that a machine cannot recognize the sentence, and not "sensory" so that a human can nevertheless recognize this sentence.

[0048] Thus, a sentence presented in vocal form is constructed, chosen in its textual form randomly from, for example, a previously constructed repertoire of sentences, with: - an implementation of a speech detection computer module for annotating and delimiting, in a voice message corresponding to a sentence of several words, the individual words of the voice message, - a sound alteration of one or more segments of the voice message, their number being a parameter whose value can be determined during the following step, - a check that the resulting sentence is not indexed by search engines either in its text form (after voice recognition of the altered voice message) or in its voice / audio form; the aforementioned number of altered segments of the voice message may be increased until the search results present a large number (above a threshold) of sentences different from the original voice message, and - the provision (in a database for example stored in the MEM memory) of the altered voice message, accompanied by a textual and vocal description of the challenge associated with the initial sentence and its resolution.

[0049] The initial voice message may be derived from the recording of a sentence previously spoken by an operator, and acquired by the microphone MIC (as described above with reference to [Fig.l]) or alternatively, audio processing may be implemented to generate a voice message from the text of an initial sentence. This alternative has the advantage of not requiring a human operator for the recording of the initial sentence. Using a human operator for the initial sound recording has the advantage of adding intonation to the audio signal, making the altered sentence more intuitively understandable for a human.

[0050] The proposed voice challenges require the most probable reconstruction of the sentence based on the altered voice message. The voice challenges may, for example, each offer a multiple-choice questionnaire (MCQ), based on different resolutions of the altered parts of the message, for example.

[0051] The DIS device of [Fig.l] thus triggers the sending of a voice challenge and collects the response from a voice interaction, in the context of providing a service between a human user and a server for example, in the typical case where the server seeks to ensure that the client is a human.

[0052] Of course, a sequencing of several voice messages can be provided, altered, to build a single challenge.

[0053] Each of the steps of constructing a sentence with alteration of its vocal version in order to make it semantically ambiguous is detailed below, according to one embodiment, with: - Alteration of a sentence at the level of its vocal representation, - Verification of the ambiguity of the result, - Use of altered sentence to make vocal challenges.

[0054] For a sentence F, we hereinafter denote F_txt its text version and F_voc its vocal version. It is possible to go from F_txt to F_voc by a voice synthesis method known per se, and from F_voc to F_txt by a voice recognition method also known. We consider that a sentence is made up of basic units of first level which are the phonemes, and of second level which are the words, the words being made up of a sequence of several phonemes. A sentence F_txt decomposed into basic units is denoted by F_txt_dec. Thus, to a character string of a certain length we can make correspond an audio signal (or vocal message) of a certain duration as illustrated in [Fig.4].

[0055] Thus, it is possible to establish a correspondence between the basic units that are the phonemes (syllables of the example sentence at the top of [Fig.4]) and specific time segments (acoustic signals corresponding to these syllables at the bottom of [Fig.4]) in the interval that corresponds to the entire sentence. It is possible to locate the beginning and the end of lexical or phonetic units in this time interval. Thanks to this correspondence, it is possible to reference and locate specific moments in the vocal representation of a sentence and act with a desired precision on its representation, in particular with a view to cutting it up.Indeed, after recovering a sentence F in its textual form F_txt and its annotation to explain its decomposition, noted dec(F) into basic units of first-level phonemes and / or second-level words, it is possible for example to generate by voice synthesis the sentence F_voc using its decomposition dec(F) in order to determine the set of time segments, noted I(F_voc), corresponding to each basic unit, i.e. {F_txt, dec(F)} becoming {F_voc, I(F_voc)}.

[0056] After this step, it is possible to carry out a computerized step of grammatical / syntactic analysis of the sentence F in order to assign to each second-level basic unit (words) a weighting coefficient reflecting the importance of this unit for the understanding of the meaning of F by a human. For example, the verb(s) of the sentence (“am”, “do” in the example of figures 4 and 5) may have a higher weight for the understanding of a sentence by a human, relative to other words. The first-level basic units (phonemes) recover the weight of the higher-level unit of which they are part.

[0057] The phonemes are then altered using a transformation / scrambling procedure presented below. An algorithm for altering the sentence F_voc can be applied to this step, which performs the following steps, for example: - set a number K of basic units to alter, - define a rule that adjusts the parameter K to optimize it: the sentence must remain understandable for a human but not for a machine, - randomly draw a list of K basic units of F_txt to be altered {p_l, ..., p_K] among the basic units with the lowest weighting coefficients, - identify the time segments corresponding to these basic units in F_voc, thus moving from the selection {p_l, ..., p_K] to the selection {I_l, ..., I_K], - apply a modification treatment of F_voc to the segments {I_l, ... , I_K}.

[0058] Several alteration schemes are possible, including substitution and permutation.

[0059] A substitution consists of substituting the sound signal in a segment Ij with a typical signal, for example white noise, or a beep, or a silence. In [Fig.5], we have illustrated the case of a substitution of certain phonemes of the sentence of [Fig.4] with silences. Permutation, on the other hand, consists of randomly permuting, or two by two, the segments I_i; I_j. If the permutations are applied to pairs of segments, these pairs can be chosen at random.

[0060] At the end of this process, we obtain a first sentence F_voc_alt which is an audio signal (voice message) corresponding to an altered version of the original sentence F_voc.

[0061] If the original sentence remains comprehensible and in particular indexed by a search engine, it is possible to repeat these steps with an increased number K, for example by altering all the phonemes with low weighting coefficients, and / or by altering phonemes with higher weighting coefficients. Such an implementation, detailed below, can be done iteratively and progressively depending on the weights assigned to the basic units to be altered.

[0062] Indeed, we can then check the level of detectability of the original sentence F in its altered vocal version F_voc_alt. We can apply for this purpose a voice recognition on F_voc_alt to obtain its textual version F txt alt, then search for the sentence F_txt_alt by a set of search engines and determine if F_txt appears among the first results of the searches with in particular a determination of the rank R (giving the aforementioned relevance score) of this occurrence in the list of proposals that the search engines return. For example, for rank thresholds si, s2 for different engines Ml, M2, with si < s2: - if R < si, then it can be determined that F is detectable; - if si < R < s2, then F is potentially detectable and - if R > s2 then F is not detectable.

[0063] As indicated previously, the aforementioned probability score can be a function of this rank R, and typically be inversely proportional to this rank (for example in 1 / R).

[0064] We can repeat the steps of successive alterations in the sentence and search by the engines to refine, for a given sentence, the number K of target alterations in order to arrive at the last result above (R > s2).

[0065] As a possible application, a call management server, for example, which transfers incoming calls to a pool of operators, may need to verify that the customers are humans and not machines. The management server then uses a voice challenge management service. A bot can then interact vocally with the customers and execute an API to request a challenge and then submit it to a customer. After receiving the response proposed by the customer, the API transmits this response to a challenge management server to determine whether the proposed response is valid or not. If it is valid, the customer is considered to be a human and their call is transferred to the requested pool of operators. Otherwise, the server can end the call or alternatively an additional challenge is launched (this process being repeated a maximum number of times before abandoning the call).

[0066] Of course, other applications are possible in any sector involving interactions with servers, in particular with voice servers, for conversational interactions for example with reservation, payment or other systems on the internet. The “acoustic” realization of the challenge can be proposed in particular but not exclusively for applications of interactions of visually impaired people with access servers via the internet.

Claims

Claims

1. Method implemented by computer means, for verifying the use of a human / machine interface by a human operator to access a service, the method comprising: - upon request for access to a service received from a computer entity, sending (S8) a challenge message based on an altered sentence, said altered sentence comprising phonemes of which at least one phoneme is modified (S4), - analyzing (S10) a response to said challenge message, received from the computer entity (S9), to grant access (SI 1) or not to the service depending on the response.

2. Method according to claim 1, comprising beforehand: - for each sentence of a plurality of sentences, cutting (S2) the sentence into phonemes and modifying (S4) at least one of the phonemes of the sentence, - testing (S5) the sentence thus altered to estimate a probability score of finding the sentence without alteration from the altered sentence, and - if the score is lower than a threshold (S5), recording (S6) the altered sentence in a memory, and in which: - upon request for access to a service received by said computer entity, accessing said memory (S7) to send a challenge message based on an altered sentence and previously recorded in said memory.

3. Method according to claim 2, in which testing said sentence thus altered (S5) comprises: - proposing the altered sentence to a search engine, the probability score being a function of a relevance score of a result of the search produced by the search engine.

4. Method according to one of claims 2 and 3, wherein, if the probability score is greater than the threshold, the method further comprises: - modifying (S4) at least one additional phoneme in the altered sentence, - testing (S5) the sentence thus altered to estimate said probability score, and - repeating the modification of an additional phoneme and the test as long as the probability score is greater than the threshold.

5. Method according to one of claims 2 to 4, in which, if the probability score is greater than the threshold, the method further comprises: - changing a part of the phonemes that are changed in the sentence, - testing the sentence including the changed part to estimate said probability score, and - repeating the change of part and said test as long as the probability score is above the threshold.

6. Method according to one of the preceding claims, in which the challenge message comprises an audio signal comprising data of the altered sentence, said audio signal being intended to be played by a human / machine interface available to a user.

7. The method of claim 6, wherein the altered phrase is spoken in the audio signal with natural intonation for a human.

8. Method according to one of the preceding claims, in which modifying at least one phoneme comprises a permutation of at least two phonemes between them.

9. Method according to one of claims 6 and 7, in which modifying at least one phoneme comprises replacing this phoneme with noise over a segment of the same duration as the modified phoneme.

10. Method according to one of claims 6 and 7, in which modifying at least one phoneme comprises replacing this phoneme with a silence over a segment of the same duration as the modified phoneme.

11. A method according to any preceding claim, wherein the challenge is to guess the altered sentence.

12. Method according to one of the preceding claims, in which, in the event of an incorrect response received to the challenge message, another altered sentence is proposed in a new challenge message.

13. Computer program comprising instructions for implementing the method according to one of the preceding claims when this program is executed by a processor.

14. Computer device, comprising a processing circuit configured for implementing the method according to one of claims 1 to 12.

15. Non-transitory recording medium readable by a computer on which is recorded a program for implementing the method according to one of claims 1 to 12 when this program is executed by a processor.

Images