Natural Language Processing with Quantum Random Number Generators

Abstract

To provide a method and system for natural language processing that allows for eliminating restrictions on the existing chatbot and providing a chatbot to offer user experience more approximate to conversation with a human.SOLUTION: A method for natural language processing includes a step S10 of receiving a sample including a natural language, a step S12 of processing the sample to generate a plurality of response hypotheses and a plurality of confidence values so that each response hypothesis is associated with a corresponding confidence value, and a step S14 of selecting a response, including randomly selecting a response from among a plurality of response hypotheses based at least partially on a corresponding confidence value by a quantum random number generator.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to techniques for natural language processing using quantum random number generators, and in particular to techniques for creating conversational chatbots with quantum random number generators. [Background technology]

[0002] A chat robot (or chatbot) is a device that simulates human conversation using natural language input, such as text or voice input. Chatbots may be implemented in dialog systems to perform a variety of practical tasks, including automated customer support or information retrieval. Traditionally, many sophisticated chatbots utilize artificial intelligence and machine learning techniques, such as deep neural networks, and may use probabilistic techniques. The ultimate goal is to present the user with an experience similar to conversing with a human, but all current chatbots still have significant limitations and fall short of that goal. In particular, many existing chatbots reveal artifacts that reveal their underlying algorithms and give the user the impression that their requests are being handled by a machine rather than by a human. Summary of the Invention [Problem to be solved by the invention]

[0003] Given the current state of the art, there remains a need for natural language processing techniques that remove the limitations of existing chatbots and provide chatbots that present a user experience that is closer to a human conversation.

[0004] This object is achieved by a method for natural language processing according to independent claim 1 and by a system for natural language processing according to independent claim 10. The dependent claims relate to preferred embodiments. [Means for solving the problem]

[0005] In a first aspect, the disclosure relates to a method for natural language processing comprising: receiving a sample comprising natural language; processing the sample, the step including generating a plurality of response hypotheses and generating a plurality of confidence values, each response hypothesis being associated with a corresponding confidence value; and selecting a response, the step including randomly selecting a response from among the plurality of response hypotheses based at least in part on the corresponding confidence value by a quantum random number generator.

[0006] Quantum objects are known to constitute the only carriers of true randomness, and quantum random number generators can exhibit truly unpredictable behavior. The use of quantum random number generators in selecting responses among multiple response hypotheses can greatly improve the capabilities of natural language processing, in particular enabling chatbots to output responses that avoid repetitive patterns, as well as mimic the response behavior and free will of a human dialogue partner much better than can be achieved with conventional natural language processing techniques.

[0007] In the context of the present disclosure, a quantum random number generator may refer to a hardware random number generator capable of generating random numbers from a quantum physical process. In particular, a quantum random number generator may be a hardware device adapted to implement a quantum physical process and further adapted to derive random numbers based on the quantum physical process.

[0008] Quantum physics predicts that certain physical phenomena are fundamentally random and cannot be predicted in principle. These phenomena may include nuclear decay, shot noise, or photons passing through a semi-transparent mirror. In general, all quantum phenomena that exhibit true unpredictability can be used in the quantum random number generator according to the present disclosure. Quantum optical processes are often very convenient to implement and relatively easy to handle, and therefore provide good examples of quantum random number generators for many practically relevant scenarios.

[0009] An overview of the fundamentals of quantum random number generators and example systems that can be used in the context of the present disclosure can be found in M. Herrero-Collantes, JC Garcia-Escartin, "Quantum Random Number Generators", Reviews of Modern Physics 89 (2017) 015004 (also available from the Cornell preprint server, arXiv:1604.03304).

[0010] According to one embodiment, the step of selecting a response includes receiving a random number from a quantum random number generator.

[0011] In particular, the step of selecting a response may include randomly selecting a response from among a plurality of response hypotheses based at least in part on the corresponding confidence value and the received random number.

[0012] According to one embodiment, the step of selecting a response comprises invoking or querying a quantum random number generator.

[0013] In response to an invocation or query of the quantum random number generator, the quantum random number generator can output at least one random number that can be used in randomly selecting a response from among a number of response hypotheses.

[0014] In other embodiments, a quantum random number generator may provide random numbers without being actively called or queried, for example, a quantum random number generator may provide a random number or numbers at regular time intervals.

[0015] In particular, the step of selecting a response may include receiving at least one random number from a quantum random number generator at regular time intervals.

[0016] According to one embodiment, the step of selecting a response includes generating a probability distribution for a number of response hypotheses with corresponding weights based on their respective confidence values.

[0017] The respective weights may be selected proportional to the respective trust values ​​on which they are based. In particular, the respective weights may be selected from the trust values ​​by normalizing the trust values ​​so that they sum to one.

[0018] The step of selecting a response may further include selecting among the plurality of response hypotheses randomly according to the corresponding weights by a quantum random number generator.

[0019] According to one embodiment, selecting a response includes comparing the corresponding confidence value to a predetermined threshold and, if the corresponding confidence value is less than the predetermined threshold, randomly selecting, by a quantum random number generator, a response from among a plurality of response hypotheses based at least in part on the corresponding confidence value.

[0020] In particular, the method may involve randomly selecting, by a quantum random number generator, a response from among a plurality of response hypotheses based at least in part on the corresponding confidence value only if the corresponding confidence value is less than a predetermined threshold.

[0021] In this way, a response can be randomly selected from among multiple response hypotheses only if all of the respective confidence values, or all of the corresponding weights, are less than a predetermined threshold.

[0022] A response may be deterministically selected from among this subset of response hypotheses if at least one of the multiple response hypotheses has a corresponding confidence value or a corresponding weighting that is equal to or greater than a predetermined threshold.

[0023] In particular, in this case, the response may be selected as the response hypothesis having the greatest confidence value or greatest weight.

[0024] Thus, according to one embodiment, the method includes deterministically selecting a response from among a plurality of response hypotheses based at least in part on the corresponding confidence value if the corresponding confidence value is greater than or equal to a predetermined threshold.

[0025] These techniques can balance deterministic and probabilistic methods for selecting responses and can ensure that responses with very high confidence values ​​are prioritized.

[0026] According to one embodiment, selecting a response may include pre-selecting a number of response hypotheses according to predetermined criteria.

[0027] Pre-selecting a plurality of response hypotheses may include discarding response hypotheses according to predetermined criteria.

[0028] For example, toxic responses or responses that do not meet the suitability requirements for an acceptable response may be discarded.

[0029] The preselecting step may include lowering the confidence value or weight of each response hypothesis, in particular setting the confidence value or weight of each response hypothesis to zero.

[0030] In the context of this disclosure, natural language may include any form of language or signal representing language, including written text and / or audio signals.

[0031] The samples may be understood to represent any string or word in a natural language and may be provided in any format amenable to electronic data processing, for example, the samples may represent utterances or questions that a human user may utter when conversing with a chatbot.

[0032] In particular, the method may include receiving a sample from a human user.

[0033] According to one embodiment, the sample may be annotated sample.

[0034] In the context of this disclosure, a sample may be referred to as an annotated sample when it is processed by a natural language processing (NLP) model, the output of which may aid in generating candidate responses or selecting a response rather than directly generating a response.

[0035] In particular, the method may include the step of annotating the sample.

[0036] According to one embodiment, samples may be annotated before generating multiple response hypotheses and generating multiple confidence values.

[0037] Annotating the samples may include spelling preprocessing and / or sentence segmentation.

[0038] In the context of this disclosure, a response hypothesis represents a candidate response to a sample.

[0039] The confidence value may represent the probability or likelihood that the corresponding response hypothesis is a significant response for the sample.

[0040] According to one embodiment, the method may further comprise the step of returning the selected response, in particular returning the selected response to a human user.

[0041] The multiple response hypotheses may be generated by any response generation technique, in particular any natural language processing technique.

[0042] According to one embodiment, processing the sample may include generating a plurality of response hypotheses by at least two different response generation techniques.

[0043] The different response generation techniques may optionally include probabilistic and / or deterministic response generation techniques.

[0044] In particular, processing the sample may include generating at least a portion of the plurality of response hypotheses by a neural network.

[0045] Any of the different response generation techniques can generate one or more response hypotheses, and the method can then select from among these response hypotheses.

[0046] According to one embodiment, generating a plurality of response hypotheses may include generating at least a portion of the plurality of response hypotheses with a quantum random number generator.

[0047] In this way, the excellent properties of quantum random number generators can be exploited not only in selecting a response from multiple response hypotheses, but also in generating the response hypotheses in the first place.

[0048] According to one embodiment, the step of generating a plurality of response hypotheses comprises randomly selecting words of the response hypotheses, in particular by means of a quantum random number generator.

[0049] In one embodiment, generating the plurality of response hypotheses includes receiving a random number from a quantum random number generator and generating at least a portion of the plurality of response hypotheses based on the received random number.

[0050] The step of generating a number of response hypotheses may include querying or invoking a quantum random number generator.

[0051] In other embodiments, a quantum random number generator may provide random numbers without being actively called or queried, for example, a quantum random number generator may provide a random number or numbers at regular time intervals.

[0052] In particular, the step of generating a plurality of response hypotheses may include receiving at least one random number from a quantum random number generator at regular time intervals.

[0053] According to one embodiment, the method may be a computer-implemented method.

[0054] In a second aspect, the present disclosure relates to a computer program, or a computer program product, comprising computer readable instructions which, when read on a computer, cause the computer to carry out a method having some or all of the above features.

[0055] A computer program product may be or may comprise a computer readable medium.

[0056] In a third aspect, the present disclosure relates to a system for natural language processing comprising: a receiving unit adapted to receive a sample comprising natural language; a processing unit adapted to process the sample, the processing unit adapted to generate a plurality of response hypotheses and to generate a plurality of confidence values, each response hypothesis being associated with a corresponding confidence value; and a selection unit adapted to select a response, the selection unit being adapted to randomly select a response from among the plurality of response hypotheses based at least in part on the corresponding confidence values ​​by a quantum random number generator.

[0057] According to one embodiment, the system, and in particular the selection unit, is coupled to a quantum random number generator and adapted to receive the random numbers from the quantum random number generator.

[0058] The selection unit may be adapted to randomly select a response from among the plurality of response hypotheses based at least in part on the corresponding confidence value and the received random number.

[0059] According to one embodiment, the selection unit is adapted to invoke or query a quantum random number generator.

[0060] In other embodiments, the quantum random number generator may provide random numbers to the system, and in particular the selection unit, without being actively called or queried, for example, the quantum random number generator may provide a random number or numbers at regular time intervals.

[0061] Thus, the system, and in particular the selection unit, may be coupled to a quantum random number generator and adapted to receive at least one random number from the quantum random number generator at regular time intervals.

[0062] In one embodiment, the system may include a quantum random number generator.

[0063] According to one embodiment, the selection unit is adapted to generate a probability distribution of a plurality of response hypotheses with corresponding weights based on the confidence values, and the selection unit is further adapted to randomly select among the plurality of response hypotheses according to the corresponding weights by a quantum random number generator.

[0064] The selection unit may be configured to compare the corresponding confidence value with a predetermined threshold and, if the corresponding confidence value is less than the predetermined threshold, to randomly select, by the quantum random number generator, a response from among the plurality of response hypotheses based at least in part on the corresponding confidence value.

[0065] In particular, the selection unit may be adapted to randomly select a response from among the plurality of response hypotheses based at least in part on the corresponding confidence value by the quantum random number generator only if the corresponding confidence value is less than a predefined threshold.

[0066] According to one embodiment, the selection unit is adapted to deterministically select a response from among the plurality of response hypotheses based at least in part on the corresponding confidence value if the corresponding confidence value is greater than or equal to a predefined threshold.

[0067] In one embodiment, the selection unit may be adapted to pre-select response hypotheses according to predetermined criteria. In particular, the selection unit may be adapted to discard response hypotheses according to predetermined criteria.

[0068] According to one embodiment, the system may further comprise an output unit adapted to return the selected response.

[0069] In one embodiment, the processing unit may be adapted to generate a plurality of response hypotheses according to different response generation techniques, which may optionally include probabilistic response generation techniques and / or deterministic response generation techniques.

[0070] According to one embodiment, the processing unit may comprise a neural network adapted to generate at least a portion of the plurality of response hypotheses.

[0071] In one embodiment, the processing unit is adapted to generate at least a portion of the plurality of response hypotheses by a quantum random number generator.

[0072] In particular, the processing unit may be adapted to select the words of the response hypotheses randomly, in particular by means of a quantum random number generator.

[0073] According to an embodiment, the processing unit may be adapted to receive the random numbers from a quantum random number generator.

[0074] In particular, the processing unit may be further adapted to generate a plurality of response hypotheses at least in part based on the received random number.

[0075] In one embodiment, the processing unit may be adapted to query or invoke a quantum random number generator.

[0076] In other embodiments, a quantum random number generator may provide random numbers to a processing unit without being actively called or queried, for example, a quantum random number generator may provide a random number or numbers at regular time intervals.

[0077] Thus, the processing unit may be adapted to receive at least one random number from the quantum random number generator at regular time intervals.

[0078] According to one embodiment, the quantum random number generator is connected to the selection unit and / or the processing unit by a communication line.

[0079] The system may further comprise an annotator unit adapted to annotate the samples, in particular to annotate the samples before generating the plurality of response hypotheses and generating the plurality of confidence values.

[0080] According to one embodiment the receiving unit and / or the processing unit and / or the selection unit and / or the output unit and / or the annotator unit may be implemented in hardware. In other embodiments the receiving unit and / or the processing unit and / or the selection unit and / or the output unit and / or the annotator unit may be implemented in software or firmware. In yet other embodiments the receiving unit and / or the processing unit and / or the selection unit and / or the output unit and / or the annotator unit may be implemented partly in hardware and partly in software or firmware.

[0081] In some embodiments, the receiving unit and / or the processing unit and / or the selection unit and / or the output unit and / or the annotator unit are separate stand-alone units that may be communicatively coupled to one another, in other embodiments at least two of these units may be incorporated into a common unit. [Brief description of the drawings]

[0082] The features and associated advantages of the present disclosure are best understood from the following detailed description of exemplary embodiments in connection with the accompanying drawings. [Figure 1] FIG. 1 is a schematic diagram of a system for natural language processing according to one embodiment. [Diagram 2] FIG. 1 is a flow diagram illustrating a method for natural language processing according to one embodiment. [Diagram 3] FIG. 1 is a schematic diagram of a specific implementation of a system for natural language processing according to one embodiment. [Figure 4] 1 illustrates a technique for generating response hypotheses according to one embodiment. [Diagram 5] 1 is a decision tree illustrating a method for selecting a response among multiple response hypotheses according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0083] The techniques of this disclosure will now be described primarily in the context of an example chatbot system 10, 10' that receives samples from a user that include natural language, such as written text, speech, or other forms of signals, processes the samples, and generates responses that can be output to the user. However, the techniques of this disclosure may also be applied to a variety of other contexts of natural language processing, or information processing in general.

[0084] 1 is a schematic diagram of a system 10 for natural language processing according to one embodiment. System 10 comprises a receiving unit 12, a processing unit 14 communicatively coupled to receiving unit 12, and a selection unit 16 communicatively coupled to processing unit 14.

[0085] The receiving unit 12 is adapted to receive samples 18 comprising natural language. For example, the samples 18 may be received from a human user (not shown) and may comprise natural language in the form of written text. In these examples, the receiving unit 12 may comprise an input front end having a keyboard or some other type of text input interface. In other embodiments, the samples 18 may comprise natural language in the form of audio signals and the receiving unit 12 may comprise an audio microphone or some other form of audio input interface. In further embodiments, the samples 18 may be received from an external data processing unit (not shown) and may comprise text representing natural language or data signals representing audio.

[0086] 1 may be adapted to receive samples 18 from the receiving unit 12, and is adapted to process the samples 18. In particular, the processing unit is adapted to generate a plurality of response hypotheses 20 and generate a plurality of confidence values ​​22, each response hypothesis 20 being associated with a corresponding confidence value 22.

[0087] According to one embodiment, response hypotheses 20 may represent candidate responses to sample 18. Response hypotheses 20 may be generated by a variety of different (deterministic or probabilistic) response generation techniques, including artificial intelligence and machine learning. Confidence values ​​22 may represent the probability or likelihood that the corresponding response hypotheses 20 are significant responses to sample 18 according to predetermined criteria.

[0088] As further shown in FIG. 1, the selection unit 16 may be communicatively coupled to a quantum random number generator 24 via a data connection 26. In the context of the present disclosure, the quantum random number generator 24 may be understood as a hardware random number generator that may be adapted to generate random numbers from a quantum physical process, such as a quantum optical process. For example, in the quantum random number generator 24, photons may be generated by a laser and sent through a beam splitter, so that the photons may end up in one of two different output branches where they may be detected by a photon detector. The probability that a photon will be detected in a given branch depends on the properties of the beam splitter, but the laws of quantum physics do not allow to predict, for each individual photon, in which output branch the photon will be detected. Thus, the outcome is probabilistic at a fundamental level, depending on the current state of physics. Thus, assigning an output value "0" to a first output branch and an output value "1" to a second output branch results in a sequence of binary random numbers. The underlying probability distribution depends on the properties of the beam splitter. For example, if the beam splitter is a 50 / 50 beam splitter, the sequence of binary output numbers is independent and identically distributed (iid). Another physical process that exhibits true randomness and can be used in a quantum random number generator 24 according to the present disclosure involves the change in electron population in a semiconductor.

[0089] Different quantum random number generators that can be used in the context of the present disclosure are described in M. Herrero-Collantes, JC Garcia-Escartin, "Quantum Random Number Generators", Reviews of Modern Physics 89 (2017) 015004 (arXiv:1604.03304), and references cited therein.

[0090] In some embodiments, the quantum random number generator 24 may be a unit external to the system 10 and communicatively coupled to the selection unit 16 by a data connection 26. In other embodiments, the quantum random number generator 24 may form part of the system 10 or may be integrated into the selection unit 16.

[0091] The selection unit 16 may be adapted to receive a plurality of response hypotheses 20 and a plurality of associated confidence values ​​22, and may be adapted to randomly select a response 28 from among the plurality of response hypotheses 20 based at least in part on the corresponding confidence values ​​22 by a quantum random number generator 24. For example, the selection unit 16 may be adapted to invoke or query the quantum random number generator 24 via the data connection 26. In response to the query, the quantum random number generator 24 may provide a random number or numbers to the selection unit 16 via the data connection 26, and the selection unit 16 may randomly select from among the plurality of response hypotheses 20 based at least in part on the corresponding confidence values ​​22 and the received random number.

[0092] The system 10 may further be adapted to output a selected response 28 to the user as a response to the sample 18 .

[0093] FIG. 2 is a flow diagram illustrating a method for natural language processing according to one embodiment, such as the method described above in connection with chatbot system 10.

[0094] In a first step S10, a sample containing natural language is received.

[0095] In a subsequent second step S12, the method includes processing the received samples, where processing the samples includes generating a plurality of response hypotheses and generating a plurality of confidence values, each response hypothesis being associated with a corresponding confidence value.

[0096] In a subsequent third step S14, the method includes a step of selecting a response, the step including randomly selecting a response from among a plurality of response hypotheses based at least in part on a corresponding confidence value by a quantum random number generator.

[0097] 1 and 2, the chatbot system 10 can enter into a dialogue with the user that includes multiple utterances and responses. The chatbot system 10's reliance on the quantum random number generator 24 ensures that the responses 28 generated by the chatbot system 10 contain an element of true randomness and unpredictable behavior that cannot be simulated by classical (pseudo)random number generators, and thus can reflect the free will of the human conversation partner.

[0098] FIG. 3 is a schematic diagram illustrating a system 10' for natural language processing at a greater level of detail.

[0099] System 10' generally corresponds to system 10 described above in connection with Figures 1 and 2, and corresponding elements have the same reference numbers.

[0100] In particular, the system 10' comprises a receiving unit 12 adapted to receive a sample 18 containing a natural language, a processing unit 14 and a selection unit 16 connected to a quantum random number generator 24 by a data connection 26, as described above in relation to Figure 1.

[0101] However, the system 10′ further comprises an annotator unit 30 downstream of the receiving unit 12 and upstream of the processing unit 14. The annotator unit 30 is adapted to receive the samples 18 from the receiving unit 12 and to annotate the samples 18. For example, the annotator unit 30 may comprise a spelling unit 30a and a segmentation unit 30b, which may be adapted to receive the samples 18 from the receiving unit 12 and to pre-process the samples 18.

[0102] The spelling unit 30a may be adapted to perform spelling pre-processing, such as to correct typos in the input sample 18. The spelling unit 30a may utilize an automatic spelling correction pipeline provided by DeepPavlov.ai, an open-source conversational AI framework developed by the Neural Networks and Deep Learning Lab at the Moscow Institute of Physics and Technology (see http: / / docs.deeppavlov.ai / en / 0.17.0 / features / models / spelling_correction.html). However, other typo correction techniques may be used as well.

[0103] The segmentation unit 30b may be adapted for sentence segmentation, in particular for segmenting the input text of the sample 18 into individual sentences. Furthermore, the segmentation unit 30b may implement techniques for named entity recognition (NER) to classify tokens in the sample 18 into predefined categories, such as person names, quantity expressions, percentage expressions, names of locations, and / or organizations. More information on these techniques is available from DeepPavlov.ai (see http: / / docs.deeppavlov.ai / en / 0.17.0 / features / models / ner.html#ner-based-model-for-sentence-boundary-detection-task). However, other techniques for sentence segmentation may be used as well.

[0104] The annotator unit 30 may provide the resulting annotated samples 32 to the processing unit 14. As discussed above in relation to Figures 1 and 2, the processing unit 14 is adapted to process the annotated samples 32, such that the processing unit 14 is adapted to generate a number of response hypotheses 20a-20e and corresponding confidence values ​​22a-22e.

[0105] To that end, the processing unit 14 may comprise a number of hypothesis generation units 34a-34e, each of which may be adapted to generate at least one, and possibly a number of, response hypotheses 20a-20e. The hypothesis generation units 34a-34e may implement different skills, i.e. different deterministic or probabilistic algorithms, for generating a response or set of responses derived from the input samples 18 and their annotations.

[0106] In the example shown in FIG. 3, the processing unit 14 comprises five different hypothesis generation units 34a-34e, however this is by way of example only and the processing unit 14 may generally comprise any number of hypothesis generation units.

[0107] The first hypothesis generation unit 34a may correspond to the Free ALICE AIML chatbot set available from the Google Code Archive at https: / / code.google.com / archive / p / aiml-en-us-foundation-alice / and generate the first response hypotheses 20a.

[0108] Similarly, hypothesis generation unit 34b may correspond to the Eliza chatbot, described in more detail by Joseph Weizenbaum in "ELIZA - A Computer Program for the Study of Natural Language Communication Between Man and Machine", Communications of the ACM, volume 9, Number 1 (January 1966) pp. 36-45, and generate second response hypotheses 20b.

[0109] The third hypothesis generation unit 34c may implement an open domain question answering (ODQA) model that answers questions based on Wikipedia entries. More information about the ODQA algorithm is available from DeepPavlov.ai, http: / / docs.deeppavlov.ai / en / 0.17.0 / features / skills / odqa.html. However, other techniques for retrieving information from Wikipedia or other online encyclopedias can be used as well. In response to the sample questions 18, the third hypothesis generation unit 34c outputs third answer hypotheses 20c.

[0110] The fourth hypothesis generation unit 34d may implement a knowledge-based question answering (KBQA) model adapted to answer any question based on the Wikidata knowledge base. More information on the KBQA technique is available at DeepPavlov.ai, http: / / docs.deeppavlov.ai / en / 0.17.0 / features / models / kbqa.html. As shown in FIG. 3, the hypothesis generation unit 34d may output two response hypotheses 20d, 20d′, both of which match the sample 18.

[0111] The fifth hypothesis generation unit 34e may employ a hypothesis generation technique based on generative pre-trained transformers (GPT), an autoregressive language model that uses deep learning techniques to generate human-type text as the fifth response hypothesis 20e. These techniques are described in more detail in "Release Strategies and the Social Impact of Language Models" by I. Solaiman et al., available from the Cornell University Preprint Server (November 13, 2019) at https: / / arxiv.org / abs / 1908.09203.

[0112] In general, any number of hypothesis generation units may be employed in the processing unit 14, each of which may implement a different algorithm to generate at least one response hypothesis. The hypothesis generation units may use both deterministic and probabilistic approaches. If one or more of the plurality of hypothesis generation units 34a-34e use probabilistic techniques to generate their respective response hypotheses 20a-20e, they may, in some embodiments, utilize a quantum random number generator, such as the quantum random number generator 24.

[0113] In particular, the techniques of the hypothesis generation unit 34e may be extended beyond conventional GPT to incorporate a quantum random number generator 24 as a source of randomness, as described in further detail below in connection with FIG.

[0114] For example, the hypothesis generation unit 34e may call or query the quantum random number generator 24 and may receive quantum random numbers from the quantum random number generator 24 in substantially the same manner as described above with reference to Figure 1 in relation to the selection unit 16. In particular, the hypothesis generation unit 34e may use the random numbers received from the quantum random number generator 24 to randomly select words or letters according to a probability distribution when generating the response hypotheses 20e.

[0115] 4 is a schematic flow diagram illustrating how the hypothesis generation unit 34e may iteratively generate response hypotheses 20e by a token prediction model.<T0,T1,T2> Assuming that has already been selected in the previous step, the token<T0,T1,T2> is an input sequence of tokens<T0,T1,T2> The distribution F(t3) is used as an input for the GPT module 36, which implements the GPT model to compute a distribution F(t3) that depends on . The distribution F(t3) is a distribution in the configuration space of symbols and groups of symbols from which a subsequent token T3 may be selected. A particular implementation of the GPT module 36 that may be selected in this context is the DialoGPT module available from Microsoft, as described in more detail in "DialoGPT: Large-Scale Generative Pre-Training for Conversational Response Generation" by Y. Zhang et al., available from the Cornell preprint server arXiv:1911.00536 v 3 (May 2, 2020). To select a subsequent token T3 according to the distribution F(t3), the GPT module 36 may invoke the quantum random number generator 24. In particular, the GPT module 36 may query the quantum random number generator 24 and in return receive a random number or multiple random numbers from the quantum random number generator 24. Here, the GPT module 36 can use random numbers to sample tokens T3 according to a distribution F(t3). <T0、T1、T2、T3、...、T M > corresponds to response hypothesis 20e.

[0116] Optionally, the distribution F(t3) may be shape-shifted by using temperature techniques described in I. Solaiman et al., "Release Strategies and the Social Impact of Language Models," available from the Cornell University Preprint Server (November 13, 2019) at https: / / arxiv.org / abs / 1908.09203.

[0117] Returning to Figure 3, the processing unit 14 further comprises a hypothesis annotator unit 38 adapted to receive the plurality of answer hypotheses 20a-20e generated by the plurality of hypothesis generation units 34a-34e. The hypothesis annotator unit 38 may be adapted to assign a confidence value 22a-22e to each of the plurality of answer hypotheses 20a-20e.

[0118] In particular, the hypothesis annotator unit 38 encodes the plurality of response hypotheses 20a-20e to provide a real number c between 0 and 1 for each of the i=1,...,n response hypotheses 20a-20e. i The text encoder unit 38a may be adapted to assign the number c i may represent confidence values ​​22a-22e, i.e., the probability that corresponding response hypothesis i is the correct response to sample 18. In the example discussed above in connection with FIG. 3, n=6, but in general, n may represent any positive integer.

[0119] According to one embodiment, the text encoder unit 38a may be selected as the PolyAI sentence encoding model ConveRT, as described in more detail in "ConverRT: Efficient and Accurate Conversational Representations from Transformers," by M. Henderson et al., work of the Association for Computational Linguistics, EMNLP 2020, pp. 2161-2174, also available from the Cornell University preprint server arXiv:1911.03688 (April 29, 2020). However, a different evaluation model may be used instead.

[0120] The hypothesis annotator 38 may also optionally comprise a toxicity unit 38b capable of implementing a Deep Pavlov-based regression model adapted to measure the level of aversion, harm, threat, etc. in the response hypotheses 20a-20e. The toxicity unit 38b may then, for each i=1,...,n, calculate a corresponding real number ti may be assigned to each of the response hypotheses 20a to 20e. i may represent the toxicity values ​​of response hypotheses 20a-20e.

[0121] As further shown in FIG. 3, the hypothesis annotator unit 38 generates tuples of corresponding response hypotheses 20a-20e, as well as corresponding confidence values ​​c i and toxicity i is provided to the selection unit 16.

[0122] As described above in relation to Figures 1 and 2, the selection unit 16 is configured to randomly select a response 28 from among a plurality of response hypotheses 20a-20e based on associated confidence values ​​22a-22e by invoking a quantum random number generator 24 connected to the selection unit 16 via a data connection 26.

[0123] In one exemplary embodiment, the selection unit 16 is

number

[0124] In some embodiments, the probability distribution (p i ) i Further rules can be added to modify

[0125] In implementing the embodiment described above in relation to FIG. 3, the selection unit 16 selects toxicity values ​​t 1 that are higher than a predefined threshold T. i and classifying the response hypotheses 20a-20e having the confidence values ​​ci Toxic response hypotheses 20a-20e may be discarded by setting x to 0, thereby effectively preventing these response hypotheses 20a-20e from being selected as responses 28. For example, the response hypothesis "I hate you" may be assigned a high toxicity value by toxicity unit 38b and therefore may be classified as toxic and may be excluded from the set of response hypotheses 20a-20e.

[0126] In another variation of the embodiment described above in relation to FIG. 3, the selection unit 16 selects the confidence value c i may be compared to a predetermined threshold d, and further i d. If this subset is not empty, the response 28 is determined to be the one with the highest confidence value c i , the response hypothesis having the highest confidence level may be selected only in the subset of samples for which processing unit 14 can determine the correct response with a high degree of confidence. This technique can ensure that the response is deterministically selected for samples for which processing unit 14 can determine the correct response with a high degree of confidence.

[0127] On the other hand, c i If the subset of response hypotheses 20a-20e that satisfies >d is empty, the selection unit 16 may be configured to randomly select a response 28 from among the response hypotheses 20a-20e by invoking a quantum random number generator 24, as described above in relation to FIG. 3.

[0128] A flow diagram illustrating the decision tree implemented in the logic of the selection unit 16 is shown in FIG.

[0129] The flow diagram in FIG. 5 illustrates the preselection of response hypotheses 20a-20e according to toxicity criteria and the selection of hypotheses with confidence values ​​c greater than a given threshold d. i , and a deterministic selection from the remaining response hypotheses 20a-20e with confidence value c iIf and only if d=0.97 is not satisfied, then a response 28 is randomly selected by invoking the quantum random number generator 24 using the normalized confidence values ​​according to equation (1) as weights in the probability distribution.

[0130] Given that the selection unit 16 randomly selects responses 28 to the samples 18 by invoking the quantum random number generator 24, it is essentially impossible to predict how the system 10' will respond next. As a further consequence, the system 10' will exhibit variability in responses and will typically respond to the same dialogue in different ways, which may reflect how a free person would respond. At the same time, the samples 18 that can identify the correct response with a high probability can be deterministically answered, thereby increasing the reliability of the chatbot.

[0131] In some embodiments, the selection unit 16 may alternatively be adapted to implement a tree-based method such as a neural network, gradient boosting (J. H. Friedman: Greedy Function Approximation: Gradient Boosting Machines. Annals of Statistics 29(5) (2001) pp. 1189-1232), random forests (T. K. Ho: Random Decision Forests. Proceedings of the 3rd International Conference on Document Analysis and Recognition (1995) pp. 278-282), or another machine learning algorithm to select among the response hypotheses 20a-20e, and the neural network may invoke a quantum random number generator 24.

[0132] For example, the selection unit 16 can use vectors that encode the context and each possible response. These tensors can be used to find the cosine similarity between the context and each given response by computing the dot product of the vector corresponding to the context and the particular response with a trainable Gram matrix. The cosine similarity and the vectors that encode the context and the possible responses can be provided as input to a machine learning algorithm. A SoftMax function can be applied to the output of the selection unit 16 to render the probability that some answer needs to be selected. The trainable parameters of the machine learning algorithm can be optimized, in particular, by minimizing a loss function that represents the distance between the predicted probability distribution and the true distribution. For example, cross-entropy loss can be used.

[0133] Returning to Figure 3, the output unit 40 may be adapted to return the selected response 28 to the user, such as in the form of a text or audio signal, or any other format. The dialogue can then continue with the user presenting the chatbot system 10' with another sample 18.

[0134] The use of a quantum random number generator 24 in selecting a response 28, and optionally also in generating multiple response hypotheses 20a-20e, can solve many of the potential problems of conventional pseudorandom number generators, including shorter than expected periods for some seed states, lack of uniformity in the distribution of large numbers of generated numbers, correlation of consecutive values, the dimensional distribution of the output sequence, or a distribution where the distance between locations where certain values ​​occur that differs from a true random sequence distribution.

[0135] The techniques of this disclosure can avoid these patterns and ensure that the chatbot system 10, 10' exhibits variability in the sense that it responds to the same interaction in different ways, just as a human being with free will would.

[0136] The description of the embodiments and drawings only serves to illustrate the technology of the present disclosure and its related advantages, and should not be understood as meaning limitations. The scope of the present disclosure should be determined from the appended claims. [Explanation of symbols]

[0137] 10,10' Natural language processing system; chatbot system 12 Receiving unit 14 Processing Unit 16 Selection Units 18 Samples 20;20a-20e Multiple Response Hypothesis 22;22a-22e Multiple confidence values 24 Quantum Random Number Generator 26 Data Connections 28 Selected Responses 30 Annotator Unit 30a Spelling Unit 30b Split Unit 32 Annotated Samples 34a~34e Hypothesis Generation Unit 36 GPT module of hypothesis generation unit 34e 38 Hypothesis Annotator Unit 38a Text Encoder Unit 38b Toxic Unit 40 Output Units

Claims

1. A computer implemented method for natural language processing, comprising: receiving a sample (18) containing natural language; - processing the samples (18), the method comprising the steps of generating a plurality of response hypotheses (20; 20a-20e) representing candidate responses to the samples (18), and generating a plurality of confidence values ​​(22; 22a-22e), each response hypothesis (20; 20a-20e) being associated with a corresponding confidence value (22; 22a-22e); selecting a response (28), comprising randomly selecting the response (28) from among the plurality of response hypotheses (20; 20a-20e) based at least in part on the corresponding confidence values ​​(22; 22a-22e) by a quantum random number generator (24), the quantum random number generator being a hardware random number generator (24) capable of generating random numbers from a quantum physical process; A method comprising:

2. 2. The method of claim 1, wherein the step of selecting a response (28) comprises the steps of generating a probability distribution over the plurality of response hypotheses (20; 20a-20e) with corresponding weights based on each of the plurality of confidence values ​​(22; 22a-22e) and randomly selecting from among the plurality of response hypotheses (20; 20a-20e) according to the corresponding weights by the quantum random number generator (24).

3. 2. The method of claim 1, wherein the step of selecting the response (28) comprises the steps of: comparing the corresponding confidence value (22; 22a-22e) to a predetermined threshold; and if the corresponding confidence value (22; 22a-22e) is less than the predetermined threshold, selecting, by the quantum random number generator (24), the response (28) randomly from among the plurality of response hypotheses (20; 20a-20e) based at least in part on the corresponding confidence value (22; 22a-22e).

4. 4. The method of claim 3, comprising deterministically selecting the response (28) from among the plurality of response hypotheses (20; 20a-20e) based at least in part on the corresponding confidence values ​​(22; 22a-22e) if the corresponding confidence values ​​(22; 22a-22e) are greater than or equal to the predetermined threshold.

5. 2. The method of claim 1, wherein the step of selecting the response (28) comprises a step of preselecting the response hypotheses (20; 20a-20e) according to predetermined criteria, and wherein the preselecting step may optionally comprise a step of discarding response hypotheses (20; 20a-20e) according to the predetermined criteria.

6. 2. The method of claim 1, wherein generating the plurality of response hypotheses (20; 20a-20e) comprises generating at least a portion of the plurality of response hypotheses (20; 20a-20e) by the quantum random number generator (24).

7. 2. The method of claim 1, wherein the step of generating a plurality of response hypotheses (20; 20a-20e) comprises a step of randomly selecting words of the response hypotheses (20; 20a-20e), in particular by means of the quantum random number generator (24).

8. 2. The method of claim 1, further comprising the step of annotating the samples (18), in particular annotating the samples (18) before generating the plurality of response hypotheses (20; 20a-20e) and generating the plurality of confidence values ​​(22; 22a-22e).

9. A computer program comprising computer readable instructions which, when read by a computer, cause the computer to carry out a method according to any one of claims 1 to 8.

10. a receiving unit (12) adapted to receive a sample (18) containing natural language; a processing unit (14) adapted to process the samples (18), generating a plurality of response hypotheses (20; 20a-20e) representing candidate responses to the samples (18), and to generate a plurality of confidence values ​​(22; 22a-22e), each response hypothesis (20; 20a-20e) being associated with a corresponding confidence value (22; 22a-22e); a selection unit (16) adapted to select a response (28), the selection unit (16) being adapted to randomly select the response (28) from among the plurality of response hypotheses (20; 20a-20e) based at least in part on the corresponding confidence values ​​(22; 22a-22e) by a quantum random number generator (24), the quantum random number generator being a hardware random number generator (24) capable of generating random numbers from a quantum physical process; A system (10; 10') for natural language processing comprising:

11. 11. The system (10; 10') of claim 10, wherein the selection unit (16) is adapted to generate a probability distribution over the plurality of response hypotheses (20; 20a-20e) with corresponding weights based on each of the plurality of confidence values ​​(22; 22a-22e), and the selection unit (16) is further adapted to randomly select from among the plurality of response hypotheses (20; 20a-20e) according to the corresponding weights by the quantum random number generator (24).

12. 11. The system (10; 10') of claim 10, wherein the selection unit (16) is adapted to compare the corresponding confidence value (22; 22a-22e) with a predetermined threshold, and further adapted to select, by the quantum random number generator (24), the response (28) randomly from among the plurality of response hypotheses (20; 20a-20e) based at least in part on the corresponding confidence value (22; 22a-22e) if the corresponding confidence value (22; 22a-22e) is less than the predetermined threshold.

13. The system (10; 10') according to claim 10, wherein the selection unit (16) is adapted to preselect the response hypotheses (20; 20a-20e) according to predetermined criteria, in particular the selection unit (16) is adapted to discard response hypotheses (20; 20a-20e) according to the predetermined criteria.

14. The system (10; 10') of claim 10, wherein the processing unit (14) is adapted to generate at least a portion of the plurality of response hypotheses (20; 20a-20e) by means of the quantum random number generator (24).

15. The system (10; 10') according to any one of claims 10 to 14, further comprising an annotator unit (30) adapted to annotate the samples (18), in particular to annotate the samples (18) before generating the plurality of response hypotheses (20; 20a-20e) and generating the plurality of confidence values ​​(22; 22a-22e).

Images