Keyboard activity analysis

Abstract

Keyboard Activity Analysis A computer-implemented method for analyzing keyboard activity of a user is provided. The method comprises obtaining (S21) the instants of key pressing and / or releasing while typing a reference text on a client computer system by the user. The reference text comprises one or more words. The method comprises determining (S22), for each respective word of the reference text, the value of a typing dynamics characteristic vector for the respective word. The characteristic vector comprises for a typed word one or more metrics each determined from the instants of key pressing and / or releasing while typing the reference text. The method comprises determining (S23) the value of a signature for the reference text from each determined characteristic vector. The analysis method improves monitoring of a user. [Fig. 1]

Description

Title of the invention: Keyboard activity analysis Technical field

[0001] The present disclosure relates to keyboard activity analysis, and more particularly to a computer-implemented method of analyzing keyboard activity, a computer-implemented method of monitoring keyboard activity, and a computer program and a server computer system for performing these methods. Technical background

[0002] Today, there are intelligent systems capable of monitoring the physical world in real time and providing high value-added services to users. With the rise and development of technologies, automated intelligent surveillance now makes it possible, particularly from complex information processing and control infrastructures and networks, to improve security and provide ubiquitous detection.

[0003] For monitoring a user on a computer system, existing monitoring solutions generally use a webcam directed towards the user to monitor them from a video stream. Such monitoring solutions can, for example, be used in virtual classrooms for distance learning, for example to help supervisors ensure the smooth running of an online exam, for example in a school or academic setting. Among the frauds sought are the use of external human assistance (this illicit assistance can be given in different ways) and the use of an unauthorized resource (the learner using a set of resources that are prohibited to them to facilitate the answering of exam questions).

[0004] Existing surveillance solutions are not sufficient. Indeed, they require sensors for capturing information (for example a camera or a fingerprint sensor) to enable user monitoring, and it is difficult from an economic and usage point of view to impose the use of such sensors.

[0005] Therefore, there is a need for an improved monitoring solution. Abstract

[0006] To this end, a computer-implemented method is proposed for analyzing a user's keyboard activity. This method is then called an "analysis method." The analysis method comprises obtaining the instants of key pressing and / or releasing while typing a reference text on a system. by the user. The reference text includes one or more words. The analysis method includes determining, for each respective word of the reference text, the value of a characteristic vector of typing dynamics for the respective word. The characteristic vector includes, for a typed word, one or more metrics each determined from the key press and / or release instants during the typing of the reference text. The analysis method includes determining the value of a signature for the reference text from each determined characteristic vector.

[0007] The one or more determined metrics may include a metric Di equal to the time, for each character i, between the key press instant Pt of the respective key corresponding to the character i and the key release instant Rt of the respective key corresponding to the character i.

[0008] The one or more determined metrics may comprise a metric D2 equal to the time, for each character i, between the instant of pressure P, of the respective key corresponding to the character i and the instant of pressure PM of a respective key corresponding to the following character i+1.

[0009] The one or more determined metrics may comprise a metric D3 equal to the time, for each character i, between the instant of pressure P t of the respective key corresponding to the character i and the instant of release R i+2 of a respective key corresponding to the second following character i+2.

[0010] The one or more determined metrics may comprise a metric D4 equal to the time, for each character i, between the instant of pressure P t of the respective key corresponding to the character i and the instant of pressure P i+3 of a respective key corresponding to the third following character i+3.

[0011] The one or more determined metrics may comprise a metric D5 equal to the time between the instant of pressing the respective key corresponding to the first character of the respective word and the instant of releasing the respective key corresponding to the last character of the respective word.

[0012] The reference text may comprise several words. The signature for the reference text may comprise an average of the characteristic vectors determined for each of the words of the reference text.

[0013] Also provided is a computer-implemented method for monitoring keyboard activity on behalf of a user. This method is then referred to as a "monitoring method." The monitoring method includes providing the value of a signature determined for the user. The monitoring method includes obtaining the instants of key pressing and / or releasing while typing online text on a client computer system for the user account. The online text comprises one or more words. The monitoring method comprises determining, for each respective word of the online text, the value of the typing dynamics feature vector for the respective word. The monitoring method comprises determining whether or not the user has been impersonated based on a comparison between one or more values ​​of the feature vector determined for the online text and the value of the signature determined for the reference text.

[0014] Determining whether or not spoofing has occurred may further be based on a comparison between a facial capture while typing online text on the client computer system on behalf of the user and an image of the user from a database.

[0015] Determining whether or not spoofing has occurred may include merging the results of comparing the one or more feature vector values ​​determined for the inline text and comparing the facial capture.

[0016] Determining whether or not the user has been impersonated may include determining, for each respective word of the online text, a distance between the value of the characteristic vector of the respective word and the value of the signature determined for the user. Determining whether or not the user has been impersonated may include determining whether or not the user has been impersonated from each determined distance.

[0017] The characteristic vector and the signature can each be a vector having coordinates for each of the one or more metrics. The distance determined for each respective word of the online text can be equal to a norm of the difference between the value of the characteristic vector of the respective word and the value of the signature determined for the user.

[0018] The monitoring method may further include, for each respective word of the online text, a comparison of the distance determined for the respective word with a confidence threshold. The monitoring method may further include a display of the comparison result for each respective word of the online text.

[0019] There is also provided a computer program comprising program code instructions for performing, when said program is executed on a computer, the steps of the analysis method and / or the monitoring method.

[0020] Also provided is a server computer system configured to communicate with a client computer system. The server computer system has a memory on which the computer program is stored. The server computer system is configured to execute the computer program. Brief description of the drawings

[0021] Non-limiting examples will be described with reference to the following figures:

[0022] [Fig.l] illustrates an example of implementation of the monitoring solution.

[0023] [Fig.2] illustrates an example of an image quality evaluation algorithm.

[0024] [Fig.3] illustrates an example of a convolutional neural network architecture for face analysis.

[0025] [Fig.4] shows an example of distances calculated throughout a video sequence when the learner is who he claims to be.

[0026] [Fig.5] shows an example of distances calculated during an attempted identity theft fraud.

[0027] [Fig.6] shows examples of keyboard interactions.

[0028] [Fig.7] shows different use cases of keyboard analysis.

[0029] [Fig.8] shows examples of metrics.

[0030] [Fig.9] shows an example of encryption of the determined signature.

[0031] [Fig.10] illustrates an example of merging the results of keyboard analysis and face analysis. References

[0032] [1] Saad, MA, Bovik, AC, and Charrier, C. (2012). Blind image quality assessment: A natural scene statistics approach in the DCT domain. IEEE transactions on Image Processing, 21(8), 3339-3352.

[0033] [2] Omkar M. Parkhi, Andrea Vedaldi and Andrew Zisserman. Deep Face Récognition. In Xianghua Xie, Mark W. Jones, and Gary K. L. Tarn, editors, Proceedings of the British Machine Vision Conférence (BMVC), pages 41.1-41.12. BMVA Press, September 2015.

[0034] [3] Z. Chen, K. Wu, Y. Li, M. Wang and W. Li, "SSD-MSN: An Improved Multi-Scale Object Détection Network Based on SSD," in IEEE Access, vol. 7, pp. 80622-80632, 2019.

[0035] [4] JIN, Andrew Teoh Beng, LING, David Ngo Chek, et GOH, Alwyn. Biohashing: two factor authentication featuring fingerprint data and tokenised random number. Pattern récognition, 2004, vol. 37, no 11, p. 2245-2255 Description détaillée

[0036] A computer-implemented method for analyzing a user's keyboard activity is provided. This method is hereinafter referred to as an "analysis method." The analysis method comprises obtaining the instants of key pressing and / or releasing while the user is typing a reference text on a client computer system. The reference text comprises one or more words. The analysis method comprises determining, for each respective word of the reference text, the value of a typing dynamics characteristic vector for the word respective. The characteristic vector comprises for a typed word one or more metrics each determined from the instants of key pressing and / or release during typing of the reference text. The analysis method comprises a determination of the value of a signature for the reference text from each determined characteristic vector.

[0037] The analysis method improves monitoring of a user.

[0038] Indeed, the analysis method makes it possible to simply and reliably identify a user. This identification can then be used to monitor a user, for example in order to confirm his identity and to detect identity theft or not. The value of the determined signature notably allows this identification of the user. Indeed, the determined signature characterizes the user's keystroke from the instants of pressing and / or releasing the key while typing the reference text, and therefore uniquely identifies this user. The analysis method therefore allows a unique, reliable and robust identification of the user. Furthermore, the identification of the user by the analysis method does not require an additional sensor for collecting information. The analysis method can therefore be implemented simply.

[0039] Also provided is a computer-implemented method for monitoring keyboard activity on behalf of a user. This method is hereinafter referred to as a "monitoring method." The monitoring method comprises providing the value of a signature to the user. The monitoring method comprises obtaining the instants of key pressing and / or releasing while typing an online text on a client computer system on behalf of the user. The online text comprises one or more words. The monitoring method comprises determining, for each respective word of the online text, the value of the typing dynamics characteristic vector for the respective word.The monitoring method comprises determining whether or not the user's identity has been impersonated based on a comparison between one or more values ​​of the characteristic vector determined for the online text and the value of the signature determined for the reference text.

[0040] The analysis method and the monitoring method may be linked and may together form a single improved monitoring solution. Indeed, the analysis method allows the determination of a signature uniquely identifying a user and the monitoring method allows the determination of whether or not this user has been usurped from the signature determined by the analysis method. The analysis method and the monitoring method are therefore based on the same principles, namely the use of a signature determined from the instants of keypress and / or release while typing a reference text for user identification.

[0041] The analysis method and the monitoring method may be included in a monitoring solution, which may comprise, during a first phase, the execution of the analysis method, and during a second phase following the first, the execution of the monitoring method. The monitoring solution may for example be a method for monitoring an exam, and the first phase may correspond to a first phase of forming the signature for authenticating the user for this exam, i.e. determining the value of the signature for the user. The second phase may correspond to a phase of verifying the identity of the user, i.e. comparing to verify that it is always the person who typed the reference text during the first phase who takes the online exam during the second phase.

[0042] For example, the first phase may take place during a first exam of a school year. In this case, the reference text may be the answer text that the user (i.e., the candidate) types during this first exam. In this case, the second phase may take place during the other exams taken by the candidate (e.g., the other exams of the school year). During these other exams, the proctoring solution may execute the proctoring method to verify the identity of the candidate from his or her signature determined by the analysis method during the first exam of the year.

[0043] Alternatively, the first phase may take place before one or more examinations of the school year. The first phase may be a particular session during which the candidate's signature is determined, before the one or more examinations are conducted. In this case, the second phase may take place during the one or more examinations, to verify the candidate's identity during these examinations with the signature determined during the first phase.

[0044] Alternatively, an examination may comprise several tests, and the first phase may take place during a first test of the examination in order to determine the signature of the candidate. During the other tests, the second phase may take place, and the monitoring method may be executed in order to verify the identity of the user taking the test (i.e., to verify that the user taking the examination is the user identified during the first test). For example, the inline text may be text that is typed by the user during the test (e.g., while answering questions on the examination), and the monitoring method may verify the identity of the user while typing the inline text (i.e., while answering questions on the examination).

[0045] The first phase may take place before the monitoring method is executed. The analysis method may comprise, once the signature has been determined, a recording of this signature (for example on a memory of the server or the client computer system). During the second phase, the provision of the value of the signature may comprise a reading of the recorded signature.

[0046] The analysis method and the monitoring method make it possible to reduce the number of invigilators during the second phase, or to reduce the monitoring "load". Indeed, the monitoring solution makes it possible to verify the identity of individuals from the way they type a text. The way people type a text on the keyboard makes it possible to identify them and to recognize identity theft. The monitoring solution allows automatic monitoring of exams from the way candidates type on the keyboard.

[0047] When the monitoring method determines the usurpation, the monitoring method may comprise sending an alert and / or a notification (for example audible or visual), for example to a supervisor, who may then for example take a closer look at the candidate in question, to confirm or not the usurpation.

[0048] During the first phase, the monitoring solution may comprise a determination of the value of the signature for a user from the analysis method. In the second phase, the monitoring solution may comprise a determination of whether or not the identity of the user has been usurped from the monitoring method and the signature determined for the user during the first phase of forming the signature for authentication.

[0049] During the second phase, the monitoring method may be repeated. For example, the monitoring method may be repeated for each word of the online text typed, or each time a number n of words of the online text are typed (n being a positive integer, for example between 2 and 10). Alternatively, the method may be repeated at time intervals which may be regular. For example, the method may be repeated every X seconds (with X being a positive number, for example between 1 and 10 seconds).

[0050] In examples, the monitoring solution may include repeating the analysis and monitoring processes for each user (e.g., for each student taking the exam). During the exam, the monitoring solution may verify that each user taking the test is the same user who typed the reference text during the first signature formation phase, and thus verify the identity of each user. The monitoring solution may thus monitor multiple users at the same time.

[0051] The server computer system (hereinafter referred to as the "server") can execute the monitoring method and the analysis method. In this case, for the reference text and for the online text, the client computer system (hereinafter referred to as the "client") on which the text is typed can be configured to send the key press and / or release times while typing the text. For example, the client can record the key press and / or release times in a file on a memory and then send this file to the server (at the end of typing or during typing, for example after each key typed). For the reference text and for the online text, obtaining the key press and / or release times can then include a reception by the server of the file sent by the client. The server can be in communication with a computer and / or a tablet of one or more proctors. When the monitoring method determines the spoofing, the monitoring method can include sending an alert and / or a notification to the computer and / or the tablet of the one or more proctors.

[0052] Alternatively, the client on which the reference text is typed may execute the analysis method and the server may execute the monitoring method. In this case, the client on which the reference text is typed may be configured to record the determined signature on a memory and send it to the server. Providing the value of the signature may then comprise a reception of the signature sent by the client. For the inline text, the client on which the inline text is typed may record the key press and / or release times in a file on a memory and then send this file to the server. Obtaining the key press and / or release times may then comprise a reception by the server of the file sent by the client.

[0053] Alternatively, the client on which the inline text is typed may execute the monitoring method. In this case, the client on which the reference text is typed or the server may execute the analysis method. The computer system executing the analysis method may then send the signature determined for the user to the client on which the inline text is typed. The provision may then comprise receiving the sent signature. The obtaining may in this case comprise detecting the instants of key pressing and / or release while typing the inline text.

[0054] Alternatively, one or more steps of the analysis method may be performed by the client on which the reference text is typed, and one or more other steps of the analysis method may be performed by the server. For example, the client may perform the steps of obtaining and determining each feature vector, and the analysis method may then include sending each determined feature vector to the server, which may then perform the determination of the signature value from each determined signature. Alternatively, the server may perform the determination of each feature vector, and in this In this case, the analysis method may include sending the moments of key pressure and / or release obtained by the client to the server.

[0055] Similarly, the monitoring method may be executed by the client on which the inline text is typed and by the server. For example, the client on which the inline text is typed may execute the steps of obtaining and determining each characteristic vector. In this case, the monitoring method may then comprise sending each determined characteristic vector to the server, which may then execute the determination of whether or not the spoofing has occurred. The monitoring method may also execute the providing step. Alternatively, the server may execute the determination of each characteristic vector, and in this case the monitoring method may comprise sending the instants of key pressing and / or release obtained by the client to the server.

[0056] The client on which the reference text is typed may be the same as the client on which the online text is typed. For example, the client may be a personal computer of the user (e.g., at the user's home), a tablet, or a computer shared by several people, for example, at a university or a company. Alternatively, the client on which the reference text is typed may be different from the client on which the online text is typed. For example, the online text may be typed on the user's personal computer and the reference text on a shared computer or on another user's personal computer. Conversely, the online text may be typed on a shared computer and the reference text on a shared computer or on another user's personal computer.

[0057] The user typing the reference text may be the same as the user typing the online text. In this case, the monitoring method may statistically determine the non-spoofing of the user's identity. Indeed, the comparison will statistically indicate that the one or more values ​​of the characteristic vector are close to the determined signature. Alternatively, the user typing the online text may be different from the user typing the reference text. In this case, the monitoring method may statistically determine the non-spoofing of the user's identity. Indeed, the comparison will statistically indicate that the one or more values ​​of the characteristic vector are different from the determined signature. The monitoring method may comprise providing a result of the comparison, for example a number statistically indicating a probability of spoofing or not.

[0058] The reference text and / or the inline text may be typed on any type of keyboard. For example, the keyboard may be a physical keyboard such as a keyboard connected to a computer or a keyboard of a laptop computer. Alternatively, the keyboard may be a virtual keyboard such as a keyboard displayed on a device with a touchscreen (e.g. a tablet or smartphone).

[0059] The reference text and / or the inline text may each comprise one or more words of one or more characters. The reference text and the inline text may be different. The reference text may comprise a number of words at least greater than or equal to 10. For example, the reference text may comprise between 20 and 40 words. The inline text may comprise a number of words at least greater than or equal to 10 (for example between 20 and 40 words).

[0060] The reference text and the inline text may be in an Indo-European language. For example, the reference text and the inline text may be in English, French, German, Spanish, Italian and / or Portuguese (or any combination of Indo-European languages).

[0061] Typing text comprises the sequential typing (or inputting via the keyboard) of each word of the text, that is, the sequential typing or inputting of each character of each word. Typing each character comprises pressing, at a first instant, a key on the keyboard corresponding to the character and then releasing, at a second instant after the first, the key corresponding to the character. The characters of each word may be typed successively, one after the other. For the reference text and / or the online text, the times of key pressing and / or release during typing may be the times of key pressing and / or release of each of the characters of the text.

[0062] For the analysis method or the monitoring method, obtaining the key pressure and / or release times may comprise a recording of the key pressure and / or release times. For example, obtaining may comprise a recording of these times in the form of two vectors P=(Pp PD) and R=[Ry Rn) containing all the pressure and release times R} of the characters i = 1, ..., 22 of the text in question (the reference text or the online text). These two vectors may be recorded in a memory.

[0063] When the text comprises several words, typing the text may include, between each word, a keystroke of a particular key (for example a space key). The instants of pressing and / or releasing this particular key may be ignored, that is to say that the determination of the value of the characteristic vector may not take into account the instants of pressing and / or releasing this key. In other words, the vectors P and R may not include the instants of pressing and / or releasing this particular key. Similarly, typing any key other than a character key may be ignored. For example, reference text and / or inline text may include one or more punctuation marks or numbers, and similarly to the space key, the typing of these keys may be ignored.

[0064] The determination of the value of the typing dynamics characteristic vector (hereinafter referred to as "feature vector") of a word is now discussed. The information given below applies both to the analysis method with each word of the reference text and to the monitoring method with each word of the online text.

[0065] The feature vector may be a vector having coordinates for each of the one or more metrics. The feature vector may include a respective coordinate for each metric. When the feature vector includes only one metric, the feature vector may include only one coordinate. When the feature vector includes multiple metrics, the feature vector may include multiple coordinates (one for each metric).

[0066] The determination of each value of the characteristic vector may comprise a calculation of the value of each metric from the recorded vector P and / or the recorded vector R. The value of each calculated metric may correspond to a time interval between instants of pressure P^ of the recorded vector P and / or instants of release R of the recorded vector R.

[0067] The feature vector may comprise one or more of the metrics discussed below. In particular, the feature vector may comprise any combination of these metrics. During the analysis process and during the monitoring process, the determined feature vectors may comprise at least one or more common metrics. For example, the determined feature vectors may comprise the same metrics. In other words, the determination of each value of the feature vector in the analysis process may be performed in the same manner as the determination of each value of the feature vector in the monitoring process.The determination of each value of the feature vector may include, for each respective word of the typed text, a determination of the value of each metric for the respective word (the typed text being the reference text for the analysis process and the online text for the monitoring process).

[0068] A first metric Di may be an average of the time to type each character of the word. The time to type each character 1 of the word may be equal to the time between the instant of pressing the respective key corresponding to the character 1 and the instant of release R^ of the respective key corresponding to the character 2. For each word of the typed text, the determination of the metric Di may include, for each character 2 of the word, a calculation of the time to type the character, and then, a calculation of an average of these calculated times.

[0069] A second metric D2 may be an average of the typing time between two characters of the word. The typing time between each character 2 and the next character of the word may be equal to the time between the instant of pressing P^ of the respective key corresponding to the character 2 and the instant of pressing Pj+i of the respective key corresponding to the character i+1. For each word of the typed text, the determination of the metric D2 may comprise, for each character 2 of the word, a calculation of the typing time between successive characters of the word, then, a calculation of an average of these calculated times. For the last character of the word, the calculation of the typing time may be between this last character of the word and the first character of the next word.

[0070] A third metric D3 may be an average of the time to type three characters of the word. The time to type three characters from character 2 may be equal to the time between the instant of pressing P^ of the respective key corresponding to character 2 and the instant of releasing R1+2 of the respective key corresponding to character i+2. For each word of the typed text, the determination of the metric D3 may comprise, for each character 2 of the word, a calculation of the time between the start of typing character 2 and the end of character i+2, then, a calculation of an average of these calculated times. For the penultimate and last character of the word, the calculation of the typing time may take into account the time until the instant of releasing the first character(s) of the following word.

[0071] A fourth metric D4 may be an average of the total typing time of three characters of the word. The total typing time of three characters of the word starting from character 2 may be equal to the time between the instant of pressing P^ of the respective key corresponding to character 2 and the instant of pressing P^ of the respective key corresponding to character i+ 3. For each word of the typed text, the determination of the metric D3 may comprise, for each character 2 of the word, a calculation of the time between the start of this character 2 and the start of character 7 + 3, then, a calculation of an average of these calculated times. For the last three characters of the word, the calculation of the typing time may take into account the time until the instant of pressing the first character(s) of the following word.

[0072] The one or more metrics may include any other metric determined from the times of key presses and / or releases while typing the text. For example, one or more metrics may include a metric that is an average of the total time to type (or between) two successive characters, four successive characters, or any number n of successive characters with n a positive integer.

[0073] In examples, the one or more metrics may comprise one or more metrics determined for the respective word directly. For example, the one or more determined metrics may comprise a D5 metric equal to the time to type the respective word. The time to type the respective word may be equal to the time between the instant of pressing the respective key corresponding to the first character of the respective word and the instant of releasing the respective key corresponding to the last character of the respective word. The D5 metric makes it possible to take into account the hesitation time for writing a text, which is a unique data linked to the person and their actual way of using their keyboard, which is not the case for passwords. The D5 metric therefore improves monitoring.

[0074] The determined signature may be a vector having coordinates for each of the one or more metrics of the feature vector. The determined signature may comprise a respective coordinate for each metric. When the feature vector comprises only one metric, the determined signature may comprise only one coordinate. When the feature vector comprises several metrics, the determined signature may comprise several coordinates (one for each metric). The determined signature and the feature vector may comprise the same number of coordinates, and these coordinates may correspond to the same metrics (i.e., in the same order on each vector).

[0075] In examples, the reference text may comprise multiple words. In this case, the signature for the reference text may comprise an average of the feature vectors determined for each of the words in the reference text. Determining the value of the signature may comprise calculating, for each metric, an average of the value of the metric over each of the words in the reference text. Alternatively, the reference text may comprise a single word. In this case, the signature for the reference text may be equal to the feature vector determined for that single word.

[0076] Determining whether the user is being impersonated may include determining whether the user typing the inline text is the user who typed the reference text. In the case where the user is the same, the monitoring method may determine whether the user is not being impersonated, and conversely when the two users are different.

[0077] The determination of whether or not there is usurpation may be based on a comparison between the value of the characteristic vector determined for each word of the online text and the value of the signature determined for the reference text. For example, the Determining whether or not the user is being spoofed may include determining, for each respective word in the online text, a distance between the value of the feature vector of the respective word and the value of the signature determined for the user. The distance may represent a deviation between the way in which the user types the word in the online text and the way in which the reference text was typed. For example, the distance may be a cosine similarity, a Euclidean distance, a Manhattan distance, or any other measure representing a distance between two vectors. Determining whether or not the user is being spoofed may then be a function of each determined distance, i.e., these deviations calculated for each word in the online text.

[0078] The determination of whether or not spoofing has occurred can be carried out in real time. For example, after each word typed during the typing of the online text, the monitoring method can comprise a determination of the distance between the value of the characteristic vector determined for the word that has just been typed with the value of the signature. The determination of whether or not spoofing has occurred can then comprise an update of the result based on this new determined distance. Alternatively, the determination of whether or not spoofing has occurred can be carried out at the end of the typing of the online text. In this case, the determination of whether or not spoofing has occurred can be a function of all the distances determined at the end of the typing of the online text.

[0079] In examples, the monitoring method may determine spoofing or not by comparing the calculated distances with a confidence threshold. For example, the monitoring method may include, for each respective word of the inline text, comparing the distance determined for the respective word with the confidence threshold. A distance greater than the threshold may mean that the user typing the word is not the one who typed the reference text.

[0080] The monitoring method may determine that there is spoofing from the calculated distances in various ways. For example, the monitoring method may determine that there is spoofing when the number of calculated distances that are greater than the threshold is greater than a predetermined number (e.g., a number n with n a positive integer). Alternatively, the monitoring method may determine that there is spoofing when the number of calculated distances that are greater than the threshold is greater on average than the number of calculated distances that are less than the threshold (e.g., after a number n of calculated distances, with n a positive integer). Alternatively, the monitoring method may determine that there is spoofing when the average of the calculated distances is greater than the threshold.

[0081] In examples, the distance determined for each respective word of the text in line can be equal to a norm of the difference between the value of the vector characteristic of the respective word and the value of the signature determined for the user. For example, the distance d(Tref, T) between the determined signature Tref and a characteristic vector T can be calculated with the formula: d(Tref, T) - √(Tref1 - T1)2 +... + (Trefm

[0082]

[0083] where Trefj and Tj correspond to the 1 coordinates of the vectors Tref and T respectively. In examples, the monitoring method may include a display of the result of the comparison for each respective word of the online text. For example, the display may include a display of a graph representing the set of determined distances. The graph may include the set of words of the online text on the abscissa and the calculated distances for each word on the ordinate. Alternatively or additionally, the display may include a display of an alert, for example when the monitoring method determines that there is spoofing. The display may be displayed on a screen, and may be updated in real time as the online text is typed. The screen may be viewed by a proctor, for example during an online exam. The proctor may then be informed whether the user's identity has been spoofed.

[0084] In examples, determining whether spoofing is occurring may further be based on facial analysis. The facial analysis may include comparing a capture of the face while typing the online text on the client computer system on behalf of the user to an image of the user from a database. The determination may include capturing the face (e.g., via a webcam), extracting the image of the user, and comparing the captured face to the image of the user (e.g., using a deep learning algorithm). The image of the user may have been taken while typing the reference text.For example, the analysis method may include capturing the face of the user typing the reference text (e.g., via a webcam) and recording the capture as an image in the database, and the monitoring method may include extracting this image from the database. The server and the clients may be connected to this database to extract and record data thereon..

[0085] In some examples, determining whether there is impersonation may include fusing the results of comparing one or more values of the characteristic vector determined for the online text (i.e., keyboard analysis) and the comparison of the face capture (i.e., face analysis). For example, the monitoring method may include simultaneously displaying the results of the keyboard analysis and the face analysis. Alternatively, the method of monitoring may include determining a common outcome based on the results of the keyboard analysis and the facial analysis. For example, the monitoring method may determine that there is spoofing when on average the keyboard analysis and the facial analysis indicate that there is spoofing, and conversely that there is no spoofing when on average the keyboard analysis and the facial analysis indicate that there is no spoofing. For example, merging may include merging the scores obtained by the keyboard analysis and by the facial analysis. The monitoring method may include providing a result of the merging, for example a number statistically indicating a probability of spoofing or not.

[0086] Examples of implementation of the analysis method and the monitoring method will now be given with reference to Figures 1 to 10.

[0087] Face analysis is discussed first. Face analysis may be used to secure user monitoring. The client on which the reference text is typed and / or the client on which the inline text is typed may include a sensor. Face analysis may include biometric analysis, for example, gaze analysis, gesture analysis, or skeleton extraction.

[0088] The client on which the reference text is typed and / or the client on which the online text is typed may each be a biometric system for detecting, extracting and analyzing data by means of a sensor (keyboard and webcam), which may record the users' model. The analysis of the learner's data may be considered as an attempt to find a relationship between the sensor's output data and the user's actual model. Among the challenges of computer vision are noise detection, lack of brightness and information loss. The analysis method and the monitoring method are divided into several steps: data acquisition, preprocessing, recognition, description and decision.

[0089] Determining whether or not to impersonate involves confirming or verifying the identity claimed by a person. The determination makes it possible to determine the identity of the person. The facial analysis may comprise a comparison of the captured image with several facial captures stored in a biometric database to find the identity of an unknown individual. Determining whether or not to impersonate may be based on biometric identifiers for security, for example classified into physiological or behavioral characteristics. Physical characteristics may include characteristics of the human body for identification purposes, such as for example fingerprint, hand, veins, iris and face. Behavioral characteristics may make it possible to recognize people from their interactive characteristic. Behavioral features may include typing dynamics, gesture analysis, and signature analysis.

[0090] [Fig.l] illustrates an example of implementation of the monitoring solution comprising the analysis method and the monitoring method. In this example, the analysis method S20 and the monitoring method S30 are integrated into a remote exam management solution S10 (creation, performance of tests and / or automatic correction). This solution S10 uses keystroke dynamics as a main basic modality to secure the authentication phase. This solution S10 also includes a face analysis S40 (via a multimodal device) in order to increase the security level of the application. The solution includes the implementation of security measures and data protection processes by obtaining an encrypted signature in the form of a secure code. The encryption algorithm is used to ensure the confidentiality of the data exchanged for the storage of the biometric templates.

[0091] The analysis method S20 comprises obtaining S21 the instants of key pressing and / or release during typing of the reference text (e.g. with pre-processing), determining S22, for each respective word of the reference text, the value of the typing dynamics characteristic vector for the respective word (feature extraction), and determining S23 the value of the signature. The analysis method S20 then comprises recording S24 the determined signature on the database, for example after encryption of the signature to obtain an encrypted signature.

[0092] The monitoring method comprises obtaining S31 the instants of key pressing and / or release during typing of the online text (for example with pre-processing), determining (S32 and S33), for each respective word of the online text, the value of the typing dynamics characteristic vector for the respective word, providing S34 the value of the signature determined for the user, and determining whether or not the identity of the user has been usurped based on a comparison S35 between one or more values ​​of the characteristic vector determined for the online text and the value of the signature determined for the reference text. In the event of usurpation, the monitoring method may comprise a display S36 of an alert, for example for an exam supervisor.

[0093] Face analysis is now discussed in more detail.

[0094] The face analysis may comprise two steps. The first step may consist of enrolling the user's signature. The second step may consist of verifying the degree of similarity between stored templates after applying certain quality requirements. Once the face image is acquired, the face analysis may comprise applying an image quality evaluation algorithm. without reference to evaluate the quality of the input data, using a statistical model of a natural scene, such as the model illustrated in [Fig.2] (reference [1]). Furthermore, the face analysis may include an adjustment of the image intensity values ​​to improve the contrast of the output image. The face analysis may include face detection and cropping to normalize the region of interest, for example by applying an object detector based on an algorithm (such as the Single Shot Multibox Detector algorithm described in reference [3]). The face analysis may include a resizing of the image by specifying the number of rows and columns to obtain a two-dimensional matrix. Finally, the face analysis may include a use of a convolutional neural network architecture (for example the one illustrated in [Fig.3], reference [2]) to extract a signature from the user. With this signature (or "template"), during the execution of the monitoring method, users (e.g. students) can access online courses and exams, for example based on the result of a comparison of this signature with a signature acquired during an initial authentication. The comparison may include a calculation of a distance between the two signatures and a comparison of this distance with a confidence threshold.

[0095] A performance analysis of the solution is now presented. The performance of the solution is analyzed by simulating attacks to access the remote exam session. A signature is compared with other signatures extracted during the connection phase. A reference image obtained on the day of the simulation or provided by the training center or university is compared with several images acquired to gain access and launch an online test. Table 1 below presents the results obtained when the verification is based on a verification of the learner's identity with face templates (translation of the English "FaceCodes") in an uncontrolled environment.The function takes a list of similarity scores between the templates (also called signatures) of the authorized learners and a list of verification scores of the imposters and generates an output containing the false positive rate (FAR) and the false negative rate (FRR) for all threshold values. After running the attack simulations, the verification achieves an error rate (EER) of 12.22% using the native signatures in an uncontrolled environment.

[0096] Table 1: Performance Evaluation (EER) in percentage. Number of signatures used to form the template or reference signature (average) Approx. not controlled (screw template age) 10 12.27% 20 12.24% 30 12.99% 40 12.28% 50 20.13% 60 20.07% 70 20.03% 80 12.22% 90 12.24% 100 12.24%

[0097] [Fig.4] shows the distances (similarity measures with signatures called "BioCodes" hereinafter) calculated throughout a video sequence when the learner is who he claims to be. [Fig.5] shows the distances calculated during an attempted identity theft fraud.

[0098] In an uncontrolled environment, the results show that the obtained EER value varies between 12% and 20%. The number of face templates to be used can be at least 20 to generate the signature (also called "reference template").

[0099] Continuous identity verification also allows the detection of identity theft attempts. [Fig.4] shows the distances (BioCodes similarity measures) calculated throughout a video sequence when the learner is who he claims to be. [Fig.5] shows the distances calculated during a fraudulent identity theft attempt. The results show that during an identity theft attempt, the similarity value is almost equal to 1, which reflects the fact that the two faces (the reference one and the captured one) are very different. In the case where the learner is indeed the person authorized to connect, this value is divided by two to be around 0.5. Face analysis can be based on a face template obtained from an average signature calculated on the first 80 face templates of the video.

[0100] Keyboard analysis is now discussed in more detail.

[0101] Keyboard analysis is based on an analysis of keyboard interactions (e.g., the flight time between two keys), which are collected during the use of a computer by an individual. [Fig.6] shows such examples of keyboard interactions. This information can, for example, describe the time a key was pressed and released when a person uses their keyboard.

[0102] Keyboard analysis uses this information for authentication of individuals or their profiling (such as recognizing the gender or age category of the individual from a password or free text). With multimodal biometrics, the monitoring solution can combine keystroke dynamics with another biometric modality in order to have a more effective identity verification system against identity theft attempts.

[0103] [Fig.7] shows different use cases of keyboard analysis. In some examples, keyboard analysis can be based on the same text: all users enter the same text, e.g. the name of a site. The entered text (or reference text) is therefore known at the design of the keyboard analysis (also called "Dynamic Keystrokes", DDF). In other examples, keyboard analysis (DDF) can be based on fixed text: users have a text associated with their account that they must enter to authenticate. This text can be secret (e.g. password), unique (e.g. login). However, such text cannot be known at the design of the system. In still other examples, keyboard analysis (DDF) can be based on free text: users can type whatever they want (e.g., a review, an article, a message). The text is generally longer than fixed text, but it is of variable length.In this case, keyboard analysis may include a transformation of this text in order to use distance functions.

[0104] The monitoring method may include continuous verification throughout the examination session for each learner. When the user writes a sentence or text on the keyboard, the monitoring method may include determining for each word the value of a characteristic vector. From several time sequences, the monitoring method may include extracting a signature (or "template") of the learner comprising several metrics.

[0105] [Fig.8] shows examples of metrics. The metric di can correspond to the time interval between pressing and releasing a key. The metric d2 can correspond to the time interval between two consecutive presses. The metric d' can correspond to the time interval between releasing a key and pressing the next key. The metric d3 can correspond to an overall duration of a sequence of characters.

[0106] The monitoring method may include processing and analyzing words with more than two letters. The monitoring method may include calculating the time intervals between pressing and releasing a key, the time intervals between two consecutive presses, and the overall duration of a sequence of characters. The monitoring method may also take into account the Learner hesitation time. Hesitation times for typing text reveal unique data related to the person and their actual keyboard usage, which is not the case for passwords. For each keystroke, the time intervals are recorded and concatenated with the duration to create a feature vector.

[0107] The analysis method can be implemented according to algorithm 1 below. Algorithm 1 Analysis of Fwüviiê daw The Eatrees to; F -- , FJ: Key press, S: F ™ ( ¾.,.., MJ ' Key release h Exits at: 7' (7],,,.. < Fj: Biometric template & Process 7: Count the number of keys pressed n and words recorded m a-. For z = 1, ..... m calculate -- .¾ — ZJ ~~ Pu-i ■ / ¼ F, - F,....... The distances (Ih, < •., ZM) are calculated for each character 1 hour end for 12-; Calculation of hesitation time - optional characteristic: :rik / ¾ F' J». / V) 14: Generate the characteristic vector T for each word: T .hMD;h DJ

[0108] The analysis method may comprise a recording of the instants of pressure and release times when using the keyboard (P and R). To obtain the biometric template (T), the analysis method may comprise a calculation of the averages of the different metrics (Db D2, D3 and D4) calculated for each character of each word, and of the metric related to the learner's hesitation (D5). For example, for the word TestWe, the metric D5 may be calculated from the formula D- = Re- PT in which Re corresponds to the instant of release of the key e and PT to the instant of pressure of the key T. The metric D5 may be calculated for all the words typed which comprise more than two characters. After this, the analysis method may comprise the determination of the user's signature Tref.

[0109] The monitoring method can be implemented according to algorithm 2 below. Algorithm 2 Calculation of the similarity score Generation of the signature for the learner using the average of the x-vectors - / Aj a The value of x-\ eetem o> for is obtained after the evaluation study to choose the optimal number of vectors. Calculation of similarities for each word u For h = 1, k calculate end for & Identity Verification Process if dt G™ r> 1 ' if d!: represents the verification threshold, generally equal to 0. Generation of the identity verification score curve for each user:

[0110] The monitoring method may comprise a calculation of the distance (for example a Euclidean distance) between the value of the characteristic vector T u determined for each word U = 1, ..., k of the typed online text and the determined signature Tref. Then, after this, the monitoring method may comprise a comparison of the result obtained with a confidence threshold. After several comparison operations during the remote examination period, the monitoring method may comprise a creation of a curve of the results obtained to assist the invigilator during the examination by alerting him if there is a case of fraud which requires intervention and verification thereof.

[0111] The analysis method may comprise an encryption of the determined signature, for example before a recording of the determined signature. The encryption makes it possible to protect the personal information (the biometric data) of the user. In this case, in the monitoring method, the provision may comprise a decryption of the determined signature. This decryption makes it possible to find the determined signature before the encryption.

[0112] The protection of biometric data is now discussed in more detail. The monitoring solution may include, for authentication, encryption of the signature determined from the method of hashing the unique biometric signature (translation from English "BioHashing"). [Fig.9] shows an example of encryption of the determined signature. Verification of a user is a security task whose action consists of limiting access to physical locations or the computer network only to authorized persons. This step can be performed by users using their biometrics.

[0113] The monitoring solution may combine two or more authentication factors to reap security benefits. An example algorithm of the raw biometric data transformation method (the hashing method) usable in the monitoring solution is shown below. This method accepts deviations from the captured data and allows for highly correlated bit strings to be obtained with the signatures. Algorithm 2 Uhdtrm the iiabant of the learner b Inputs 2 ™ ■ - >2”) template. & : secret code 4c Exit Z? -- (£4..... £^,) : BlbGxle 5: (Generation with the œde / iG of m jî^ndo-random vectors 14,..., K» of length n has vectors with Cram's algorithm» Schmidt, 7; For "~ L..... m calculate T, 14 >. 8: Bnpow 8: Calculate BCCcde: where r is a given threshold, gcza normally equal to 0

[0114] Cryptography has significant functional advantages over biometrics. It allows for the elimination of false acceptance rates without having a real effect on false rejection rates (as explained in reference [4]). In addition, it is difficult to obtain the user-specific data without having the random data and the user's native signature.

[0115] The hashing method makes it possible to protect learners' biometric signatures in distance tests.

[0116] Single-modality biometric systems face a variety of challenges such as noisy data, intra-class variation, non-universality, impersonation attacks, and unacceptable error rates. Fusing the results of keyboard analysis and face analysis overcomes these limitations of single-modality biometric systems. Indeed, such a multimodal biometric system allows integrating evidence presented by multiple sources of information.

[0117] [Fig. 10] illustrates an example of merging the results of the keyboard analysis and the face analysis. The merging may comprise one or more of the following different steps. A first step may comprise a merging S51 at the level of feature extraction (e.g. metrics). A second step may include S52 fusion at the matching score level (fusion at the template comparison phase and / or fusion at the classifier level to optimize similarity points between intra-class templates). A third step may include S53 fusion at the decision-making level (spoofing or not). The fusion may include only S52 fusion of scores.

[0118] The monitoring solution may include fusion at different stages and multiple levels to have a multimodal biometric system with integration strategies that may be adopted to consolidate the information. For example, the system (server and / or client) running the monitoring solution may include four modules. The system may include a sensor module that captures information from the physical world and transforms and records this information as biometric data. The system may include a feature extraction module that analyzes the data to extract a set of features to obtain a representation of the event. The system may include a compare and match module that uses a classifier to compare the extracted set of features with models in the database to generate similarity scores.The system may include a decision module that uses the match scores to verify an identity or validate a claimed identity.

[0119] The fusion may comprise a fusion of facial recognition as well as the dynamics of keystrokes at the level of feature extraction (metrics), at the level of generation of similarity scores between the determined signature and each value of the characteristic vector acquired during the test session and / or at the level of decision-making (spoofing or not).

[0120] The monitoring method may comprise an extraction of the characteristics and a comparison of each value of the characteristic vector obtained on the day of the examination with the determined signature recorded on the day of the simulation (a simulation which will take place at least one day before the examination for example).

[0121] The monitoring method may use multimodal fusion by combining two signatures for each modality (face and keystroke analysis). After feature extraction, the monitoring method may include concatenating the two signatures to form a complex signature. In a second step, the monitoring method may implement fusion at the resemblance score level. Indeed, it is relatively easy to access and combine the values ​​obtained using the different modalities. Fusion at the decision-making level may be performed by majority voting.

[0122] The fusion of the results of the keyboard analysis and the face analysis can comprise the following different steps. A first step can comprise a fusion of the attributes (metrics): the data set coming from several sensors / sources are merged, for example the average of several face templates or typing dynamics. The fusion can be done by the concatenation of a series of vectors obtained after the conversion of the raw data. A second step can comprise a fusion of the scores: the scores generated by several classifiers of different modalities are combined, the fusion can be done at the level of the scores coming from the comparison modules.Fusion at this level is actually well known in the literature as Multiple Classifier Systems using hybrid intelligent systems that help overcome the limitations of traditional single classifier-based approaches. A third step may include decision fusion: Decision-level fusion can be used for its simplicity. It may include providing a binary decision in the form (Yes or No) represented by 0 and 1. Fusion may then include consolidation of the final result of multiple outputs via techniques such as majority voting.

[0123] Results of the monitoring solution are now presented.

[0124] The data below were collected in an operational context. To obtain biometric data, information is collected from the individual via a built-in camera of a computer ("ThinkPad", registered trademark, S440). The objective is to see the performance of the proctoring solution in real conditions by analyzing its different components from data acquisition to decision-making and to determine the most sensitive stage to the environment of the candidates taking the test, in an environment where learners often do not take into account the instructions that managers request, such as hiding their face involuntarily during an exam conducted remotely, which can be considered as an attempt at cheating. The process is activated throughout the test using a GUI application developed with QT and C++.

[0125] An application collects data on keystroke dynamics. The application allows the collection of flight time between each keystroke as well as the release time of the keys. A button can be used when a participant enters personal information (password, PIN etc.) to avoid the collection of sensitive data.

[0126] In the step called continuous authentication, the monitoring solution must decide whether the user provides the necessary template; the features extracted from the current template are compared to the stored features. More precisely, Performance evaluation is based on the similarity score between the registered user's signature and the signature that the system saved during the prerequisite phase.

[0127] In the case of continuous verification of the individual, the monitoring solution may comprise a calculation of a signature from several characteristic vectors. To improve the performance of the solution, it is possible to take into account the influence of the number of characteristic vectors (the face templates and keyboard activity templates) used to calculate the reference signature on the EER value of the system. Thus, for each person, the average of the first (n) face codes / keyboard code (n varying from 10 to 100 in steps of 10) is calculated and will serve as a reference model. The signature can then be generated from this model thus constructed.

[0128] The experimental results obtained in the operational context are now presented.

[0129] Table 2 shows the results obtained when using keystroke analysis and face analysis to continuously authenticate the individual. The number of signatures to be used is at least 40 to generate the optimal reference template since a fusion process based on the set of both signatures is then applied.

[0130] Fusion at the feature extraction level poses several problems: non-homogeneous data, various distributions and vectors of different sizes. After transforming the data and implementing normalization techniques, the monitoring solution achieves an error rate of 11%.

[0131] Table.2 - Performance measure (EER) in percentage of the proposed multimodal system, calculated after merging the scores. Reference templates (average) Face template Keycode (DDF) Merging of the two signatures 10 0.12% 0.372% 0.492% 20 0.18% 0.4% 0.58% 30 0.09% 0.392% 0.482% 40 0.09% 0.376% 0.467% 50 0.12% 0.40% 0.490% 60 0.12% 0.407% 0.527% 70 0.12% 0.392% 0.512% 80 0.12% 0.392% 0.512% 90 0.12% 0.45% 0.57% 100 0.12% 0.407% 0.512%

[0132] Table 3 - Performance measure (EER) in percentage of the proposed multimodal system, calculated after merging the decisions by majority vote. Modality Reference template Error rate Face and DDF & (average) (average) 40 signatures 0.09%

[0133] The monitoring solution may comprise an application of a majority vote for each signature saved in the database (or a summation of two scores obtained to verify the identity of the learners). For this, the monitoring solution may comprise a use of a resemblance score from the face analysis after having applied a deep learning process by obtaining a characteristic vector of each learner and a similarity score between the characteristic vectors determined during the keyboard activity.

[0134] A significant decrease in the EER value obtained at 0.09% compared to the values ​​obtained for the modalities alone (0.38% for the DDF and 0.09% for the face using fusion at the decision level) is observed, which demonstrates the interest of the multimodal approach of the monitoring solution.

Claims

Claims

1. A computer-implemented method for analyzing a user's keyboard activity, the method comprising: - obtaining (S21) the instants of key pressing and / or release during typing of a reference text on a client computer system by the user, the reference text comprising one or more words;characterized in that the method comprises: - a determination (S22), for each respective word of the reference text, of the value of a characteristic vector for the respective word, the characteristic vector comprising for a typed word one or more metrics each determined from the instants of pressing and / or releasing the key during typing of the reference text, the one or more determined metrics comprising a metric D5 equal to the time between the instant of pressing the respective key corresponding to the first character of the respective word and the instant of releasing the respective key corresponding to the last character of the respective word; and - a determination (S23) of the value of a signature for the reference text from each determined characteristic vector.;

2. The method of claim 1, wherein the one or more determined metrics comprise: - a metric Di equal to the time, for each character i, between the instant of pressure P t of the respective key corresponding to the character i and the instant of release R t of the respective key corresponding to the character i, - a metric D2 equal to the time, for each character i, between the instant of pressure P t of the respective key corresponding to the character i and the instant of pressure P t +7 of a respective key corresponding to the following character i+1, - a metric D3 equal to the time, for each character i, between the instant of pressure P t of the respective key corresponding to the character i and the instant of release R i+2 of a respective key corresponding to the second following character i+2, and / or - a metric D4 equal to the time, for each character i, between the instant of pressure P t of the respective key corresponding to the character i and the instant of pressure P t +3 of a respective key corresponding to the third following character i+3.

3. A method according to claim 1 or 2, wherein the reference text comprises several words, the signature for the reference text comprising an average of the characteristic vectors determined for each of the words of the reference text.

4. A computer-implemented method of monitoring keyboard activity on behalf of a user, the method comprising: - providing (S34) the value of a signature determined according to any one of claims 1 to 3 for the user; - obtaining (S31) the instants of key pressing and / or release during typing of an online text on a client computer system on behalf of the user, the online text comprising one or more words; - determining (S32, S33), for each respective word of the online text, the value of the characteristic vector for the respective word; and - determining whether or not the identity of the user has been usurped based on a comparison (S35) between one or more values ​​of the characteristic vector determined for the online text and the value of the signature determined for the reference text.

5. The method of claim 4, wherein determining whether spoofing is occurring is further based on a comparison (S40) between a facial capture while typing online text on the client computer system on behalf of the user and an image of the user from a database.

6. The method of claim 5, wherein determining whether spoofing occurs comprises merging the results of comparing the one or more feature vector values ​​determined for the inline text and comparing the facial capture.

7. A method according to any one of claims 4 to 6, wherein determining whether or not the user has been impersonated comprises: - determining, for each respective word of the online text, a distance between the value of the characteristic vector of the respective word and the value of the signature determined for the user; and - determining whether or not the user has been impersonated from each determined distance.

8. The method of claim 7, wherein the feature vector and the signature are each a vector having coordinates for each of the one or more metrics, the distance determined for each respective word of the online text being equal to a norm of the difference between the value of the characteristic vector of the respective word and the value of the signature determined for the user.

9. A method according to claim 7 or 8, the method further comprising: - for each respective word of the online text, a comparison of the distance determined for the respective word with a confidence threshold; and - a display of the result of the comparison for each respective word of the online text.

10. A computer program comprising program code instructions for executing, when said program is executed on a computer, the steps of the method according to any one of claims 1 to 3 and / or the method according to any one of claims 4 to 9.

11. A server computer system configured to communicate with a client computer system, the server computer system having a memory on which the computer program according to claim 10 is recorded, the server computer system being configured to execute the computer program.

Images