Wrist-worn device for biometric user identity verification, biometric user identity verification method and computer-readable storage medium

Abstract

The invention relates to a device (1) worn on the wrist (7) for biometric user identity verification, comprising a skin texture sensor (2) and a central unit (5) connected thereto for receiving and processing sensor data, held on the wrist (7) by means of a device carrier element (1), and characterized in that it further comprises an inertial sensor (3) connected to the central unit (5). The central unit (5) is configured for biometric user verification based on a combined analysis of sensor data received from the skin texture sensor (2) and the inertial sensor (3). The invention also relates to a method for biometric user identity verification, comprising: recording (101) a texture image; recording (102) inertial data; classifying (103) the recorded skin texture image as matching or not matching a previously recorded skin texture pattern and storing, respectively, a positive or negative partial verification; classifying (104) the recorded inertial data as matching or not matching a previously recorded inertial data pattern and storing, respectively, a positive or negative partial verification; repeating the recording (101, 102) and classifying (103, 104) steps at predetermined time intervals or as a result of specific events, and accumulating successive partial verifications (105, 106) separately for skin texture images and inertial data. Positive verification and user authentication (107) occur when M consecutive positive partial verifications are obtained among the last accumulated partial verifications of skin texture images, where M ≥ 1; and at least Nprog positive partial verifications are obtained among the last Npoz accumulated partial verifications of inertial data, where Nprog ≤ Npoz, whereas otherwise negative verification and user non-authentication (108) occur. The invention further relates to a computer-readable storage medium comprising program instructions for carrying out the method according to the invention.

Description

[0001] 18 March 2025

[0002] WRIST- WORN DEVICE FOR BIOMETRIC USER IDENTITY VERIFICATION, BIOMETRIC USER IDENTITY VERIFICATION METHOD AND COMPUTER- READABEE STORAGE MEDIUM

[0003] This invention relates to a device and a method for biometric user identity verification, and in particular, to a multi-sensor device intended for placement on the wrist and a multimodal method for user identity verification using biometric data recorded by this device, as well as a computer-readable storage medium containing instructions for implementing the user identity verification method.

[0004] STATE OF THE ART

[0005] In recent years, there has been an increased interest, both scientific and market-driven, in the use of wearable devices for automatic, biometric user identity verification. This application is based on the ability of worn devices to record multiple physical, behavioral, or cognitive features, from which characteristic patterns can be extracted and used for automatic identification or differentiation of users, as described, among others, in the publication E. Maiorana, ‘A survey on biometric recognition using wearable devices’, Pattern Recognit. Lett., vol. 156, pp. 29-37, Apr. 2022, doi: 10.1016 / j.patrec.2022.03.002.

[0006] Numerous scientific papers show that many of the available solutions, whether marketready or prototypes, exhibit a very limited ability to distinguish users. The Equal Error Rate (EER) often exceeds 10%. As indicated in the aforementioned publication by E. Maiorana, these results are influenced by: imperfections in the devices used, a small study group, and the specificity of the physical feature used. Therefore, existing worn devices offering identity verification functionality, such as NymiTM and Flywalletpay®, utilize the popular fingerprint, which involves the necessity of placing the other hand on the sensor, and due to the single and known verification mechanism, may be more susceptible to forgery attempts - although this susceptibility depends on the type of sensor used in the device.

[0007] In the work by H. Kim et al., ‘A Wearable Wrist Band-Type System for Multimodal Biometrics Integrated with Multispectral Skin Photomatrix and Electrocardiogram Sensors’, Sensors, vol. 18, no. 8, p. 2738, Aug. 2018, doi: 10.3390 / sl8082738, it was shown that identity verification based on information about the structure of the palmar side of the wrist, extended with physiological information in the form of an ECG signal, allows achieving an EER of 0.1% (thus, the key security factor of False Acceptance Rate FAR < 0.1% is also achievable). The structural information was obtained using light absorption sensors of various wavelengths, and the ECG signal was recorded when the finger of the other hand was placed on the wrist-worn device.

[0008] Technical solutions in the form of watch clasps containing electronic circuits, including various biometric sensors, are known. Sony's product: Wena 3, offering "smartwatch" functionality in the clasp itself, has biometric sensors in the form of photoplethysmographic sensors on the inner side, used to monitor user activity.

[0009] Patent application EP3366155A1 discloses a watch band buckle comprising a modular wearable electronic device and encompasses the general concept of embedding an electronic circuit in a symmetrical butterfly clasp "surrounding" the electronic circuit enclosed in a waterproof housing, which may contain one or more sensors enabling: step counting, sleep monitoring, body temperature measurement, humidity measurement, heart rate measurement, UV radiation measurement.

[0010] U.S. Patent No. US 10898135B2 describes a device for measuring biometric information, a method for managing biometric information, and a program for managing biometric information. The disclosed solution includes the combined use of any biometric sensor located on the palmar side of the wrist for identity verification and a body contact sensor to maintain authentication, the result of which is stored in the device's built-in memory.

[0011] U.S. Patent No. US10398370B2 relates to a wearable electrical device comprising a wrist biometric sensor for acquiring skin texture pattern images and related methods. The device described in claim 1 includes, among other things, a device body, a band connected to the body, and a wrist biometric sensor, comprising an infrared (IR) thermal imaging sensor, contained in the body or band. A processor connected to the wrist biometric sensor cooperates with the IR thermal imaging sensor to acquire skin texture pattern images from nearby portions of the user's wrist and, based on this, can perform an authentication function. In the event of a successful match, the processor can, for example, activate the functions of the wearable electronic device. An electric field sensor or an optical sensor with sensor pixels can be used as the biometric sensor.

[0012] U.S. Patent Application No. US2016283808A1 relates to authentication based on skin texture. The disclosed skin texture-based authentication system comprises a recording module, an analysis module, and an authentication module. An input representation of a user's skin received by the first module in a device worn by the user is analyzed by the second module to identify a set of features, and the user is authenticated by the third module based on this. The imaging mechanism can be located on the underside of the wrist, near the clasp. The disclosed solution uses a single physiological feature of the user for authentication.

[0013] U.S. Patent No. US10146921B2 describes the use of a butterfly clasp as a carrier for a device for identity verification based on a wrist vein image recorded using a near-infrared (NIR) camera. The vein image is recorded either from a distance, before closing the clasp, or up close, in fragments and then combined into an image of the area necessary to assess the vein pattern under the skin. At the same time, a closed-state sensor is used in the clasp to activate / deactivate authentication. The disclosed solution uses a single physiological feature of the user for authentication.

[0014] U.S. Patent Application No. US2015242605A1 relates to a method for conducting continuous identity verification based on data from various sources. The claims do not relate to the devices used, although the solution description mentions a wide list of various devices that can implement the reported method in their software. The patent does not cover the construction of devices dedicated to implementing the method. The list of devices does not include a butterfly or snap clasp for a watch band.

[0015] U.S. Patent No. US10075846B1 relates to continuous user authentication using worn devices. The solution describes continuous verification based on a worn device that includes multiple sensors used to perform a registration phase and a verification phase. Sensors include, among others, an accelerometer and a gyroscope providing 3D data. With the increasing popularity of wearable electronic devices worn by users on their hands, particularly on the wrist, user identification based on their individual biometric features is beginning to play a very important role due to the risk of access to many sensitive information and functions in the event of loss, theft, or the mentioned electronic devices falling into the wrong hands. Solutions based on biometric sensors of wrist skin topography or texture as single authentication factors, known from the aforementioned solutions disclosed in US10146921B2, US 10898135B2, and US10398370B2, despite providing a basic security mechanism against use by an unauthorized person, still require further improvements to significantly increase the level of security while ensuring cyclical authentication and user verification, operating in the most automated way possible without unnecessary user involvement, such as the need to perform additional actions. Therefore, there is a need to develop solutions based on biometric sensors of skin topography or texture with additional degrees of authentication and verification. The aim of the present invention is to provide a device and method for biometric user identity verification that meet the above requirements.

[0016] According to the invention, a wrist-worn device for biometric user identity verification is provided, comprising a skin texture sensor and a central unit connected thereto for receiving and processing sensor data, held on a wrist by means of a device carrier element, characterized in that it further comprises an inertial sensor connected to the central unit, wherein the central unit is configured for biometric user verification based on a combined analysis of sensor data received from the skin texture sensor and the inertial sensor.

[0017] In one advantageous embodiment, the device further comprises a skin texture sensor operation management system connected between the skin texture sensor and the central unit.

[0018] In another advantageous embodiment, the inertial sensor comprises a three-axis accelerometer and / or a three-axis gyroscope.

[0019] In yet another advantageous embodiment, the skin texture sensor is a capacitive, ultrasonic, or optical sensor comprising an array of sensor elements.

[0020] In another advantageous embodiment, the device further comprises a wireless communication system connected to the central unit for transmitting data recorded by the skin texture sensor and / or the inertial sensor and / or the result of the identity verification to an external portable device.

[0021] In another advantageous embodiment, the device carrier element is the body of a wrist- worn band clasp, wherein the skin texture sensor is located on the underside of the clasp facing the wrist skin.

[0022] In another advantageous embodiment, the clasp is a butterfly or snap clasp.

[0023] In another advantageous embodiment, the device further comprises a clasp closed-state sensor connected to the central unit, for initiating and terminating biometric user verification.

[0024] In another advantageous embodiment, the central unit is configured to initiate biometric user verification based on a clasp snapping event recorded by the inertial sensor.

[0025] In another advantageous embodiment, the device carrier element is a band worn on the user's wrist, wherein the skin texture sensor is located on the underside of the band facing the wrist skin. In another advantageous embodiment, the device carrier element is a watch worn on the user's wrist, wherein the skin texture sensor is located on the underside of the watch facing the wrist skin, and the device shares the central unit and / or the wireless communication system with the watch.

[0026] According to the invention, a method for biometric user identity verification is also provided, characterized in that it comprises: recording a skin texture image using a skin texture sensor placed on the user's wrist skin; recording inertial data using an inertial sensor placed on the user's wrist; classifying the recorded skin texture image as matching or not matching a previously recorded skin texture pattern and storing, respectively, a positive or negative partial verification; classifying the recorded inertial data as matching or not matching a previously recorded inertial data pattern and storing, respectively, a positive or negative partial verification; repeating the recording and classifying steps at predetermined time intervals or as a result of specific events, and accumulating successive partial verifications separately for skin texture images and inertial data; positive verification and user authentication in the event that:

[0027] M consecutive positive partial verifications are obtained among the last accumulated partial verifications of skin texture images, where M > 1; and at least Nprogpositive partial verifications are obtained among the last Npozaccumulated partial verifications of inertial data, where Nprog< Npoz; and negative verification and user non-authentication otherwise.

[0028] In one advantageous embodiment, the recording of inertial data is performed using a three-axis accelerometer and / or a three-axis gyroscope.

[0029] In another advantageous embodiment, the method is initiated and terminated depending on the state of the clasp located on a band worn on the user's wrist, indicated by the clasp closed- state sensor, and / or depending on the detection of contact between the skin texture sensor and the skin.

[0030] In yet another advantageous embodiment, the recording of inertial data is continuous, and the classification of recorded inertial data and the determination of partial verifications are performed for successive time intervals of a predetermined length, wherein the previously recorded pattern constitutes one or more actions recorded by the inertial sensor.

[0031] In yet another advantageous embodiment, one or more actions are selected from the group comprising: putting on the hand, fastening on the wrist, typing on a keyboard, walking, climbing stairs, descending stairs.

[0032] In yet another advantageous embodiment, after the method is initiated, the first partial verification of inertial data is based on a pattern in the form of the actions of putting on the hand and / or fastening on the wrist. The invention further relates to a computer-readable storage medium comprising computer program instructions which, when executed by a central unit and / or a portable device, cause the execution of the method described above.

[0033] The operation of the device consists in that, in the first step, the device recognizes the moment of putting it on the hand by detecting contact of the skin texture sensor with the body surface or detecting a change in the clasp state to closed (in embodiments equipped with a hardware clasp closed-state sensor or embodiments in which the snapping of a snap or butterfly clasp is detected by the inertial sensor). Immediately after putting it on, it records an image of the wrist skin texture and the courses of inertial signals occurring during the actions of putting on and fastening (time fragments of signals are recorded, covering a time segment starting before the moment of fastening detection but ending after that moment). These data are used to carry out the first stage of identity verification, the result of which constitutes the first part of the accumulated verification results.

[0034] While the device remains on the user's wrist, it performs successive data recordings: wrist skin texture, which can take place periodically - with a predetermined frequency or as a result of specific conditions / events, e.g., event 1 in the form of a request submitted by an external application or event 2 in the form of immobility detection by the central unit based on signals from the inertial sensor (immobility favors high-quality image recording) or event 3 consisting of the end of a high-activity period detected by the central unit based on signals from the inertial sensor. These data are transmitted to a method (classifier) performing identity verification; inertial signals, which can proceed continuously, with a predetermined frequency, or as a result of specific conditions / events, e.g., event 1 in the form of an external application request or event 2 consisting of the central unit detecting, based on signals from the inertial sensor, the end of an immobility period or event 3 consisting of the central unit detecting motion parameter values recorded by the inertial sensor that meet the criteria adopted for a given application. Time segments of these signals are transmitted to a method (classifier) recognizing selected actions and then, if one of the predefined actions is recognized, to a method (classifier) performing identity verification.

[0035] The above-mentioned conditions / events, which may influence the moments of recording the skin texture image or inertial signals by the appropriate sensors, are merely examples. A person skilled in the art will also be aware of other conditions / events not directly indicated here. The term "external application" should be understood as, for example, an application running on an external device that communicates with the device according to the invention, preferably wirelessly, so that the operation of the device according to the invention can be adapted to a specific external device and related application.

[0036] Successive partial verification results are accumulated with the results obtained at the moment of putting the device on the hand. When the set of collected results becomes sufficient to confirm identity with the reliability adopted in a given embodiment, the user is considered authenticated.

[0037] Authentication is maintained until the device is removed from the hand (interruption of contact between the skin texture sensor and the body surface or detection by the clasp closed- state sensor of its opening). Authentication can also be interrupted by software as a result of specific conditions / events (e.g., when the connected portable device connects to an unsecured Wi-Fi network or recognizes unusual events, e.g., a strong shock recorded by the inertial sensor that may accompany an accident, followed by a period of immobility, or detection by the central unit or connected portable device of an unauthorized connection attempt with the device (so- called BlueBorne attack) or the occurrence of too many negative partial verifications.

[0038] The operation according to the above description requires that each new user of the device first carry out, with the help of this device, the process of registering patterns of their biometric data, with which the data recorded during normal use will then be compared.

[0039] Automatic, biometric user identity verification is possible immediately after putting on a watch, bracelet, or other wrist-worn device, as well as its multiple confirmations and maintaining the obtained authorization until the moment of unfastening and removing the device from the wrist.

[0040] The solution is intended for use, for example, with standard watches and watch bands, and thus allows the user to secure access to selected resources in a discreet and automatic way. The device is invisible, and the identity verification process does not require any additional or non-standard actions.

[0041] This authorization can enable previously defined functions, such as the contactless payment function built into the same clasp, and / or can be transmitted via a short-range link (active, e.g., Bluetooth, or passive, e.g., by activating the NFC system) to any cooperating device (phone, tablet, electronic lock, etc.).

[0042] The invention will become better understood upon reviewing the detailed description of its exemplary embodiments. The subject of the invention in the examples of execution is shown in the drawing, in which:

[0043] Fig. 1 presents the general structure and principle of operation of the device according to the invention;

[0044] Fig. 2A presents the device according to the invention integrated into a watch band clasp according to the first example of the invention;

[0045] Fig. 2B presents in more detail the first example of the device according to the invention in the form of a butterfly clasp in a top view and in a cross-sectional view A-A;

[0046] Fig. 2C presents the first example of the device according to the invention from Fig. 2B in a top view, in an unfastened arrangement;

[0047] Fig. 2D presents a diagram of the connections of the electronic circuit elements of the device in the first example of the invention;

[0048] Fig. 3A presents the device according to the invention integrated into a wristwatch according to the second example of the invention;

[0049] Fig. 3B presents a diagram of the connections of the electronic circuit elements of the device in the second example of the invention;

[0050] Fig. 4A presents the device according to the invention in the form of an electronic circuit made on a flexible printed circuit board (so-called flex), integrated into a watch band according to the third example of the invention; Fig. 4B presents a diagram of the connections of the electronic circuit elements of the device in the third example of the invention;

[0051] Fig. 5 presents a diagram of the operation of the multimodal, automatic method of biometric identity verification using biometric data recorded by the device according to the invention.

[0052] Fig. 1 presents the general structure and principle of operation of the device 1 according to the invention. The device 1 comprises a skin texture sensor 2 adjacent during use to the wrist skin 7 of the user's hand, an inertial sensor 3 (inertial measurement unit - IMU) comprising at least a single- or multi-axis accelerometer or an accelerometer and a gyroscope, a skin texture sensor operation management system 4, a central unit 5 (e.g., a microcontroller or other circuit capable of receiving and processing data from sensors), and a wireless communication system 6 enabling the transmission of data or the result of identity verification to an external connected portable device 8, which facilitates the operation of the device according to the invention but is not essential for its operation. The device 1 is intended for use with any wrist-worn device or piece of jewelry (in particular, a watch and band) and automatically performs biometric identity verification. Communication with an external device, such as a portable device 8 (e.g., a smartphone), can be implemented using wireless or wired connectivity and thus provide the user with access to protected digital resources or resources protected using digital security. Advantageous forms of the device 1 according to the invention are shown in the following examples of execution described and explained in more detail later in this application.

[0053] First Example Embodiment - Circuit Integrated into a Butterfly Clasp for a Watch Band

[0054] Fig. 2A presents the device 1 according to the invention in the first example embodiment. As shown in the figure, a watch 10 is placed on the user's wrist 7, fastened with a band 9 equipped with a clasp, which in the presented example is in the form of a butterfly clasp, shown in more detail in Figures 2B and 2C. Integrated into the clasp is a skin texture sensor 2, an inertial sensor 3, and a clasp closed-state sensor 1.9. The skin texture sensor 2 is located on the underside of the butterfly clasp and has a shape adapted in such a way that after putting on the watch and fastening the band 9 with the butterfly clasp, the mentioned skin texture sensor 2 adheres with its entire surface to the wrist skin 7 on the inner (palmar) side.

[0055] The butterfly clasp is shown in more detail in Figs. 2B and 2C, showing respectively its cross-sectional view and top view, in an unfastened state with visible component elements of the device according to the invention. In addition to the previously mentioned skin texture sensor 2 placed externally on the underside of the clasp, an electronic module is integrated into the body of the butterfly clasp, comprising an inertial sensor 3 (3-axis accelerometer and 3-axis gyroscope), a clasp closed-state sensor 1.9, and electronic components including a skin texture sensor operation management system 4, a central unit 5, and a wireless communication system 6, which in this example uses the Bluetooth standard. The internal space of the clasp body, in which the mentioned electronic module is integrated, is tightly closed to protect the electronic components from the effects of moisture or the ingress of contaminants such as dust. Alternatively or additionally, the electronic module itself may be waterproof. The butterfly clasp can be freely unfastened and fastened without affecting the structure of the electronic module integrated into its body. After putting on the watch 10 on the hand and fastening the butterfly clasp connected to the band 9, the skin texture sensor 2 covering the surface of the butterfly clasp facing the user's body adheres with its entire surface to the palmar side of the user's wrist 7, thus enabling the recording of the wrist skin texture or topography. Similar to fingerprints, the topography or texture of the wrist skin is individual to each person, and therefore the image of the wrist skin topography or texture recorded using the skin texture sensor 2 can be used for biometric authentication. In the context of this invention, the terms "skin texture" or "skin topography" should therefore be understood as the pattern of epidermal wrinkles together with other features such as birthmarks, pores, spots, freckles, warts. These wrinkles, also referred to as sulci cutis or glyphic patterns, are grooves that intersect to form diverse, person-unique polygons.

[0056] In connection with Figs. 2A, 2B, and 2C, Fig. 2D shows a diagram of the connections of the electronic module elements. In addition to the previously presented and discussed elements: skin texture sensor 2, inertial sensor 3, hardware clasp closed-state sensor 1.9, dedicated skin texture sensor operation management system 4, central unit 5 in the form of a suitably selected microcontroller equipped with memory, and wireless communication system 6 (in this example, Bluetooth), the waterproof electronic module in the first example embodiment also includes a battery 1.1, a power management system 1.2, and an inductive charging system 1.3.

[0057] In the presented example, a flexible capacitive skin texture sensor 2 is used, the active elements of which (capacitor array) are applied to a film, enabling its mounting (gluing) to the profiled underside surface of the clasp. The profiling aims to match the shape of the clasp and sensor 2 to the typical curvature of the wrist section 7, as is the case with a typical butterfly and snap clasp. Its dimensions and fixed mounting location mean that each time it is put on, the skin texture sensor 2 is in the same location, recording the same area of the wrist skin 7. An important advantage of this example embodiment is that fastening the butterfly or snap clasp, due to its operating principle, involves pressing it against the wrist surface, which results in significantly better conditions for recording the wrist skin texture image by the skin texture sensor 2 when putting the device on the hand.

[0058] The capacitive skin texture sensor 2 is connected to a dedicated skin texture sensor operation management system 4 to enable the recording of skin texture images from an area of 10 x 24 mm with a resolution of 200 x 480 pixels (508 DPI). The recorded images are transmitted via an SPI link from the system 4 to the microcontroller constituting the central unit 5 of the device 1.

[0059] The hardware clasp closed-state sensor 1.9 is connected to the central unit (microcontroller) 5 and enables immediate detection of the moment of removing the device 1 from the wrist 7 and termination of user authentication. The clasp closed-state sensor 1.9 can be made as any electrical circuit in which the mechanism latching / holding the clasp in the closed state causes a short circuit of contacts in the circuit, which is recorded on a selected input of the central unit 5, or by using a proximity sensor (inductive or capacitive) detecting the direct proximity of the moving parts of the clasp.

[0060] The digital inertial sensor 3 includes a three-axis accelerometer and a three-axis gyroscope. As a result of movements made by a user wearing the device 1 according to the invention on their wrist 7, the inertial sensor 3 records inertial signals related to this movement, which it transmits directly to the microcontroller 5 connected to it via an SPI link. The recorded skin texture data (in the form of an image formed by the signal values of individual capacitors in the array) and movement activity (6 signals) are transmitted from the central unit (microcontroller) 5, via the wireless communication system 6 (Bluetooth) connected to the central unit (microcontroller) 5 using an SPI link, to an external connected portable device 8, e.g., a smartphone, where user identity verification is performed using them, e.g., by comparison with patterns of these data previously recorded during the user's "learning" stage (new user registration). In the absence of a connection to the portable device 8, the data are stored in the memory of the central unit 5, and are transmitted after the connection is reestablished. The result of user authentication is stored in the memory of the connected portable device 8 and from there can determine the user's access to specific resources.

[0061] The battery 1.1 is connected via a power stabilizing and charging control system 1.2 to elements 3, 4, 5, 6 to provide electrical power and to an inductive charging system 1.3 enabling the charging of the battery 1.1 from an external power source. For clarity, the mentioned elements 1.1, 1.2, and 1.3 are not shown in Figs. 2a and 2b. In the presented example embodiment, they are placed in the clasp body together with the rest of the device elements.

[0062] The advantage of this example embodiment is that watch and bracelet clasps are the most universal parts of these jewelry (equipment) items. With an extremely wide range of watches, bands, and bracelet designs, clasps are offered in a small number of variants, primarily including different colors and widths of supported bands. Therefore, implementing the solution in this part provides the greatest opportunities to combine it with other, individually selected elements.

[0063] Second Example Embodiment - Circuit Integrated into a Wristwatch

[0064] Fig. 3A presents the device 1 according to the invention in the second example embodiment. As shown in the figure, a watch 10 is placed on the user's wrist 7, fastened with a band 9 equipped with a clasp. In this example embodiment, the skin texture sensor 2 covers the surface of the watch 10 facing the user's body, and the inertial sensor 3, along with the entire electronic module (not shown in Fig. 3A), is integrated into the watch 10. After putting the watch 10 on the wrist 7, the skin texture sensor 2 adheres to the dorsal side of the wrist 7. In this example embodiment, an ultrasonic skin texture sensor 2 with a diameter of 28 mm is used.

[0065] In connection with Fig. 3A, Fig. 3B shows a diagram of the connections of the electronic module elements. In the watch 10, on a separate PCB or integrated with other watch elements, there are electronic components including an inertial sensor 3 and a dedicated system 4 connected to the ultrasonic skin texture sensor 2 operating it. Further data processing is performed by the built-in microcontroller 10.5 (acting as the central unit) of the watch 10 connected to the system 4 and the inertial sensor 3 via SPI links and having a built-in Bluetooth wireless communication function. The systems are powered by the watch's built-in battery 10.1 controlled by the built-in power management and stabilization and charging system 10.2, which uses a standard built-in contact link 10.4 for charging.

[0066] Placing the ultrasonic skin texture sensor 2 on the dorsal side of the wrist 7, which is approximately flat, ensures its good adhesion to the skin despite its mounting on the flat underside surface of the watch case 10. Its surface area and fixed mounting location mean that each time it is put on, the sensor 2 is in the same location, recording the same area of the wrist skin 7. An important advantage of this example embodiment is that the act of fastening the band with a classic buckle with a pin involves a momentary stronger tension of this band, which results in pulling the watch case against the wrist surface, thus providing significantly better conditions for recording the wrist skin texture image when putting the device on the hand.

[0067] The ultrasonic skin texture sensor 2 simultaneously provides control of the continuity of adhesion of the skin texture sensor 2 to the body surface, thanks to which it is possible to immediately detect the moment of removing the device from the hand and terminate user authentication. The moment of separation of the skin texture sensor 2 is detected by the occurrence of a significant change in the intensity of the image recorded by the skin texture sensor, which is recorded immediately thanks to the function of triggering by a change in the state of the recorded signals used in this type of sensor. In a similar way, the detection of the moment of putting the watch 10 on the user's wrist 7 can take place, when the skin texture sensor 2 comes into contact with the skin surface again. The event related to putting on or removing the watch 10 (or more generally related to putting on or removing the skin texture sensor 2 (adhesion to the skin or its absence)) from the wrist 7 can be used in some embodiments to, respectively, initiate and terminate the user verification process.

[0068] The ultrasonic skin texture sensor 2, in combination with the dedicated skin texture sensor operation management system 4, enables the recording of skin texture images from an area with a diameter of 25 mm with a resolution of 10 pixels / mm (254 DPI). The recorded images are transmitted via an SPI link to the built-in microcontroller 10.5 of the watch 10, which constitutes the central unit of the device. The ultrasonic skin texture sensor is a two- dimensional array of fine ultrasonic transducers of any kind - piezoelectric, capacitive, or others, in which the transducers first emit short ultrasonic pulses and then record reflected / scattered echoes, which carry information about the three-dimensional structure of the skin's top layer - its folding, but also the spatial distribution of mechanical properties. The advantage of using an ultrasonic sensor over resistive or optical sensors is that its response is influenced not only by the shape of the skin surface itself, but also by the spatial distribution of its mechanical properties (including density and elasticity), thanks to which this type of sensor provides significantly greater resistance to fraud attempts by using a copy of the recorded skin surface impression. The advantage of an ultrasonic sensor over a capacitive one is its higher accuracy, lower sensitivity to external conditions, faster response, and even higher resistance to fraud attempts.

[0069] The digital inertial sensor 3 located inside the watch 10 includes a three-axis accelerometer and a three-axis gyroscope. As a result of movements made by a user wearing the watch 10 on their wrist 7, the inertial sensor 3 records inertial signals related to this movement, which it transmits via an SPI link directly to the microcontroller 10.5 acting as the central unit.

[0070] Data concerning skin texture (image) and movement activity (6 signals) are transmitted from the microcontroller 10.5 (i.e., the central unit), via the built-in Bluetooth wireless communication system to an external device, e.g., a phone 8, where user identity verification is performed using them, e.g., by comparison with patterns of these data recorded during the registration ("learning") stage of a new user. In the absence of a connection to the portable device 8, the data are stored in the memory 10.11 of the watch connected to the microcontroller 10.5, and are transmitted after the connection is re-established. Alternatively, verification can take place directly in the central unit (microcontroller 10.5). The battery 10.1 is connected via a power stabilizing and charging control system 10.2 to elements 3, 4, 10.5 to provide electrical power and to a contact link 10.4 enabling the charging of the battery 1.1 from an external power source. For clarity, the mentioned elements 10.1, 10.2, 10.4, 10.11 are not shown in Fig. 3A. In the presented example embodiment, they are placed in the watch case together with the rest of the device elements.

[0071] The advantage of this example embodiment is that the circuit is integrated into an existing device that also performs other functions. Where such integration is possible, a significant reduction in the costs of implementing the circuit and method and a significantly higher degree of integration of the biometric security offered by the circuit with the protected functions available in the watch can be achieved. Sharing electronic resources and batteries further reduces the costs of implementing the solution. A higher performance battery allows the use of an ultrasonic skin texture sensor, which provides a higher level of biometric security.

[0072] Third Example Embodiment - Circuit Integrated into a Watch Band

[0073] Fig. 4A presents the device 1 according to the invention in the third example embodiment. As shown in the figure, a watch 10 is placed on the user's wrist 7, fastened with a band 9 equipped with a clasp. In this example embodiment, the watch band 9 contains a built- in electronic module performing the relevant functions. After putting the watch 10 together with the band 9 on the wrist 7, the skin texture sensor 2 adheres to a portion of the wrist circumference 7. The flexible capacitive skin texture sensor 2 with dimensions of 12 x 25 mm (3) covers the surface of the watch band facing the user's body.

[0074] In connection with Fig. 4A, Fig. 4B shows a diagram of the connections of the electronic module elements. Inside the band 9, on a separate flexible PCB (so-called flex), there are electronic components including an inertial sensor 3 and a dedicated system 4 operating the capacitive skin texture sensor and a microcontroller 5 with a Bluetooth communication system, a battery 1.1, a power management and stabilization and charging system 1.2, and a contact connector 1.4 enabling the charging of the battery 1.1.

[0075] The use of a flexible capacitive skin texture sensor 2 attached (glued) to a flexible watch band 9 ensures good adhesion of the sensor to the user's skin. Its surface area and fixed mounting location mean that each time it is put on, the sensor 2 is in the same location, recording the same area of the wrist skin 7. An important advantage of this example embodiment is that the act of fastening the band with a classic buckle with a pin involves a momentary stronger tension of this band, which results in pulling the entire band circumference against the wrist surface, thus providing significantly better conditions for recording the wrist skin texture image when putting the device on the hand.

[0076] The capacitive skin texture sensor 2 simultaneously provides control of the continuity of adhesion of the skin texture sensor 2 to the body surface, thanks to which it is possible to immediately detect the moment of removing the device from the wrist 7 and terminate user authentication (as explained earlier).

[0077] The capacitive skin texture sensor 2, in combination with the dedicated skin texture sensor operation management system 4, enables the recording of skin texture images from an area with dimensions of 11 x 23 mm with a resolution of 220 x 460 pixels (508 DPI). The recorded images are transmitted via an SPI link to the microcontroller 5, which constitutes the central unit of the device. The advantage of using a capacitive sensor over resistive or optical sensors is that its response is influenced not only by the shape of the skin surface itself, but also by the spatial distribution of its properties (dielectric permittivity), thanks to which this type of sensor provides significantly greater resistance to fraud attempts by using a copy of the recorded wrist skin impression. The advantage of a capacitive sensor over an ultrasonic one is its simpler design, lower cost, and lower power consumption.

[0078] The digital inertial sensor 3 located in the watch band 9 of the watch 10 includes a three- axis accelerometer and a three-axis gyroscope. As a result of movements made by a user wearing the band 9 on their wrist 7, the inertial sensor 3 records inertial signals related to this movement, which it transmits via an SPI link directly to the microcontroller 5.

[0079] Data concerning skin texture (image) and movement activity (6 signals) transmitted to the microcontroller 5 (central unit) are classified directly in the microcontroller 5 by dedicated software implemented in it. In this case, patterns characterizing the authorized user, obtained during the authorized user's "learning" process, are stored in the microcontroller 5's memory. The authentication result is transmitted through the wireless communication function (built into the microcontroller 5) to an external device 8.

[0080] The battery 1.1 is connected via a power stabilizing and charging control system 1.2 to elements 3, 4, 5 to provide electrical power and to a contact connector 1.4 enabling the charging of the battery 1.1 from an external power source. For clarity, the mentioned elements 1.1, 1.2, 1.4 are not shown in Fig. 4A. In the presented example embodiment, they are placed in the watch band together with the rest of the device elements.

[0081] The advantage of this example embodiment is that it provides the greatest convenience of use, which may be a solution addressed to recipients for whom the use of a butterfly or snap clasp is cumbersome due to the fact that such a clasp is stiffer and thicker than a standard buckle / clasp with a pin.

[0082] Although the above examples describe specific forms of the device according to the invention, it will be obvious to those skilled in the art that the invention is not limited to the details presented. For example, the skin texture sensor is not limited to a capacitive or ultrasonic sensor, and the invention also includes the use of any other type of skin texture sensor, e.g., an infrared sensor, an optical sensor, or another type of imaging sensor, provided it enables effective acquisition of a skin texture image. Optical sensors may include arrays of photosensitive elements that record light reflected from the skin surface. Light for detection purposes, with various wavelengths covering the visible range or also infrared, is emitted by elements that are part of the sensor. The use of light in the infrared band increases the resistance of this type of sensor to fraud attempts because the recorded image results not only from the shape of the skin surface, but also from its material properties.

[0083] Furthermore, depending on the requirements, the processing of data recorded by the sensors may take place in the central unit, in the connected external device, or partly in the central unit and partly in the connected external device. In other words, in the most basic configuration, communication with the external device and related systems can be dispensed with, and one can limit oneself to connecting the sensors directly to the central unit (e.g., in the form of a suitably programmed microcontroller or other suitable dedicated system). This also applies to the previously mentioned skin texture sensor operation management system, whose function in the most basic configuration can also be taken over by the central unit. Communication with an external device can use various wireless communication standards. Other configurations of placing the device elements around the user's wrist are also possible, as long as they enable the recording of relevant data.

[0084] The device according to the invention is intended to be placed around the user's wrist. To ensure proper operation of the device, the skin texture sensor should adhere to the wrist skin, and the inertial sensor should record the user's wrist movements. According to the presented examples, various configurations of the device elements (including the central unit and other elements) around the wrist are possible, such as in a watch band, in a watch band clasp, or in a watch case. All such support means that enable the retention (e.g., by embedding, adhering, screwing, fastening, etc.) of the device elements around the wrist and their proper functioning are referred to in this application and claims as the "support element" of the device. The support element is not limited to the presented forms (band, clasp, watch case) and can have an equivalent form, provided it fulfills the above-mentioned function, e.g., a flexible band, a support element adhered to the skin using an appropriate adhesive agent, a piece of jewelry worn on the wrist, etc. Furthermore, the support element may consist of more than one part; for example, some device elements may be placed, e.g., in a watch band, and others in a clasp or watch case - in such a case, the band, watch, and clasp constitute a multi-part support element of the device. The support element may also be referred to as a carrier component, support means, carrier, etc.

[0085] Fig. 5 shows a diagram of the operation of the multimodal, automatic method of biometric identity verification using biometric data recorded by the sensors 2, 3 of the device 1 according to the invention. The subsequent steps of the method are indicated in Fig. 5 by labels 101-108. The mentioned data include wrist skin texture data recorded by the skin texture sensor 2 and inertial data on user movement obtained by the inertial sensor 3, and in particular by the accelerometer and gyroscope. Below, with reference to the diagram in Fig. 5, two example embodiments of the method according to the invention implemented by the device 1 according to the invention, shown respectively in Figs. 2A-2D and Figs. 4 A, 4B, are presented.

[0086] First Example - Implementation of the Method of Identity Verification / User Authentication in a Butterfly Clasp for a Watch Band

[0087] The method of identity verification / user authentication begins when the user puts on the watch 10 shown in Fig. 2A and fastens the butterfly clasp with the built-in device 1. The device 1 recognizes the moment of putting on the watch 10 on the wrist 7 thanks to the built-in clasp closed-state sensor 1.9 (as explained earlier). At the moment of putting on (the moment when the clasp is pressed against the skin surface and the clasp snaps shut), the device 1 records (step 101) an image of the skin texture of the palmar side of the wrist 7 using the skin texture sensor 2 and records (step 102) the waveforms of inertial signals (also referred to in this application as inertial data) occurring during the act of putting on and fastening using the inertial sensor 3 (fragments of 3 acceleration signals and 3 angular velocity signals starting 5 seconds before and ending 5 seconds after the moment of clasp closure are recorded). These data are encrypted by the software of the central unit 5 and then transmitted wirelessly, via the wireless communication system 6, to the connected portable device 8. Dedicated software installed in the connected portable device 8 classifies them using a texture classifier and a motion classifier as matching or not matching patterns previously recorded in memory (steps 103 and 104) and on this basis determines the identity verification result, which is the first part of the accumulated wrist skin texture verification results (step 105) and the accumulated motion verification results (step 106).

[0088] While the device 1 remains on the user's wrist 7, it performs further data recordings (steps 101, 102) using the relevant sensors 2, 3:

[0089] Wrist skin texture - a skin texture image is recorded every 5 minutes (step 101), which is then transmitted in encrypted form to the connected portable device 8, where its matching with the previously recorded pattern is verified using a texture classifier (step 103).

[0090] Inertial signals - the signals are recorded continuously (step 102) and transmitted to the central unit 5 of the authentication device 1, where the built-in software divides them into intervals (time ranges) of a predetermined duration, e.g., 8 seconds each, and classifies them using the HAR (Human Activity Recognition) method of recognizing selected activities (typing on a keyboard, walking, climbing stairs, descending stairs). The operation of the method is based on machine learning models that have been previously trained using the same type of data labeled with reference information about the type of movement they correspond to. The intervals (time segments) of input signals are fed directly into the method's input when using so-called convolutional neural networks (CNN) or so-called feature vectors are calculated for them (including, e.g., amplitude, frequency, statistical, energy parameters, etc.) and these feature vectors constitute the input parameters of classifiers (e.g., so-called decision trees, random forests, neural networks). Regardless of the classifier model used, it predicts the assignment of input data to one of the previously defined classes, in this case including a set of recognized activities, necessarily extended by the "other" class, to which cases that do not fit into one of the sought classes are assigned. If one of these activities is recognized in a given interval, then this interval is transmitted, in encrypted form, to the connected mobile device 8, where its validation against the previously recorded pattern of the relevant one or more activities for a given user is verified using a motion classifier (step 104).

[0091] Successive partial verification results are accumulated (steps 105, 106) by dedicated software installed in the connected portable device 8 with the results obtained at the moment of putting the authentication device 1 on the wrist 7. The user is recognized as authenticated (step 107) by the software when a positive result has been obtained for a) the number M of the last verifications based on the skin texture image, in this example M = 2, and b) at least NprOg from the number Npozof the last verifications based on inertial data, in this case Nprog= 8 from the last NPoz = 10 verifications. The information about successful authentication is sent back to the authentication device 1 and stored there. Otherwise, the user is not authenticated (step 108).

[0092] Authentication is maintained until the authentication device 1 is removed from the wrist 7 (interruption of contact between the skin texture sensor 2 and the body surface or detection by the clasp closed-state sensor 1.9 of the fact that it has been opened (as explained earlier)). The fact of unfastening the clasp is recorded in hardware in the authentication device 1 and automatically, without sending information to the connected mobile device 8, disables authentication. Authentication can also be interrupted by its deliberate deactivation in the dedicated program installed in the connected portable device 8 by the user.

[0093] The process of registering a new user involves the user repeating the act of putting on and fastening the clasp on the hand several times and recording the appropriate time of performing the activities recognized by the method: typing on a keyboard, walking, climbing stairs, descending stairs. Information about the performance of these activities can be entered manually by the user into the program installed on the connected portable device 8 and confirmed by an individual PIN number entered into the portable device 8.

[0094] The advantage of this example implementation is that, thanks to the use of a hardware clasp closed-state sensor, the change of its state is detected immediately and does not require any computational process by the device software. Transferring the process of classifying / verifying the matching of the recorded data with the pattern to the connected portable device significantly reduces the requirements for computing power and power consumption in the wearable device itself. However, it should be noted that some or all of the functions performed by the external device in another configuration of the method could also be performed by the central unit, as explained earlier.

[0095] Second Example - Implementation of the Method of Identity Verification / User Authentication in a Watch Band

[0096] The method of identity verification / user authentication begins when the user puts on the watch 10 shown in Fig. 4A and fastens the band 9 on the wrist 7. The device 1 recognizes the moment of putting on the watch 10 on the wrist 7 by a change in the state received by the skin texture sensor 2 cooperating with the skin texture sensor operation management system 4. Immediately after putting on, the authentication device 1 records an image of the skin texture (step 101), using the skin texture sensor 2, on the part of the wrist circumference 7 to which it adheres, and records the waveforms of inertial signals (step 102) occurring during the act of putting on and fastening using the inertial sensor (fragments of 3 acceleration signals and 3 angular velocity signals starting 4 seconds before and ending 5 seconds after the moment of clasp closure are recorded). The dedicated built-in software implemented in the central unit 5 classifies them using a texture classifier (step 103) and a motion classifier (step 104) as matching or not matching patterns of the relevant one or more activities for a given user previously recorded in the memory of the central unit 5 and on this basis determines the identity verification result, which is the first part of the accumulated wrist skin texture verification results (step 105) and the accumulated motion verification results (step 106).

[0097] While the authentication device 1 remains on the user's wrist 7, it performs further data recordings (steps 101, 102) using the relevant sensors 2, 3:

[0098] Wrist skin texture - when the inertial signals indicate immobility, no more often than every 30 seconds, a maximum of 5 times, or until 3 consecutive positive matching verification results are obtained (step 103) with the previously recorded pattern (including the recording made at the moment of putting the device 1 on the wrist 7).

[0099] Inertial signals - the signals are recorded continuously and transmitted to the central unit 5 of the authentication device 1, where the dedicated software divides them into intervals of a predetermined duration, e.g., 7 seconds each, and classifies them using a method of recognizing selected activities (walking, typing on a keyboard, using a mobile phone). If one of these activities is recognized in a given interval, then this interval is transmitted to the method implemented in the software of the central unit 5, which performs validation (step 104) of the data contained in the interval of 6 inertial signals with the previously recorded pattern.

[0100] Successive partial verification results are accumulated (steps 105, 106) with the results obtained at the moment of putting the authentication device 1 on the wrist 7. The user is recognized as authenticated (step 107) by the software when a positive result has been obtained for a) M = 3 consecutive verifications based on the skin texture image and b) at least Nprog= 10 from the last Npoz= 15 verifications based on inertial data. Otherwise, the user is not authenticated (step 108). Information about successful authentication is stored in the authentication device 1, where it can be used to unlock selected functions or resources, and is sent to the connected portable device 8 - immediately or after establishing a connection.

[0101] Authentication is maintained until the authentication device 1 is removed from the wrist 7, which is detected by interrupting the contact of the skin texture sensor 2 with the body surface. Authentication can also be interrupted by its deliberate deactivation in the dedicated program installed in the connected portable device 8 by the user, which requires an active wireless connection with the authentication device 1.

[0102] The process of registering a new user requires an active connection with the connected portable device 8 and involves the user repeating the act of putting on and fastening the clasp on the hand several times and recording the appropriate time of performing the activities recognized by the method: walking, typing on a keyboard, using a mobile phone. Information about the performance of these activities is entered manually by the user into the program installed on the connected portable device 8 and confirmed by an individual PIN number entered into the portable device 8.

[0103] The advantage of this example implementation is that it enables the implementation of the biometric verification process of a registered user without the need to maintain a wireless connection with the connected portable device.

[0104] Although the above example implementations describe specific sequences of the method according to the invention, it will be obvious to those skilled in the art that the invention is not limited to the details presented. For example, the intervals or ranges of reading data from individual sensors can be adjusted as needed, and the overall verification can take place taking into account arbitrarily selected numbers of verification repetitions independently for both sensors. Similarly, the threshold values M, Nprog, Npozfor positive verification of data obtained from both sensors can be set independently for each sensor.

Claims

Claims1. A device (1) worn on a wrist (7) for biometric user identity verification, comprising a skin texture sensor (2) and a central unit (5) connected thereto for receiving and processing sensor data, held on the wrist (7) by means of a support element of the device (1), characterized in that it further comprises: an inertial sensor (3) connected to the central unit (5); wherein the central unit (5) is configured for biometric user verification based on a combined analysis of sensor data received from the skin texture sensor (2) and the inertial sensor (3).

2. The device (1) according to claim 1, characterized in that it further comprises a system (4) managing the operation of the skin texture sensor (2) connected between the skin texture sensor (2) and the central unit (5);3. The device (1) according to claim 1 or 2, characterized in that the inertial sensor (3) comprises a three-axis accelerometer and / or a three-axis gyroscope.

4. The device (1) according to any one of claims 1-3, characterized in that the skin texture sensor (2) is a capacitive, ultrasonic, or optical sensor, comprising an array of sensor elements.

5. The device (1) according to any one of claims 1-4, characterized in that it further comprises a wireless communication system (6) connected to the central unit (5) for transmitting data recorded by the skin texture sensor (2) and / or the inertial sensor (3) and / or the identity verification result to an external portable device (8).

6. The device (1) according to any one of claims 1-5, characterized in that the support element of the device (1) is a body of a clasp of a band worn on the user's wrist (7), wherein the skin texture sensor (2) is placed on an underside of the clasp facing the wrist (7) skin.

7. The device (1) according to claim 6, characterized in that the clasp is a butterfly or snap clasp.

8. The device (1) according to claim 6 or 7, characterized in that it further comprises a clasp closed-state sensor (1.9) connected to the central unit (5), for initiating and terminating biometric user verification.

9. The device (1) according to claim 7, characterized in that the central unit (5) is configured to initiate biometric user verification based on a clasp snapping event recorded by the inertial sensor (3).

10. The device (1) according to any one of claims 1-5, characterized in that the support element of the device (1) is a band (9) worn on the user's wrist (7), wherein the skin texture sensor (2) is placed on an underside of the band (9) facing the wrist (7) skin.

11. The device (1) according to any one of claims 1-5, characterized in that the support element of the device (1) is a watch (10) worn on the user's wrist (7), wherein the skin texture sensor (2) is placed on an underside of the watch (10) facing the wrist (7) skin, and the device (1) shares the central unit (10.5) and / or the wireless communication system with the watch (10).

12. A method of biometric user identity verification, characterized in that it comprises:recording (101) a skin texture image using a skin texture sensor (2) placed on the user's wrist (7) skin; recording (102) inertial data using an inertial sensor (3) placed on the user's wrist (7); classifying (103) the recorded skin texture image as matching or not matching a previously recorded skin texture pattern and storing, respectively, a positive or negative partial verification; classifying (104) the recorded inertial data as matching or not matching a previously recorded inertial data pattern and storing, respectively, a positive or negative partial verification; repeating the recording (101, 102) and classifying (103, 104) steps at predetermined time intervals or as a result of specific events and accumulating successive partial verifications (105, 106) separately for skin texture images and inertial data; positively verifying and authenticating (107) the user if: o M consecutive positive partial verifications have been obtained among the last accumulated partial verifications of skin texture images, where M > 1; and o at least Nprogpositive partial verifications have been obtained among the last Npozaccumulated partial verifications of inertial data, where Nprog< Npoz; and negatively verifying and not authenticating (108) the user otherwise.

13. The method according to claim 12, characterized in that the recording (102) of inertial data is performed using a three-axis accelerometer and / or a three-axis gyroscope.

14. The method according to claim 12 or 13, characterized in that the method is initiated and terminated depending on the state of a clasp arranged on a band (9) worn on the user's wrist (7) indicated by a clasp closed-state sensor (1.9) and / or depending on the detection of contact between the skin texture sensor (2) and the skin.

15. The method according to any one of claims 12-14, characterized in that the recording (102) of inertial data is continuous, and the classifying (104) of the recorded inertial data and the determining of partial verifications is performed for their successive time intervals of a predetermined length, wherein the previously recorded pattern constitutes one or more activities recorded by the inertial sensor (3).

16. The method according to claim 15, characterized in that one or more activities are selected from the group comprising: putting on the hand, fastening on the wrist, typing on a keyboard, walking, climbing stairs, descending stairs.

17. The method according to claim 16, characterized in that after initiating the method, the first partial verification of inertial data is based on a pattern in the form of the activities of putting on the hand and / or fastening on the wrist.

18. A computer-readable storage medium containing computer program instructions which, when executed by a central unit (5) and / or a portable device (8), cause the execution of the method as defined in any one of claims 11-16.

Images