Image manipulation to detect the state of materials related to an object.

Abstract

A computer implementation method for extracting the state of materials related to an object, the method comprising the following steps: a) A step of receiving at least one pattern image, wherein the at least one pattern image shows at least a portion of an object that is illuminated by patterned electromagnetic radiation b) A step of modifying and / or deleting at least a portion of the spatial information of a pattern image, c) A step of determining the state of the material related to the object based on at least one pattern image, d) A step of providing the state of the material related to the object.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for extracting the state of a material related to an object, a computer-implemented method for training a data-driven model, the use of a data-driven model for extracting the state of a material, the use of the state of a material related to an object in an authentication process, the use of at least one pattern image for extracting the state of a material related to an object, and / or the use of the state of a material for predicting the presence of a living body.

Background Art

[0002] Authentication processing can be disguised by things such as masks and images that represent the characteristics of users. Impersonation is becoming more realistic, and it is becoming difficult to distinguish between impersonation targets and humans. Therefore, it is necessary to reliably distinguish between humans and impersonation items.

Summary of the Invention

Problems to be Solved by the Invention

[0003] An object of the present disclosure is to provide a robust, reliable, and secure method for authenticating humans.

Means for Solving the Problems

[0004] Overview In one aspect, the present disclosure relates to a computer-implemented method for extracting the state of a material related to an object, the method comprising the following steps: a) receiving at least one pattern image, the at least one pattern image showing at least a part of an object under illumination by patterned electromagnetic radiation; b) modifying and / or deleting at least a part of the spatial information of the pattern image; c) determining the state of the material related to the object based on the at least one pattern image; and d) providing the state of the material related to the object. Includes. [Modes for carrying out the invention]

[0005] In another aspect, the disclosure relates to a computer implementation method for training a data-driven model for extracting material states associated with an object, the method comprising the following steps: a) A step of receiving a training dataset which includes at least one pattern image having altered and / or removed spatial information and the state of the material associated with the object, b) Steps to train a data-driven model according to the training dataset, and c) A step of providing a trained data-driven model.

[0006] In another aspect, the disclosure relates to the use of a data-driven model for extracting material states associated with an object, which is trained on a training dataset that includes at least one pattern image having altered and / or removed spatial information and material states associated with the object.

[0007] In another aspect, the disclosure relates to the use of material states associated with an object in an authentication process for initiating and / or verifying user authentication. In another aspect, the disclosure relates to the use of at least one pattern image having altered and / or removed spatial information for extracting material states associated with an object and / or for using material states associated with an object to predict the presence of a living organism.

[0008] In another aspect, the disclosure relates to the use of at least one pattern image having altered and / or removed spatial information for extracting the state of material associated with an object and / or using the state of material associated with an object to predict the presence of living organisms.

[0009] In another aspect, the present disclosure relates to a computer program element having instructions configured to perform steps of the method described herein when executed on a processing device.

[0010] In another aspect, the disclosure relates to a non-transient, computer-readable data medium for storing a computer program that includes instructions for performing steps of the method described herein.

[0011] This disclosure provides means for an efficient, robust, stable, and reliable method for extracting the material state associated with an object. The material state associated with an object is of crucial importance in detecting impersonation attacks in authentication processes. Commercial authentication systems can be easily impersonated with silicone masks. Blood perfusion can be used as an effective feature to distinguish an impersonation mask from a real human. Blood perfusion may also be detected by detecting the material state associated with the object. Spatial features such as the shape of the nose, nails, bones, and body parts are easily recognizable, but are also easily imitated, especially with recent hyperrealistic masks and similar impersonation objects. Therefore, authentication that relies solely on spatial features carries security risks. Furthermore, by suppressing at least some of the spatial information in an image, attention can be diverted from the spatial information, making it easier to determine the material state. This is particularly important in the case of data-driven models that learn from training data and distinguish between real people and impersonation objects. A method is provided for suppressing and / or removing spatial features that obscure essential information for determining the state of a material associated with an object, by modifying and / or removing at least a portion of the spatial information of a pattern image. Thus, user recognition and / or authentication is enhanced by predicting the likelihood that the object is biological, thereby determining the state of the material. Furthermore, using images with suppressed spatial information allows for more effective training of models, particularly data-driven models.

[0012] Based on the state of materials associated with the object, it is possible to reliably determine whether a living organism has been presented to the camera. A particular advantage of this method is that it is robust against impersonation, for example, by presenting a mask that replicates the face of an authorized user to the camera. A further advantage of the method according to this disclosure is that pattern images can be recorded using standard equipment, such as a standard laser and a standard camera, such as a charge-coupled device (CCD) and / or complementary metal-oxide-semiconductor (CMOS) sensor element. Furthermore, an efficient and robust method for monitoring the state of a living organism is provided. Using the feature contrasts of multiple pattern images, state measures such as heart rate, blood pressure, and suction level can be determined. For example, heart rate is a sensitive indicator of the state of a living organism, particularly the stress level indicated by the state measure. Therefore, by monitoring a living organism, it is possible to identify situations in which the organism is, for example, under stress, and to take appropriate action based on the determined state measurement. Identifying such situations is particularly important when critical state measures pose health or security risks. Examples of such situations include drivers controlling vehicles, users using virtual reality headsets, or individuals who must make long-term decisions. Identifying such situations reduces security and health risks associated with them. Furthermore, this disclosure utilizes inexpensive hardware for monitoring living organisms. In addition, the disclosed methods, systems, computer hardware, and applications are easy to integrate and implement, and at the same time provide reliable results without requiring direct contact with living organisms. Thus, living organisms are not restricted by the monitoring, and by irradiating them with infrared light, they do not need to be aware that they are being monitored. Therefore, with the use of the methods, systems, computer-readable storage media, and signals disclosed herein, living organisms are not distracted by the light or feel that they are being observed.

[0013] Embodiment This disclosure takes into account that bodily fluids or moving particles within a living organism, such as blood cells, particularly red blood cells, interstitial fluid, transcellular fluid, lymph, ions, proteins, and nutrients, can cause motion blur in reflected light. Stationary objects do not cause motion blur. Therefore, when coherent electromagnetic radiation is reflected by moving scattering particles such as red blood cells, the speckle pattern fluctuates and the spots become blurred. As a result of this blurring, the speckle contrast decreases. Thus, the speckle pattern and the speckle contrast derived from the speckle pattern contain information about whether or not the illuminated object is a living organism. Speckle contrast values ​​generally range between 0 and 1, and when illuminating an object, a value of 1 represents no motion, and a value of 0 represents the fastest particle movement, which can cause the most pronounced blurring of the speckles. Since the state of the material associated with the object may be determined based on the speckle contrast, a lower speckle contrast value indicates a higher certainty that the corresponding state of the substance associated with the object indicates the presence of a living organism. Conversely, the higher the speckle contrast value, the greater the certainty that the corresponding state of the substance associated with the object indicates the presence of a non-living object.

[0014] The following sections outline the terms and / or the technical fields of this disclosure as used herein, by definition and / or by examples. Where examples are provided, it should be understood that this disclosure is not limited to those examples.

[0015] In this specification, the terms "living organism" and "living species" are used interchangeably.

[0016] Methods, systems, applications, and computer program elements may be used to predict the presence of living organisms. Thus, methods, systems, applications, and computer program elements may also be applied to non-living organisms. To distinguish between living and non-living organisms, it is advantageous to apply methods, systems, applications, and computer program elements to all kinds of objects.

[0017] In one embodiment, the patterned electromagnetic radiation may be coherent electromagnetic radiation. Coherent electromagnetic radiation may represent electromagnetic radiation that can exhibit interference effects. It may also include partial coherence, i.e., imperfect correlation between phase values. In some embodiments, the coherent electromagnetic radiation may be in the infrared range, particularly the near-infrared range. The coherent electromagnetic radiation may be generated by a light source. The light source may be part of an apparatus and / or system. As an example, the coherent electromagnetic radiation may have wavelengths of 300 to 1100 nm, particularly 500 to 1100 nm. In addition or alternatively, light in the infrared spectral range, e.g., light in the range of 780 nm to 3.0 μm, may be used. Specifically, a portion of the near-infrared region to which silicon photodiodes are applicable, particularly coherent electromagnetic radiation in the range of 700 nm to 1100 nm, may be used. By using coherent electromagnetic radiation in the near-infrared region, it becomes possible to detect coherent electromagnetic radiation that is undetectable or only weakly detectable by the human eye, but still detectable by silicon sensors, especially standard silicon sensors.

[0018] In one embodiment, patterned electromagnetic radiation may include a speckle pattern. The speckle pattern may include at least one speckle. The speckle pattern may represent any known or predetermined arrangement including at least one speckle of any arbitrary shape. The speckle pattern may include periodic or aperiodic arrangements of speckles. The speckle pattern can be at least one of the following: at least one quasi-random pattern, at least one Sobol pattern, at least one quasi-periodic pattern, at least one dot pattern, in particular a pseudo-random dot pattern, at least one line pattern, at least one stripe pattern, at least one checkerboard pattern, at least one triangular pattern, at least one rectangular pattern, at least one hexagonal pattern, or a pattern consisting of convex inclines. The speckle pattern may also be an interference pattern generated from coherent electromagnetic radiation reflected from an object, e.g., reflected from the outer surface of the object or reflected from the inner surface of the object. Speckle patterns typically occur from diffuse reflection of coherent electromagnetic radiation such as laser light. Within a speckle pattern, the spatial intensity of coherent electromagnetic radiation can vary randomly due to interference of coherent wavefronts. Speckle is at least part of a speckle pattern. Speckle may include symbols of arbitrary shape, at least partially. A symbol can be any one of the following: at least one point; at least one line; at least two lines, such as parallel or intersecting lines; at least one point and one line; at least one array of periodic speckles; or at least one speckle pattern of arbitrary shape.

[0019] In one embodiment, the object may represent any object. The object may include living organisms such as humans and animals. The state of the material may be determined relative to the object in the image. The image may include more than just the object whose state of material is determined. The image may further include a background. The background may represent an object for which vital sign measurements have not been determined. The pattern image may include at least a part of the object and / or background. Preferably, the object may be a body part. The body part may be an external body part. For example, the head, forehead, face, cheeks, chin, eyes, ears, nose, mouth, arms, legs, feet, etc.

[0020] In one embodiment, the state of the material relating to the object may represent information relating to the material relating to the object. The state of the material may be used to classify the material and / or the object. The state of the material may be the chemical state of the material, the physical state of the material, the type of the material, and / or the biological state of the material. The state of the material may include information relating to the chemical state of the material, the physical state of the material, and / or the biological state of the material. The chemical state of the material may indicate the composition of the material. Examples of composition may be iron oxide or PVC. The type of material may indicate the kind of material. The type of material may be derived from the chemical state of the material, the physical state of the material, and / or the biological state of the material. Examples of types of material may be silicon, metal, wood, etc. Skin may include multiple chemical compounds. Skin may be determined based on multiple chemical compounds. Skin may be determined, for example, based on the biological state of the material, for example, blood perfusion.

[0021] The physical state of the material may indicate the structure of an object related to the material, the orientation of the object, and / or the like. The structure of the object may include information related to the surface structure and / or the internal structure. For example, the structure of the object may include topological information, periodic arrangements, and / or the like. The biological state of the material may indicate vital signs, vital sign measurements, state measurements, the type of biological material, and / or the like. The type of biological material may be skin such as human and / or animal-specific skin. The state of the material may include at least one numerical value.

[0022] In one embodiment, the vital sign may indicate the presence of a living body. The vital sign may represent information related to the presence of a living body. In other words, the vital sign is any sign suitable for distinguishing a living body from a non-living body. In particular, the vital sign may be related to the presence of moving body fluids or moving particles in the body of a living body. In this regard, blood flow, for example, the presence of red blood cells, may be a preferred vital sign. However, other moving body fluids or moving particles present in the body of a living body, such as interstitial fluid, transcellular fluid, lymphatic fluid, ions, proteins, and nutrients, may also be used as vital signs. Preferably, the vital sign may be detectable by analyzing a speckle pattern, for example, by detecting speckle blur caused by moving body fluids or moving particles moving in the body of a living body. This blur may reduce the speckle contrast compared to the case where there are no moving body fluids or moving particles.

[0023] In one embodiment, at least one state of the material associated with the object may represent at least one indicator indicating whether the object exhibits vital signs. The at least one vital sign indicator may be determined based on speckle contrast. Thus, the at least one vital sign indicator may depend on the determined speckle contrast. As the speckle contrast changes, at least one vital sign measurement derived from the speckle contrast may change accordingly. The at least one vital sign measurement (scale) may be at least one single numerical and / or value representing the likelihood that the object is a living organism. In addition or alternatively, the vital sign measurement may include a Boolean value, for example, true for living organisms and false for non-living organisms, or vice versa.

[0024] In one embodiment, spatial information may include information about the spatial orientation of an object, and / or information about the contour of an object, and / or information about the edges of an object. Spatial information may be classified through a model, particularly a classification model. Spatial information may be associated with spatial features. Spatial features of a face may be facial features. Spatial features may be represented as vectors. Vectors may contain at least one numerical value. Examples of spatial features of a face may include at least one of the following: nose, eyes, eyebrows, mouth, ears, chin, forehead, contours such as wrinkles and scars, and cheekbones. Other examples of spatial information may include fingers, nails, etc. At least a portion of the spatial information may be removed by deleting at least a portion of the spatial features and / or modifying at least a portion of the spatial features. Deleting at least a portion of the spatial features may be associated with deleting data related to the spatial features. Modifying and / or deleting at least a portion of the spatial features may be associated with modifying data related to the spatial features.

[0025] In one embodiment, modifying and / or deleting at least a portion of spatial information may include performing at least one image enhancement technique, in particular any combination of at least two image enhancement techniques. Performing at least one image enhancement technique may include changing the distance between at least two spatial features, representing the object as a two-dimensional plane, modifying at least a portion of an image, deleting at least a portion of an image, rearranging at least a portion of an image, generating at least one partial image, and / or any combination thereof. Changing the distance between at least two spatial features may include changing at least two positions associated with at least two spatial features. Representing an object as a two-dimensional plane may include generating a UV map, a flattened representation of the object, a distorted image, a distorted image, and / or these. Representing an object as a two-dimensional plane may result in a pattern image in which the distances between features in the pattern image differ from the generated pattern image. For example, the pattern image may show a face, and the pattern image may be sheared. As a result, in the sheared pattern image, the eyes may be further apart and the nose may be closer to the right eye compared to the generated pattern image. As a result, the model may be unable to recognize faces due to unknown and / or unexpected spatial expansion of facial features. Modifying at least a portion of an image may include modifying at least one spatial feature, changing the distance between at least two parts of at least one spatial feature, changing the brightness, changing the contrast, blurring the image, deleting at least a portion of at least one spatial feature, changing the shape of at least one spatial feature to an arbitrary shape, changing the size of at least one spatial feature, and / or similar actions. Deleting at least a portion of an image may include deleting at least a portion of at least one spatial feature. Rearranging at least a portion of a pattern image may include changing the position of at least a portion of the pattern image. The portion images may be of any size and / or shape.When at least a part of the spatial information of the pattern image is changed and / or removed, an operated pattern image with different, less, or no spatial information may be obtained. The change and / or deletion of at least a part of the spatial information of the pattern image may be related to the change and / or deletion of data related to the spatial information. Operating on an image, particularly a pattern image, may be referred to as changing and / or removing at least a part of at least a part of the spatial information of the image. The operated image may be an image in which at least a part of the spatial information of the pattern image is changed and / or deleted.

[0026] In particular, performing at least one image augmentation technique may include changing the distance between at least two spatial features, representing the object as a two-dimensional plane, deleting at least a portion of the image, rearranging at least a portion of the image, generating at least one partial image, and / or a combination thereof. More preferably, performing at least one image augmentation technique may include changing the distance between at least two spatial features, deleting at least a portion of the image, rearranging at least a portion of the image, and / or generating at least one partial image. More preferably, performing at least one image augmentation technique may include changing the distance between at least two spatial features, deleting at least a portion of the image, and / or generating at least one partial image. More preferably, performing at least one image augmentation technique may include changing the distance between at least two spatial features, and / or generating at least one partial image. More preferably, performing at least one image augmentation technique may include deleting at least a portion of the image, and / or generating at least one partial image. More preferably, performing at least one image augmentation technique may include deleting at least a portion of the image and / or changing the distance between at least two spatial features. More preferably, performing at least one image enhancement technique may include deleting at least a portion of the image and / or modifying at least a portion of the image, in particular modifying at least a portion of the spatial features.

[0027] In one embodiment, the image enhancement technique may include at least one of the following: scaling, cropping, rotation, blurring, warping, shearing, resizing, folding, contrast modification, brightness modification, noise addition, multiplication of at least some pixel values, dropout, color adjustment, convolution application, embossing, sharpening, inversion, averaging of pixel values, etc. The image enhancement technique may be performed to modify and / or remove at least some of the spatial information. For example, at least some of the spatial information may be modified and / or removed by shearing the pattern image. Shearing the pattern image may result in, for example, changing the distance between at least two spatial features and / or changing at least one spatial feature. Thus, at least some of the spatial information of at least one pattern image may be modified and / or removed by performing at least one of the image enhancement techniques. At least some of the spatial information of a pattern image may be modified and / or removed by performing at least one of the image enhancement techniques, in particular any combination of the image enhancement techniques. For a non-extensive list of image augmentation techniques, see Advanced Graphics Programming - A volume (2005) by TOM McREYNOLDS and DAVID BLYTHE, ISBN 9781558606593, https: / / doi.org / 10.1016 / b978-1-55860-659-3.50030-5.

[0028] In one embodiment, at least a first image augmentation technique may be performed on at least one pattern image. In addition or alternatively, a combination of at least two image augmentation techniques, including at least a first image augmentation technique and at least a second image augmentation technique, may be performed on at least one pattern image. Furthermore or alternatively, a combination of at least two image augmentation techniques, including at least a second image augmentation technique, may be performed on at least one pattern image.

[0029] In one embodiment, at least one image augmentation technique may be varied. Varying at least one image augmentation technique may include performing at least one image augmentation technique differently on at least one pattern image's first portion compared to at least one pattern image's second portion. Performing at least one of the image augmentation techniques may be randomized by randomly selecting at least one of the image augmentation techniques. Performing at least one of the image augmentation techniques may be varied based on a predetermined sequence of image augmentation techniques. Randomly performing at least one of the image augmentation techniques may include randomly selecting a sequence of at least two image augmentation techniques. Furthermore, randomly performing at least one of the image augmentation techniques may include selecting parameters suitable for defining how the image augmentation techniques may be applied randomly. Examples of parameters suitable for defining how the image augmentation techniques are applied may include rotation angles, scaling parameters, shear coefficients, parameters defining warping, and parameters defining the position, number, and length of cuts. Randomly altering or removing spatial features is advantageous because it creates images in which the shape of the object deviates. Doing so alters the shape of the object in the image, which is then not taken into account when analyzing the state of the material.

[0030] Spatial information may be recognized by a model, particularly a data-driven model. If an image contains multiple types of information that the model can focus on, the model will focus on the most prominent feature. Pattern images contain information related to vital signs and spatial information. Models, particularly data-driven models, may tend to focus too much on spatial information, which is less effective for detecting living organisms, rather than on relevant features, such as information related to the state of the material, like vital signs. By deleting and / or modifying at least some of the spatial information, the model can be focused on information related to vital signs. Thus, the model can be trained more efficiently, and the model's results / outputs will be more accurate. Deleting at least some of the spatial information may also mean modifying at least some of the spatial information. Modifying at least some of the spatial information may also mean deleting at least some of the spatial information. Different image augmentation techniques may be selected depending on the object in the image and the state of the material related to the object being determined. Different image augmentation techniques may have different effects on the degree of change or removal.

[0031] In one embodiment, the processor may represent any logic circuit configured to perform basic operations of a computer or system, and / or, generally, a device configured to perform calculations or logical operations. In particular, the processor, or computer processor, may be configured to process basic instructions that drive the computer or system. This may be a semiconductor-based processor, a quantum processor, or other type of processor configured to process instructions. As an example, the processor may be a central processing unit ("CPU") or include a central processing unit. The processor may be a graphics processing unit ("GPU"), a tensor processing unit ("TPU"), a complex instruction set computing ("CISC") microprocessor, a reduced instruction set computing ("RISC") microprocessor, a very long instruction word ("VLIW") microprocessor, or a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. The processing means may also be one or more special-purpose processing devices, such as application-specific integrated circuits ("ASICs"), field-programmable gate arrays ("FPGAs"), composite programmable logic devices ("CPLDs"), digital signal processors ("DSPs"), and network processors. The methods, systems, and apparatus described herein may be implemented as software within a DSP, microcontroller, or other side processor, or as hardware circuitry within an ASIC, CPLD, or FPGA. The term "processor" may refer to one or more processing devices, such as a distributed system of processing devices deployed across multiple computer systems (e.g., cloud computing), and should be understood that, unless otherwise specified, it is not limited to a single device.

[0032] In one embodiment, the inputs and / or outputs may include one or more serial or parallel interfaces or ports, USB, Centronics ports, FireWire, HDMI®, Ethernet, Bluetooth®, RFID, Wi-Fi, USART, or SPI, or one or more analog interfaces or ports such as ADCs or DACs, or standardized interfaces or ports to further devices.

[0033] In one embodiment, memory represents physical system memory, which may be volatile, non-volatile, or a combination thereof. Memory may include non-volatile mass storage devices such as physical storage media. Memory may be computer-readable storage media such as RAM, ROM, EEPROM, CD-ROM, or other optical storage devices, magnetic disk storage devices, or other magnetic storage devices, non-magnetic disk storage devices such as solid-state disks, or any other physical and tangible storage media that can be used to store desired program code means in the form of computer-executable instructions or data structures and can be accessed by a computing system. Furthermore, memory may also be a computer-readable medium (also called a transmission medium) that carries computer-executable instructions. Furthermore, program code means in the form of computer-executable instructions or data structures may be automatically transferred from the transmission medium to the storage medium (or vice versa) upon reaching various computing system components. For example, computer-executable instructions or data structures received via a network or data link may be buffered in RAM within a network interface module (e.g., "NIC") and then finally transferred to the RAM of the computing system and / or to a non-volatile storage medium in the computing system. Therefore, it should be understood that storage media can be included in computing components that also (or primarily) utilize transmission media.

[0034] In one embodiment, the system may include at least one computing node. A computing node may represent any device or system including at least one physical and tangible processor and physical and tangible memory capable of having computer-executable instructions executed by the processor. A computing node may be, for example, a handheld device, production equipment, sensors, monitoring systems, control systems, consumer electronics, laptop computers, desktop computers, mainframes, data centers, or devices not traditionally considered computing nodes, such as wearables (e.g., glasses, watches). The memory may take any form, depending on the nature and form of the computing node.

[0035] In one embodiment, at least one wireless communication protocol may be used. The wireless communication protocol may include any known network technology such as LTE technology using standards such as GSM, GPRS, EDGE, UMTS / HSPA, 2G, 3G, 4G, or 5G, and the wireless communication protocol may further include a wireless local area network (WLAN), such as Wireless Fidelity (Wi-Fi).

[0036] In one embodiment, the system may be a distributed computing environment. Distributed computing may be implemented. Distributed computing may represent any computing that utilizes multiple computing resources. Such utilization may be achieved by virtualizing physical computing resources. An example of distributed computing is cloud computing. "Cloud computing" may represent a model that enables on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, services). In the distributed case, the cloud computing environment may be distributed internationally within and / or across multiple organizations.

[0037] In one embodiment, the computer-readable data medium may represent any suitable data storage device or computer-readable memory on which one or more instruction sets (e.g., software) embodying one or more of the methodologies or functions described herein are stored. The instructions may also reside, all or at least partially, in the main memory and / or processing device during their execution by the computer, main memory, and processing device, which may constitute the computer-readable storage medium. The instructions may further be transmitted or received over a network via a network interface device. Examples of computer-readable data media include hard drives on servers, USB storage devices, CDs, DVDs, or Blu-ray discs. The computer program may contain all the functions and data necessary for the execution of the method according to this disclosure, or it may provide an interface for processing a portion of the method on a remote system, such as a cloud system. The term non-transient may mean that the purpose of the data storage medium is to permanently store the computer program without requiring a permanent power supply in particular.

[0038] In one embodiment, the steps of the method described herein may be performed by an apparatus. The apparatus may be a mobile device and / or a stationary device. A mobile device may include a tablet, laptop, telephone, watch, etc. A stationary device may be a device suitable for permanent installation in a fixed location. A stationary device may be, for example, a computer, desktop computer, server, cloud environment, etc.

[0039] In one embodiment, determining the state of a material related to an object based on at least one pattern image may include determining the speckle contrast of at least a portion of the pattern image.

[0040] In one embodiment, at least one further pattern image and an indication of at least one interval between at least two different time points in time in which at least two pattern images were generated may be received, the speckle contrast is determined for at least two pattern images, and the state of the material related to the object is determined based on the speckle contrast and the indication of at least one time interval.

[0041] In one embodiment, at least two pattern images and an instruction for at least one interval between at least two different time points in time in which the at least two pattern images were generated are received, the speckle contrast is determined for the at least two pattern images, and the state of the material related to the object is determined based on the speckle contrast and the instruction for at least one time interval.

[0042] In one embodiment, at least one pattern image may include image data suitable for representing at least one pattern image. The image may be a pattern image. A partial image may be a (single) image. A pattern image may include at least one pattern which includes at least one pattern feature. The pattern feature may be speckle. A pattern image is not limited to an actual visual representation of an object. Instead, a pattern image may include data generated while illuminating an object with patterned electromagnetic radiation. A pattern image may be contained within a larger pattern image. A pattern image may show and / or include at least one speckle pattern. The pattern may be a speckle pattern which includes at least one speckle. A larger pattern image may be a pattern image which includes more pixels than the pattern images which are contained within it. Dividing a pattern image into at least two parts may yield at least two pattern images. The at least two pattern images may include different data generated based on light reflected by an object which is illuminated with coherent electromagnetic radiation. A pattern image may be suitable for determining speckle contrast. Speckle contrast may be determined for at least one speckle. The pattern image may contain multiple pixels. These multiple pixels may include at least two pixels, preferably more than two. To determine the speckle contrast, at least one pixel related to reflective features and at least one pixel not related to speckle may be suitable. The pattern image can represent any data that can construct an actual visual representation of the imaged object. For example, the data may correspond to the assignment of color or grayscale values ​​to image positions, where each image position may correspond to a position within or on the imaged object. The pattern image may be, for example, two-dimensional, three-dimensional, or four-dimensional, where a four-dimensional image is understood as a three-dimensional image evolving over time, and similarly, a two-dimensional image evolving over time may be considered a three-dimensional image.If the data is digital data, the pattern image can be considered a digital image, in which case the image position may correspond to pixels or voxels of the image and / or image sensor.

[0043] In one embodiment, the speckle pattern may include at least two speckles, preferably at least three. One or more speckles provide more information compared to one speckle. More information is advantageous because it yields more results and, overall, more accurate results. Therefore, providing more information improves the accuracy of the corresponding results.

[0044] In one embodiment, determining the state of a material may include providing at least one pattern image to a data-driven model for determining the state of a material, based on at least one pattern image. The data-driven model may be parametric and / or trained to receive at least one pattern image and determine the state of a material from the pattern image. The data-driven model may be parametric and / or trained according to a plurality of pattern images and corresponding state of material.

[0045] In one embodiment, the state of the material related to the object may be determined based on the speckle contrast of at least a portion of the pattern image.

[0046] In one embodiment, determining the state of a material may include applying at least one image filter. Determining the state of a material may include applying an image filter, in particular a material-dependent image filter. The image filter can be applied to at least a portion of the pattern image. Thus, the image filter can be applied to at least one speckle. Such techniques are described, for example, in WO 2020 / 187719 A1, which are described below. The disclosure of WO 2020 / 187719 A1 is incorporated herein by reference.

[0047] Speckle contrast may represent a measure of the average contrast of the intensity distribution within a region of a speckle pattern. Speckle contrast may be determined for the first speckles of at least two pattern images, and for the second speckles of at least two pattern images. Speckle contrast may also be determined separately for at least two speckles of at least two pattern images.

[0048] In particular, the speckle contrast K across the pattern region is the average speckle intensity. The ratio of the standard deviation σ to can be expressed as, i.e.,

number

[0049] Speckle contrast values ​​generally range from 0 to 1. Speckle contrast may include and / or be associated with speckle contrast values. Speckle contrast may be determined based on at least one speckle. Subsequently, at least two values ​​for speckle contrast may be determined based on at least two speckles.

[0050] When referring to determining speckle contrast or the state of the material associated with an object, and / or determining at least one value of the state of the material associated with an object and / or speckle contrast, speckle contrast is included.

[0051] In some embodiments, the entire speckle pattern of the pattern image may be used to determine the speckle contrast. Alternatively, a portion of the speckle pattern may be used to determine the speckle contrast. The cross section of the speckle pattern preferably represents an area smaller than the area of ​​the speckle pattern. The region may have any shape. The cross section of the speckle pattern may be obtained by cropping the pattern image. The speckle contrast may differ for different parts of the object. Different parts of the object may correspond to different parts of the pattern image. Therefore, the speckle contrast may differ for different parts of the pattern image.

[0052] A condition scale is a scale suitable for determining the condition of an organism. The state of the organism may be a physical and / or mental state. A physical state may be a physical stress level, fatigue level, agitation level, or the organism's suitability for performing a specific task. A mental state may be related to a mental stress level, attention, concentration, agitation level, or the organism's suitability for performing a specific task. Such specific tasks may require the organism to have concentration, attention, alertness, calmness, or similar characteristics. Examples of such tasks include controlling machinery, vehicles, mobile devices, manipulating other organisms, sports activities, games, emergency tasks, and decision-making. A condition scale indicates the state of the organism. A condition scale may be one or more of the following: heart rate, blood pressure, suction level, etc. In some embodiments, the state of the organism may be a critical level corresponding to a high value on the condition scale, and the state of the organism may be a non-critical level corresponding to a low value on the condition scale. Next, the critical state scale in these embodiments may be equal to or lower than the threshold, and the non-critical state scale may be lower than the threshold. In other embodiments, the state of the organism may be a critical state corresponding to a low value on the state scale, and the state of the organism may be a non-critical state corresponding to a high value on the state scale. Next, the critical state scale in these embodiments may be equal to or higher than the threshold, and the non-critical state scale may be lower than the threshold. The critical state scale may be associated with high stress levels, low attention, low concentration levels, high fatigue, high excitement, low aptitude of the organism to perform a specific task, etc. The non-critical state scale may be associated with low stress levels, high attention, high concentration, low fatigue, low excitement, high aptitude of the organism to perform a specific task, etc.

[0053] In one embodiment, the state measure may be determined based on the speckle contrast of at least a portion of the pattern image. The biological state measure may be determined based on the movement of bodily fluids, preferably blood, most preferably red blood cells. The movement of bodily fluids is not constant over time and changes due to the activity of a part of the body, such as the heart. Such changes in movement may be judged based on the change in speckle contrast over time. A large difference in the speckle contrast values ​​at different time points may indicate a rapid change in movement. A small difference between the speckle contrast values ​​at different time points may be associated with a slow change in movement. Changes in the movement of bodily fluids, preferably blood, may be periodically associated with a corresponding motion frequency. Thus, the speckle contrast may change periodically with the corresponding motion frequency. The motion frequency may correspond to the length of the period associated with the periodic change in speckle contrast. In some embodiments, at least two pattern images may contain half of the period. In other embodiments, at least two pattern images may contain one or more periods. Preferably, speckles associated with the same part of the body may be used to determine the state of the organism. This is advantageous due to the fact that blood perfusion, and therefore speckle contrast, changes across different parts of the organism. In some embodiments, at least one state measure may be determined based on the speckle contrast.

[0054] At least two pattern images include a time series. The time series may include pattern images separated by a constant or varying time interval related to the imaging frequency. Preferably, the time series is configured such that the imaging frequency is at least twice the motion frequency. This is known as the Nyquist theorem. For higher resolutions, more pattern images than required by the Nyquist theorem may be received.

[0055] In one embodiment, an instruction for the interval between different time points in which at least two pattern images are generated may be received. The instruction for the interval includes a measure suitable for determining the time between different time points in which at least two pattern images are generated.

[0056] The frequency is the reciprocal of the period length. The period length may be determined by the interval between two pattern images, each containing a heartbeat or cardiac cycle. In a normal, resting human, the heart beats 60–80 times per minute, and the resting heart rate is 60–80 beats / min (bpm). The resting heart rate may be lower, for example, during sports or if the person has bradycardia. When a person is active, the heart rate can rise to 230 bpm. The heart rate of animals is 6–1000 bpm. The pattern images may be generated according to the expected heart rate of the organism being examined. The interval between pattern images may be selected up to a maximum of 10 seconds. In the case of humans, the interval may be selected up to 2 seconds. Subsequently, the imaging frequency may be selected so that at least 12 pattern images are acquired per minute, or at least 60 pattern images per minute in the case of humans. As an example, this method can be used to determine the heart rate of a human. For this purpose, an imaging frequency of 60 images per minute may be selected. As the speckle contrast is determined, it may become apparent that the imaging frequency may be too low. In such cases, the imaging frequency may be increased so that the condition scale is determined. Alternatively, the imaging frequency for imaging a human may be selected to be a high frequency, such as 460 frames per minute. Heart rate may be determined based on at least two pattern images and a representation of the interval between at least two different time points, indicating an interval of 0.13 seconds. In this example, the human has a heart rate in the range of 60–80 bpm. Subsequently, the imaging frequency may be adjusted according to the expected and / or predetermined condition scale.

[0057] For example, the interval may include the length of a half-cycle, a full cycle, or twice the cycle. The interval may also be between at least two pattern images. Subsequently, at least two pattern images may be separated by a half-cycle, a full cycle, twice the length, or the like. In the case of three exemplary pattern images, one or two different interval indications may be received. If two or more pattern images are received, the interval indication may include the interval between the first pattern image and the second pattern image, and / or the interval between the first pattern image and the third pattern image (or every other pattern image if three or more pattern images may be received), and / or the interval between the second pattern image and the third pattern image (or every other pattern image if three or more pattern images may be received). This can be appropriately applied to other scenarios with different amounts of pattern images, as will be recognized by those skilled in the art. Means for specifying the interval may be at least two time points corresponding to different time points at which at least two pattern images are generated, and / or the time elapsed between the different time points, and / or the imaging frequency associated with the generation of the pattern images. The at least two time points may be determined based on the timestamps of at least two pattern images. The imaging frequency may include a selected value. The imaging frequency may be selected based on an expected state measure, for example, an expected heart rate. Alternatively, the imaging frequency may be determined using the image frequency of the video. The expected heart rate may include the heart rate associated with the organism being monitored. In some embodiments, estimation of a state measure may be used to select the imaging frequency. Estimation of the state measure may take into account the organism and its surroundings.

[0058] By monitoring living organisms, it is possible to identify situations in which the organism is experiencing stress, for example, and to take appropriate action based on a determined state scale. Identifying such situations is particularly important when a critical state scale poses health or security risks. Examples of such situations include drivers controlling vehicles, users of virtual reality headsets, or individuals who must make long-term decisions. Identifying such situations can reduce security and health risks in those situations. Furthermore, the state monitoring method and system of the present invention utilize inexpensive hardware for monitoring living organisms. In addition, the state monitoring method and system of the present invention are easy to implement and carry out, do not require direct contact with living organisms, and can provide reliable results. Moreover, by irradiating with infrared light, living organisms do not need to be aware that they are being monitored. Therefore, with the use of the methods, systems, computer-readable storage media, and signals disclosed herein, living organisms are not distracted by the light or feel that they are being monitored.

[0059] These and other objectives, which will become clear from the following description, are addressed by the subject matter of the independent claims. Dependent claims refer to embodiments of the invention.

[0060] In one embodiment, multiple states of a material associated with an object may be determined based on multiple pattern images, including partial pattern images. In particular, at least two portions of the pattern images may be associated with different spatial locations on the object. The different spatial locations may represent overlapping or distinct spatial locations on the object. Based on at least two portions of at least two pattern images, at least two states of the material associated with the object may be determined in relation to two different spatial locations. Furthermore, at least two states of the material associated with the object may be provided as a material state map. The material state map can be generated from a speckle contrast map. The speckle contrast map may include a number of speckle contrast values, each associated with a location, for example, a location on the object on which the corresponding pattern image was recorded. The speckle contrast map can be represented using a matrix having the speckle contrast values ​​as matrix entries. The material state map may be represented using a matrix containing the individual states of the material associated with the object as matrix entries. The material state map may represent the spatial distribution of the determined states of the material associated with the object. Each state of material associated with an object may be associated with a different location on the object irradiated with coherent electromagnetic radiation in order to record the corresponding speckle pattern. Thus, associating the state of material associated with an object with a location on the object can be beneficial. To generate a map of material states associated with an object, the location associated with each state of material associated with the object can be considered. Therefore, the map may represent, for example, the spatial distribution of material states associated with the object in the coordinate system of the illuminated object. For example, the location associated with a material state associated with an object may be represented by spatial coordinates in the coordinate system of the illuminated object. A map of material states associated with an object is advantageous because at least two states of material associated with the object are determined for different parts of the object, and therefore, accuracy can improve with an increasing number of tests. An example of a material state map is a vital sign measurement map. Another example is a material type map.A speckle contrast map may contain multiple speckle contrast values. The speckle contrast map may be represented similarly to blood perfusion maps 510-530 by representing values ​​associated with speckle contrast. A material state map can be generated by assigning speckle contrast to material states associated with an object. Assigning speckle contrast to material states associated with an object may include comparing the speckle contrast to a predetermined threshold. Several thresholds may be used to obtain a stepped representation, as shown in Figure 5.

[0061] The values ​​may include speckle contrast, vital sign measurements, vital signs, and the condition of materials related to the object.

[0062] In one embodiment, the material type map may indicate material types at at least two different spatial locations within the pattern image. The object may contain multiple materials. Different spatial locations may be associated with different material types. In this example, one part of the pattern image may indicate one material type, and another part may indicate a different material type. Doing so generates more information about the object and can describe different states of material associated with the object at different spatial locations of the object.

[0063] In some embodiments, the state of the material related to the object may be determined using a model. The model may be a deterministic model, a data-driven model, or a hybrid model. A deterministic model preferably reflects the physical phenomenon in mathematical form, including, for example, a first-principles model. A deterministic model may include a set of equations that describe the interaction between matter and patterned electromagnetic radiation, thereby yielding state measurements, vital sign measurements, etc. A hybrid model may be a classification model that includes at least one machine learning architecture having deterministic or statistical adaptations and model parameters. Statistical or deterministic adaptations may be introduced to improve the quality of results, as they provide a systematic relationship between empiricism and theory. Statistical or deterministic adaptations may include constraints on intermediate or final results determined by the classification model, and / or additional inputs for (re)training the classification model. A hybrid model may be more accurate than a purely data-driven model, because a purely data-driven model may tend to overfit, especially with small datasets, which can be avoided by introducing knowledge in the form of a deterministic adaptation.

[0064] In one embodiment, the data-driven model may be a classification model. The classification model may include at least one machine learning architecture and model parameters. For example, the machine learning architecture may be one or more of the following: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifier, support vector machine, naive Bayes classification, nearest neighbor, neural network, convolutional neural network, generative adversarial network, support vector machine, or gradient boosting algorithm, or may include these. In the case of a neural network, the model may be, but is not limited to, a multiscale neural network or a recurrent neural network (RNN) such as a gated recurrent unit (GRU) recurrent neural network or a long-short memory (LSTM) recurrent neural network.

[0065] A data-driven model may be parametricized according to a training dataset. A data-driven model may be trained based on a training dataset. Training a model may include parametricizing the model. The term training may also be written as learning. This term may also refer to the process of building a classification model, in particular the process of determining and / or updating the parameters of the classification model, though not limited to this. Updating the parameters of a classification model may also be called retraining. Retraining may be included wherever training is referred to herein. In one embodiment, the training dataset may include at least one pattern image having modified and / or removed spatial information, and at least one material state, such as at least one vital sign measurement. The vital sign measurement may be a vital sign measurement associated with at least one pattern image.

[0066] A classification model may be at least partially data-driven. Training a data-driven model may include providing the model with training data. The training data may include at least one training dataset. The training dataset may include at least one input and at least one desired output. During training, the data-driven model may be adjusted to achieve a best fit with the training data, for example, by associating at least one best-fitting input value with at least one desired output value. For example, if the neural network is a feedforward neural network such as a CNN, a backpropagation algorithm may be applied to training the neural network. In the case of an RNN, a gradient descent algorithm or a backpropagation-through-time algorithm may be employed for the purpose of learning. Training a model may include, or without limitation refer to, calibrating the model.

[0067] In one embodiment, at least one pattern image may further show at least a portion of the background under illumination by patterned electromagnetic radiation. At least a portion of the pattern image associated with an object may be determined by identifying at least a portion of the object in at least one pattern image. In addition and / or alternatively, at least one flood image may be received, and at least a portion of the pattern image associated with at least a portion of the object may be determined by identifying an object in at least one flood image. The object may be identified via an algorithm that implements a model. The object may be identified based on spatial features associated with the object. Methods for identifying objects in an image are known in the art. For example, implementations for identifying faces in an image are known in the art. An object may also be identifiable by a particular combination of edges and / or a particular distance between edges. For example, a face may be identified by facial features. Facial features may represent the edges of the face. For example, facial features may include a nose, chin, eyes, eyebrows, glasses, etc. Subsequently, the object may be identified through the spatial features of the object represented in the image. The speckles referencing the object may be determined by identifying the portion of the image that references the object. At least a portion of the pattern image that references the object may be determined by comparing the spatial position of the pattern image with the spatial position of the object. For this purpose, a pattern image or a flood image may be used. The flood image may be used to reference the spatial position of the speckles. The flood image may be an RGB image representing the object illuminated with electromagnetic radiation in the visible range, and / or a flood IR image representing the object illuminated with electromagnetic radiation in the infrared range, preferably the near-infrared range.

[0068] In one embodiment, identifying an object in at least one pattern image and / or at least one flood image may include determining at least one spatial feature associated with the object in at least one pattern image and / or at least one flood image.

[0069] In one embodiment, at least a portion of the pattern image related to at least a portion of the background may be removed.

[0070] In one embodiment, modifying and / or removing at least a portion of the spatial information of a pattern image may include representing an object as a two-dimensional plane.

[0071] In one embodiment, the electromagnetic radiation may be in the infrared range.

[0072] In one embodiment, modifying and / or removing at least a portion of the spatial information may include generating at least two sub-images. The at least two sub-images may be generated based on at least one pattern image. The material state associated with the object may be determined based on at least one of the sub-images. In some embodiments, the material state associated with the object may be determined based on at least two of the sub-images. Since the material state is extracted from sub-images generated by manipulating the pattern image of the object, computing resources required to identify the material can be reduced. Furthermore, by providing sub-images to a data-driven model, potential backgrounds, not just the complete pattern image, can also be used to extract the material state. Moreover, in the training process of the data-driven model, parts necessary to train the discriminative model that do not ignore potentially highly variable backgrounds are avoided, which leads to a reduction in the required training data. In particular, since the data size of each sub-image (which is numerous to achieve a well-trained model) may be smaller than or equal to the data size of each corresponding pattern image (which is numerous to achieve a well-trained model), the size of the training dataset can be significantly reduced. Therefore, accurate identification of the material state of the object, and / or identification and / or authentication of the object, is provided, and even in complex situations, such as when the object is composed of or covered with various, possibly undesirable, materials and the user is authorized as a user of the device to perform at least one operation on the device that requires authentication, less training data is used to train the data-driven model, and the requirements in terms of technical effort and resources, cost and computational expenses are reduced in particular. Consequently, the data-driven model requires fewer parameters, resulting in, for example, fewer neurons in the first layer. As a result, the layer size is smaller. This further leads to a reduction in the problem of overfitting.Since the methods, apparatus, data media, and applications of the present invention are trained so that the model can be used by the methods, apparatus, data media, and applications of the present invention after training, all the technical effects described apply to the methods, apparatus, data media, and applications of the present invention.

[0073] In one embodiment, the model may be provided with at least one pattern image in which at least a portion of the spatial information has been modified and / or removed, and / or the model may be used to determine the state of the material associated with the object.

[0074] In one embodiment, at least one pattern image in which at least a portion of the spatial information is modified and / or removed is provided to a data-driven model, and / or the state of the material associated with the object is determined using the data-driven model.

[0075] In one embodiment, the state of material associated with an object may be compared to a threshold. A living organism may be detected if the state of material associated with the object is greater than the threshold. A non-living organism may be detected if the state of material associated with the object is less than or equal to the threshold. Comparing vital sign measurements to a threshold may be part of an authentication process. Detection of a living organism may initiate and / or verify the authentication process. Detection of a non-living organism may reject the authentication process. Comparing the state of material associated with an object to a threshold may be suitable for predicting the presence of a living organism. For example, a method for extracting the state of material associated with an object may be used as part of an authentication process implemented on the device to provide access control to a user attempting to access the device. The device may be, for example, a telephone, tablet, laptop computer, watch, etc.

[0076] In one embodiment, determining the state of a material may include determining information relating to the material associated with at least two spatial locations of the object. The at least two spatial locations may be different. The at least two spatial locations may be associated with the same material or different materials. The at least two spatial locations may represent locations in a generated pattern image. The spatial locations may change when at least some of the spatial information of the pattern image is modified and / or removed. At least one first state of the material may be associated with a first spatial location of at least two spatial locations, and at least one second state of the material may be associated with a second spatial location of at least two spatial locations. The state of the material may be determined based on at least one first state of the material and at least one second state of the material. Considering multiple parts of an image to determine whether a presented object is a human impersonator enhances the security of the authentication process, which includes determining the state of the material. Furthermore, determining the state of a material may be based on at least one first state of the material associated with at least one first spatial location, at least one second state of the material associated with a second spatial location, and at least one first spatial location and at least one second spatial location. In this way, the state of a material may be determined based on one or more states of material and the locations associated with the multiple states of material. For example, in a face, the state of material associated with the nose may be determined, and the state of material associated with the cheeks may be determined. The states of material of the nose and cheeks may be different, for example, because of different blood perfusion. The locations associated with the states of material in the generated pattern image may introduce spatial relationships between states of material associated with different spatial locations in order to determine the overall state of the material. The nose and cheeks may have characteristic differences in the state of material, for example, in blood perfusion, and may have characteristic differences in spatial location. The combination of the state of material and the spatial information associated with the states of material may represent the very unique properties of actual human skin, and the risk of impersonation may be further reduced.In this embodiment, by combining information about the state of the material and spatial information based on the distance between a position related to the nose and a position related to the cheek, it is possible to further identify the user, which may contribute to determining the presence or absence of a human being.

[0077] In one embodiment, the method preferably further includes, as part of the authentication process, a step of predicting the presence of a living organism based on the state of the material associated with the object. Preferably, the step of predicting the presence of a living organism based on the state of the material associated with the object includes at least one of the following substeps.

[0078] - A substep in which a confidence score is determined based on the determined state of the material associated with the object. - A step of comparing the confidence score with a predefined confidence threshold, - A step to predict the presence of living organisms based on that comparison.

[0079] A confidence score can be generated from the state of the material associated with the object (e.g., represented by a single numerical value or a material state map, e.g., a matrix of values ​​relating to the state of the material associated with the object). The confidence score may also represent the confidence level of the presence of a living organism. The confidence score may be represented by a single numerical value.

[0080] Preferably, the confidence score is determined by comparing the determined state of the material associated with the object with a criterion preferably associated with a particular confidence score.

[0081] Alternatively, the confidence score may be determined using a neural network trained to take a determined state of the material associated with the object as input and provide a confidence score as output. The neural network may be trained using historical data representing past states of the material associated with the object and associated confidence scores.

[0082] A confidence threshold may be predetermined to guarantee a certain degree of confidence that the object is truly a living organism. A confidence threshold may be predetermined depending on a specific application, such as the security level required to provide access to the device. For example, the confidence threshold may be set so that the confidence score represents a relatively high level of confidence that the object presented to the camera is a living organism, e.g., 90% or higher, or even 99% or higher. The presence of a living organism is only acknowledged if, as a result of comparison with the confidence threshold, the confidence score is sufficiently high, i.e., exceeds the confidence threshold. If the confidence score falls below the confidence threshold, access to the device is denied. Denied access may trigger a repetition of new measurements, such as the method for extracting the state of material associated with the object and the method for using the state of material associated with the object to predict the presence of a living organism, as described above. Optionally, an alternative authentication process is also possible. This allows verification that the requester attempting to access the device is actually a living organism and not a spoofed attachment. Furthermore, it is possible to verify that the requester is authorized for the specific request.

[0083] The above-described method for extracting the state of material associated with an object, and the above-described method for predicting the presence of a living organism using the determined state of material associated with an object, may be part of an authentication process, and may further include, in particular, biometric authentication, such as facial recognition and / or fingering sensing.

[0084] The authentication process may include the following steps: - For example, a step of performing biometric authentication of a user by using the user's face presented to a camera or by determining the user's fingerprint using a fingerprint sensor; - A step of providing a detector signal from a camera, wherein the detector signal represents an image of the user's features, such as fingerprints or facial features; - Steps to generate a low-level representation of the image, - A step to verify user authentication based on a low-level representation of an image and a saved low-level representation template. - If biometric authentication is successful, preferably, the step of determining the state of the material associated with the object by performing the step of a method for extracting the state of the material associated with the object as described above, - Based on the determined state of the material related to the object, preferably by the following method, the presence of living organisms is predicted. - A step of determining a confidence score based on the determined state of the materials related to the object, - A step of comparing the confidence score with a predefined confidence threshold. - A step to predict the presence of living organisms from that comparison, - A step of providing a positive authentication output signal if the presence of a living organism is confirmed.

[0085] Upon receiving a positive authentication output signal, the user may be granted access to the device. Otherwise, if no biological information is detected, a negative authentication output signal may be provided. In other words, generally, an authentication output signal indicating whether a biological entity was presented to the camera may be provided. Furthermore, if the result of biometric authentication is already negative, a negative authentication output signal may be output without determining the subject's vital signs.

[0086] In another authentication process, the state of the object and the materials associated with it is determined first, and then, if the presence of a living organism is successfully confirmed, biometric authentication, such as facial recognition or fingerprint sensing, is performed.

[0087] In one embodiment, pattern images may be separated by a fixed or changing time interval related to the imaging frequency. In another embodiment, the imaging frequency may be at least twice the motion frequency associated with the expected periodic motion of the bodily fluids. The expected periodic motion of the bodily fluids may be associated with the expected periodic motion of the bodily fluids. The motion of the bodily fluids may be estimated based on the organism and its surroundings.

[0088] In some embodiments, the organism may be moving relative to the camera, and this movement may not coincide with the movement of blood. In such cases, the image may be motion-corrected. Such correction aims to correct the pattern image for movements unrelated to blood perfusion. This can be done by tracking the movement of the organism. By tracking the movement, a correction factor suitable for subtracting from the feature contrast of the pattern image can be determined. The greater the movement of the organism unrelated to blood perfusion, the larger the correction factor to subtract. The correction factor may differ depending on the part of the pattern image. In this way, the non-uniform movement on the pattern image can be taken into account by the correction factor. This has the advantage that it is possible to use pattern images in which the organism was moving, eliminating the need to discard pattern images and thus eliminating the need to generate more pattern images. As a result, less data is generated, processed, and stored, and consequently energy consumption can be reduced.

[0089] In some embodiments, the step of providing a conditional scale can be replaced by the following: - A step of generating a signal indicating a state-based action based on a comparison between the state of the material related to the object and a threshold related to an important state measure. - A step of providing a signal that indicates a condition-based action.

[0090] Condition-based actions are the result of comparing a condition scale with a threshold. The condition scale may be determined as described herein. Signals indicating condition-based actions may be generated based on the comparison. Signals may be received by a state control system. State-based actions may be advisory actions and / or interference actions. In some embodiments, the signal may indicate no action. If the state measurement of the organism falls below the threshold, the organism should not be restricted. Advisory actions provide advice to the organism. In some embodiments, an advisory action may provide advice to another organism as the organism from which a pattern image was generated. In exemplary scenarios such as monitoring children, animals, people with health problems, elderly people, people with disabilities, etc., the person responsible for caring for the organism may be notified. The person responsible for care may be parents, caregivers, doctors, veterinarians, etc. Advisory actions may include any form of providing advice in a form suitable for the organism to recognize. Such advice may include, for example, advising the organism to rest, to drink, and / or to eat, or to change the organism's conditions and / or surroundings, such as voice input, temperature, visual input, air circulation, etc. Examples of forms of advisory action include visual through display functions, auditory through sound generation, such as warning signals, or tangible through vibration. Intervention may include any form of external modification. This is advantageous when the organism's condition is critical and can be improved by performing an intervention. Such modification may include, for example, modifying the organism's conditions and / or surroundings (e.g., voice input, temperature, visual input, air circulation, etc.), limiting the time the organism can operate and / or control a mobile device, and / or performing a specific task. In some scenarios, preceding advisory actions may be ignored by the organism requesting the disruptive action. Furthermore, measures for very critical conditions are high-risk and may be better handled with disruptive actions. Measures for slightly critical conditions may be adequately addressed with advisory actions. [Brief explanation of the drawing]

[0091] The present disclosure will be further described below with reference to the enclosed drawings. The drawings and the same reference numerals in this disclosure are intended to refer to the same or similar elements, components, and / or parts. [Figure 1] Figure 1 shows an example of an embodiment of a device and system for extracting the state of materials related to an object. [Figure 2] Figures 2a and 2b illustrate exemplary embodiments of images in which at least a portion of the spatial information 200 has been modified and / or deleted. [Figure 3] Figure 3 is a flowchart illustrating an example of one embodiment of a method for extracting the material state related to an object. [Figure 4] Figure 4 shows an example of an embodiment of a method for extracting the state of materials related to an object. [Figure 5] Figure 5 shows an example of an embodiment illustrating the temporal changes in a vital signs measurement map. [Figure 6] Figure 6 shows an example of a pattern image after at least a portion of the spatial information has been modified and / or removed.

[0092] [Detailed explanation] The following embodiments are merely examples of how to carry out the methods, systems, or application devices disclosed herein, and should not be considered limiting.

[0093] Figure 1a shows an exemplary embodiment of a device 101 for extracting the material state associated with an object. The device may be a mobile device, e.g., a smartphone, laptop, smartwatch, tablet, etc., and / or a non-mobile device, e.g., a desktop computer, an authentication point such as a gate, etc. The device may be suitable for performing operations and / or methods as described in Figures 2-5. The device may be a user device. The device may include a processor 114, an imaging unit, an input 115, an output 116, and / or such. Data may be supplied to the processor 114 via the input 115. The input 115 may use a wireless communication protocol. The data may be provided to the user, for example, via the output 116. The output 116 may include a graphical user interface for providing information to the user. The output 116 may be suitable for providing data to another device, for example. The output 116 may use a wireless communication protocol. The device may further include a display 113 for displaying information to the user. The device 101 may be a display device. The imaging unit may be suitable for generating images, particularly images of objects. The processor 114 may be connected to the input 115 and the output 116.

[0094] Figure 1b shows an exemplary embodiment of a system for extracting material states related to an object. The system may be a replacement for the apparatus 101 described in Figure 1a. The system may include components of apparatus 101 as described in Figure 1a. The components of apparatus 101 may be distributed along the computing resources of the system.

[0095] The system may be a distributed computing environment. In this example, the distributed cloud computing environment 102 may include computing resources such as a device 101, data storage 120, an application 121, a server 122, and a database 123. The cloud computing environment 102 may be deployed as a public cloud 124, a private cloud 126, or a hybrid cloud 128. The private cloud 124 is owned by an organization, and only members of the organization with appropriate access rights can use the private cloud 126, and the data in the private cloud is at least confidential. In contrast, data stored in the public cloud 126 may be made public to anyone via the internet. The hybrid cloud 128 may be a combination of both the private cloud 124 and the public cloud 126, where some data is kept confidential and other data is made publicly available. The components of the distributed computing environment 102 may perform at least one step of the method described herein. In a non-limiting example, the device 101 may generate images and / or include an input for receiving pattern images. Alternatively or additionally, pattern images may be received from database 123 and / or data storage devices 120 and / or clouds 124-128 by a processor for executing the steps of the method. The processor may be server 122, clouds 124-128, or may be included in server 122, clouds 124-128. Application 121 may include instructions for executing the steps of the method as described in the context of Figures 2-4.

[0096] Figures 2a and 2b show exemplary embodiments of images obtained by modifying and / or deleting at least a portion of the spatial information 200. The generated image may be 210. The generated pattern image 210 may represent any object. In Figure 2a, the object may be a cube. In Figure 2b, a three-dimensional representation of a face is illustrated.

[0097] In Figure 2a, examples i to vii represent different embodiments of an image in which at least a portion of the spatial information 200 has been modified and / or deleted. Image i may be an image in which at least a portion of the image has been modified, an image in which at least a portion of the image has been deleted, an image in which at least one partial image has been generated, and / or a combination thereof. An example of the image augmentation technique performed may be cutting. Image ii may be an image in which at least a portion of the image has been rearranged, an image in which the distance between at least two spatial features has been changed, and / or a combination thereof. An example of the image augmentation technique performed may be cutting, rotation, and / or a combination thereof. Image iii may be an image in which at least a portion of the image has been modified. An example of the image augmentation technique performed may be scaling, shearing, folding, and / or a combination thereof. Image iv may be an image in which the distance between at least two spatial features has been changed, an image in which at least a portion of the image has been modified, an image in which at least a portion of the image has been deleted, an image in which at least a portion of the image has been rearranged, an image in which at least one partial image has been generated, and / or a combination thereof. Examples of image enhancement techniques performed may include scaling, cropping, resizing, and / or combinations thereof. Image v may be an image having at least a part of it altered, an image having at least a part of it rearranged, and / or combinations thereof. Examples of image enhancement techniques performed may include rotation, contrast alteration, brightness alteration, blurring, and / or combinations thereof. Image vi may be an image representing the object as a two-dimensional plane, an image having at least a part of it altered, an image having at least a part of it deleted, and / or combinations thereof. Examples of image enhancement techniques performed may include warping, folding, and / or combinations thereof.

[0098] Modifying and / or removing at least a portion of the spatial information of at least one pattern image may include changing the distance between at least two spatial features, representing an object as a two-dimensional plane, modifying at least one spatial feature, deleting at least one spatial feature, rearranging at least a portion of the image, generating a blurred pattern image, generating an image with different contrast, generating an image with different brightness, and / or all of these combinations. Image enhancement techniques can modify data associated with an image.

[0099] Figure 2b shows an object (in this example, a face) from which facial features such as eyebrows, eyes, nose, and mouth have been removed. Facial features such as the jaw and overall contour may be present in the upper image. Figure 2b is a superposition of a pattern image and a flood image. Therefore, this image may be an example of modifying and / or removing at least some of the spatial information of at least one pattern image. The flood image may be included for the visualization of the object. The pattern image may be sufficient to extract the state of the material.

[0100] In this image, since the face is still recognizable, the image may be further manipulated. For this purpose, the flood image may be used as a reference image to modify and / or remove at least some of the spatial information, for example, by changing the distance between facial features, removing rotation, and / or shearing. In principle, the flood image can be manipulated so that the object is represented as a two-dimensional plane. Examples of representing an object as a two-dimensional plane, particularly as a face, include partial or complete UV maps.

[0101] Another example of representing an object as a two-dimensional plane may be the lower image in Figure 2b. Here, the face may be flattened and / or distorted. Furthermore, the face in the exemplary pattern image and flood image may be distorted (deformed). This is beneficial because distorting the two-dimensional representation of an object by image enhancement techniques such as rotation, blurring, and shearing may change and / or remove at least some of the spatial information, or even remove the spatial information completely. As noted above, the flood image may be used as a criterion for removing at least some of the spatial information, and the pattern image may be manipulated based on this. The flood image can serve as a reference because its spatial features are visible to the user. Therefore, it can be a direct implementation for changing and / or removing at least some of the spatial information of at least some of the pattern image. When using other images as a reference for manipulating the pattern image, image enhancement techniques may be applied to the other images to produce images with little or no spatial information, or the same image enhancement techniques with the same relevant parameters may be applied to the pattern image. In this way, it is possible to reliably change and / or remove the desired degree of spatial information.

[0102] Another option for manipulating pattern images may be to manipulate them according to known procedures. These known procedures may be a clear selection of image enhancement techniques with corresponding parameters. This is particularly advantageous when an object can be recognized in the pattern image and / or when it is known which image enhancement techniques can be applied and how to remove at least some of the spatial features. These spatial features may include landmarks. Feature detection is known in the art, particularly in the case of facial feature detection. Spatial information may be modified and / or removed by processing devices such as processors.

[0103] Figure 3 is a flowchart illustrating an example of one embodiment of a method for extracting the material state associated with object 300. A pattern image is received in 310. The pattern image may be received via a communication interface. The pattern image may be received from an image generation unit such as a camera. The pattern image may include at least one speckle pattern having at least one speckle. The pattern image may be generated from coherent electromagnetic radiation reflected from at least a portion of the object. The pattern image may show at least a portion of the object under illumination by the patterned electromagnetic radiation. The patterned electromagnetic radiation may be coherent electromagnetic radiation associated with a speckle pattern. The pattern image may be generated by a camera, for example, a camera in a device, particularly a mobile device. The patterned electromagnetic radiation may be generated by an illumination source. The illumination source may be part of a device, particularly a mobile device. For example, a user may initiate an authentication process. Detecting the material state associated with the object may be part of the authentication process. The pattern image may be received from a camera and / or a camera-equipped device. The spatial information of the received pattern image is at least partially modified and / or removed in 320. The modification and / or removal of at least some of the spatial information of the pattern image may be as described in the context of Figure 2.

[0104] The state of the material associated with the object may be determined based on a pattern image 330. For example, the state of the material associated with an object may be provided to a part of a system including another device 101 and / or a processing unit. The processing unit and / or device may be suitable for determining the state of the material associated with the object based on an manipulated image. In an embodiment, the device 101 may be suitable for generating a pattern image, manipulating the pattern image, and / or determining the state of the material associated with the object based on the manipulated pattern image. The device may be a smartphone suitable for performing an authentication process. A user may initiate an authentication process, including the detection and / or prediction of a biological entity, on the smartphone. The state of the material associated with the object may be determined based on speckle contrast. The speckle contrast may be calculated as disclosed herein. Motion may be detected based on speckle contrast. Electromagnetic radiation reflected from a moving part of an object may appear more blurred than when reflected from a stationary part of an object. The motion may be caused by blood perfusion. Thus, the detection and / or prediction of a biological entity may represent the detection of blood perfusion. Multiple speckles may be used to determine the state of the material associated with the object. In response to determining the state of the material associated with the object, the state of the material associated with the object is provided. The state of the material associated with the object may be provided for use in the authentication process. The state of the material may indicate the probability of detecting a living organism and / or whether a living organism is present or not. The state of the material may be compared to a threshold. If the state of the material exceeds the threshold, it may indicate that a living organism may be present. If the state of the material is below or equal to the threshold, it may indicate that a living organism is not present.

[0105] The material status related to the object may be provided to 340. The material status related to the object may be provided to another processing device for further processing. The material status may be provided, for example, to a user or system control via a communication interface.

[0106] Figure 4 shows an exemplary embodiment of a method for extracting the material state associated with object 400. A pattern image may be received 410 as described in the context of Figure 3. The garment-resistant material may be identified in the pattern image and / or flood image 420. Objects in the pattern image may be identified due to spatial features. Spatial features may be features of an object. Implementations for identifying objects may be publicly known in the art. For example, codes for identifying faces in an image may be readily available. The features of an object may be determined based on spatial features associated with the object. The characteristics of an object may represent its shape, boundaries with other objects, color, brightness, contrast, edges, distance from other objects, relationships between two spatial features of the object, and / or such.

[0107] At least a portion of the pattern image associated with the object may be determined based on identifying the object 430. Identifying the object may include determining the object's boundaries. Based on the object's boundaries, at least a portion of the pattern image associated with the object may be determined. In one embodiment, at least a portion of the pattern image may represent speckles. The speckles associated with the object may be speckles reflected from the object. Determining the speckles associated with the object may include comparing the spatial location of the speckles with the spatial location of the object. The spatial location of the object may be determined by its boundaries. At least a portion of the pattern image may be manipulated as described in the context of Figures 2 and 3.

[0108] The contrast of the speckles included in the pattern image may be determined to be 450, as explained in Figure 5.

[0109] The material state associated with an object shown in a pattern image may be determined based on a speckle contrast of 460. The material state associated with an object may include numerical values. The material state associated with an object may indicate the possibility of living organisms being present. The material state associated with an object may be derived from the speckle contrast. The speckle contrast may be low when there is a high probability of living organisms being present. Low speckle contrast may be caused by the part of the object reflecting electromagnetic radiation being in motion. In particular, low speckle contrast may be associated with blood perfusion. High speckle contrast may be caused by the part of the object reflecting electromagnetic radiation being in a stationary state. In particular, low speckle contrast may be associated with non-living organisms. Non-living organisms do not necessarily experience blood perfusion.

[0110] As mentioned above, possible speckle contrast values ​​generally fall between 0 and 1, where 0 represents the maximum blur and therefore the highest probability favorable to the presence of living organisms, and 1 represents the minimum blur or no blur at all and therefore the highest probability of the absence of living organisms. Thus, the determined state of the material associated with the object may indicate a certain probability that living organisms are presented to the camera, for example, based on the speckle contrast value obtained using a mechanical model. For example, if the determined speckle contrast is 0, the state of the material associated with the object may indicate a 100% probability of living organisms being present. However, if the determined speckle contrast is 1, the state of the material associated with the object may indicate a 0% probability of living organisms being present. A speckle contrast of 0.5 may result in a state of the material associated with the object indicating a 50% probability of living organisms being present. Of course, it may not be necessary for the speckle contrast to be converted one-to-one to the corresponding percentage represented by the state of the material associated with the object. For example, a speckle contrast value of 0.6 could indicate a material state associated with the object, suggesting at least a 75% probability that the object presented to the camera is a living organism.

[0111] As explained in the context of Figure 3, the state of the material related to the object can be provided 470.

[0112] Figure 5 shows an example of an embodiment of the temporal change of a material state map 500. In this example, the material state map may be a vital sign map. One image can be evaluated to obtain vital sign measurements. Multiple pattern images may be used to determine the state scale. Speckle contrast is determined from the pattern images. If at least two pattern images can be received, speckle contrast may be determined for at least two pattern images. Multiple speckle contrasts may be combined in a vital sign measurement map represented, for example, by a matrix. Speckle contrast is determined by the standard deviation of illumination divided by the average intensity. Speckle contrast may be in the range between 0 and 1, where speckle contrast is 1 when there is no speckle blurring, i.e., no movement is detected in the illuminated volume of the object, and speckle contrast is 0 when there is maximum speckle blurring due to the movement of particles detected in the illuminated volume of the object, such as red blood cells. Thus, the speckle pattern and the speckle contrast obtained therefrom are sensitive to movement within the illuminated volume. Living organisms, such as humans and animals, possess a circulatory system for transporting blood cells within their bodies; therefore, the movement of this movement within the illuminated volume indicates that the object is living. If blood circulation is not detected, it can be expected that the object is not living. The stronger the speckle blur, the lower the speckle contrast. Consequently, the lower the speckle contrast, the more reliably vital signs of the object can be detected. Speckle contrast can be determined from the full pattern image or from a portion of the pattern image obtained, for example, by cropping the full pattern image. Optionally, motion correction can be applied to the pattern image. This is advantageous if the object moved while the pattern image was being captured.

[0113] Vital sign measurements can be determined from speckle contrast. If a speckle contrast map is determined from a series of pattern images, a vital sign measurement map can be generated. The vital sign measurement map may also be generated from a single pattern image by dividing the pattern image into numerous sub-pattern images and determining the speckle contrast for each sub-pattern image. The corresponding vital sign measurements may be determined based on each speckle contrast used to perform the speckle contrast map. The vital sign measurements thus determined can be combined into a vital sign measurement map. This allows for a more accurate match of vital sign measurements to specific locations on the object. In other words, it is possible to find the contribution of a portion of the object to the total vital sign measurements. For example, a portion of an object exhibiting relatively high fluid motion is expected to contribute more significantly to the sum of vital sign measurements related to the entire volume illuminated by coherent electromagnetic radiation.

[0114] The determined vital sign measurements or vital sign measurement map are then provided, for example, to the user, to another component of the same device, or to another device or system for further processing. For example, vital sign measurements may be used to predict the presence of a living organism as part of an authentication process implemented in a device, for example, as described with reference to Figure 3. In particular, vital sign measurements indicate objects exhibiting vital signs. Therefore, vital sign measurements can be used to evaluate whether an object presented to the camera is a living organism.

[0115] This can be represented by a blood perfusion map 510-530, as shown in Figure 5. Maps 510-530 can be colored according to feature contrast, with high feature contrast values ​​represented in blue and low feature contrast values ​​in red. The spatial orientation of each pixel in the image corresponds to the spatial orientation of the pixels in the pattern image having the determined corresponding feature contrast. The blood perfusion map may also be a vital signs map. The vital signs map may be determined based on speckle contrast.

[0116] The first pattern image may be generated at time t. The second pattern image may be generated at time t+p, where p is the period length of the cardiac cycle related to the heart rate. Furthermore, an indication of the interval between different time points in which pattern images may be generated is received. For one pattern image at point t and another at point t+p, the interval indication is suitable for determining the length of the interval p. The feature contrasts at point t and point t+p may be equal, where the term equal should be understood in terms of limitations of measurement uncertainty and / or biomimetic variability. The condition scale is underpinned by biomimetic variability because hemoperfusion may deviate according to various criteria. A data-driven model may be trained to compensate for uncertainty and / or biomimetic variability. A deterministic model may be suitable for compensating for uncertainty and / or biomimetic variability. Two or more pattern images may be received for determining the condition scale. In time-series images, the temporal changes in hemoperfusion are periodic due to the periodic heart rate. Placing a third pattern image between two images separated by a time period of one cycle may be advantageous in ensuring changes in motion over that period. For example, the pattern image may be generated or received using a virtual reality headset or a vehicle. In either scenario, monitoring of the living organism may be necessary to ensure the safe use of virtual reality (VR) technology and the safe control of the living organism over the vehicle, but particularly in the context of driver monitoring, the security aspect is extended to the area surrounding the living organism. This does not require direct contact with the living organism, nor does it restrict the movement of the living organism (preferably a human).

[0117] As an example, Figure 5 shows the temporal changes in hemoperfusion of hand 500. By generating multiple pattern images at different time points, blood flow and heart rate can be visualized. Part 510 of Figure 5 is a map representation of vital sign measurements at time t showing a hand with reduced hemoperfusion. Due to the activity of the heart, which is responsible for pumping blood into the blood vessels, hemoperfusion increases to its maximum at time t+p / 2 520, when half of the cardiac cycle has elapsed. After reaching the maximum, hemoperfusion decreases to the initial level 530. Changes in hemoperfusion are visualized by changes in color, where a large amount of red corresponds to a large amount of hemoperfusion with a low feature contrast value, and a large amount of blue corresponds to a small amount of hemoperfusion with a high feature contrast value. The elapsed time between t and t+p / 2 540 is equal to the time between t+p / 2 and t+p 550, where p corresponds to the length of the period. Both intervals have a length of p / 2. This is represented in Figure 5 by arrows between points in time. Another interval that provides a sufficient indicator of the time elapsed between at least two pattern images is the interval of length p 560 between the first pattern image 510 and the last pattern image 530. By recognizing the contrast change from minimum to maximum hemoperfusion, the interval of length p / 2 can be used to determine the frequency of the heart rate. Alternatively, the interval of the entire cycle of length p can also be used to determine the frequency of the heart rate, also called the exercise frequency.

[0118] Figure 6 shows an exemplary embodiment of a pattern image after at least some of the spatial information has been modified and / or removed.

[0119] In one embodiment, modifying and / or removing at least a portion of the spatial information associated with a pattern image may include removing at least a portion of the image within a predefined distance from one or more pattern features associated with the pattern image. Thus, the pattern image after modifying and / or removing at least a portion of the spatial information associated with the pattern image may represent one or more parts of the pattern image, the number of parts may be equal to the number of pattern features. This enhancement may be independent of the spatial features associated with the pattern image. Therefore, determining the state of the material from the pattern image may be independent of the spatial features. Thus, a reliable determination of the state of the material becomes possible, and it becomes possible to determine whether a living organism has been presented to the camera.

[0120] This disclosure has been described in connection with preferred embodiments and examples. However, other modifications can be understood by those skilled in the art from a study of the drawings, this disclosure and the claims and can be used to carry out the claimed invention. In particular, any of the presented steps can be performed in any order, i.e., the present invention is not limited to a particular order of these steps. Furthermore, it is not required that different steps be performed in a particular location, i.e., each step may be performed in a different location using a different data processing device.

[0121] As used herein, “determine” also includes “initiate or cause to determine,” “generate” also includes “initiate and / or cause to generate,” and “provide” also includes “initiate or cause to perform a determination, generation, selection, transmission, and / or reception.” “Initiate or cause to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

[0122] In the claims and specification, the word “including” does not exclude other elements or steps, and the indefinite articles “a” or “an” do not exclude plurals. A single element or other unit may perform the functions of multiple entities or items described in the claims. The mere fact that certain means are described in different dependent claims does not imply that combinations of these means cannot be used in advantageous embodiments.

[0123] The disclosures and embodiments described herein relate to the methods, systems, computer program elements used, and vice versa. Advantageously, any advantages provided by any embodiment and example are equally applicable to all other embodiments and examples, and vice versa.

Claims

1. A computer implementation method for extracting the state of materials related to an object, comprising the following steps: a) A step of receiving at least one pattern image, wherein the at least one pattern image shows at least a portion of the object under illumination by patterned electromagnetic radiation. b) A step of modifying and / or removing at least a portion of the spatial information of the pattern image, wherein the spatial information includes information relating to the spatial orientation of the object and / or information relating to the contour of the object and / or information relating to the edges of the object. c) A step of determining the state of the material related to the object based on the at least one pattern image, and d) A step of providing the state of the material related to the object, A method that includes this.

2. The method according to claim 1, wherein the patterned electromagnetic radiation is coherent, and the state of the material relating to the object is determined based on the speckle contrast of at least a portion of the pattern image, wherein the speckle contrast represents a measure of the average contrast of the intensity distribution within the region of the speckle pattern.

3. The method according to claim 1 or 2, wherein the at least one pattern image further shows at least a portion of a background under illumination by patterned electromagnetic radiation, and at least a portion of the pattern image relating to the object is determined by identifying at least a portion of the object in the at least one pattern image, and / or at least one flood image is received, and at least a portion of the pattern image relating to at least a portion of the object is determined by identifying the object in the at least one flood image.

4. The method according to claim 3, wherein identifying the object in the at least one pattern image and / or the at least one flood image comprises determining at least one spatial feature associated with the object in the at least one pattern image and / or the at least one flood image.

5. The method according to claim 1 or 2, wherein the object is a living organism.

6. The method according to claim 1 or 2, wherein performing at least one image enhancement technique involves modifying and / or removing at least a portion of the spatial information of the pattern image.

7. The method according to claim 1 or 2, wherein the electromagnetic radiation is within the infrared range.

8. The method according to claim 1 or 2, wherein at least two pattern images and instructions for at least one interval between at least two different time points in time in which the at least two pattern images were generated are received, a speckle contrast is determined for the at least two pattern images, and the state of the material relating to the object is determined based on the speckle contrast and the instructions for at least one time interval, wherein the speckle contrast represents a measure of the average contrast of the intensity distribution within the region of the speckle pattern.

9. The method according to claim 1 or 2, wherein at least one pattern image having at least a portion of the spatial information modified and / or removed is provided to a data-driven model and / or the state of the material relating to the object is determined using the data-driven model, the data-driven model may be parametricized according to a training dataset, the training dataset may include at least one pattern image having the modified and / or removed spatial information and at least one state of the material.

10. The method according to claim 1 or 2, wherein modifying and / or removing at least a portion of the spatial information of the pattern image includes representing the object as a two-dimensional plane.

11. A computer implementation method for training a data-driven model for extracting the material state related to an object, the following: a) A step of receiving a training dataset which includes at least one pattern image having modified and / or removed spatial information and the state of the material associated with the object, b) A step of training a data-driven model according to the training dataset, c) A step of providing a trained data-driven model, Computer implementation methods including

12. Use of the state of material obtained by claim 1 or 2 relating to an object in an authentication process for initiating and / or verifying user authentication.

13. The use of at least one pattern image having modified and / or removed spatial information for the purpose of extracting the state of material associated with an object, and / or predicting the presence of living organisms, and for the use of the state of material associated with an object.

14. A computer program element having instructions, configured to perform a step of the method according to claim 1 or 2 when executed on a processing device.

15. A system or apparatus for extracting the state of materials related to an object, the following: a) An input unit for receiving at least one pattern image, wherein the at least one pattern image shows at least a portion of an object illuminated by patterned electromagnetic radiation, b) A processor that modifies and / or removes at least a portion of the spatial information of a pattern image and determines the state of the material associated with the object based on the at least one pattern image. c) Output unit for providing the state of the material related to the object, A system or device that includes such a system or device.

Images