System and method for determining the material of an object - Patents.com

Abstract

The present invention relates to a system for determining a material of an object. The system comprises an image providing unit, a material score determining unit, a material determining unit and an output unit. The image providing unit is configured to provide at least two images, each showing a part of an object. The material score determining unit is configured to determine a material score for each of the at least two images, said material score being indicative of a presence of a predefined material in the respective image. The evaluation unit is configured to evaluate the determined material score for each of the at least two images. The material determining unit is configured to determine the material of the object based on the evaluation. The output unit is configured to output the determined material of the object.

Description

[Technical field]

[0001] The present invention relates to a system and a method for determining the material of an object. Furthermore, the present invention relates to a computer program for determining the material of an object, and to a non-transitory computer-readable data medium having said computer program stored thereon. The present invention is particularly directed to determining the material of an object as part of an authentication process implemented to prevent unauthorized access to electronic devices. However, the present invention can also be used in other technical fields, such as security technology, production technology, safety technology, documentation or technical purposes, maintenance, agriculture, cosmetics, medical technology or science. [Background technology]

[0002] In various situations it may be necessary to determine the material of an object, for example in certain situations it may be necessary to distinguish biological material, such as human skin, from non-biological material, as described inter alia in US 2016 / 155006 A1.

[0003] In WO2020 / 187719A1, a detector for identifying at least one material property is disclosed. The detector comprises at least one sensor element and at least one evaluation device. The detector is configured for determining and / or verifying whether an object to be examined or a surface under test is or comprises a living tissue, in particular human skin, and / or for distinguishing living tissue, in particular human skin, from other tissues, in particular other surfaces, and / or for distinguishing between different types of living tissue, for example different types of human tissue, such as muscle, fat, organs, etc.

[0004] However, situations may arise where the material of an object is determined, but parts of the object are covered with a different material. For example, a face may be partially covered by glasses. In such a situation, it may happen that the material of the part that actually covers the object of interest is determined and then incorrectly assigned to the object itself. This may result in denial of access to an electronic device when considering an authentication process involving face recognition, because the material of the part that covers the object, rather than the material of the object itself, has been determined. Another scenario may be that access is denied during the authentication process because the material of the object cannot be determined with sufficient certainty.

[0005] It would therefore be advantageous to be able to more reliably determine the material of an object even when several parts of the object are covered with different materials. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] US2016 / 155006A1 [Patent Document 2] WO2020 / 187719A1 Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention aims to provide an improved or at least alternative system, method and computer program for determining the material of an object. Preferably, the system, method and computer program allow for a more reliable determination of the material of an object, in particular even when some parts of the object are covered with another material. Thus, the system, method and computer program are relatively robust against disturbances, in particular when the disturbances are limited to a small part of the image of the object. More preferably, the system, method and computer program require relatively little data, thereby providing a relatively high accuracy in determining the material of the object. [Means for solving the problem]

[0008] According to the present invention, there is proposed a system for determining a material of an object, which comprises an image providing unit, a material score determining unit, an evaluation unit, a material determining unit, and an output unit.

[0009] The image providing unit is configured to provide at least two images, each showing a portion of the object. The material score determining unit is configured to determine a material score for each of the at least two images. The material score indicates a presence of a predefined material in the respective image. The evaluation unit is configured to evaluate the determined material score for each of the at least two images. The material determination unit is configured to determine a material of the object based on the evaluation. The output unit is configured to output the determined material of the object.

[0010] The present invention includes the recognition that when determining the material of an object based on an image showing the object, situations may arise where parts of the object are covered with a different material, such as the face covered by the glasses mentioned above, in which the material covering the object may be erroneously determined to be the material of the object itself.

[0011] With the system according to the invention, the risk of assigning the wrong material to an object can be reduced. The system can thus determine the material of the object with a relatively high reliability. The system has the particular advantage that it can determine the material of the object in a relatively robust manner, even in the presence of disturbances in the image. In particular, the influence of small disturbances present in some parts of the object (e.g. the presence of a material covering the object) is reduced when determining the material of the object. A further advantage of the system is that a relatively small amount of data, e.g. a smaller number of images, is required to determine the material of the object. For example, it is sufficient for a single target area image to be divided into a number of partial images that are used by the system to determine the material of the object. In this way, the determination of the material of the object can be achieved by the system relatively quickly and with reduced effort. At the same time, the system can determine the material of the object with a relatively high accuracy.

[0012] This is achieved by the system determining individual material scores for at least two images, each showing a part of the object. Each of the material scores may be independent of the other material scores and indicates the presence of a predefined material in its respective image. In this way, the system can be used to provide multiple material scores related to different parts of the object. In particular, it is of particular advantage that each of the material scores is associated with a specific part of the object, reducing errors resulting from averaging over the whole object. Thus, considering again the example of a face partially covered by glasses, the system can determine a material score for the part of the face covered by the glasses, and one or more further material scores related to other parts of the face that are not covered.

[0013] The evaluation unit can be used to evaluate multiple material scores to obtain an evaluation result. In particular, each of the material scores contributes individually to obtain the evaluation result. In other words, various parts of the object and the corresponding material scores can be taken into account to generate the evaluation result. For example, the evaluation result represents a certain probability that the object contains a certain material. The material determination unit can be used to determine the material of the object while taking into account the evaluation result. For example, the evaluation result is compared with a reference value or a threshold value for determining the material of the object. In this way, the system can be used to determine the material of the object more reliably. In particular, the system can be used to reduce the risk that a material covering some part of the object contributes significantly to the process of determining the material of the object. In this way, assigning an incorrect material to the object is avoided.

[0014] It is particularly preferred that the material of the object determined using the system is further used in the authentication process: for example, if a person's face needs to be recognized, as part of the authentication process, the determination that the material is human skin can be used, for example in combination with facial recognition, to avoid spoofing, for example by providing the camera with a silicone mask that resembles the authenticated user's face.

[0015] The system is able to determine the material of an object more reliably, especially when at least two images show different parts of the object, although it is of course also possible that the at least two images show different parts of the object that overlap.

[0016] The system can be used to determine different types of materials. This is achieved by considering the interaction of light with the object, for example to determine a material score associated with a certain part of the object. When light is incident on a certain material, such as human skin, it is expected that the incident light penetrates the object and is reflected from the inside of the object. When other materials, such as metals, are considered, the incident light is expected to be almost completely reflected at the outer surface of the object. Furthermore, the light spectrum of the reflected light captured by the camera is taken into account to determine the material score associated with a certain part of the object. This allows taking into account that a certain material reflects light within a certain spectral range. Alternatively or additionally, the properties of the specular and / or diffuse reflection, such as the intensity distribution or the spread of the peaks, are taken into account, for example to determine a material score associated with a certain part of the object. The latter property may be particularly relevant when the object is illuminated with patterned light, i.e. with a large number of light spots captured by the camera.

[0017] The material determined by the system may be biological material, including living cells. In particular, the biological material may be or include human tissue or parts thereof, such as skin, hair, muscle, fat, organs, etc. The biological material may be or include animal tissue or parts thereof, such as skin, fur, muscle, fat, organs, etc. The biological material may be or include tissue or parts thereof of a plant, such as a tree. The biological material may include, for example, cotton or silk used to manufacture textiles, such as cloth or carpets.

[0018] The material to be determined may be an inorganic material, such as a metal, or a plastic such as polyethylene terephthalate (PET) or polyvinyl chloride (PVC), or a synthetic fiber (such as those used in fabric or carpet manufacturing).

[0019] The system may be adapted to distinguish biological material, such as human, animal or plant tissue or parts thereof, from one or more of inorganic tissue, metal surfaces or plastic surfaces. The system may be adapted to distinguish food and / or beverages from plates and / or glasses. The system may be adapted to distinguish different types of food, such as fruit, meat and fish. The system may be adapted to distinguish cosmetics and / or applied cosmetics from human skin. The system may be adapted to distinguish human skin from foam, paper, wood, displays, screens. The system may be adapted to distinguish human skin from fabrics. The system may be adapted to distinguish maintenance products from machine part materials, such as metal parts. The system may be adapted to distinguish organic from inorganic. The system may be adapted to distinguish human biological tissue from man-made or non-living surfaces.

[0020] The system may be a stationary device or a mobile device. Examples of mobile devices include mobile phones or smartphones, tablet computers, laptop computers, portable gaming devices, portable Internet devices, and other handheld devices, and wearable devices, such as smart watches, smart glasses, headphones, pendants, ear pieces, etc. Furthermore, the system may be a standalone device or may form part of another device, such as a computer, a vehicle, or any other device. Furthermore, the system may be a portable device. Other embodiments of the system are possible.

[0021] The system, and in particular the evaluation unit, comprises at least one database containing lists and / or tables containing a number of predefined materials and associated material names and / or material groups.

[0022] Preferably, the at least two images are partial images each showing a part of the object area contained in the object area image. In other words, preferably, the at least two images are partial images generated from the object area image. In this case, the partial images each show a part of the object area image. Preferably, the object area image is a floodlight image or a pattern image or is generated from the floodlight image or the pattern image. To obtain the partial image, the partial image can be extracted or generated from the object area image, for example by image processing. Thus, the partial image is an image obtained from the object area image and showing a part or a part of the object area image.

[0023] Preferably, the target area image is a single shot image. Preferably, the target area image is an image that substantially shows a selected or predefined target area of ​​the object, for example a face or a portion of a carpet on a floor. The target area image can be generated by capturing the object with a camera such that only the target area is recorded. Alternatively, the target area image can be generated by cropping the original image such that the target area image substantially shows the selected target area. Thus, the target area image can be divided into a number of partial images. For example, the target area image may be divided into a number of partial images such that when the partial images are joined together the target area image can be reconstructed.

[0024] To generate the at least two images provided by the image providing unit, the target area image may be divided into at least two partial images representative of the at least two images provided by the image providing unit, preferably each of the at least two images showing a part of the target area shown in the target area image.

[0025] Of course, the region of interest image can be divided into a larger number of sub-images, for example between 10 and 250 sub-images, such as 16, 32, 64 or 128 sub-images. In some applications, an even larger number of sub-images, for example 500 or more or 1000 or more, can be beneficial.

[0026] The object area image may be an RGB image of the object. Alternatively, the object area image may show the object illuminated by the point cloud. In particular, the object area image may be an infrared (IR) image.

[0027] Thus, the at least two images may be RGB images of a portion of the object, or the at least two images may show a portion of the object illuminated by the point cloud, in particular the at least two images may be IR images.

[0028] Preferably, the material score determination unit is configured to determine an individual material score for each of the partial images representative of the at least two images provided by the image providing unit, each material score preferably representing the likelihood that an associated part of the object shown in the respective image is made of a predefined material, e.g. a target material.

[0029] Generally, the material score may be represented by a single value between 0 and 1, especially when only one material is determined. For example, it may be determined whether the object contains human skin or not. Preferably, a material score of 0 indicates a 0% probability that the object contains the predefined material. Thus, a material score of 1 indicates a 100% probability that the object contains the predefined material.

[0030] The material score associated with one of the at least two images may be represented by a vector or a matrix. Each vector element or each matrix element may have a value between 0 and 1. Such a representation of the material score as a vector or a matrix is ​​feasible when multiple materials are determined for an image. An example of a situation where multiple materials are determined is based on an image showing a carpet laid on a floor. For example, it can be determined whether the floor is made of PET, PVC or wood, and whether the carpet is made of organic or synthetic fibers. After determining the material score for each of the determined materials, the resulting material scores can be provided as a vector or a matrix of material scores.

[0031] In particular, based on the likelihood that certain predefined material is present in a respective one of the at least two images, the material score determination unit is configured to determine a material score for each of the at least two images.

[0032] Preferably, for determining the material score, the material score determination unit comprises a material score determination model configured to receive as input at least two images each showing a part of the object, and to determine a material score for each of the at least two images. The material score determination model may be a classification model, such as a data-driven model, or may be a mechanistic model. The material score determination model may be parametrized and / or trained based on a training data set comprising a plurality of images and corresponding material scores associated therewith. Preferably, the material score determination model is parametrized and / or trained to output a material score for each of the at least two images.

[0033] It is therefore particularly preferred that the material score determination unit comprises a data-driven model configured to determine a material score for each of the at least two images. In addition to or instead of the data-driven model, the material score determination unit may comprise a mechanistic model configured to determine a material score for each of the at least two images. Preferably, the data-driven model is a neural network trained to determine a material score for each of the at least two images using the at least two images as input.

[0034] The trained neural network can be a multi-scale neural network or a recurrent neural network (RNN), such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. Alternatively, the neural network can be a convolutional neural network (CNN).

[0035] The neural network may be trained using training data including, for example, images showing a part of an object and an associated material score. For example, if the neural network is a feed-forward neural network such as a CNN, a backpropagation algorithm may be applied to train the neural network. In the case of an RNN, a gradient descent algorithm or a backpropagation through-time algorithm may be employed for training purposes. As a result of the training, operating parameters of the neural network circuitry are generated such that, when receiving as input at least two images showing a part of an object, the trained neural network outputs an associated material score as a prediction. As mentioned above, to determine a material score for an image showing a part of an object, the manner in which the part of the object interacts with light may be taken into account.

[0036] The neural network may be trained to identify materials present in a certain portion of an object represented by a respective one of the at least two images. For example, the neural network may be trained to classify the determined materials. Material classes may be, for example, human skin, PVC, PET, silicone, metal, wood, etc. The trained neural network outputs a material score indicating the likelihood that a material belonging to a certain predefined class is present in each image.

[0037] Preferably, the evaluation unit may be further configured to evaluate the determined material score by setting the determined material score relative to a reference amount, such as a predefined threshold or reference (e.g. a reference material score). The threshold may be a likelihood value, and the determined material score may be compared to the threshold to obtain an evaluation result. Additionally or alternatively, the evaluation unit may be further configured to evaluate the determined material score by setting the determined material score relative to a reference material score. The reference material score may be associated with a reference image showing the known material. For example, the determined material score may be compared to the reference material score to obtain an evaluation result. Additionally or alternatively, the evaluation of the material score may include classifying the determined material score. For example, the evaluation of the material score may be performed using a data-driven model, such as a classification model, such as a neural network or a vision transformer. For example, the classification model may be configured to evaluate the material score by providing a prediction whether the object contains or does not contain a predefined material.

[0038] Preferably, the evaluation unit is further configured to evaluate the determined material scores for each of the at least two images by forming an average material score of the determined material scores. This has the advantage that the material of the object is not determined based on a single outlier. Instead, the material of the object is determined based on the most prevalent material scores. To form the average material score, the evaluation unit may be configured to ignore the outliers in a predefined manner.

[0039] Preferably, the material determination unit is further configured to compare the average material score with a predefined threshold. If the average material score is above the predefined threshold, the object is identified, for example, by the material determination unit as being of the predefined material. If the average material score is below the predefined threshold, the object is identified, for example, by the material determination unit as not being of the predefined material.

[0040] Preferably, the predefined threshold is set such that the false positive rate and the false negative rate in the test data set are approximately the same. This has the advantage that the material of the object can be determined more reliably. In particular, by setting the threshold high, it is possible to avoid too many false negative results. Similarly, by setting the threshold low, it is possible to avoid too many false positive results. By setting the predefined threshold such that the false positive rate and the false negative rate in the test data set are approximately the same, it is possible to take into account that the training, evaluation and test data sets are often not complete in order to prepare the system for all possible situations that it may have to handle.

[0041] In particular, when forming an average material score, the evaluation unit may be further configured to evaluate the material scores determined for each of the at least two images by giving a weight to each of the at least two images. For example, the material scores of each image may be multiplied by a weight to obtain a weighted material score. The weight preferably indicates how important each image is for the material determination. Taking face recognition as an example, typically, the area of ​​the face close to the hair is less important, since it can be assumed that this area is often hidden by the hair. However, the area around the nose and mouth is considered to be of increased importance. Thus, in the face example, images showing the mouth or nose area are assigned a higher weight than images showing areas close to the hair.

[0042] Preferably, the material determination unit is further configured to determine the material of the object based on a comparison of the average material score with a predefined threshold. In particular, the material determination unit may determine that the object contains the predefined material if the average material score exceeds the threshold. Correspondingly, the material determination unit may determine that the object does not contain the predefined material if the average material score is less than the threshold.

[0043] Preferably, the evaluation unit may be further configured to evaluate the material score by forming an average material score of the determined material scores based on a weight assigned to each of the at least two images. The weighted average material score may be calculated by multiplying each element of the material score vector with a corresponding weight value associated with each part of the object. For example, segmented images are provided showing segments of the object with associated weight values ​​indicating the importance of the particular segment for the material determination. Thereby, parts of the object that are expected to be of increased importance for the material determination of the object will contribute more to the formed weighted average material score.

[0044] Preferably, the material determination unit may be further configured to compare a weighted average material score based on the weighted material scores with a predefined threshold.

[0045] Additionally or alternatively, the evaluation unit is further configured to compare the material scores determined for the at least two images with a reference. The reference preferably comprises at least one reference material score determined from a reference image showing a known reference material. For example, the reference can be a reference material score vector comprising reference material scores as vector elements. For example, the material score is represented by a material score vector or matrix. In particular, the reference can be a stored material score vector or matrix from an image where the material of a part of an object is known. For example, the known material can be a target material determined for the object. As an example, the reference can be associated with a known material "human skin" so that by comparing the material scores of the at least two images with the reference, it can be determined whether the part of the object shown in the image actually shows human skin. Taking face recognition as an example, the reference can be available from an enrollment process where a user scans his face to generate a reference image for setting up the recognition system.

[0046] For example, the evaluation unit may be configured to evaluate the material scores by element-by-element comparison of each of the material scores determined for the at least two images with a reference.

[0047] In addition to or instead of the element-by-element comparison, to evaluate the material score, the evaluation unit may comprise a neural network trained to receive as input the material scores of at least two images and the reference, and to output a prediction of the material of the object based on the input. The neural network may be trained to output a binary information of whether a part of the object in a particular image is indicative of the material of the reference. Alternatively, the neural network may be trained to output a likelihood of whether a part of the object in a particular image is indicative of the material of the reference. The likelihood may be converted into a decision, for example by setting a threshold value depending on the use case by the material determination unit.

[0048] The trained neural network of the evaluation unit may be a multi-scale neural network or an RNN, such as, but not limited to, a GRU recurrent neural network or an LSTM recurrent neural network. Alternatively, the neural network may be a CNN. The neural network may be trained using training data (e.g., including material scores along with reference and related known materials).

[0049] The material determination unit may be configured to determine the material of the object based on a piece-by-piece comparison of the material scores of the at least two images with a reference. Alternatively or additionally, the material determination unit may be configured to determine the material of the object by comparing a prediction of the material of the object provided by the trained neural network with a predefined use case threshold.

[0050] It is advantageous if the image providing unit is configured to provide each of the at least two images together with position information indicative of a relative position on the object. The evaluation unit may include a neural network trained to receive as input the material scores of the respective images together with the position information of said images and to output a prediction of the material of the object based on the input. The position information of the images may be provided as 3D information representing a position in space, for example in a coordinate system of the object.

[0051] Preferably the neural network is trained to know which locations on the object have material scores that are more relevant to a material determination than other locations. Preferably the neural network is trained to output a binary decision or likelihood that the material of the object matches a predefined material.

[0052] It is particularly preferred that the trained neural network is a point net neural network. Alternatively, the trained neural network of the evaluation unit may be a multi-scale neural network or an RNN, such as, but not limited to, a GRU recurrent neural network or an LSTM recurrent neural network. Alternatively, the neural network may be a CNN. The neural network may be trained using training data (e.g., including a material score for each image together with position information of the image and related known material).

[0053] The system may further comprise an authentication unit configured to authenticate the object using the determined material of the object. The authentication unit may be configured to combine the determined material of the object information with other authentication processes, such as facial recognition, to provide an increased level of security for verifying the authenticity of a claimant.

[0054] The invention also relates to a method for determining the material of an object, said method comprising the steps of: - providing at least two images, each showing a portion of an object; - determining a material score for each of at least two images, said material score being indicative of a presence of predefined material in the respective image; - evaluating the determined material score for each of the at least two images; - determining a material of the object based on the evaluation; and - A step of outputting the determined material of the object. Includes.

[0055] The method may be implemented using a system for determining the material of an object as described above.

[0056] The invention also relates to a computer program for determining the material of an object, said computer program comprising instructions for carrying out the steps of the method defined above, when said computer program is executed on a computer. The invention also relates to a non-transitory computer-readable data medium storing said computer program.

[0057] A non-transitory computer readable data medium having a computer program stored thereon may be part of a system for determining the material of an object as described above.

[0058] The invention also relates to an authentication process, for example a biometric authentication process, comprising the following steps: - carrying out a biometric recognition of the user, for example by presenting his face to a camera or by determining his fingerprint with a fingerprint sensor, preferably comprising the following sub-steps: - providing a detection signal from a camera, said detection signal representing an image of a feature of a user (e.g. a fingerprint or a face); - generating a low-level representation of the image; - verifying the authentication of the user based on the low-level representation of the image and the stored low-level representation template; and - if the biometric recognition is successful, a step of determining the facial material, preferably comprising the following sub-steps: - determining the facial material shown in the image; - comparing the determined material with stored predefined materials. and - providing a positive authentication output signal if the determined material matches the stored predefined material. Alternatively, if the determined material does not match the predefined material, a negative authentication output signal is provided. In other words, generally, an authentication output signal is provided that indicates whether the determined material matches the stored predefined material.

[0059] The material may be determined based on one or more material scores determined for at least two images of the object, and in particular by implementing the method for determining the material of an object described above.

[0060] If the verification step has already produced a negative result, a negative authentication output signal is provided without determining the material.

[0061] In another biometric authentication process, first the material of the object (e.g. the face presented to the camera) is determined, and then a facial recognition process is performed if the determined material matches successfully with a predefined material.

[0062] It is to be understood that the above-mentioned aspects, and in particular the system of claim 1, the method of claim 13 and the computer program of claim 14, have similar and / or identical preferred embodiments, in particular as defined in the dependent claims.

[0063] It is to be understood that a preferred embodiment of the invention can also be any combination of the dependent claims or the above embodiments with the respective independent claim.

[0064] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. [Brief description of the drawings]

[0065] [Figure 1] FIG. 1 shows a schematic and exemplary image recording device. [Diagram 2] FIG. 2 illustrates, in a schematic and exemplary manner, an image processing device that can be used to generate at least two images of an object. [Diagram 3] FIG. 3 illustrates, in a schematic and exemplary manner, a system for determining the material of an object. [Figure 4] FIG. 4 shows a flow chart illustrating a method for determining the material of an object. [Diagram 5] FIG. 5 shows a flow chart depicting the steps performed to evaluate material scores by forming an average material score. [Figure 6] FIG. 6 shows a flow chart depicting the steps performed to evaluate a material score, including comparing the material score to a reference. [Figure 7] FIG. 7 shows a flow chart depicting the steps performed to evaluate the material score by forming a weighted average material score. [Figure 8] FIG. 8 shows a flow chart depicting the steps performed to evaluate material scores based on location information provided for each of the material scores. [Figure 9] FIG. 9 shows a flow chart illustrating the authentication process, which includes determining the material of the user to be authenticated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0066] Detailed Description of the Preferred Embodiments 1 shows a schematic and exemplary image recording device 100. The image recording device 100 may be, for example, a mobile phone. The image recording device 100 comprises a camera 102 and two projectors, a first projector 104 for illuminating a floodlight (e.g., LED) and a second projector 106 for illuminating a light pattern (e.g., a VCSEL (Vertical Cavity Surface Emitting Laser) array). Thus, with the image recording device 100 it is possible to capture an RGB image of a scene and / or an image from the scene illuminated by a point cloud, i.e. a pattern image.

[0067] The camera 102 of the image recording device 100 can capture at one time an object illuminated by floodlight using a first projector 104 and at another time an object illuminated by a light pattern generated by a second projector 106. These images, the pattern image and the floodlight image, are then transmitted to an image processor, e.g., as described with reference to FIG.

[0068] Camera 102 may include one or more image sensors for capturing digital images. The image sensor may be an array of sensors. The sensors of the sensor array may include, but are not limited to, charge-coupled device (CCD) and / or complementary metal-oxide semiconductor (CMOS) sensor elements to capture IR images or other non-visible electromagnetic radiation. Camera 102 may include multiple image sensors to capture multiple types of images. For example, camera 102 may include both IR and RGB (red, green, and blue) sensors. In certain embodiments, the image sensor of camera 102 is an IR image sensor, and the image sensor is used to capture infrared images used for face detection, face recognition authentication, material detection, and / or depth detection.

[0069] The first and second projectors 104, 106 preferably comprise at least one light source, for example a plurality of light sources. The first and second projectors 104, 106 comprise an artificial illumination source, in particular at least one laser source, and / or at least one incandescent lamp, and / or at least one semiconductor light source, for example at least one light emitting diode, in particular an organic and / or inorganic light emitting diode. By way of example, the light emitted by the first and second projectors 104, 106 has a wavelength of 300-1100 nm, in particular 500-1100 nm. Additionally or alternatively, light in the infrared spectral range, such as in the range of 780 nm to 3.0 μm, may be used. In particular, light in the near infrared range part, in particular in the range of 700 nm to 1100 nm, where silicon photodiodes are applicable, may be used. By using light in the near infrared range, the light is not detected or is only weakly detected by the human eye, but is still detectable by silicon sensors, in particular standard silicon sensors.

[0070] The first and second projectors 104, 106 may be adapted to emit light at a single wavelength. In other embodiments, the first and second projectors 104, 106 may be adapted to emit light having multiple wavelengths that allow additional measurements in other wavelength channels. The first and second projectors 104, 106 may be or include at least one multi-beam light source. For example, the first and second projectors 104, 106 may include at least one laser source and one or more diffractive optical elements (DOEs).

[0071] Specifically, the first and second projectors 104, 106 may have at least one laser and / or laser source. Various types of lasers may be employed, such as semiconductor lasers, double heterostructure lasers, external cavity lasers, separate confinement heterostructure lasers, quantum cascade lasers, distributed Bragg reflector lasers, polariton lasers, hybrid silicon lasers, extended cavity diode lasers, quantum dot lasers, volume Bragg grating lasers, indium arsenide lasers, transistor lasers, diode pumped lasers, distributed feedback lasers, quantum well lasers, interband cascade lasers, gallium arsenide lasers, semiconductor ring lasers, extended cavity diode lasers, or vertical cavity surface emitting lasers. Additionally or alternatively, non-laser light sources such as LEDs and / or light bulbs may be used. The first and second projectors 104, 106 may have one or more diffractive optical elements (DOEs) adapted to generate an illumination pattern.

[0072] In particular, for face detection, the projector 104 may provide flood IR illumination to illuminate a subject with IR illumination, and the image sensor may capture an image of the flood IR illuminated subject. The flood IR illumination image may be, for example, a two-dimensional image of the subject illuminated by IR light.

[0073] In particular, for depth sensing or material detection, the projector 106 may provide IR illumination with a pattern. The pattern may be a pattern of light with a known and controllable configuration and pattern that is projected onto a subject. The pattern may be regularly arranged with a structured pattern of light patterns or may be irregularly arranged. In a particular embodiment, the pattern is a speckle pattern. Patterns include, but are not limited to, dots, speckles, stripes, dashes, nodes, edges, and combinations thereof.

[0074] The image sensor of the camera 102 and the projectors 104, 106 may be included in a single or separate chip packages. In some embodiments, the image sensor and the projectors 104, 106 are disposed in separate chip packages. In addition to or instead of the first and second projectors 104, 106, the image recording device 100 may further include a visible light projector, such as a flash illuminator, an RGB light projector, and / or an infrared light projector.

[0075] Images captured by recording device 100 may be processed, for example, by an image processing device as described with reference to FIG.

[0076] 2 illustrates, in a schematic and exemplary manner, an image processing device 200 that can be used to generate at least two images of an object. The image processing device 200 may be configured to receive images captured by a camera 202 (not part of the image processing device 200). The camera 202 may be part of the recording device 100 as described with reference to FIG. 1. The camera 202 may thus provide a floodlight image showing an object illuminated by a floodlight and / or a pattern image showing an object illuminated by a light pattern.

[0077] The image processing device 200 includes an image processor 204 that may include circuitry suitable for processing images received from the camera 202. The image processor 204 may include hardware and / or software, such as implementing program instructions, such as a trained neural network 205, that can process or analyze images captured by the camera 202.

[0078] The image processing device 200 further comprises a secure processor 206. The secure processor 206 is provided for sensitive operations, e.g., part of an authentication process. The secure processor 206 may be a secure circuit configured to authenticate an active user. The secure processor 206 may be a circuit that protects isolated internal resources from being directly accessed by external circuits. The internal resources may be memory that stores sensitive data, such as biometric information, encryption keys, etc.

[0079] The secure processor 206 may perform a facial recognition authentication process based on images captured by the camera 202 and processed by the image processor 204. A single processor may perform the functions of the image processor 204 and the secure processor 206.

[0080] The image processing device 200 may perform an enrollment process during which the camera 202 captures or collects images and / or image data from an authenticated user and then authenticates the user using a facial recognition authentication process. From the images of the enrollment process, a template may be generated and stored in the secure processor 206.

[0081] During face authentication, the camera 202 communicates image data to the secure processor 206 via a secure channel. The secure channel may be, for example, either a dedicated path for communicating data (i.e., a path shared only with the intended participants) or a dedicated path for communicating data encrypted with an encryption key known only to the intended participants. The secure processor 206 may run one or more machine learning models. To run the machine learning models, one or more neural network modules 207 may be used. The neural network modules may be located in the secure processor 206. The secure processor 206 compares the image features to stored templates for each type of image and generates an authentication score according to a matching score or other ranking of matches between the user in the captured image and the user in the stored templates. The authentication scores of the images, such as the flood IR image and the pattern illumination image, may be combined to determine the identity of the user and, if authenticated, allow the user to use the device (e.g., unlock the device).

[0082] Figure 3 shows, in a schematic and exemplary manner, a system 300 for determining the material of an object (not part of the system 300), which may be part of the image processing apparatus 200 as described with reference to Figure 2. For example, elements of the system may be implemented as part of the image processor 204 and / or the secure processor 206.

[0083] The system 300 comprises an image providing unit 302 configured to provide at least two images 304, 306 each showing a part of an object. The image providing unit 302 receives the at least two images 304, 306 from an image recording device, for example as described with reference to FIG.

[0084] Typically, the at least two images 304, 306 are partial images of the region of interest, i.e. each shows a part of the region of interest. The at least two images 304, 306 showing a part of the object may be generated in different ways.

[0085] For example, a camera may be used to capture a target area image showing a selected or predefined target area of ​​the object. The image may be a floodlight image or a pattern image. The image may be split into at least two images, each showing a portion of the target area image. The at least two images may be transmitted to an image providing unit 302 of the system 300 for determining the material of the object.

[0086] It is also possible to capture images with the camera that show some environment other than the object and the target. Such images are raw images that can be pre-processed, preferably cropped, to generate a target area image that shows only the selected target area or that substantially shows the selected target area. Cropping may be based on identifying a peak of the pattern and cropping a certain size around the peak. In such an embodiment, the peak may be in the center of the image. The sub-images may have a fixed size and do not overlap. In another embodiment, the cropping may be random with respect to the location of the peak. The sub-images may have any size. The sub-images may overlap. The sub-images may include at least one pattern feature, e.g. an intensity peak, or more pattern features. The sub-images may include parts of the pattern features or outlier signatures of the pattern features. Another option is single shot detection (SSD; region-based neural network or mask recurrent neural network) to provide a bounding box for the segmentation of the sub-image. The manipulation may therefore be based on segmentation via anchor points based on the pattern features of the pattern image. The region of interest image generated by cropping the original image is split into at least two images 304 , 306 showing parts of the object and sent to an image providing unit 302 of the system 300 .

[0087] It is also possible to directly capture the at least two images 304, 306 using a camera, for example by scanning the target area of ​​the object. In this case, each of the at least two images shows only a smaller part of the selected target area of ​​the object, whose material is to be determined. Thus, in this case, the at least two images constitute partial images of the target area. The at least two images may be directly captured by the camera and transmitted to the image providing unit 302 of the system 300 for determining the material of the object.

[0088] The system 300 comprises a material score determination unit 308 configured to determine a material score 310, 312 for each of the at least two images 304, 306. The material score represents the likelihood that a portion of the object shown in the images 304, 306, respectively, contains the target material. The material score may be represented by a single value ranging from 0 to 1, whereby 0 indicates a 0% likelihood that the object exhibits the material in the respective portion and 1 indicates a 100% likelihood that the object exhibits the material in the respective portion.

[0089] The determined material scores 310, 312 are passed to an evaluation unit 314 configured to evaluate the determined material scores 310, 312 for each of the at least two images 304, 306. The evaluation unit 314 provides an evaluation result 316 indicative of an evaluation of the material scores 310, 312. For example, the evaluation result 316 is obtained by forming an average or sum of the material scores 310, 312. Alternatively, the evaluation unit 314 may comprise a trained neural network configured to provide the evaluation result 316 based on the material scores 310, 312. In general, the process of evaluating the material scores 310, 312 can be implemented in various ways, which will be described with reference to Figures 5, 6, 7 and 8.

[0090] Based on the evaluation result 316, a material determination unit 318 of the system 300 can determine the material 320 of the object. For example, to determine the material 320, the material determination unit 318 compares the evaluation result 316, e.g., an average of the material scores 310, 312, with a threshold value to determine the material 320 of the object. The determined material 320 is then output by an output unit 322 of the system.

[0091] 4 shows a flow chart illustrating a method for determining the material of an object, which may be implemented, at least in part, using the system 300 described with reference to FIG.

[0092] First, an image of an object is received showing a target area of ​​the object (e.g. a face) (step S1). The target area image is divided into a number of sub-images (step S2), each sub-image representing a different part of the target area image. The sub-images represent, for example, at least two images provided to an image providing unit of a system for determining the material of an object as described with reference to FIG. 3.

[0093] For each partial image, an individual material score is determined (step S3). The material score of a partial image indicates the presence of a predefined material in the respective image. The material score of a partial image may be a value between 0 and 1 if only one material is detected, for example when detecting whether a part of a face shown in the partial image is made of human skin or not. The material score of a partial image may be a vector or a matrix if multiple materials are determined in the partial image (for example an image in which a floor with carpet is PET, PVC or wood). Doing this for all partial images results in a material score for each image. The result is provided by a material score vector or matrix. The material score vector or matrix is ​​then compiled and / or evaluated in a way that best suits the particular use case (step S4). Some methods of evaluating the material scores are described with reference to Figs. 5, 6, 7 and 8, and may be included in the method for determining the material of an object described with reference to Fig. 4.

[0094] Based on the evaluation of the material score, the material of the object is determined. The determined material is then matched against a target material, e.g., a predefined determined material, to determine whether the determined material is indeed the target material (step S5) or whether the determined material is different from the target material (step S6).

[0095] The method described with reference to Figure 4 may be performed by a computer program comprising instructions for performing the steps of the method when executed on a computer. The computer program may be stored on a non-transitory computer readable data medium which may be part of the system 300 described with reference to Figure 3 and / or part of the image processing device described with reference to Figure 2.

[0096] The material of an object determined by performing the method steps described with reference to FIG. 4 can be used to authenticate the object, for example in the authentication process described with reference to FIG.

[0097] Different methods of evaluating the material scores determined for at least two images, e.g. sub-images of a target area image, are described below with reference to Figures 5 to 8. Each method may be included in the method for determining the material of an object described with reference to Figure 4.

[0098] 5 shows a flow chart illustrating the steps performed to evaluate a material score by forming an average material score. In particular, in this method, the average value of each element of a material score vector or matrix is ​​formed.

[0099] For this purpose, the determined material scores are provided (step T1), for example expressed as a vector or matrix. Based on the determined material scores, an average material score is formed (step T2). The average material score thus obtained is compared with a preset threshold (step T3). If the average material score is above the threshold, the object is identified as a target material (step T4). Otherwise, the object is indicated as not being a target material (step T5). The determination of the threshold is important for the reliability of the recognition: if the threshold is too high, there will be too many false negative results; if the threshold is too low, there will be too many false positive results. In many cases, the training, evaluation and test data sets are not complete to prepare the system for all possible situations that it may have to handle. However, the inventors have found that good results are obtained if the threshold is selected such that the false positive and false negative rates of the test data set are approximately the same.

[0100] FIG. 6 shows a flow chart depicting the steps performed for evaluation of a material score, including comparing the material score to a reference.

[0101] First, a material score of an image is provided (step V1), for example expressed as a vector or matrix. The image may be a sub-image of a target region image. Furthermore, a reference is provided (step V2). This reference is preferably a stored material score vector from an image where the material of the target region is known. In the face recognition example, the reference may be a reference image available from an enrollment process where a new person scans their face to set up the recognition system.

[0102] In the method, the material score of the image is compared to a reference (step V3). In this particular example, the comparison is performed by a trained neural network configured to receive the material score vector and the reference as input. The neural network may be trained to output a binary information if the object is a material of interest (step V4) or not (step V5). Alternatively, the neural network may be trained to output a likelihood that is converted into a decision by setting a threshold depending on the use case. However, the step of comparing the material score vector to the reference may be realized in another way, such as by performing a simple element-wise differencing.

[0103] FIG. 7 shows a flow chart depicting the steps performed to evaluate the material score by forming a weighted average material score.

[0104] First, a material score vector is provided (step W1), the elements of which are the material scores of parts of an object. In the method, a segmented image is provided, the image containing segments of the object with associated weight values ​​indicating how important each segment is to the material determination (step W2). In the example of face recognition, typically, areas close to hair are less important since they are often hidden by hair, while areas around the nose and mouth are more important.

[0105] Each element of the material score vector can be multiplied with a corresponding weight value of the segmented image. This is possible because each material score is associated with a particular part of the object and can therefore be matched with a segmented image that shows the object.

[0106] Of the weighted material scores, a weighted average material score is formed (step W3). The weighted average material score is compared to a pre-defined threshold as described for the evaluation process of Fig. 5. If the weighted average material score is above the threshold, the object is identified as being of target material (step W4). Otherwise, the object is indicated as not being of target material (step W5).

[0107] FIG. 8 shows a flow chart depicting the steps performed to evaluate material scores based on the location information provided for each material score.

[0108] The method provides a material score vector (step X1) and 3D information (step X2) that describes the position in space, for example in the coordinate system of an object, of each image with an associated material score.

[0109] Both the material score vector and the 3D information are provided as input to a trained neural network (step X3). Preferably, the neural network is a point-net neural network. During the training process, the neural network has learned which locations have material scores that are more relevant to the overall estimation than other locations. The trained neural network outputs a decision whether the object material determined for each image matches the target material (step X4) or not (step X5) based on the associated material scores.

[0110] FIG. 9 shows a flow chart illustrating an authentication process that includes determining the material of the object to be authenticated.

[0111] In the method, a detection signal representative of a feature of the object to be authenticated (e.g., representative of the face or fingerprint of a user to be authenticated) is provided from a camera (step M1). For example, for authentication, a floodlight image is analyzed for facial features.

[0112] From the detected features of the user to be authenticated, a low-level representation is generated (step M2). Options for constructing the low-level representation (e.g. of a subimage) are Fast Fourier Transform (FFT), wavelets, deep learning such as convolutional neural networks, energy models, regularized flows, vision transformers, or autoregressive image modeling.

[0113] Authentication is performed based on the generated low-level representation (step M3). For this purpose, a template of the low-level representation is provided (step M4). For example, the analyzed facial features are compared with the template. The template is provided in order to obtain a matching score. In a particular embodiment, the template space includes templates of the enrollment profile of the authenticated user on the device, for example templates generated during the enrollment process. The matching score may be a difference score between the facial feature and the corresponding feature in the template space, for example a feature vector of the authenticated user generated during the enrollment process. The matching score may be higher the closer the feature vector is to the feature vector in the template space, for example the shorter the distance or the smaller the difference.

[0114] Comparing the feature vector from the template space to the template to obtain a corresponding matching score may include classifying and evaluating the difference between the generated feature vector and the feature vector from the template space using one or more classifiers or classifiable networks. Examples of different classifiers that can be used include, but are not limited to, linear, piecewise linear, non-linear classifiers, support vector machines, and neural network classifiers. In some embodiments, the matching score is evaluated using a distance score between the feature vector and the template from the template space.

[0115] For authentication, the matching score can be compared to an unlock threshold of the device (step M5). The unlock threshold represents, for example, a minimum difference in feature vectors between the face of an authenticated user according to the template and the face of a user attempting to unlock the device. For example, the unlock threshold can be a threshold that determines whether the unlock feature vector is sufficiently close to the template vector associated with the authenticated user's face.

[0116] If the matching score is below the unlock threshold, the user's face in the image captured for unlocking does not match the authenticated user's face, in which case a signal indicating a negative authentication is provided (step M6).

[0117] However, if the matching score is above the unlock threshold, the user's face in the image captured for unlocking matches the face of an authenticated user, in which case a second authentication process is initiated.

[0118] This second authentication process includes determining some material of the user presented to the camera (step M7). For determining the user's material the method described with reference to figure 4 can be used. For example the method described with reference to figure 4 can be implemented using the system described with reference to figure 3.

[0119] Once the material has been determined, it is checked whether the determined material matches a predefined target material (step M8). For example, in combination with a face recognition process, it is determined whether the material presented to the camera is really human skin or, for example, the silicone of a spoof attachment that mimics the face of an authenticated user.

[0120] If the result of the comparison is that the determined material does not match the target material, a signal is provided indicating a negative authentication (step M9).

[0121] However, if the user also passes the second authentication step, ie the determined material matches the predefined target material, a signal indicating positive authentication is provided (step M10).

[0122] As a result, the user is authenticated as an authorized user of the registration profile on the device, and the device is unlocked, allowing the user to use the device and / or to access selected features of the device, such as applications running on the device, unlocking payment system features or making payments, accessing personal data, enlarging notifications, etc.

[0123] Alternatively, the authentication process may be implemented such that the material of the presented object is first determined and only if that material matches a predefined target material is a separate second authentication process then performed, such as biometric authentication including facial recognition or fingerprint sensing.

[0124] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0125] For the processes and methods disclosed herein, the operations performed in the processes and methods may be performed in different orders. Moreover, the schematic operations are provided by way of example only, and some of the operations may be optional, combined with fewer steps and operations, supplemented with further operations, or expanded with additional operations, without detracting from the essence of the disclosed embodiments.

[0126] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0127] A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0128] Procedures such as providing at least two images each showing a part of the object, determining a material score for each of the at least two images, evaluating the determined material scores for each of the at least two images, determining the material of the object based on the evaluation, etc. may be performed by one or more units or devices and may be performed by any number of other units or devices. These procedures may be implemented as program code means of a computer program and / or as dedicated hardware.

[0129] The computer program product may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium, provided together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0130] The units described herein may be processing units that are part of a conventional computing system. The processing unit may include a general-purpose processor, and may include a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other specialized circuit. Any memory may be a physical system memory, and may be volatile, non-volatile, or a combination of the two. The term "memory" includes any computer-readable storage medium, such as a non-volatile mass storage device. If the computing system is distributed, the processing and / or memory capabilities may also be distributed. The computing system may include multiple structures as "executable components." The term "executable components" is a well-understood structure in the computing field that may be software, hardware, or a combination thereof. For example, when implemented in software, one skilled in the art will understand that the structures of executable components include software objects, routines, methods, etc., that run on the computing system. This includes both executable components in the heap of the computing system or on a computer-readable storage medium. The structure of the executable components resides on a computer-readable medium such that, when interpreted by one or more processors, e.g., processor threads, of a computing system, it causes the computing system to perform a function. Such structure may be directly computer readable by a processor, e.g., where the executable components are binary, or may be structured to be interpretable and / or compilable, e.g., to generate binary directly interpretable by a processor, whether in a single stage or multiple stages. In other examples, the structure may be hard-coded or hard-wired logic gates that are implemented exclusively or nearly exclusively in hardware, such as in a field programmable gate array (FPGA), application specific integrated circuit (ASIC), or any other specialized circuitry.Thus, the term "executable component" is a term that describes a structure well understood by one of ordinary skill in the art of computing, whether implemented in software, hardware, or a combination thereof. Any embodiment herein is described with reference to acts performed by one or more processing units of a computing system. When such acts are implemented in software, one or more processors direct the operation of the computing system in response to executing the computer-executable instructions that make up the executable component. A computing system may also include communication channels that enable the computing system to communicate with other computing systems, for example, over a network. A "network" is defined as one or more data links that enable the transmission of electronic data between computing systems and / or modules and / or other electronic devices. When information is transferred or provided to a computing system over a network or other communications connection (e.g., either hardwired, wireless, or a combination of hardwired or wireless), the computing system properly regards the connection as a transmission medium. A transmission medium may include a network and / or data links that can be used to carry desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computing system or combination thereof. Although not all computing systems require a user interface, in some embodiments a computing system includes a user interface system for use in interfacing with a user. The user interface serves as an input or output mechanism to a user, for example via a display.

[0131] Those skilled in the art will appreciate that at least a portion of the present invention may be practiced in a networked computing environment having many types of computing system configurations, such as personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, cell phones, PDAs, pagers, routers, switches, data centers, wearables such as glasses, etc. The present invention may also be practiced in a distributed system environment where local and remote computing systems are linked through a network, for example, by either a hardwired data link, a wireless data link, or a combination of hardwired and wireless data links, and both computing systems perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[0132] Those skilled in the art will also appreciate that at least a portion of the present invention may be implemented in a cloud computing environment. The cloud computing environment may be distributed, but this is not required. If distributed, the cloud computing environment may be distributed internationally within an organization and / or may have components owned across multiple organizations. For purposes of this specification and the claims that follow, "cloud computing" is defined as a model that allows on-demand network access to a shared pool of configurable computing resources, e.g., networks, servers, storage, applications, and services. The definition of "cloud computing" is not limited to any of the many other advantages that are obtained when such a model is deployed. The computing system of the figures includes various components or functional blocks that may implement various embodiments disclosed herein, as described. The various components or functional blocks may be implemented on a local computing system or on a distributed computing system that includes elements that reside in the cloud or that implement aspects of cloud computing. The various components or functional blocks may be implemented as software, hardware, or a combination of software and hardware. The computing system depicted in the figures may include more or fewer components than shown, and some of the components may be combined depending on the circumstances.

[0133] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A system (300) for determining a material (320) of an object, comprising: an image providing unit (302) configured to provide at least two images (304, 306) each showing a part of said object; a material score determination unit (308) configured to determine a material score (310, 312) for each of said at least two images (304, 306), said material score (310, 312) being indicative of a presence of predefined material in the respective image; an evaluation unit (314) configured to evaluate the material scores (310, 312) determined for each of said at least two images (304, 306); a material determination unit (318) configured to determine the material (320) of the object based on the evaluation, and an output unit (322) configured to output said determined material (320) of said object; The system (300) has:

2. The system (300) of claim 1, wherein the at least two images (304, 306) are partial images each showing a portion of a target area included in a target area image showing the target area of ​​the object.

3. 3. The system (300) of claim 1 or 2, wherein the material score determination unit (308) comprises a data-driven model configured to determine the material score (310, 312) for each of the at least two images (304, 306), or the material score determination unit (308) comprises a mechanistic model configured to determine the material score (310, 312) for each of the at least two images (304, 306), preferably the data-driven model is a neural network trained to determine the material score (310, 312) for each of the at least two images (304, 306) using the at least two images (304, 306) as inputs.

4. 3. The system (300) of claim 1 or 2, wherein the evaluation unit (314) is further configured to evaluate the determined material scores (310, 312) for each of the at least two images (304, 306) by forming an average material score of the determined material scores (310, 312).

5. 5. The system of claim 4, wherein the evaluation unit is further configured to evaluate the determined material scores for each of the at least two images by assigning a weight to each of the material scores and by forming a weighted average material score of the determined material scores based on the weight assigned to each of the material scores.

6. 5. The system (300) of claim 4, wherein the material determination unit (318) is further configured to determine the material (320) of the object based on a comparison of the average material score or the weighted average material score with a predefined threshold.

7. 3. The system (300) of claim 1 or 2, wherein the evaluation unit (314) is further configured to compare the material scores (310, 312) determined for the at least two images (304, 306) with a reference, the reference comprising at least one reference material score determined from a reference image representing a known reference material.

8. 8. The system (300) of claim 7, wherein the evaluation unit (314) is configured to evaluate the material scores (310, 312) by element-wise comparing each of the material scores (310, 312) determined for the at least two images (304, 306) with the reference.

9. 8. The system (300) of claim 7, wherein to evaluate the material scores (310, 312), the evaluation unit (314) comprises a neural network trained to receive the material scores (310, 312) of the at least two images (304, 306) and the reference as inputs, and to output a prediction of the material (320) of the object based on the inputs.

10. 8. The system (300) of claim 7, wherein the material determination unit (318) is configured to determine the material (320) of the object based on an element-wise differencing of the material scores (310, 312) and the reference, or the material determination unit (318) is configured to determine the material (320) of the object by comparing a prediction of the material (320) of the object provided by the trained neural network with a predefined use case threshold.

11. 2. The system (300) of claim 1, wherein each of the at least two images (304, 306) is provided with location information indicating a relative location on the object, and the evaluation unit (314) has a neural network trained to receive as input the material scores (310, 312) of each image along with the location information of the image, and to output a prediction of the material (320) of the object based on the input.

12. The system (300) of claim 1, further comprising an authentication unit configured to authenticate the object using the determined material (320) of the object.

13. A method for determining the material (320) of an object, comprising the steps of: - providing at least two images (304, 306) each showing a portion of said object; determining a material score (310, 312) for each of said at least two images (304, 306), said material score (310, 312) being indicative of the presence of predefined material in the respective image; - evaluating the material scores (310, 312) determined for each of said at least two images (304, 306), determining the material (320) of the object based on the evaluation; and - outputting said determined material (320) of said object; The method includes:

14. A computer program for determining the material (320) of an object, the computer program comprising instructions for carrying out the steps of the method according to claim 13, when the computer program is executed on a computer.

15. A non-transitory computer readable data medium storing the computer program of claim 14.

Images