Method, processing system and program for determining whether two images represent same entity

Abstract

The invention relates to a method, a processing system and a program for determining whether two images represent the same entity. A method of verifying, by a processing system, whether a user of a device corresponds to a pre-verified user. The processing system is accessible to the first image and the second image. The first image is an image of the identity document and comprises an image of a pre-verified user. The second image is an image captured by the device. The method includes comparing the first image and the second image to determine whether they are images of the same user; and, if it is determined that the first image and the second image are images of the same user: designating one of the first image and the second image as a higher quality image; and storing the specified image as a verification image of a pre-verified user.

Description

[0001] This application is a divisional application of Chinese patent application No. 201480030908.5. Technical Field

[0002] This invention relates to methods, systems, and computer programs for comparing images. Background Technology

[0003] The demand for service providers to deliver their services via devices such as PCs, tablets, and mobile phones is increasing. However, for many service providers, it is important to verify the credentials of the users they provide services to. For some online service providers, there is a need to ensure that their users are above a certain age. For example, online banking service providers need to ensure that the user's identity is reliably verified before allowing access to user-restricted banking services. For instance, verifying user identity via devices presents unique challenges compared to verifying an individual's identity through face-to-face transactions. Summary of the Invention

[0004] According to a first aspect of the invention, a method is provided for determining whether a user of a mobile device corresponds to a pre-verified user, the user being pre-verified by an identification document comprising: a photographic image of the pre-verified user visible on the identification document; and an integrated circuit element storing data representing a digital image of the pre-verified user. The method includes: enabling a chip reader connected to or integrated with the mobile device to access the integrated circuit element to retrieve data representing the digital image of the pre-verified user; enabling a camera connected to or integrated with the mobile device to capture a first image corresponding to a portion of the identification document containing the photographic image visible on the identification document; enabling a camera connected to or integrated with the mobile device to capture a second image corresponding to the user of the mobile device; and arranging the retrieved data and data representing the first and second images to be compared to determine whether the first image, the second image, and the digital image represent the same user; and if it is determined that the first image, the second image, and the digital image represent the same user, an association is formed between the pre-verified user and the mobile device.

[0005] The step of comparing the retrieved data with the data representing the first and second images can be performed based on each possible permutation of the retrieved data and the data representing the first and second images. Alternatively, a less processor-intensive process can be employed, wherein the retrieved data is compared separately with the data representing the first image and the data representing the second image.

[0006] By establishing an association between a pre-authenticated user and a mobile device, the mobile device is effectively authenticated as belonging to the pre-authenticated user. This association can be used for several purposes.

[0007] As in the first example, a third party providing access to a user-restricted service or resource via a mobile device can use this association. More specifically, the third party can use this association to determine on which mobile device access is provided to a pre-authenticated user who has requested the service / resource. In this case, once it is determined that the first image, the second image, and the digital image represent the same user, the third party can be notified which mobile device the pre-authenticated user is associated with. In this way, the third party can ensure that the services or resources they provide are supplied to the mobile device held by the pre-authenticated user.

[0008] As another example, this association can be used to identify suspicious user behavior. For instance, if the user of the first mobile device claims to be a pre-verified user; however, the pre-verified user is associated with a different mobile device and has only ever verified themselves on that device, then the user of the first mobile device can be identified as a suspicious user. In this case, a more rigorous verification check can be performed on the identification documents provided by the user of the first device.

[0009] As another example, this association can be used to verify the user of the mobile device in a later verification event for that device. More specifically, once it is determined that the first image, the second image, and the digital image represent the same user, data representing the second image and / or data representing the digital image retrieved from the chip of the identity document can be stored together with the association of the mobile device as a verification image for the pre-verified user.

[0010] In subsequent mobile device verification events, stored images of pre-verified users associated with the mobile device can be retrieved and compared with newly captured images of the mobile device user. In this way, it can be determined whether the current user of the mobile device is a pre-verified user associated with the mobile device without requiring the user to provide identification documents. Therefore, the association between the mobile device and the pre-verified user is actually an association between the mobile device and an image that has already been verified as belonging to the pre-verified user.

[0011] In a specific arrangement where the second image (i.e., an image of the pre-verified user captured via a mobile device) is stored in a storage device remotely from the mobile device as a verification image of the pre-verified user, this association between the mobile device and the second image has a specific purpose. This is because a particular user will typically have multiple mobile devices on which they verify themselves. Therefore, the remote storage device can store multiple "second" images of the pre-verified user; each of them was captured via a different mobile device. By storing the association between each of the second images and the mobile device that captured the second image, when a user attempts to verify themselves via one of these devices, the "correct" second image can be retrieved from the storage device. In other words, when a user attempts to verify themselves via a specific mobile device, the second image captured via that mobile device can be retrieved from the storage device and compared with a newly captured image by the user of that mobile device. By comparing images captured by the same device, the reliability of the image matching results can be improved.

[0012] Regardless of how this association is used, an association can be formed between a pre-authenticated user and a mobile device, for example, by storing an association between a unique device identifier and data that uniquely identifies the pre-authenticated user. As mentioned above, the data that uniquely identifies the pre-authenticated user may include data representing a digital image of the pre-authenticated user. The unique device identifier and the data that uniquely identifies the pre-authenticated user can be stored on a storage device located away from the mobile device.

[0013] By determining whether the first image, the second image, and the digital image represent the same user, it is possible to determine whether the user reaching the mobile device is a pre-verified user with a high level of trust.

[0014] More specifically, the reliability of image matching results is improved by performing three comparisons between the retrieved data and the data representing the first and second images, compared to performing two comparisons, for example, between the retrieved data and the second image.

[0015] Furthermore, the validity of the identity document can be verified by comparing data representing the first image (i.e., an image of a pre-verified user's photograph visible on the identity document) with a digital image of the pre-verified user stored on an integrated circuit element. For example, changes in the photograph visible on the identity document can be detected. Additional validity checks can also be performed.

[0016] Advantageously, for the purpose of verifying the validity of an identification document, the method may include arranging for the retrieved data and data representing a first image to be compared. For example, the identification document may further include first data, in which case the method may further include arranging for the first data to be derived from the identification document, such that the verification of the validity of the identification document is performed based on the first data.

[0017] In one instance, at least some of the first data may be stored in an integrated circuit element, and the steps of deriving the first data from the identity document may include enabling a chip reader connected to or integrated with a mobile device to access the integrated circuit element to retrieve the first data. As in a specific example, the first data stored in the integrated circuit element may include data marked by the issuing authority. In this case, the steps of verifying the validity of the identity document based on the first data may include verifying the data marked by the issuing authority.

[0018] Alternatively or additionally, at least some of the first data may be visible on the identity document, and the first image includes the portion of the identity document containing the first data. In this case, arranging the steps to derive the first data from the identity document includes any one of the following: analyzing features within the first image, or sending the first image to a remote processing system configured to analyze features within the first image, thereby deriving the first data from the identity document.

[0019] For a specific example, some initial data may be stored in an integrated circuit element, and some may be visible on an identification document. The data stored in the integrated circuit element can be encrypted using a key derived from the data visible on the identification document. In this case, the steps to verify the validity of the identification document may include deriving the visible data from the identification document and using the visible data to derive a key for decrypting the data stored in the integrated circuit element. In this way, for example, it can be verified that the visible data and / or the data on the integrated circuit element has not been tampered with.

[0020] As another example, at least some of the first data may include a unique identifier of the user associated with the identification document. In this arrangement, the unique identifier can be used to further retrieve user-related data from a storage device located remotely from the identification document. The retrieved data can be used to verify the validity of the identification document. As a specific example, the retrieved data may include a user image associated with the identification document, and the validity of the identification document can be verified by comparing the retrieved image with an image on the identification document (i.e., the "first image") and / or data representing a digital image stored on the chip, to verify that the identification document has not been tampered with. Furthermore, or alternatively, the retrieved image may be compared with a user image on the device (i.e., the "second image"). Doing so improves the reliability of the user identification verification result.

[0021] In one arrangement, the identification document may further include second data relating to a pre-verified user, and the method may further include arranging to derive the second data from the identification document. The second data may be data identifying the pre-verified user, such as name, date of birth, and / or user address. If the first image, the second image, and the digital image are determined to represent the same user, this second data may be stored along with the identifier of the pre-verified user for use in subsequent verification events. Therefore, this data does not need to be provided in subsequent verification events.

[0022] In one instance, at least some of the second data is stored in an integrated circuit element, and the steps of deriving the second data from the identification document include enabling a chip reader connected to or integrated with a mobile device to access the integrated circuit element to retrieve the second data.

[0023] Furthermore, or alternatively, at least some of the second data may be visible on the identity document, and the first image may include a portion of the identity document containing the second data. In this case, the step of arranging to derive the second data from the identity document may include arranging to extract the second data from the first image using optical character recognition.

[0024] The second set of data can be a subset of the first set of data.

[0025] In one setup, a chip reader can access integrated circuit components using a near-field communication protocol.

[0026] In some arrangements, the step of comparing data representing a pre-verified user's digital image and data representing a first image and a second image may include sending the data representing the pre-verified user's digital image and the data representing the first image and the second image to a remote processing system configured to perform the comparison. Alternatively, the data representing the pre-verified user's digital image and the data representing the first image and the second image may be compared via a mobile device's processing system.

[0027] According to a second aspect of the invention, a processing system is provided for determining whether a user of a mobile device corresponds to a pre-verified user, the user being pre-verified by an identification document comprising: a photographic image of the pre-verified user visible on the identification document; and an integrated circuit element storing data representing a digital image of the pre-verified user. The processing system is configured to: enable a chip reader connected to or integrated with the mobile device to access the integrated circuit element to retrieve data representing the digital image of the pre-verified user; enable a camera connected to or integrated with the mobile device to capture a first image corresponding to a portion of the identification document containing the photographic image visible on the identification document; enable a camera connected to or integrated with the mobile device to capture a second image corresponding to the user of the mobile device; and arrange for comparison of the retrieved data and the data representing the first and second images to determine whether the first image, the second image, and the digital image represent the same user; and if it is determined that the first image, the second image, and the digital image represent the same user, an association is formed between the pre-verified user and the mobile device.

[0028] According to a third aspect of the invention, a computer program is provided for determining whether a user of a mobile device corresponds to a pre-verified user, the user being pre-verified by an identification document, the identification document comprising: a photographic image of the pre-verified user visible on the identification document; an integrated circuit element storing data representing a digital image of the pre-verified user; and a computer program comprising, when executed on a processing system, configuring the processing system to execute instructions according to the method of the first aspect.

[0029] According to a fourth aspect of the invention, a method is provided for verifying whether a user of a device corresponds to a pre-verified user by means of a processing system, the processing system being able to access a first image and a second image, the first image being an image of an identification document including an image of the pre-verified user and data identifying the pre-verified user, the identification document being verified by a trusted authority regarding the pre-verified user to pre-verify the user, and the processing system being configured to derive data identifying the pre-verified user from the identification document, wherein the second image is an image captured by the device, the method comprising: comparing the first image and the second image to determine whether they are images of the same user; and, if it is determined that the first image and the second image are images of the same user: designating one of the first image and the second image as a higher quality image; storing the designated image as a verification image of the pre-verified user and an identifier for the pre-verified user in a storage device, the designated image being designated for subsequent verification events of the pre-verified user; and arranging the derived data to be stored together with the identifier for the pre-verified user, such that the data can be retrieved in subsequent verification events of the pre-verified user.

[0030] By storing the association between a specified image and a pre-verified user in the storage device, a higher-quality specified image can be used as a replacement for a lower-quality image to verify whether the user of the device corresponds to a pre-verified user in a subsequent verification event.

[0031] Identification documents typically include a user-associated image; however, these images are often of low quality for facial image matching. Current identification documents are usually issued in the form of cards or other photocopying media; however, the present invention is equally applicable, for example, to identification documents with an electronic identity component. An example of this electronic identity component could be a chip within the identification document that stores data such as a digital representation of the user's image. As another example, the electronic identity component could be a storage device located remotely from the identification document, and the identification document stores data such as a digital image of the user.

[0032] Typically, the second image will be of higher quality for facial image matching; it is an image captured by the device. Therefore, by storing the second image and using it, which is superior to the image of the identification document in subsequent verification events, the reliability of subsequent verification results can be improved.

[0033] As mentioned, identification documents also include data that identifies which user they are associated with. For example, this data may include information such as name, date of birth, and / or user address. By storing this data along with the identifier of the pre-verified user, it can be retrieved later in a subsequent user verification event. Therefore, this data does not need to be provided in subsequent verification events.

[0034] In some cases, data may be printed or otherwise presented on the surface of the identification document. For example, in this arrangement, optical character recognition can be used to derive the data from an image of the identification document.

[0035] Alternatively, the identification document may include a chip storing data identifying a pre-verified user. In this case, the data can be derived from the identification document using, for example, near-field communication (NFC). More specifically, the device for capturing an image of the identification document may include a NFC reader configured to retrieve data stored in the chip of the identification document when very close to it. In effect, in this arrangement, the processing system indirectly derives the data stored in the chip via the device's NFC reader.

[0036] The data stored in the chip within the ID card may include a digital image of a pre-verified user. In this case, the digital image can be retrieved and compared with a first image to verify that the first image is from a valid ID card. Through this method, the processing system can determine whether the ID card, and specifically the image of the pre-verified user on the ID card, has been tampered with. The digital image from the chip may also be stored along with other data derived from the ID card and an identifier used for pre-verification of the user.

[0037] Data derived from identification documents may additionally or alternatively include a unique identifier for the user associated with the identification document. In this arrangement, the unique identifier can be used to further retrieve user-related data from storage devices located remotely from the identification document. The processing system can use the retrieved data to verify the validity of the identification document. As a specific example, the retrieved data may include an image of the user associated with the identification document, and the validity of the identification document can be verified by comparing the retrieved image with an image on the identification document (i.e., the "first image") to confirm that the image on the identification document has not been tampered with. Furthermore, or alternatively, the retrieved image may be compared with a user image on the device (i.e., the "second image"). This improves the reliability of the user identification verification results.

[0038] In addition, or alternatively, in order to use the retrieved data to verify the validity of identity documents, the processing system may arrange to store the retrieved data so that it can be used in subsequent verification events.

[0039] Advantageously, in subsequent verification events for the device user, the method includes comparing an image subsequently captured by the device as part of the subsequent verification event with the designated image to determine whether the device user is the pre-verified user. Using a higher-quality designated image for comparison improves the reliability of the results of subsequent verification events.

[0040] If, in a subsequent verification event, it is determined that the device user is a pre-verified user, in one arrangement, the method may include: retrieving details from a first image using the identifier of the pre-verified user; and sending the details, along with the identifier, to a system remote from the processing system.

[0041] In one arrangement, the method includes encoding the second image using a one-way encoding algorithm before storing the encoded second image in the step of storing the second image.

[0042] A pre-authenticated user can be associated with a unique user identifier, and if it is determined that the first image and the second image represent the same user, the method may further include storing the association between the unique user identifier and the second image. If the pre-authenticated user subsequently wishes to verify themselves in a subsequent verification event, the unique user identifier can be used to retrieve the second image from the storage device.

[0043] Furthermore, if it is determined that the first image and the second image are images of the same user, the method may further include sending the identifier to a system remote from the processing system, thereby indicating that the device user has been verified as the user associated with the identifier. For example, the system remote from the processing system could be a system associated with a service provider, through which the device user wishes to verify themselves.

[0044] In another arrangement, the device is associated with a device unique identifier, and if it is determined that the first image and the second image are images of the same user, the method further includes storing the association between the device unique identifier and the second image.

[0045] If it is determined that the first image and the second image are images of the same user, the method may further include sending the device's unique identifier along with an indication that the user of the device has been verified to a system remote from the processing system. As mentioned above, it may be particularly useful that the system remote from the processing system is associated with a service provider, and the service provider provides services directly to the device.

[0046] In one arrangement, the step of designating one of the first and second images as the higher-quality image includes comparing the image quality of the first image with that of the second image. Alternatively, it is assumed that the second image is the higher-quality image by default, and no comparison is performed. However, performing an image quality comparison between the first and second images ensures that the higher-quality image can always be used in subsequent verification events.

[0047] According to a fifth aspect of the invention, a processing system for comparing two images to determine whether they represent the same entity is provided. The processing system is configured to: evaluate the image quality of each of a plurality of portions of a first image, thereby assigning image quality to each of the plurality of portions of the first image; for at least a portion of the first image determined to have an image quality different from other image qualities of the plurality of portions of the first image; perform an image processing procedure comprising: configuring a matching algorithm for that portion of the first image based on the image quality assigned to that portion; comparing the portion of the first image with corresponding portions of a second image of the two images using the configured matching algorithm, thereby generating an output; and using the output to determine whether the first image and the second image represent the same entity.

[0048] According to a sixth aspect of the invention, a computer program is provided for verifying whether a user of a device corresponds to a pre-verified user. The computer program includes, when executed on a processing system capable of accessing a first image and a second image, instructions for executing a method according to a third aspect, wherein: the first image is an image of an identification document including an image of the pre-verified user and data identifying the pre-verified user, the identification document having been verified relative to the pre-verified user by a trusted authority, thereby pre-verifying the user; and the processing system is arranged to derive data identifying the pre-verified user from the identification document; and the second image is an image captured by the device.

[0049] According to a seventh aspect of the invention, a method is provided for comparing two images by a processing system to determine whether they represent the same entity, the method comprising: evaluating the image quality of each of a plurality of portions of a first image in the two images, thereby assigning image quality to each of the plurality of portions of the first image; for at least a portion of the first image determined to have an image quality different from other image qualities of the plurality of portions of the first image; performing an image processing procedure comprising: configuring a matching algorithm for the portion of the first image according to the image quality assigned to the portion; and comparing the portion of the first image with a corresponding portion of a second image of the two images using the configured matching algorithm, thereby generating an output; and using the output to determine whether the first image and the second image represent the same entity.

[0050] Determining the amount of detail contained in a portion of the first image that is useful for performing an image matching process on another image can help evaluate the quality of that portion. A portion of the first image containing little detail useful for face matching can bias the overall comparison result between the first and second images. By considering the image quality of the first image piece by piece, portions of the first image containing little such detail can be identified and taken into account when configuring the matching algorithm, thus reducing the bias effect of these portions.

[0051] In one arrangement, the image quality of the portion is determined by identifying features within that portion and comparing the characteristics of those features with the characteristics of a predetermined set of training features. The training image setup may include one or more sets of images with “ideal” features and one or more sets of images with “non-ideal” features. The set of images with ideal features may consist of images suitable for image comparison. The set of images with non-ideal features may include, for example, images with poor lighting or blurry features.

[0052] Alternatively, or further, the image quality of the portion can be determined by identifying features within it, and the sharpness of the identified features can be determined. A portion with features having relatively high determined sharpness can be assigned higher image quality compared to a portion with features having relatively low determined sharpness. For example, the sharpness of a feature can be evaluated by determining the variation in pixel density over a given region. A large variation in pixel density over a relatively small region can reveal relatively sharp features, but a small variation over a larger region can reveal relatively blurry features. Portions of an image containing sharp features typically contain details that can aid in image matching.

[0053] In one arrangement, the matching algorithm for a given portion of a first image is configured to: convert the portion of the first image and the corresponding portion of the second image into a first numerical representation and a second numerical representation of the portion of the first image and the second image, respectively, the first numerical representation and the second numerical representation representing the characteristics of the features within the portion of the first image and the second image, respectively; and compare the first numerical representation and the second numerical representation to determine whether the first image and the second image represent the same entity.

[0054] In one particular arrangement, the method may include performing an image processing procedure for a plurality of said portions of a first image to produce a plurality of outputs, each corresponding to a comparison of a portion of the first image; and using the corresponding outputs of the portions of the first image to determine whether the first image and the second image represent the same entity.

[0055] Furthermore, the configured matching algorithm can be configured to combine the various outputs, the combination including: assigning relatively high weights to the output corresponding to the comparison of the portion of the first image having relatively high assigned image quality; assigning relatively low weights to the output corresponding to the comparison of the portion of the first image having relatively low assigned image quality; and combining the weighted outputs.

[0056] In this arrangement, the weighted outputs may be combined to give a value indicating the probability that the first image and the second image represent the same entity, and the step of determining whether the first image and the second image represent the same entity may include comparing the value with a predetermined threshold.

[0057] Optionally, the first image may be an image captured by a processing system or in conjunction with a processing system. Alternatively or additionally, the second image may be an image captured by a processing system or in conjunction with a processing system. The processing system is a component of a user device (such as a mobile device), for example, the first and / or second images may be captured by the image capture component of the user device. Alternatively, the processing system may be, for example, a remote server, and the first and / or second images may be captured by a device remote from the processing system, wherein the processing system operation determines whether the first and second images represent the same entity.

[0058] If it is determined that the first image and the second image do not represent the same entity, the method may include continuously capturing additional images by a processing system or in combination with a processing system, and comparing each of the additional images with the first image to determine whether they represent the same entity. This arrangement is particularly advantageous when the second image is an image captured by a mobile device, because the image capture conditions of the mobile device can be varied (brightness, for example, greatly depending on the device's position and orientation). Therefore, if the first image and the second image are erroneously determined to not represent the same entity due to poor image quality of the second image, additional images can be captured and compared with the first image. If the additional images have appropriately improved image quality, the likelihood of correcting the user's comparison results of the additional images can be increased.

[0059] In one arrangement, the method further includes comparing the overall image quality of two images and designating the image with lower overall image quality as the first image and the image with higher overall image quality as the second image. The lower-quality image may contain a greater number of parts that are of little use to the image in terms of details, and therefore, the effectiveness of the aforementioned method in improving the reliability of image matching results can be increased by designating the lower-quality image as the first image.

[0060] According to an eighth aspect of the invention, a processing system for comparing two images to determine whether they represent the same entity is provided. The processing system is configured to: evaluate the image quality of each of a plurality of portions of a first image, thereby assigning image quality to each of the plurality of portions of the first image; for at least a portion of the first image determined to have an image quality different from other image qualities of the plurality of portions of the first image; perform an image processing procedure comprising: configuring a matching algorithm for that portion of the first image based on the image quality assigned to that portion; comparing the portion of the first image with corresponding portions of a second image of the two images using the configured matching algorithm, thereby generating an output; and using the output to determine whether the first image and the second image represent the same entity.

[0061] According to a ninth aspect of the invention, a computer program is provided for comparing two images to determine whether they represent the same entity, the computer program including, when executed on a processing system capable of accessing the two images, causing the processing system to be configured to execute instructions according to the method of the seventh aspect.

[0062] According to a tenth aspect of the invention, a method is provided for verifying whether a user of a device is a user who has been pre-verified, wherein the device is capable of accessing a plurality of images and capturing at least two of the plurality of images for a user within a continuous time period, the method comprising: performing a difference detection process for the at least two images, the difference detection process comprising: comparing the at least two images to detect a difference between them; and determining whether the detected difference is sufficient to indicate that the at least two images correspond to an active user, thereby outputting an activity indicator; and selectively comparing one of the at least two images with a pre-verified image of the pre-verified user in an image matching process based on the activity indicator, so as to determine whether the compared image corresponds to the pre-verified user.

[0063] This difference detection process is performed before comparing the image captured for the user with a pre-verified image of a pre-verified user, ensuring that the image captured for the user is of an active user (i.e., the user holding the device), and not, for example, an image of a user who does not hold the device. If the difference between the two images is insufficient, the image matching process can be omitted, and the user can be excluded from verification as a pre-verified user.

[0064] If the detected differences are determined to be insufficient to indicate that the at least two images correspond to an active user, the method includes repeating the difference detection process for two different images captured for the user within the consecutive time period, thereby outputting an activity indicator for the two different images, and selectively performing an image matching process for one of the two different images based on the activity indicator. This is useful when the user of the device remains specifically to allow more time for moving the user to be detected.

[0065] Furthermore, the method may further include repeating the difference detection process on multiple different images already captured within the said consecutive time window. Thus, in effect, the user of the device is given a predetermined time during which they can "prove" they are an active user (i.e., given a predetermined time to display movement indicating "activity"). If the difference detection process does not find sufficient differences between the images captured within that time window, it can be determined that the image is not an image of an active user, and therefore, the image matching process cannot be performed on that user.

[0066] In one arrangement, the step of detecting the difference between the at least two images includes comparing pixels within a portion of a first image of the at least two images with pixels within a corresponding portion of a second image of the at least two images, said portion being identified as including facial features. In this way, changes in user expression can be detected, and these changes can be used to identify images of active users.

[0067] Alternatively or additionally, the step of detecting the difference between the at least two images may include comparing pixels within a portion of a first image of the at least two images with pixels within a corresponding portion of a second image of the at least two images, said portion being determined to include both facial features and background features. In this way, changes in user position can be detected with respect to the background, and these changes can be used to identify images of active users.

[0068] According to an eleventh aspect of the present invention, a processing system is provided for verifying whether a user of a device is a user who has been pre-verified with respect to the device. The processing system is capable of accessing multiple images, at least two of which have been captured by the device for a user within a consecutive time period. The processing system is configured to: perform a difference detection process for the at least two images, the difference detection process comprising: comparing the at least two images to detect a difference between them; and determining whether the detected difference is sufficient to indicate that the at least two images correspond to an active user, thereby outputting an activity indicator; and, based on the activity indicator, selectively comparing one of the at least two images with a pre-verified image of the pre-verified user in an image matching process to determine whether the compared image corresponds to the pre-verified user.

[0069] According to a twelfth aspect of the invention, a computer program is provided for verifying whether a user of a device is a user who has been pre-verified with respect to the device. The computer program includes, when executed on a processing system capable of accessing multiple images, at least two of the multiple images have been captured by the user within a consecutive time period, such that the processing system is configured to execute instructions according to the method of the tenth aspect.

[0070] Any of the processing systems described above may include at least one processor and at least one memory including computer program instructions, wherein the at least one memory and the computer program instructions are configured using the at least one processor to perform the methods of the first, fourth, seventh, and tenth aspects described above. While some of the above-described functions may be embodied in a server system, the processing system may be embodied in a user terminal device such as a mobile device, in which case an image may be received by the server system from a device remote from the server system. Furthermore, the invention described herein may be embodied in a non-volatile computer-readable storage medium storing the computer program instructions. Attached Figure Description

[0071] Figure 1 An exemplary first captured image is schematically shown according to an embodiment of the present invention;

[0072] Figure 2 An exemplary second captured image is schematically shown according to an embodiment of the present invention;

[0073] Figure 3 An exemplary device configured to perform a method according to an embodiment of the present invention is illustrated schematically;

[0074] Figure 4 A flowchart illustrating a method according to an embodiment of the present invention is shown schematically;

[0075] Figure 5 A close-up of an exemplary first captured image according to an embodiment of the present invention is shown schematically;

[0076] Figure 6 An exemplary device configured to perform a method according to an embodiment of the present invention is illustrated schematically; and,

[0077] Figure 7 These are exemplary processing systems and exemplary devices configured to perform methods according to embodiments of the present invention. Detailed Implementation

[0078] The traditional way to verify an individual's identity and / or credentials is to require that person to provide documents proving their identity and / or credentials. For example, an individual might be asked to provide a valid photo ID (such as a passport or driver's license) as proof of their identity. In this case, to verify the person's identity, two separate checks are typically performed. First, the validity of the photo ID is checked, and second, the person providing the photo ID is compared with the image on the photo ID to verify that the photo ID belongs to that person. These checks are usually performed by a human.

[0079] There are known techniques for verifying the validity of identification documents (such as photographic IDs) via devices. For example, by configuring the device to look for certain features in an image to a reasonable level of certainty, an image of an identification document can be verified as a valid identification document. Such features may include, for example, inclusions of certain verification digits within a machine-readable area on the identification document (which can be read by the device using optical character recognition (OCR) technology), or inclusions of an image of a human face located in the expected position of other features on the document. Other validity indicators include, for example, inclusions of watermarks or holograms, and inclusions of special fonts used.

[0080] The inventors have identified that if a secondary verification can be performed via the device—that is, a comparison is made between the device user's face and an image of a human face on the captured ID, held by the device user—then the device user can be verified in this manner. The inventors have also recognized that the device can be configured to capture both the device user's image and an image of the ID held by the device user, and to compare the device user's image with an image of a human face on the ID to determine whether they represent the same entity. Figure 1 and Figure 2 Two examples of this type of image capture, 100 and 200, are shown.

[0081] like Figure 1 As schematically shown, the first image 100 is an image of an identity document 110 associated with a person. The identity document 110 contains an image 120 of the person associated with it. Typically, the identity document 110 will include details 130 that can be used to identify the person associated with it and / or other credentials. Some identity documents 110 may also include a chip that stores additional information about the person associated with it and can be retrieved by a chip reader via a suitable protocol. The chip may store, for example, biometric information, such as a digital image of the person associated with it and / or other identifying information about that person, such as name, address, etc., as well as data related to the authority that issued the identity document 110.

[0082] For example, identification documents are typically issued by a trusted institution (such as a government). This trusted institution has pre-verified that image 120 is an image of the person associated with identification document 110, and verifies that this person is the person associated with details 130. Identification documents can be physical documents, such as ID cards, passports, or certificates, or they can be electronic documents, such as digital photographs and associated identity data.

[0083] like Figure 2 As illustrated schematically, the second image 200 is an image of the device's user 210, captured as if by a camera on the device. By comparing the first image 100 and the second image 200, it is possible to verify whether the device's user 210 is the person associated with the identity document 110 when the second image 200 is captured.

[0084] Numerous facial recognition and matching techniques exist in this field. For reliable matching to be performed, most of these techniques require high-quality facial images that contain sufficient distinguishing details to determine if they represent the same person. Factors typically affecting the reliability of facial matching between two images include image resolution (which can be effectively quantified as the number of pixels between that person's eyes) and the brightness of the individual's face. Images with too much brightness appear faded, making only strong facial features (such as eyes and nose) stand out, while images with too little brightness have only very limited contrast, and therefore the aforementioned strong facial features are less visible.

[0085] The images 120 on the identification document 110 are typically of low quality. For example, they are often small, overexposed, and have low resolution. Furthermore, many identification documents 110 have visible security features printed on the images 120, which can blur facial details, making facial matching difficult. If the identification document 110 is subsequently imaged, the quality of the face in question is reduced.

[0086] Current face matching technologies do not perform well enough to reliably compare a low-quality captured image 120 from image 100 of an identity document 110 with an image 200 captured by the device user 210. Therefore, aspects of the present invention relate to providing an image matching method that can reliably compare a low-quality image with another image to determine whether they represent the same entity.

[0087] Figure 3 A block diagram of a device 300 arranged to perform a comparison according to an exemplary embodiment of the present invention is shown. For example, device 300 may be a mobile phone, computer, or tablet computer. In this example, device 300 includes a processing system 310 and an image capture component 320, such as a camera. The image capture component 320 may be integrated with device 300, or it may be separate but communicable to device 300.

[0088] In this arrangement, device 300 is configured to capture both a first image 100 of an identification document 110 associated with a pre-verified user and a second image 200 of the user 210 of device 300. Figure 3 As schematically shown by the arrows, images 100 and 200 are provided to processing system 310. In an alternative arrangement, processing system 310 may be located remotely from device 300, in which case device 300 may transmit the first image 100 and the second image 200 to processing system 310 via, for example, a wired or wireless network. (See reference...) Figure 7 The arrangement will be discussed in more detail below.

[0089] In another arrangement, the first image 100 may be pre-processed and stored in a storage device, and the processing system 310 may be arranged to retrieve the first image 100 from the storage device.

[0090] The processing system 310 is configured to compare the first image 100 with the second image 200 to determine whether they represent the same user (i.e., to determine whether the user 210 represented in the second image 200 is a pre-verified user associated with the identity document 110). Figure 4 A flowchart illustrating the steps included in this comparison process according to an embodiment of the present invention is shown.

[0091] In step 400, the processing system 310 is configured to evaluate the image quality of each of a plurality of portions of the first image 100, thereby assigning image quality to each of the evaluated portions of the first image 100. Figure 5 A close-up of the first image 100 is shown, illustrating a captured image 120 of a pre-verified user associated with identification document 110. Two exemplary portions 500 and 510 of the image are indicated by dashed lines, with portion 500 covering the eye area and portion 510 covering the cheek area. In this example, each of these portions 500 and 510 is assigned an image quality. The assigned image quality may correspond to the suitability of the portion used for face matching, which can be influenced by a number of factors as discussed above.

[0092] Typically, an image consists of an array of pixels with varying densities. In one arrangement, such as the first section 500, the quality of a portion can be evaluated using wavelet processing to identify variations in pixel density among pixels within a given region of that section.

[0093] More specifically, for example, considering the first portion 500, the grid of the wavelet can utilize the interlacing of the pixels that make up the first portion 500 to provide a response representing the variation in pixel density over the area covered by the wavelet. By using wavelets of different sizes, features of the image can be identified and the “sharpness” of those features can be determined.

[0094] For example, a large variation in pixel density over a relatively small area will display relatively sharp features, while a small variation over a larger area will display relatively blurry features. The portion of an image containing sharp features typically contains details that can be used for face matching. For example, an image of eyes, which typically contains many details over a relatively small area, usually contains relatively large variations in pixel density over that small area. On the other hand, an image of cheeks, which typically contains fewer details useful for face matching, usually contains very few variations in pixel density across the entire area. Therefore, the portion of image 100 with sharper features (i.e., larger variations in pixel density per unit area) can be assigned higher quality compared to the portion with fewer sharp features. In this case, the first portion 500 will most likely be assigned a higher image quality than the second portion 510.

[0095] Furthermore, the sharpness of features identified in a given portion, and / or other properties of the identified features, can be compared with the properties of features within a set of training images. This set of training images may include one or more sets of images with “ideal” features and one or more sets of images with “non-ideal” features. These sets of images can be used to further evaluate the quality of portions of image 100. For example, where portions of image 100 with sharp features have been identified, the test image can be used to determine whether those sharp features are likely facial features or whether they are likely non-facial features, such as anti-counterfeiting marks printed on the face. The set of images with ideal features may consist of a set of images of human faces suitable for facial comparison. For example, the set of images with non-ideal features may include a set of images of human faces with features such as anti-counterfeiting marks imposed on them. The training image set can also be used to train processing system 310 to distinguish between the faces of individuals with bright skin captured under low lighting conditions and those with darker skin.

[0096] By using the training image, the processing system 310 can be trained to distinguish between the characteristics of ideal facial features and those of non-ideal features. In other words, when performing image comparisons, the training image can be used to identify portions of an image that are likely to be of most use. In this arrangement, portions 500, 510 that have been identified as including ideal features can be assigned high image quality for portions with fewer ideal features and / or portions with more non-ideal features.

[0097] As another example, the quality of a portion may be selectively or additionally evaluated by determining the number of pixels per unit area in a given portion. It should be recognized that other indicators of image quality exist, and these may be used as alternative or additional indicators to assign image quality to portions of the first image 100.

[0098] After the processing system 310 assigns image quality to multiple portions of the first image 100, the processing system 310 then performs an image processing procedure for at least one portion of the first image 100, which is determined to have an image quality different from that of other evaluated portions of the first image 100. The processing procedure includes steps 410 and 420. Considering the first portion 500, for example, in step 410, the processing system 310 configures a matching algorithm for the first portion 500 based on the image quality assigned to that portion 500.

[0099] In step 420, the processing system 310 uses the configured matching algorithm to compare the first part 500 with the corresponding part of the second image 200 (i.e., the part of the second image 200 that covers the same facial portion as the first part 500 of the first image 100).

[0100] In each of the first image 100 and the second image 200, a portion of the second image 200 corresponding to a first portion 500 of the first image 100 can be determined using conventional facial recognition techniques, such as those described above, to identify key features of a human face, such as the eyes, nose, and mouth. These features can be used as anchor points to fit a grid of portions to each of the images 100 and 200, such that each portion in the grid covers a predetermined portion of the face.

[0101] The output of the matching algorithm for the first portion 500 of the first image 100 can represent the probability that the first portion 500 represents a part of a face, which also exists in a corresponding part of the second image 200. The matching algorithm can be configured to compare the first portion 500 with the corresponding part of the second image 200 by comparing features (or characteristics of features) within the first portion 500 with features (or characteristics of features) within the corresponding part of the second image 200 to determine whether a match exists.

[0102] More specifically, in one arrangement, the processing system 310 can compare a first portion 500 of the first image 100 with a corresponding portion of the second image 200 by first analyzing the changes in pixel density as discussed above. The changes in pixel density across the first portion 500 can be represented numerically. This process can be repeated for the corresponding portion of the second image 200, thereby generating a numerical representation of the changes in pixel density across that portion of the second image 200. These two numerical representations can then be compared to determine whether the two portions have the same characteristic properties.

[0103] To accelerate the comparison process and reduce the computational requirements on the processing system 310, in one arrangement, discriminant analysis can be used to reduce the size of the numerical representation of that part.

[0104] The image processing procedure can be repeated for multiple parts of the first image 100. In this case, the matching algorithm is configured according to the image quality assigned to the multiple parts of the first image 100 and is used to compare those parts of the first image 100 with the corresponding parts of the second image 200, thereby generating multiple outputs.

[0105] Finally, in step 430, the processing system 310 is configured to use one or more outputs from the matching algorithm to determine whether the first image 100 includes the image of the user 210.

[0106] When assessing the importance of a very good match (or lack thereof) between a given portion of those portions and a corresponding portion of the second image 200, evaluating the image quality of portions of the first image 100 and configuring the matching algorithm based on the image quality assigned to at least one of those portions means that the image quality of different portions can be taken into account.

[0107] In instances where the first portion 500 of the first image 100 is assigned a higher image quality than the second portion 510, for example, a very good match (or lack of match) between the first portion 500 and the corresponding portion may be more meaningful than a very good match (or lack of match) between the second portion 510 and the corresponding portion of the second image 200.

[0108] This is particularly useful when the first image 100 is typically of low quality (as discussed above, typically in the case of a captured image of picture 120 on ID document 110). This is because, if the first image 100 is compared to the second image 200 as a whole, strong correspondences (or lack thereof) between portions of the first image 100 with little detail (such as cheeks) can bias the overall comparison result, leading to an incorrect determination about whether images 100 and 200 represent the same user 210. In other words, by considering the quality of the first image 100 piece by piece, the matching algorithm can be configured to account for the bias effect of portions of the image with low image quality.

[0109] In one specific arrangement, the matching algorithm may be configured to take these bias effects into account by weighting portions of the first image 100 and then combining the weighted outputs to produce values ​​representing the probability that the first image 100 and the second image 200 represent the same user 210. The weighting of the output for the portions of the first image 100 with higher image quality may be set higher than the weighting for the portions with lower image quality. The combined weighted outputs may then be compared with a threshold to determine whether the images represent the same user.

[0110] In one arrangement, the matching algorithm can be configured for portions of a first image that have been determined to have image quality above a predetermined threshold. In this case, those portions with assigned image quality above the threshold are compared with corresponding portions of the second image 200, and portions with assigned image quality below the threshold are not compared with the second image 200. This reduces the computational requirements on the processing system 310 and prevents extremely strong similarities or discrepancies between low-quality portions and corresponding portions of the second image 200 from negatively impacting the overall comparison results.

[0111] If it is determined that the first image 100 and the second image 200 are images of the same user 210, then user 210 of device 300 can be verified as a user associated with identity document 110. Before user 210 is verified as a user associated with identity document 110, as will be described in more detail below, processing system 310 may perform additional steps to verify that image 100 is an image of a valid identity document.

[0112] Although the above-described method for comparing two images to determine whether they represent the same user 210 has already been described in the context of comparing image 120 in the first image 100 of the identity document 110 with the second image 200 of the user 210 of the device 300, it should be recognized that the method can be applied to comparing any two images to determine whether they represent the same entity. As mentioned above, the method is particularly useful when the first image 100 is a low-quality image, such as any image pre-captured on a copying medium other than an image directly associated with the processing system 310, thus reducing the bias effect of the low-quality portions of the image on the overall comparison result.

[0113] Therefore, the method typically includes evaluating the overall image quality of the two images to be compared, and preparatory steps such as designating the image with lower overall image quality as the first image 100 and the image with higher overall image quality as the second image 200 before performing the image matching process as described above.

[0114] When it is known that one of the two images to be compared is image 100 of ID card 110, it can be assumed that the image of ID card 110 is a low-quality image (as mentioned above, for the purpose of face matching, image 120 on ID card 110 usually has extremely poor quality).

[0115] However, generally speaking, the methods described above are used to compare any two facial images, and image quality can be assessed regarding the usability of the images used for facial comparison. Factors affecting the usability of an individual's image used for facial comparison include: whether the person was stationary when the image was captured, whether the person was looking directly at the camera (or other image capture device) when the image was captured, whether the person had their eyes open, and whether the person was wearing items that blur their face, such as glasses. As mentioned above, other factors include image resolution and the brightness of the individual's face.

[0116] In one arrangement, the training image set described above can be used to evaluate the quality of images 100 and 200. As discussed above, the training images are used to train the processing system 310 to identify certain “ideal” features and distinguish them from other similar “non-ideal” features. For this purpose, for example, the training images can be used to train the processing system 310 to identify images where the lighting is suboptimal. Thus, the processing system 310 can determine which of the two images to be compared is the lower-quality image by determining which of those images has the most “ideal” features.

[0117] As an alternative or other preparatory step, processing system 310 can compare the image quality of two images with a threshold quality; for example, if the image quality of one of the images is below the threshold quality, an alternative image can be requested. This is particularly useful for comparing the quality of the second image 200 with the threshold quality, because a better second image can be captured for user 210 by, for example, instructing user 210 of device 300 to discover better lighting conditions.

[0118] Alternatively, if the first image 100 is determined not to be an image of user 210 represented in the second image 200, the device 300 may be configured to capture another image of user 210 and compare that image with the first image 100. Again, this can provide guidance for user 210 on how to improve the quality of the second image 200. Therefore, if the first image 100 is actually an image of user 210, but is determined not to be an image of user 210 due to poor image quality of the second image 200, capturing another second image with appropriately improved image quality can increase the likelihood of correcting the comparison result of the other image of user 210.

[0119] As described above, before user 210 is verified as the user associated with identity document 110, processing system 310 may perform the step of verifying that image 100 is an image of a valid identity document. In an exemplary embodiment, identity document 110 may include a chip storing data about the identity of the user associated with identity document 110, and this data can be used to verify that image 100 is an image of a valid identity document. Specifically, this data may include a digital image of the user associated with identity document 110 and / or other data of the user, such as the name, address, and / or date of birth of the user associated with identity document 110. Typically, this data is encrypted within the chip.

[0120] In one arrangement, device 300 may be configured to retrieve data from the chip and transmit that data to processing system 310. The processing system can then use this data to verify identity document 110. In effect, the processing system is thus configured to derive data for verifying identity document from the chip of the identity document via device 300. This data is encrypted and can be decrypted by processing system 310 before being used to verify the identity document.

[0121] In a specific example, the data stored in the chip of the ID card 110 includes a digital image of the user associated with the ID card 110. The processing system 310 can be configured to compare the digital image from the chip with a first image 100. Through this method, the processing system 310 can determine that the image 120 on the ID card 110 has been tampered with (e.g., replaced with an image of a different user). If the first image 100 and the image from the chip are determined to represent the same user, the processing system can determine that the ID card 110 in the first image 100 is valid.

[0122] Alternatively, or further, the digital image derived from the chip of ID document 110 can be compared with the second image 200 (i.e., image 200 of user 210 of device 300). This comparison can be performed in place of the aforementioned comparison between the first image 100 and the second image 200, or in addition to the comparison between the first image 100 and the second image 200. Performing this comparison in addition to the comparison between the first image 100 and the second image 200 can improve the reliability of the user verification method. In a specific arrangement, the comparison result of the second image 200 and the first image 100 can be combined with the comparison result of the second image 200 and the image derived from the chip. The combined result can be used to determine whether user 210 of device 300 is likely a user associated with ID document 110.

[0123] In one arrangement, near field communication (NFC) can be used to retrieve data stored in the chip of the identity document 110. In this arrangement, device 300 may include an NFC reader component configured to retrieve data stored in the chip when very close to it. Alternatively, device 300 may be communicatively connected to a separate NFC reader via, for example, a USB port.

[0124] In one specific embodiment, the identification document 110 may be an electronically readable travel document (eMRtd) or a similar identification document 110 conforming to the ICAO (International Civil Aviation Organization) eMRtd standard. This identification document includes a chip, which can be used in particular to verify the validity of the identification document 110. As described in detail below, there are several methods that can be used to verify the validity of the identification document 110 using this chip. However, the eMRtd or a similar identification document 110 conforming to the ICAO eMRtd standard will be described in more detail first.

[0125] EMRtd's chip stores first data in a "logical data structure". The first data may include data, such as data visible on the surface of the ID card 110. As a specific example, the first data may include data corresponding to data encoded in optical character recognition (OCR) format in the machine-readable area (MRZ) of the ID card 110.

[0126] The chip also stores a "file security object" used to verify the validity of identification documents. The file security object includes a first data hash. As will be described in more detail below, it may also include the public key of identification document 110.

[0127] The document security object is signed by the issuing authority; that is, the document security object is encrypted using the issuing authority's private key. For example, the issuing authority could be a government.

[0128] To verify the validity of the identity document 110, device 300 can be configured to read first data and a document security object from the chip of the identity document 110. This data can be read via, for example, a chip reader integrated with or connected to device 300 (such as a near-field communication reader). This data can then be sent to processing system 310. Once received, processing system 310 can be configured to identify the issuing authority of the identity document 110 and obtain their public key.

[0129] The issuing authority can be identified from data derived from the identity document 110. For example, the issuing authority can be identified through data encoded in a machine-readable area of ​​the identity document 110. In this case, for example, the processing system 310 can be configured to analyze the first image 100 and extract data for identifying the issuing authority using optical character recognition technology.

[0130] Once the issuing authority is identified, the public key can then be obtained from, for example, a public keybook held by a trusted third party.

[0131] Alternatively, the public key can be stored on the chip along with the first data and file security object, and can be read by device 300 and sent to processing system 310.

[0132] Alternatively, the processing system 310 can be pre-configured for the issuing authority using a public key.

[0133] Regardless of how the public key is retrieved, the processing system 310 can be configured to verify the validity of the identity document 110 by first decrypting the file security object using the issuing authority's public key. Thus, the processing system 310 can verify that the file security object is a valid file security object.

[0134] Once decrypted, the processing system 310 can be configured to compare the decrypted file security object with the first data hash. If a match is found, the processing system 310 can verify that the first data has not been tampered with and that the identity document 110 is valid.

[0135] In addition to the above, data stored in the chip (i.e., first data, file security objects, and any other data stored thereon) can be encoded. In one specific embodiment, the data can be encoded using a key derived from data visible on the surface of the identity document. For example, such visible data may include data encoded in OCR format in the MRZ of identity document 110.

[0136] Therefore, in order to read the first data and document security object from the chip of ID card 110, device 300 may first need to derive the visible data from the surface of the ID card. This data can be derived, for example, from the surface of ID card 110 or directly from the first image of ID card 100 using OCR technology.

[0137] If the processing system 310 can successfully decode the data stored on the chip using the data visible on the surface of the ID card 110, it can be determined that the chip of the ID card 110 has not been replaced, and / or the visible data on the surface of the ID card 110 has not been changed.

[0138] The chip may further include a secure element containing a private key for the identity document 110. In this case, the device can send a challenge to the chip, causing the chip to respond to a signature using the private key of the identity document 110.

[0139] Once a signed response is received, the processing system 310 can be configured to verify, using the public key of the identity document 110, that the response has been signed using the private key of the identity document 110. This ensures that the data stored on the chip of the identity document 110 has not been copied from another chip.

[0140] As should be understood, the processing system 310 is a component of the device 300 and will verify the validity of the identity document through the device 300 itself. The processing system 310 is located remotely from the device 300 and will perform the verification remotely, and the device 300 is configured to send data from the identity document 110 required by the processing system 310 to verify the validity of the identity document 110.

[0141] Alternatively, or in addition to the above-described validity checks, validity checks can be performed using data stored on the chip of the ID card 110 and / or data visible on the surface of the ID card 110, by using data stored in a storage device remotely to the ID card 110. For example, the stored data may include an image of the user associated with the ID card 110, and this image can be retrieved from the remote storage device, and one or both of the first image 100 and the second image 200 can be compared to verify the validity of the ID card 110 and improve the reliability of the user verification result. For example, the remote storage device could be a storage device maintained by a government entity that stores verification images of citizens.

[0142] In one specific instance, an image stored on a remote storage device can be retrieved using data derived from ID document 110, which uniquely identifies the user associated with ID document 110. In other words, the image can be retrieved using a unique user identifier derived from the ID document. As mentioned above, this unique identifier may include, for example, a unique user identification code (such as a passport number or social security number) and can be derived from the surface of ID document 110 and / or the chip in ID document 110.

[0143] In this example, the user associated with ID document 110 can first be identified by retrieving an image stored on the remote storage device by sending the derived unique user identifier to the remote storage device. The remote storage device can then use the unique user identifier to retrieve the image of the user of ID document 110, and the retrieved image can be sent to device 300 and / or processing system 310.

[0144] According to another aspect of the invention, such as Figure 6 As shown, in one embodiment, the processing system 310 is able to access the storage device 600. For example... Figure 6As shown, once it is determined that the first image 100 of the ID card 110 and the second image 200 of the user 210 of the device 300 represent the same user, the second image 200 of the user 210 can be stored in the storage device 600 as a verification image of the user associated with the ID card 110.

[0145] As described above, typically, the second image 200 captured by device 300 will have higher quality than the first image 100 of the ID card 110. In one arrangement, if a user of device 300 later wishes to verify themselves on device 300 as a pre-verified user 210 associated with ID card 110, processing system 310 can capture a subsequent image 200* of the user of device 300 and can compare the subsequent image 200* with the valid second image 200 to determine if they are images of the same user. Images can be compared according to the method described above, or alternatively, images can be compared using conventional face matching algorithms.

[0146] If it is determined that the two images 200 and 200* represent the same user, the processing system 310 can verify the user of device 300 as the pre-verified user 210 associated with identity document 110.

[0147] Therefore, in this embodiment, once user 210 has been verified using image 100 of ID document 110, user 210 does not need to provide any additional image of ID document 110 in subsequent verification events. Instead, user 210 can verify himself using the stored verification image 200.

[0148] Furthermore, for example, storing a second image 200 that is superior to the first image 100 can improve the reliability of subsequent verification events. This is because the second image 200 will generally have a higher quality than the first image 100, and therefore subsequent verification events are performed by comparing the two relatively high-quality images 200, 200* with each other, rather than comparing a very low-quality image (the first image 100) with a higher-quality image 200*. In fact, it can be said that the second image 200 is designated as the higher-quality image when stored. The valid second image 200 can be used for all subsequent verification events of user 210.

[0149] Alternatively, in one arrangement, if it is determined that the image 200* subsequently captured by the user of device 300 represents the pre-verified user 210 represented in the valid second image 200, then processing system 310 may also store the subsequently captured image 200* as a verification image of the pre-verified user 210 in storage device 600. (The storage shown is schematically illustrated.) Figure 6Two exemplary subsequently captured images 200** and 200*** in the storage device 600 are used to pre-verify the second image 200 as images of a user associated with the identity document 110.

[0150] In one arrangement, processing system 310 can compare the quality of a valid second image 200 with that of a subsequently captured image 200* and designate one as the higher quality image. Subsequently, in a subsequent verification event, processing system 310 can select the designated higher quality image from storage device 600 and use that image in the subsequent verification event, thereby further improving the reliability of the comparison results in that subsequent verification event. In one arrangement, processing system 310 can assign image quality to each stored verification image, and processing system 310 verifies the user of device 300 each time by selecting the highest quality verification image from storage device 600 to verify the user.

[0151] The stored verification images 200, 200**, and 200*** can be encoded using a one-way encoding algorithm before they are stored. In other words, images 200, 200**, and 200*** can be stored as numerical representations from which the original images cannot be derived. When a subsequently captured image 200* by a user of device 300 is compared with the encoded images 200, 200**, and 200*** stored in storage device 600, the subsequently captured image 200* is similarly encoded before being compared with the stored encoded images. As described above, when comparisons are performed between numerical representations of images (e.g., encoded images) rather than between the original images themselves, the computation required on processing system 310 is lower, and therefore, the comparison can be performed faster by encoding the images before comparing them.

[0152] In one arrangement, processing system 310 may generate or otherwise derive a unique user identifier 610 associated with identity document 110 for user 210, and may store this identifier 610 together with second image 200 and any other stored verification images 200**, 200*** of user 210. Processing system 310 may use this identifier 610 to retrieve the verification image of user 210 from storage device 600 in subsequent verification events of user 210.

[0153] In a specific instance, the unique user identifier 610 of user 210 could be a hash value derived from details about user 210. For example, these details could include user 210's first and last name, as well as user 210's date of birth. These details can be derived from identity document 110 by processing system 310 (e.g., using optical recognition or other applicable technologies).

[0154] Subsequently, in order to identify user 210 in a subsequent verification event, processing system 310 only needs to have details about the verified user 210, from which a unique user identifier 610 can be derived. Processing system 310 may selectively or additionally store certain user details that are associated with but separate from the unique user identifier.

[0155] In one instance, processing system 310 may also send a unique user identifier 610 of user 210 to a server remote from processing system 310, thereby displaying user 210 of device 300, who has been verified as the user associated with the unique user identifier 610, to the server. This is useful, for example, when a user of device 300 requests access to a service provided by a remote server via device 300, and the remote server needs to verify the identity of user 210 of device 300 before providing the service.

[0156] In one configuration, processing system 310 may store details about the identity of user 210 associated with ID document 110 along with a verification image 200 of user 210 and / or user identifier 610. In one instance, these details may be derived from image 100 of ID document 110. For example, where the ID document contains details 130 printed in text form or otherwise presented on the surface of ID document 110, these details can be extracted and stored using optical character recognition.

[0157] Alternatively, this detail may originate from data stored in the chip of the identity document 110. More specifically, as described above, the identity document 110 may include a chip storing data identifying the user associated with the identity document 110 (e.g., the user's name, address, and / or digital image). The device 300 may be configured (e.g., using NFC) to retrieve data from the chip and transmit this data to the processing system 310 for storage. In other words, the processing system 310 may store data originating from the chip of the identity document 110 via the device 300.

[0158] Alternatively, data derived from identity documents can be used to retrieve some or all of the stored details from a remote storage device. More specifically, in one arrangement, processing system 310 is configured to derive data from identity document 110, which uniquely identifies the user associated with identity document 110. In other words, processing system 310 can be configured to derive a unique user identifier from identity document 110. Processing system 310 can then send the unique user identifier to the remote storage device, and the remote storage device can use the unique user identifier to retrieve details about the user associated with identity document 110 and send the retrieved details to processing system 310.

[0159] Alternatively or concurrently, when initially verifying user 210 using the first image 100 of ID document 110, the stored details may be provided by user 210 of device 300.

[0160] Alternatively, regarding the local storage of data derived from identity documents in processing system 310, processing system 310 may arrange to store details about the identity of user 210 along with the user 210's verification image 200 and / or user 210's identifier in a storage device 600 located remotely from processing system 310. For example, the remote storage device 600 could be a service provider's storage device used by the user attempting to verify themselves.

[0161] In any subsequent verification event, when the user of device 300 is verified as a pre-verified user 210 associated with identity document 110, these details can be retrieved from storage device 600. In one instance, these details can be sent to a server remote from processing system 310, thereby identifying the verified user to the server.

[0162] As referenced above Figure 3 The described implementation can be used to verify users on multiple devices. When a captured image 200* of a user on a given device is determined to represent a pre-verified user 210 (by comparing the captured image 200* with the image 100 of the ID card 110 or by comparing the captured image 200* with stored images 200, 200**, 200***, which have been pre-verified by the processing system 310 as the pre-verified user 210), the processing system 310 can store the device's unique identifier together with the captured image 200*. As will be clearly seen in the following description, this device unique identifier can be used in multiple ways.

[0163] Device unique identifiers can be used to identify suspicious user behavior. For example, if a user of a device attempts to authenticate himself as a given pre-authenticated user, but the given pre-authenticated user has already authenticated himself on a different device, the processing system 310 can determine from the device's device unique identifier that the authentication is suspicious.

[0164] Furthermore, when a user of the device wishes to authenticate himself to a server remote from the processing system 310 via the device, a unique user identifier can be sent to the server, thereby identifying the server on the device on which the user has already authenticated.

[0165] According to another aspect of the invention, before comparing the image 200 of the user 210 of the device 300 captured by the device 300 with the image of the pre-verified user (i.e., the image of the user's identity document 110 or the image that has been pre-verified as the user's image by the processing system 310), it can be verified that the second image 200 is an image of the actual individual ("active" user) rather than, for example, a still photograph of that person.

[0166] The verification may include a series of images of a user captured by the capture device 300, and a step of comparing consecutive images to find differences between them, where the consecutive images show the active user's image. Once two consecutive images captured are sufficiently different to show that the image is the active user's image, the processing system 310 can use one of these images as a second image 200 during the comparison process described above.

[0167] Performing this check will prevent the user of device 300 from verifying himself as a different user by keeping photos of different users in front of the image capture unit 320.

[0168] In one setup, before performing a comparison between two consecutively captured images, the differences between them can be sought by analyzing the images to determine that some parts of one image represent a human face and others represent the background (as mentioned above, a training image set can be used for this analysis). In this setup, at least a portion of one image, including facial and background features, can be compared with a corresponding portion of the other image to look for facial motion with respect to the background. This comparison can be performed on a pixel-by-pixel basis.

[0169] Alternatively or additionally, at least a portion of one of the images identified as including only facial features is compared with a corresponding portion of another image. This comparison can look for differences between images representing facial movements (such as blinking).

[0170] Subsequently captured image pairs may not be compared until a pair of subsequently captured images is identified as sufficiently different to indicate that the image belongs to an active user, or until a predetermined number of subsequently captured image pairs have been compared. Alternatively, subsequently captured image pairs may not be compared until a predetermined time has elapsed.

[0171] As mentioned above, please refer to the specific details. Figure 3 In one arrangement, at least a portion of the processing system 310 may be located remotely from the device 300. Figure 7 An exemplary remote processing system 310 in this arrangement is schematically illustrated. The processing system 310 is communicatively connected to multiple devices. Figure 7 The image shows two devices (300 and 300*) of this device.

[0172] In one example, a user of the first device 300 initiates a user authentication event on the first device 300, causing the first device 300 to capture an image of the user. As discussed above, the first device 300 may also capture an image 100 of an identification document 110 associated with the user 210. In this arrangement, the first device 300 then sends the two captured images 100 and 200 to a processing system 310, and upon receipt, the processing system 310 determines whether the two images 100 and 200 are images of the same user. The processing system 310 may perform actions such as... Figure 4 The steps shown are to determine whether the image represents the same user.

[0173] The first device 300 may also optionally retrieve data from the chip of the identity document 110 and may send the retrieved data to the processing system 310.

[0174] The verification event can be associated with a verification event identifier. The identifier can be generated by the processing system 310 or the first device 300, but in any case, the verification event identifier is shared between the two components 300 and 310, thereby identifying the verification event to both components 300 and 310.

[0175] Once the processing system 310 determines whether the image represents the same user, the processing system 310 can send an indication along with a verification event identifier to the first device 300 to confirm the result of the verification event, thereby displaying to the first device 300 whether the user of the first device 300 is the user 210 represented in the identity document 110 of the verification event.

[0176] In the arrangement where the first device 300 sends data retrieved from the chip of the ID card 110 to the processing system 310, the processing system 310 can perform further verification using the data retrieved from the chip before confirming the result of the verification event to the first device 300. Specifically, the data retrieved from the chip includes an image of the user associated with the ID card 110, which the processing system can compare with one or both of the first image 100 and the second image 200 as described above. This is useful in verifying the validity of the ID card 110 and in increasing the reliability of the verification result.

[0177] In an alternative arrangement, where processing system 310 has pre-verified user 210, processing system 310 may have already stored one or more verification images 200**, 200*** of user 210 in storage device 600. In this case, the first device 300 may not send the image 100 of the identity document 110 associated with user 210 to processing system 310, but instead may send details identifying the user to processing system 310, which can use these details to identify user 210 and retrieve user 210's verification image from storage device 600.

[0178] As described above, in one arrangement, the processing system 310 may store the verification images 200**, 200*** of user 210 together with the user identifier 610 of user 210. In this arrangement, the details sent from the first device 300 to the processing system 310 may include the user identifier 610 of user 210, or alternatively, the details may include details from which the user identifier 610 can be derived. The latter is permissible, for example, where the user identifier 610 is the one referenced above. Figure 6 The hash value under discussion.

[0179] Once images 200** and 200*** of user 210 are retrieved from storage device 600, processing system 310 compares the image 200 of user 210 received from first device 300 with the pre-validated images 200** and 200*** of user 210, thereby verifying whether the user of device 300 is the pre-validated user 210.

[0180] Furthermore, the verification event can be associated with a verification event identifier, and the processing system 310 can display the verification result together with the verification event identifier to the first device 300.

[0181] As should be understood, users typically have more than one device, each with components for capturing images. Therefore, while the first device 300 is used to capture images of "active" users, the second device 300* can capture images 100 of the identification document 110. For example, it may be useful if the second device 300* can capture images of higher quality than those captured by the first device 300. In this arrangement, the aforementioned verification event identifier can be provided to devices 300, 300*, allowing the processing system 310 to identify images received from two different devices involving the same verification event.

[0182] Once two images 100 and 200 are received, the processing system can be configured, as described above, to verify the two images 100 and 200 that are associated with the same verification event identifier before comparing them, thereby determining whether they represent the same user.

[0183] As mentioned above, a given verification image 200 of a pre-verified user 210 can be stored in conjunction with details about the device used to capture an image on which the pre-verified user 210 verifies himself. Thus, where the pre-verified user 210 has multiple devices 300, 300*, and verifies himself via multiple devices 300, 300*, multiple verification images 200**, 200*** of the user 210 can be stored in a remote storage device 600.

[0184] In one arrangement, during a subsequent verification event for a pre-verified user 210, the processing system 310 can select, at least based on the device's unique identifier, a pre-validated image 200**, 200*** of the pre-verified user 210 that the pre-verified user 210 wishes to verify himself on that device (i.e., the "verifying" device). For example, the processing system 310 can select a pre-validated image 200**, 200*** of the user 210 captured by the verification device to verify the user of the verification device. This improves the reliability of face matching results because the two images to be compared may be similar since they were captured by the same device. As discussed above, when verifying a user based on a specified image quality, the processing system 310 can also determine which pre-stored verification image 200**, 200*** to use. For example, if the pre-verified image 200**, 200*** has a significantly higher quality than a verification image captured by the verification device, the processing system 310 can use a pre-validated image 200**, 200*** captured by a device different from the verification device.

[0185] The above embodiments are to be understood as exemplary examples of the present invention. Another embodiment of the invention is also contemplated. For example, regarding an aspect of the invention, in which verification images are stored in storage device 600, processing system 310 may be configured to evaluate the image quality of each verification image and may store the association between these images and their determined image quality. In a subsequent verification event, processing system 310 may select the highest quality image from storage device 600 and compare it with a user image of the device, thereby verifying the user. Alternatively, if a captured image has a higher quality than a pre-verified user's verification image to which it is compared, processing system 310 may only store the captured image. If the captured image has higher quality, processing system 310 may use the captured image to replace the pre-verified image, such that only one verification image for a given user is stored at any given time.

[0186] It should be understood that any feature described with respect to any embodiment may be used alone or in combination with other described features, and may also be combined with one or more features of any other embodiment or any use of any other embodiment. Furthermore, equivalents and modifications not described above may also be used without departing from the scope of the invention as defined by the appended claims.

[0187] Although at least some aspects of the embodiments described herein with reference to the accompanying drawings include computer processes executed in a processing system or processor, the invention also extends to computer programs, particularly computer programs suitable for implementing the invention on or in a carrier. The program may be in the form of non-volatile source code, object code, intermediate source code, and object code, such as in a partially compiled form, or in any other non-volatile form suitable for implementing the process according to the invention. The carrier may be any entity or device capable of carrying the program. For example, the carrier may include storage media such as solid-state drives (SSDs) or other semiconductor-based RAM; ROMs, such as CD ROMs or semiconductor ROMs; magnetic recording media, such as floppy disks or hard disks; typically optical storage devices; etc.

[0188] It should be understood that the processing system described herein may actually be provided by a single chip or integrated circuit or multiple chips or integrated circuits, optionally provided as a chipset, application-specific integrated circuit (ASIC), active programmable gate array (FPGA), digital signal processor (DSP), etc. The chip or multiple chips may include circuitry (and possibly firmware) embodying at least one or more of a data processor or multiple data processors, a digital signal processor or multiple digital signal processors, baseband circuitry, and radio frequency circuitry, which are configurable to operate according to exemplary embodiments. In this regard, exemplary embodiments may be implemented at least in part by computer software stored in (non-volatile) memory, and may be executed by a processor or hardware or a combination of tangible storage software and hardware (and tangible storage firmware).

Claims

1. A method for comparing two images using a processing system to determine whether they represent the same entity, the method comprising: The image quality of each of a plurality of portions of the first image is evaluated, thereby assigning image quality to each of the plurality of portions of the first image; For at least a portion of the first image that is determined to have image quality different from that of other portions of the plurality of portions of the first image: perform an image processing procedure, the image processing procedure comprising: The matching algorithm for the portion of the first image is configured according to the allocated image quality; and The configured matching algorithm is used to compare a portion of the first image with the corresponding portion of the second image of the two images, thereby generating an output; The output is used to determine whether the first image and the second image represent the same entity, wherein the image quality of the portion is determined by identifying features within the portion and comparing the characteristics of the features with the characteristics of a predetermined group of trained features.

2. The method according to claim 1, wherein, The image quality of a portion is determined by identifying features within that portion and determining the sharpness of those identified features, wherein portions of features with relatively high determined sharpness are assigned higher image quality than portions of features with relatively low determined sharpness.

3. The method according to claim 1, wherein, The matching algorithm for a given portion of the first image is configured as follows: A portion of the first image and a corresponding portion of the second image are respectively converted into a first numerical representation and a second numerical representation of the portion of the first image and the second image, respectively representing the characteristics of the features within the portion of the first image and the second image; and, The first numerical representation and the second numerical representation are compared to determine whether the first image and the second image represent the same entity.

4. The method according to claim 1, comprising: An image processing procedure is performed on multiple portions of the first image to produce multiple outputs, each of which corresponds to a comparison of a portion of the first image; and The outputs for the portion of the first image are used to determine whether the first image and the second image represent the same entity.

5. The method according to claim 4, wherein, The configured matching algorithm is configured to combine the various outputs, the combination including: The relatively high weights are assigned to the output corresponding to the comparison of the portion of the first image with the relatively high assigned image quality; The relatively low weights are assigned to the corresponding output in the comparison with the portion of the first image that has a relatively low assigned image quality; and The output is based on the aforementioned weights.

6. The method according to claim 5, wherein, The weighted outputs are combined to give a value indicating the probability that the first image and the second image represent the same entity, and the step of determining whether the first image and the second image represent the same entity includes comparing the value with a predetermined threshold.

7. The method according to claim 1, wherein, In response to the determination that the first image and the second image represent the same entity, the method further includes storing the second image or the representation of the second image together with an identifier associated with the entity in a storage device.

8. The method of claim 7, further comprising deriving the identifier from the first image or the second image.

9. The method according to claim 1, wherein, The first image is not only an image directly related to the processing system, but also an image pre-captured on the copying medium.

10. The method according to claim 1, wherein, The first image is an image captured by the processing system or in combination with the processing system.

11. The method according to claim 1, wherein, The second image is an image captured by the processing system or in combination with the processing system.

12. The method according to claim 1, wherein, If it is determined that the first image and the second image do not represent the same entity, the method includes continuously capturing additional images by the processing system or in combination with the processing system, and comparing each of the additional images with the first image to determine whether they represent the same entity.

13. The method of claim 1, further comprising comparing the overall image quality of the two images, designating the image with lower overall image quality as the first image, and designating the image with higher overall image quality as the second image.

14. A processing system for comparing two images to determine whether they represent the same entity, said processing system being configured to perform the method according to any one of claims 1 to 13.

15. A computer program for comparing two images to determine whether they represent the same entity, the computer program comprising instructions such that when the computer program is executed on a processing system capable of accessing the two images, the processing system is configured to perform the method according to any one of claims 1 to 13.

16. A method for determining whether a user of a device corresponds to a pre-verified user, the pre-verified user being pre-verified via an identification document including an integrated circuit component storing data representing a digital image of the pre-verified user, the method comprising: A camera connected to or integrated with the device captures a first image, the first image being an image of the identity document, and including first data encoded in an optical character recognition format in a machine-readable area of ​​the identity document; A reader connected to or integrated with the device retrieves second data from the integrated circuit assembly, the second data including the digital image; Arrange a portion of the first data and / or the second data to be compared with third data retrieved from a storage device remote from the device to determine whether a portion of the first data and / or the second data matches the third data; A camera connected to or integrated with the device captures a second image, which is an image of the user of the device; The second image is compared with a digital image retrieved from the integrated circuit component to determine whether they represent the same entity; and If it is determined that the second image and the digital image represent the same entity and that a portion of the first data and / or the second data matches the third data, then the user of the device is verified as the pre-verified user.

17. The method according to claim 16, wherein, The storage device is owned by a government entity.

18. The method according to claim 16, wherein, The third data is retrieved from the storage device using a unique user identifier derived from the identity document.

19. The method of claim 16, comprising: Retrieve the file security object from the integrated circuit component; The issuing authority is identified using data encoded in the machine-readable area. Obtain the public key of the issuing authority; and The file security object is decrypted using the public key of the issuing authority; and Verifying that the user of the device is the pre-verified user also relies on the decryption of the file security object.

20. The method according to claim 19, wherein, The issuing authority's public key is obtained from a public keybook maintained by a trusted third party.

21. The method of claim 16, comprising: Retrieve the file security object from the integrated circuit component; Retrieve the issuing authority's public key from the integrated circuit assembly; The file security object is decrypted using the public key of the issuing authority; and Verifying that the user of the device is the pre-verified user also relies on the decryption of the file security object.

22. The method according to claim 19 or 21, further comprising: Compare the decrypted file security object with the hash value of the second data; and When the decrypted file security object corresponds to the hash value of the second data, it is determined that the second data has not been tampered with.

23. The method according to claim 19 or 21, wherein, The second data and the file security object are encoded, and the method further includes: Derive the key from the first data encoded in the machine-readable area; Use the key to decode the second data and the file security object; and If the key successfully decodes the second data and the file security object, it is determined that the integrated circuit component has not been replaced and / or the first data encoded in the machine-readable area has not been modified.

24. The method of claim 16, wherein, The integrated circuit component includes a security element, the security element including a private key for an identification document, and the method further includes: Send a challenge to the integrated circuit component; Receive a response to the challenge signed using the private key from the integrated circuit component; and Verify whether the response was signed using a private key using the public key of the identity document.

25. A processing system for determining whether a user of a device corresponds to a pre-verified user, the pre-verified user being pre-verified via an identity document comprising an integrated circuit component storing data representing a digital image of the pre-verified user, the processing system being configured to perform the method according to any one of claims 16 to 24.

26. A computer program for determining whether a user of a device corresponds to a pre-verified user, the pre-verified user being pre-verified via an identification document comprising an integrated circuit component storing data representing a digital image of the pre-verified user, the computer program comprising instructions such that, when the computer program is executed on a processing system, the processing system is configured to perform the method according to any one of claims 16 to 24.

27. A method for determining whether a user of a mobile device corresponds to a pre-verified user, the pre-verified user being pre-verified via an identity document including an integrated circuit component storing data representing a digital image of the pre-verified user, the method comprising: Enables a reader connected to or integrated with a mobile device to access the integrated circuit component to retrieve data representing a digital image of the pre-authenticated user; A camera connected to or integrated with the mobile device captures a first image, the first image corresponding to a user of the mobile device; The retrieved data is compared with the data representing the first image to determine whether the first image and the digital image represent the same user; and If it is determined that the first image and the digital image represent the same user, then: Arrangements are made to archive at least one of the retrieved image data and the data representing the first image as verification image data; and Establish an association between the verification image data, the mobile device, and the pre-verified user; The method further includes: The camera captures a subsequent image, which corresponds to the current user of the mobile device; The data representing the subsequent image is compared with the verification image data to determine whether the subsequent image and the verification image data represent the same user; and If it is determined that the subsequent image and the verification image data represent the same user, then the current user of the mobile device is verified as the pre-verified user.

28. The method according to claim 27, wherein, The association verifies that the mobile device is the mobile device of the pre-verified user.

29. The method according to claim 27, wherein, The association is formed using the unique device identifier of the mobile device.

30. The method according to claim 29, wherein, If it is determined that the first image and the digital image represent the same user, the method further includes sending the unique device identifier along with an indication that the user of the device has been verified to a remote server.

31. The method according to claim 27, wherein, The pre-verified user is associated with a unique user identifier, and if it is determined that the first image and the digital image represent the same user, an association is formed between the verification image data, the mobile device, the pre-verified user, and the unique user identifier.

32. The method of claim 31, further comprising retrieving the verification image data using the unique user identifier.

33. The method according to claim 31, wherein, If it is determined that the first image and the digital image represent the same user, the method further includes sending the unique user identifier to a remote server to indicate that the user of the mobile device has been verified as the user associated with the identifier.

34. The method of claim 31, comprising: The camera captures a second image, which corresponds to the image of the identity document; and The unique user identifier is derived by extracting text from the second image using optical character recognition.

35. A processing system for determining whether a user of a mobile device corresponds to a pre-verified user, the pre-verified user being pre-verified via an identity document comprising an integrated circuit component storing data representing a digital image of the pre-verified user, the processing system being configured to perform the method according to any one of claims 27 to 34.

36. A computer program for determining whether a user of a mobile device corresponds to a pre-verified user, the pre-verified user being pre-verified via an identity document comprising an integrated circuit component storing data representing a digital image of the pre-verified user, the computer program comprising instructions such that when the computer program is executed on a processing system, the processing system is configured to perform the method according to any one of claims 27 to 34.

Images