METHOD AND DEVICE FOR OBJECT IDENTIFICATION

Description

Technical Field

[0001] The present invention relates to target object identification, and in particular to target identification implemented based on a surface texture of a target.Background Art

[0002] Currently, the image recognition techniques are commonly used to recognize different objects or targets, for example, human faces. Because there are significant differences in the faces of different people, for example, from a planar and stereoscopic profile perspective, macroscopic or stereoscopic imaging and analysis techniques may be employed to implement facial recognition. In some applications, however, the objects needing to be distinguished look alike, for example, batches of products of the same model, such as computer keyboards. Since the macroscopic features of the objects are substantially the same, identification and verification of each object are achieved by employing analysis of the microscopic features of the object, such as texture features. However, due to the variation of ambient light and other factors, the extracted texture features are very random, which brings troubles for target identification. Document US2019 / 286902 discloses optical techniques used to authenticate a pre-characterized object according to its latent structural characteristics. For example, an image stack can be generated by aligning enrollment images acquired from an enrollment object according to different optical geometries. Enrollment basis functions can be computed from the stack that describe latent structural characteristics of the enrollment object, and enrollment magnitudes can be extracted from those basis functions. Subsequently, another image stack can be generated by aligning authentication images acquired from an authentication object according to different optical geometries. Authentication basis functions can be computed from the stack to describe latent structural characteristics of the authentication object, and authentication magnitudes can be extracted from those basis functions. A mathematical correspondence can be computed between the enrollment and authentication magnitudes, from which a determination can be made as to whether the authentication object is the enrollment object.Summary of the Invention

[0003] The present invention provides an improved target identification technology, which uses a light source with fixed irradiation intensity to irradiate a surface of a target so as to guarantee ambient light stability, and simultaneously ensures that the irradiation light source and an image sensor are always relatively fixed, such that images obtained at a plurality of photographing angles truly reflect light scattering features on a surface texture of the target, thereby truly reflecting the features of the surface texture. Therefore, the target identification and verification are implemented based on the texture features.

[0004] According to one aspect of the present invention, there is provided a target identification method, comprising: irradiating a target (300) to be identified by using a light source (200) with constant light intensity and simultaneously capturing images of the target (300) by using a camera (100) to generate a plurality of image frames (PN ) at different photographing angles (0i), wherein an angle difference (ΔΘi) between photographing angles (0i, Θi-1 ) of two consecutive image frames (Pi, Pi-1 ) is less than a predetermined value (ΘT ) , and the light source (200) and the camera (100) maintain a fixed relative position relation at the different photographing angles (Θi) ; registering the plurality of image frames (Pi) through homography transformation, wherein the registering the plurality of image frames (PN ) comprises: registering a current frame (Pi ) of image in the plurality of image frames (PN ) to an adjacent previous frame (Pi-1 ) of image that serves as a reference image; or taking one frame of image in the plurality of image frames (PN ) as a reference image and registering the remaining image frames in the plurality of image frames (PN ) to the reference image, and calculating a differential angle image (ΔPi) of a current frame (Pi ) of image relative to the reference image after registration, wherein the differential angle image (ΔPi) is a difference between the registered image frame (Pi') and the reference image and represents a scattering difference of irradiation light on a surface texture of the target (300) to be identified at different photographing angles (8i); and processing a plurality of differential angle images (ΔPi) generated based on the plurality of image frames (PN ) to generate a unique identification code (SN) indicating the target (300) to be identified, wherein processing the plurality of differential angle images (ΔPi ) to generate an identification code (SN) indicating the target (300) to be identified comprises: summing the plurality of differential angle images (ΔPi) to generate a total differential angle image (SDAI ) representing the target (300) to be identified at different photographing angles (8i); and processing the total differential angle image (SDAI ) by using a trained target identification model to output the identification code (SN), wherein the trained target identification model is obtained through training by using a sample collected from a target to be identified, in maintaining said fixed relative position relation of the light source (200) and the camera (100) at different photographing angles.

[0005] According to another aspect of the present invention, there is provided an electronic device, having an image sensor and a light source for providing auxiliary light for photographing integrated thereon, the electronic device further comprising a target identification module for executing the method according to the present invention.Brief Description of the Drawings

[0006] Fig. 1 schematically shows a target imaging and photographing process; Fig. 2 shows a flow chart of a target identification method according to an example of the present invention; Fig. 3 schematically shows a schematic diagram for generating a differential angle image; and Fig. 4 schematically shows a sample generation process during model training. Detailed Description of Embodiments

[0007] The technical solution of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention, and clearly, the described embodiments are for the purpose of illustrating the present invention only and are not intended to be limiting.

[0008] In reading texture features of a surface of a target, it is usually necessary to irradiate the surface of the target to be identified by using a light source, and at the same time, use an image sensor for photographing. In order to accurately read these texture features, in accordance with the embodiments of the present invention, the light source and the image sensor are required to maintain a fixed relative position relation, for example, both fixed in an electronic device, for example, a flashlight and an image sensor in a mobile phone are used, wherein the flashlight is kept on throughout the capturing of an image of the target, thereby ensuring that each obtained image is under an identical lighting condition.

[0009] Fig. 1 schematically shows a schematic diagram of photographing a surface of a target 300 by using a flashlight and an image sensor located in a mobile phone. In the figure, a triangle symbol 100 denotes the image sensor, and a circle symbol 200 denotes the flashlight. The flashlight 200 and the sensor 100 have fixed positions and orientations due to being fixed inside the frame of the mobile phone, for example, as shown in Fig. 1, in a particular electronic device, incident light emitted from the flashlight 200 to the surface of the target 300 maintains a fixed angle relative to reflected light reflected from the surface of the target to the image sensor 100. However, the angle between the incident light and the reflected light may vary depending on different electronic devices. Due to the use of the light source and the image sensor with the fixed relative position relation, the following situation may be avoided: the difference in the changed positions, for example, angles, etc., between the image sensor and the light source affects the scattering feature exhibition of the texture, and then accurate reflection on the texture features of the surface may be interfered with.

[0010] According to an embodiment of the present invention, a series of image frames of the target 300 at different photographing angles are obtained by moving the mobile phone in a certain rotation direction (for example, the clockwise direction as shown in the figure). In the moving process, the flashlight of the mobile phone is kept on throughout to obtain N frames of images. Herein, N is greater than a predetermined value, for example, 50. It should be noted that in the photographing process, a photographing angle difference between any two adjacent frames of images of the obtained N frames of images is less than a predetermined threshold θ T , the threshold θ T generally depends on a focal length of the image sensor and a distance between the image sensor and the flashlight, and the threshold θ T may be set to, for example, 5 degrees on the basis of statistics of commercially available mobile phones. However, the present invention is not limited thereto, and the threshold θ T may be set according to other practical requirements. As shown in Fig. 1, the photographing angle is represented by an included angle θ between a normal line of a lens surface of the image sensor and the surface of the target, wherein when the mobile phone is in a first position, the photographing angle is θ 1 , and a first image frame P 1 is obtained; and when the mobile phone is in an adjacent second position, the photographing angle is θ 2 , and a second image frame P 2 is obtained, such that a photographing angle difference between the two image frames P 2 and P 1 is θ 2 - θ 1 , which needs to be less than or equal to θ T , and N frames of images are obtained by continuously changing the photographing angle in the same direction in this way. According to this embodiment of the present invention, the N frames of images can be conveniently obtained by moving the mobile phone instead of moving the target. By processing the N frames of images obtained in the manner described above, a feature quantity is generated to uniquely indicate the texture feature of the target, for example, a feature code SN is generated. It should be noted herein that the generated N frames of images may be brightness or intensity images or grayscale images.

[0011] According to one example, the photographing angles θ 1 , θ 2 , ..., θ N of the N frames of images may be directly measured by using an angle sensor, etc. built in the mobile phone. For example, a continuous photographing function of the mobile phone may be utilized, and in the process of continuously photographing the surface of the target by the mobile phone, angle change information of three axes in the angle sensor may be simultaneously read while the image sensor performs photographing, such that a plurality of image frames and an angle θ of each frame may be obtained. Thus, the threshold θ T may be used to filter out images with angle differences between adjacent images greater than the threshold from these images so as to extract the image frames satisfying a threshold requirement. Further, it should be noted herein that the N frames of images used for subsequent processing may be either from the same continuous photographing operation or from different continuous photographing operations, as long as it is ensured that the plurality of image frames obtained in each continuous photographing operation satisfy the above threshold requirement.

[0012] A flow of the target identification method according to the present invention will be described below in conjunction with Fig. 2. In this example, keyboard identification is used as an example to illustrate the flow of the method according to the present invention.

[0013] In step 201, a keyboard is held stationary on an operation platform, the flashlight of the mobile phone is turned on, a first frame of image P 1 of the keyboard is taken in a first position L1, then the mobile phone is moved to a second position L2 in a clockwise direction, and a second frame of image P 2 is taken, such that a total of N frames of images, that is, {(P 1 , θ 1 ), (P 2 , θ 2 ), ..., (P i , θ i ), ..., (P N , θ N )} are obtained by continuously changing photographing angles in the direction, where P i represents an i th< frame of image obtained at the photographing angle θ i , and 1 < i ≤ N. When the mobile phone is moved to change the photographing angle, a photographing angle difference between any two adjacent image frames θ i+1 - θ i is less than or equal to θ T . As shown in Fig. 3, it is assumed that a total of four keyboard images (P 1 , P 2 , P 3 , and P 4 ) are taken.

[0014] In step 202, a differential angle image (DAI) between every two adjacent images in the N frames of images is calculated. To this end, according to an example of the present invention, any one of the N frames of images, for example, P 1 , is designated as a reference image, and the remaining N-1 frames of images are geometrically registered to the image P 1 . To implement image registration, a feature point-based visual feature extraction algorithm commonly used in the prior art, for example, an oriented FAST and rotated BRIEF (ORB) algorithm, a scale-invariant feature transform (SIFT) algorithm, etc., may be employed herein. For example, for the SIFT algorithm, the identical feature points on two images to be registered are determined by detecting and describing local features, for example, corner points, bright points in dark regions, etc., in the images. As an example, the feature points may be extracted around a shift key on the keyboard, comprising position coordinates [x, y] and description vectors of the feature points, and the feature points matching each other on different images are determined by matching description vectors of feature points in different positions, wherein the matching of the description vectors may be represented by calculating a relative distance between the description vectors of different feature points. It is assumed that a feature point [x, y] on the image P 1 and a feature point [x', y'] on the image P 2 match, then x ′ y ′ 1 = h 11 h 12 h 13 h 21 h 22 h 23 0 0 1 x y 1

[0015] Herein, I I = h 11 h 12 h 13 h 21 h 22 h 23 0 0 1 is a homography transformation matrix. Thus, each coefficient h in the homography matrix H may be solved by finding a plurality of pairs of [x', y'] and [x, y].

[0016] Based on the established homography transformation matrix H, the entire P 2 is registered to P 1 , thereby forming a first transformed image frame P 2 '. Similarly, a homography transformation matrix between P 1 and P 3 is established, and the entire P 3 is registered to P 1 , thereby forming a second transformed image frame P 3 '. In this way, transformed image frames (P 1 ', P 2 ', P 3 ', ..., P i ', ..., P N ') of the first to N th< frames of images (P 1 , P 2 , P 3 , ..., P i , ..., P N ) may be obtained, wherein the first image frame P 1 remains unchanged before and after transformation, that is, P 1 ' = P 1 . An image angle difference between each of the N transformed image frames and the first image frame P 1 may be according to the following formula: ΔP i = P i ' = P 1 , where 1 ≤ i ≤ N, thereby obtaining N differential angle images ΔP 1 to ΔP N , where ΔP 1 = 0. As previously mentioned, ΔP represents an intensity difference between the two image frames P i+1 and P i under the condition that the image P is represented in intensity, while ΔP represents a grayscale difference between the two image frames P i+1 and P i under the condition that the image P is represented in grayscale.

[0017] In step 203, a photographing angle difference between each pair of adjacent image frames (P i-1 , P i ) is calculated as Δθ i = θ i - θ i-1 , and accordingly, a weight γ i is assigned to the image angle difference ΔP i based on the photographing angle difference Δθ i . As an example, roughness of the surface of the target is taken into account while assigning the weight. For example, when the roughness of the surface is low, a difference can be observed with a larger angle change, and under the condition that the roughness is high, a smaller angle change can better describe the change. Therefore, the weight of each photographing angle difference may be set based on different situations.

[0018] In step 204, the plurality of differential angle images ΔP are multiplied by the respective assigned weights to calculate the total differential angle image S DAI , that is, S DAI = ∑ i = 1 N γi • ΔPi .

[0019] In step 205, the total differential angle image S DAI obtained in step 204 is processed by using a target identification model TIM to generate a classification indication output for the image, while the output value is defined as an identification code or identifier SN of the current keyboard to be identified. In an example of the present invention, the target identification model TIM may be a neural network model that implements binary classification, for example, an activation function sigmoid or tanh is used to output a classification probability value, and the probability value may be used as an identification code SN.

[0020] In the present invention, the target identification model TIM may be a neural network model which is used for image classification and is obtained through training by using a sample collected from a target to be identified, for example, a keyboard, and may also be implemented by means of other algorithms obtained based on big data processing and machine learning.

[0021] In another embodiment, during the calculation of a differential angle image (DAI) between two of the N frames of images, the differential angle image (DAI) between each pair of frames may also be calculated in the manner that the previous frame of image adjacent to the current frame of image serves as a reference image and the current frame of image is registered to the previous frame of image. For example, the image P 2 is registered to P 1 to form a registered image P 2 ', and a differential angle image between the images P 2 and P 1 is calculated as ΔP 2 = P 2 ' - P 1 . Similarly, the image P i+1 is registered to P i to form a registered image P i+1 ', and a differential angle image between the images P i+1 ' and P i is calculated as ΔP i+1 = P i+1 ' = P i . Thus, N differential angle images ΔP 1 to ΔP N may also be generated, where ΔP 1 = 0.

[0022] Fig. 4 shows a schematic diagram of training a neural network model by using samples collected from a target to be identified according to an example of the present invention. In this example, a keyboard is still used as an example to illustrate a model training process. Firstly, as shown in Fig. 2, in step 201, M keyboards are selected as sample sources, and N frames of images of each of the M keyboards are taken. Specifically, each keyboard is held stationary on the operation platform, the flashlight of the mobile phone is turned on, a first frame of image P 1< 1 of the first keyboard is taken in the first position, then the mobile phone is moved to the second position in the clockwise direction, and a second frame of image P 1< 2 is taken, such that a total of N frames of images (P 1< 1 , P 1< 2 , ..., P 1< i , ..., P 1< N ) are obtained by continuously changing the photographing angles in the direction, where P 1< i represents an i th< frame of image taken from the first keyboard, and similarly, a photographing angle difference Δθ between any two adjacent image frames is less than or equal to θ T . In this way, N frames of images (P 2< 1 , P 2< 2 , ..., P 2< i , ..., P 2< N ) of the second keyboard are obtained. It should be noted that when an image frame of the second keyboard is taken, a direction of moving the mobile phone may be different from the direction of moving the mobile phone when the first keyboard is photographed, for example, an anticlockwise direction is used; and it is also possible that the directions are identical, and the clockwise direction is still used. Meanwhile, the photographing angle of each image frame of the second keyboard does not need to be identical to that of the first keyboard, as long as the adjacent angle difference Δθ is less than or equal to θ T . In this way, an image array of the M keyboards is obtained, which is shown below: P 1< 1 , P 1< i , ..., P 1< N P 2< 1 , P 2< 2 , ..., P 2< i , ..., P 2< N ... ... P M< 1 , P M< 2 , ..., P M< i , ..., P M< N

[0023] Then, based on steps 202 to 204, the total differential angle images S 1< DAI , S 2< DAI , ..., S M< DAI are determined for the M keyboards. The total differential angle images S 1< DAI , S 2< DAI , ..., S M< DAI are used as training samples for training a neural network model to generate a keyboard classification model. In the examples of the present invention, a neural network used for learning texture features of the keyboard may be any neural network engine implementation known in the art. For example, a convolutional neural network (CNN) or a recurrent neural network (RNN) may be used to learn these texture features and implement classification by means of activation functions. How to design the structures of the neural networks CNN and RNN and each unit in each RNN may be determined by those skilled in the art depending on actual situations. For example, in a more preferred embodiment, the RNN employs a bidirectional recurrent neural network (BRNN), and each node may be implemented by using a different unit, for example, a long short-term memory (LSTM) neural network or a gated recurrent (GRU) neural network. It should be noted herein that the target identification model TIM is not limited to a neural network model, but may be any big data machine learning model expressed by other algorithms or mathematical representations, as long as such algorithms or mathematical representations are capable of learning texture scattering features. Therefore, after the target identification model TIM is trained, an architecture and parameters of the model are stored for the identification of the keyboard to be identified as described above.

[0024] The target identification method according to the present invention may be implemented by using any electronic device integrating a light source and an image sensor. In addition to a mobile phone, such an electronic device includes a tablet computer, etc., and the light source therein may always maintain an on state in a photographing process so as to provide constant light for photographing of each frame of image. Further, in addition to integrating the light source and the image sensor in the electronic device, a target identification module in the form of a software module, for example, may be installed to implement the target identification method described in the present invention.

[0025] The present invention has been shown and described in detail above by means of the accompanying drawings and preferred embodiments, however, the present invention is not limited to these disclosed embodiments, and any modifications, including combinations, substitutions, additions, deletions, etc. of features, may be made by those skilled in the art on the basis of the above detailed disclosure. For example, in the method example shown in Fig. 2, not all steps are necessary, and modifications may be made on this basis by those skilled in the art. For example, the step of assigning a weight to the image angle difference ΔP between image frames on the basis of the photographing angle difference Δθ i in step 203 may be omitted in another embodiment, such that such a solution is also considered part of the present disclosure, and the scope of protection of the present invention is therefore defined by the appended claims.

Claims

1. Target identification method, comprising: irradiating a target (300) to be identified by using a light source (200) with constant light intensity and simultaneously capturing images of the target (300) by using a camera (100) to generate a plurality of image frames (PN) at different photographing angles (θi), wherein an angle difference (Δθi) between photographing angles (θi, θi-1) of two consecutive image frames (Pi, Pi-1) is less than a predetermined value (θT), and the light source (200) and the camera (100) maintain a fixed relative position relation at the different photographing angles (θi); registering the plurality of image frames (Pi) through homography transformation, wherein the registering the plurality of image frames (PN) comprises: registering a current frame (Pi) of image in the plurality of image frames (PN) to an adjacent previous frame (Pi-1) of image that serves as a reference image; or taking one frame of image in the plurality of image frames (PN) as a reference image and registering the remaining image frames in the plurality of image frames (PN) to the reference image, and calculating a differential angle image (ΔPi) of a current frame (Pi) of image relative to the reference image after registration, wherein the differential angle image (ΔPi) is a difference between the registered image frame (Pi') and the reference image and represents a scattering difference of irradiation light on a surface texture of the target (300) to be identified at different photographing angles (θi); and processing a plurality of differential angle images (ΔPi) generated based on the plurality of image frames (PN) to generate a unique identification code (SN) indicating the target (300) to be identified, wherein processing the plurality of differential angle images (ΔPi) to generate an identification code (SN) indicating the target (300) to be identified comprises: summing the plurality of differential angle images (ΔPi) to generate a total differential angle image (SDAI) representing the target (300) to be identified at different photographing angles (θi); and processing the total differential angle image (SDAI) by using a trained target identification model to output the identification code (SN), wherein the trained target identification model is obtained through training by using a sample collected from a target to be identified, in maintaining said fixed relative position relation of the light source (200) and the camera (100) at different photographing angles.

2. Target identification method of Claim 1, wherein the light source (200) is integrated with the camera (100) on the same electronic device.

3. Method of Claim 1 or 2, further comprising: selecting the same reference object on the plurality of image frames (Pi) and extracting a feature representation of the reference object; and processing the feature representation to generate a transformation matrix for the homography transformation.

4. Method of Claim 1, wherein the summing the plurality of differential angle images (ΔPi) comprises: assigning a weight (γi) to each differential angle image (ΔPi) based on a photographing angle difference (Δθi) corresponding to the differential angle image (ΔPi); and summing the plurality of weighted differential angle images (γiΔPi) to calculate the total differential angle image (SDAI).

5. Method of Claim 1, wherein the target identification model is a trained neural network model.

6. Electronic device, having an image sensor (100) and a light source (200) for providing auxiliary light for photographing integrated thereon, the electronic device further comprising a target identification module for executing the method of any one of Claims 1 to 5.