Target identification method and equipment

Abstract

The present disclosure relates to a target identification method and electronic device, the method comprising: using a light source with a constant light intensity to illuminate a target to be identified and simultaneously using a camera to capture an image of the target, so as to generate a plurality of image frames at different shooting angles, wherein the angle difference between the shooting angles of two consecutive image frames is less than a predetermined value, wherein the light source and the camera maintain the same relative position relationship at the different shooting angles; using a homography transformation, aligning the plurality of image frames and calculating a differential angle image of a subsequent frame image after alignment relative to a previous frame image, wherein the differential angle image represents the scattering difference of the surface texture of the identification target to the irradiated light at different shooting angles; and processing the multiple differential angle images generated based on the plurality of image frames to generate a unique identification code representing the target to be identified.

Description

Technical Field

[0001] The present invention relates to target object identification, and in particular to target identification based on the surface texture of the target. Background Art

[0002] At present, image recognition technology is often used to identify different objects or targets, such as face recognition. Since there are significant differences in the faces of different people, such as from the perspective of plane or three-dimensional contour, macroscopic or stereoscopic imaging and analysis technology can be used to achieve face recognition. However, in some applications, the appearance of the objects to be distinguished is very similar, such as batches of products of the same model, such as computer keyboards. Since their macroscopic features are basically the same, the identification and verification of each object is achieved by analyzing their microscopic features such as texture features. However, due to factors such as changes in ambient light, the extracted texture features are very random, which brings troubles to the identification of the target. Summary of the invention

[0003] The present invention provides an improved target identification technology, which ensures the stability of ambient light by using a light source with a fixed illumination intensity to illuminate the target surface, and at the same time ensures that the illumination source and the image sensor are always relatively fixed, so that the images obtained at multiple shooting angles truly reflect the light scattering characteristics of the target surface texture, and realize the true reflection of the surface texture characteristics. In this way, the identification and verification of the target is realized based on the texture characteristics.

[0004] According to one aspect of the present invention, a target identification method is provided, comprising: illuminating a target to be identified using a light source with a constant light intensity and simultaneously capturing an image of the target using a camera to generate a plurality of image frames at different shooting angles, wherein the angle difference between the shooting angles of two consecutive image frames is less than a predetermined value, and wherein the light source and the camera maintain the same relative position relationship at the different shooting angles; utilizing a homography transformation to align the plurality of image frames and calculate a differential angle image of a subsequent frame image relative to a previous frame image after the alignment, wherein the differential angle image represents the difference in scattering of the surface texture of the identification target to the irradiated light at different shooting angles; and processing the multiple differential angle images generated based on the plurality of image frames to generate a unique identification code representing the target to be identified.

[0005] According to another aspect of the present invention, an electronic device is provided, on which an image sensor and a light source for providing auxiliary light for photographing are integrated. The electronic device also includes a target identification module for executing the method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 The target imaging shooting process is schematically shown;

[0007] Figure 2 A flow chart of a target identification method according to an example of the present invention is shown;

[0008] Figure 3 A schematic diagram schematically illustrates the generation of a differential angle image;

[0009] Figure 4 The sample generation process during model training is schematically shown. DETAILED DESCRIPTION

[0010] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only for the purpose of illustrating the present invention and are not restrictive.

[0011] When reading the texture features of the target surface, it is usually necessary to illuminate the target surface to be identified with a light source and use an image sensor to take a picture. In order to accurately identify these texture features, according to an embodiment of the present invention, it is required that the light source and the image sensor maintain a fixed relative position relationship, such as being fixed in an electronic device, such as using a flashlight and an image sensor in a mobile phone, wherein when capturing the target image, the flashlight is kept turned on throughout the process, thereby ensuring that each image obtained is under the same lighting conditions.

[0012] Figure 1 The schematic diagram of photographing the surface of a target 300 using a flashlight and an image sensor located in a mobile phone is shown. In the figure, the triangular symbol 100 represents the image sensor, and the circular symbol 200 represents the flashlight. Since the flashlight 200 and the sensor 100 are fixed inside the frame of the mobile phone, they have a fixed position and orientation. For example, Figure 1 As shown, in a specific electronic device, the incident light emitted by the flash 200 and projected onto the surface of the target 300 and the reflected light reflected from the target surface to the image sensor 100 maintain a relatively fixed angle. However, the angle between the incident light and the reflected light may be different between different electronic devices. The use of a light source and an image sensor with a fixed relative position relationship can avoid the influence of the scattering characteristics of the texture due to the difference in position, such as angle, between the image sensor and the light source, which may interfere with the accurate reflection of the texture surface characteristics.

[0013] According to an embodiment of the present invention, a series of image frames of the target 300 at different shooting angles are obtained by moving the mobile phone along a certain rotation direction (e.g., clockwise direction as shown in the figure). During the movement, the flash of the mobile phone is kept turned on to obtain N frames of images. Here, N is greater than a predetermined value, such as 50. It should be noted that during the shooting process, the shooting angle difference between any two adjacent frames of the N frames of images obtained is less than a predetermined threshold value θ T, threshold θ T Usually it depends on the focal length of the image sensor and the distance between the image sensor and the flash. Based on the statistics of mobile phones available on the market, the threshold θ T The threshold value θ is set to, for example, 5 degrees; however, the present invention is not limited thereto, and the threshold value θ may be set according to other practical requirements. T .like Figure 1 As shown, the angle θ between the normal line of the lens surface of the image sensor and the target surface represents the shooting angle. When the mobile phone is in the first position, the shooting angle is θ1, and the first image frame P1 is obtained; when the mobile phone is in the adjacent second position, the shooting angle is θ2, and the second image frame P2 is obtained. Therefore, the shooting angle difference between the two image frames P2 and P1 is θ2-θ1, which should be less than or equal to θ T In this way, by continuously changing the shooting angle along the same direction, N frames of images are obtained. According to an embodiment of the present invention, it is not necessary to move the target in the process of obtaining N frames of images, and it can be conveniently completed by moving the mobile phone. By processing the N frames of images obtained in the above manner, a feature quantity that uniquely characterizes the texture feature of the target is generated, such as generating a feature code SN. It should be pointed out here that the generated N frames of images can be either brightness or intensity images, or grayscale images.

[0014] According to an example, the photographing angles of N frames of images are θ1, θ2, ..., θ N The angle sensor built into the mobile phone can be used for direct measurement. For example, the continuous shooting function of the mobile phone can be used to continuously shoot the target surface with the mobile phone. While the image sensor is taking pictures, the angle change information of the three axes in the angle sensor can be read at the same time, so that multiple image frames and the angle θ in each frame can be obtained. Therefore, the threshold θ can be used to T , filter out the images whose angle differences between adjacent images are greater than the threshold, thereby extracting image frames that meet the threshold requirements. In addition, it should be pointed out here that the N frames of images used for subsequent processing can be from the same continuous shooting operation or different continuous shooting operations, as long as the multiple image frames obtained in each continuous shooting operation meet the above threshold requirements.

[0015] Combine the following Figure 2 The method flow of identifying a target of the present invention is described. In this example, keyboard identification is taken as an example to illustrate the method flow of the present invention.

[0016] In step 201, the keyboard is kept on the operating table and the flash of the mobile phone is turned on to shoot the first frame image P1 of the keyboard at the first position L1. Then, the mobile phone is moved in the clockwise direction to the second position L2 and shoots the second frame image P2. Thus, the shooting angle is continuously changed along this direction to obtain a total of N frames of images, namely {(P1, θ1), (P2, θ2), … (P i, θ i ), … (P N , θ N )}), where P i represents the i-th frame image obtained at the photographing angle θ i , where 1 < i ≤ N. When the mobile phone is moved to change the photographing angle, the difference θ i+1 - θ i between any two adjacent image frames is less than or equal to θ T . As shown in Figure 3 , it is assumed that a total of 4 keyboard images (P1, P2, P3, P4) are taken here.

[0017] In step 202, calculate the differential angle image DAI between every two adjacent images among the N-frame images. For this purpose, according to an example of the present invention, any one frame image among the N-frame images, such as P1, is specified as the reference image, and the remaining N - 1 frame images are geometrically registered to the image P1. To achieve image registration, a visual feature extraction algorithm based on feature points commonly used in the prior art can be used here, such as the ORB ((Oriented FAST and Rotated BRIEF)) algorithm, the scale-invariant feature transform (SIFT) algorithm, etc. For example, for the SIFT algorithm, it determines the same feature points on the two images to be registered by detecting and describing local features in the image, such as corner points, or bright points in dark areas, etc. As an example, feature points can be extracted around the shift key on the keyboard, including the position coordinates [x, y] of the feature points and the description vector. The description vectors of the feature points at different positions are matched to determine the mutually matching feature points on different images. Here, the matching of the description vectors can be represented by calculating the relative distance of the description vectors of different feature points. Assume that the feature point [x, y] on the image P1 matches the feature point [x′, y′] on the image P2, then

[0018]

[0019] where is the homography transformation matrix. Thus, by finding multiple pairs of [x′, y′] and [x, y], the coefficients h in the homography matrix H can be solved.

[0020] Based on the established homography transformation matrix H, the entire P2 is registered to P1, thereby forming the first transformed image frame P2′. Similarly, a homography transformation matrix between P1 and P3 is established and the entire P3 is registered to P1, thereby forming the second transformed image frame P3′. In this way, the transformed image frames (P1′, P2′, P3′ … P i , … P N ) of the 1st to the Nth frame images (P1, P2, P3 … P i ’, … PN '), where the first image frame P1 remains unchanged before and after the conversion, that is, P1'=P1. Therefore, the image angle difference ΔP between the N converted image frames and the first image frame P1 can be calculated according to the following formula i =P i '-P1, where 1≤i≤N, thus N differential angle images ΔP1~ΔP can be obtained N , where ΔP1 = 0. As mentioned above, when the image P is represented by intensity, ΔP represents two image frames P i+1 With P i The intensity difference between the two frames P is shown in grayscale. i+1 With P i The grayscale difference between .

[0021] In step 203, the calculation of each adjacent pair of image frames (P i-1 ,P i ) between the camera angles Δθ i =θ i -θ i-1 , based on the shooting angle difference Δθ i , is the image angle difference ΔP i Assign weight γ i As an example, the roughness of the target surface is also considered when assigning weights. For example, when the surface roughness is low, then a relatively large angle change can observe the difference. If the roughness is high, a relatively small angle change can better describe the change. Therefore, the weights of each shooting angle difference can be set based on different situations.

[0022] In step 204, the plurality of differential angle images ΔP are multiplied by their respective assigned weights to calculate a total differential angle image S DAI ,Right now

[0023] In step 205, the total differential angle image S obtained in step 204 is processed using the target identification model TAM. DAI , to generate a classification indication output for the image, and define the output value as the identification code or identity SN of the keyboard to be identified. In an example of the present invention, the target identification model TAM can be a neural network model that implements binary classification, for example, using an activation function sigmoid or tanh function to output a classification probability value, which can be used as the identification code SN.

[0024] In the present invention, the target identification model TAM can be a neural network model for image classification obtained by training using samples collected from the target to be identified, such as a keyboard, or can be implemented by other algorithms based on big data processing and machine learning.

[0025] In another embodiment, when calculating the differential angle image DAI between two images in N frames, the previous frame image adjacent to the next frame image can also be used as a reference image, and the next frame image is registered to the previous frame to calculate the differential angle image DAI between each pair of frames. For example, image P2 is registered to P1 to form a registered image P2', and the differential angle image ΔP2=P2'-P1 between images P2 and P1 is calculated. Similarly, image P i+1 Align to P i To form the registration image P i+1 ', and thus calculate the image P i+1 'With P i The difference angle image ΔP i+1 =P i+1 '-P i Thus, N differential angle images ΔP1 to ΔP can be generated. N , where ΔP1=0.

[0026] Figure 4 A schematic diagram of training a neural network model using samples collected from a target to be identified according to an example of the present invention is shown. In this example, a keyboard is still used as an example to illustrate the model training process. First, according to Figure 2 In step 201, M keyboards are selected as sample sources, and N frames of images of each of the M keyboards are captured. Specifically, each keyboard is kept on the operating table, the flashlight of the mobile phone is turned on, and the first frame of image P of the first keyboard is captured at the first position. 1 1, then move the phone clockwise to the second position and capture the second frame of image P 1 2, thus, the shooting angle is continuously changed along this direction to obtain a total of N frames of images (P 1 1,P 1 2,…P 1 i ...P 1 N ), where P 1 i represents the i-th frame image captured for the first keyboard. Similarly, the difference Δθ between the shooting angles of any two adjacent image frames is less than or equal to θ T In this way, N frames of images of the second keyboard are obtained (P 2 1,P 2 2,...P 2 i ,…P 2 N). It should be noted here that when shooting the second keyboard image frame, the direction of moving the mobile phone can be different from the direction of moving the mobile phone when shooting the first keyboard, for example, using a counterclockwise direction; or the directions can be the same, still using a clockwise direction. At the same time, the shooting angle of each image frame of the second keyboard does not have to be the same as the angle of the first keyboard, as long as the adjacent angle difference Δθ is less than or equal to θ T In this way, an image array of M keyboards is obtained, as shown below:

[0027] P 1 1,P 1 2,…P 1 i ...P 1 N

[0028] P 2 1,P 2 2,...P 2 i ,…P 2 N

[0029] …

[0030] P M 1,P M 2,...P M i ,…P M N

[0031] Then, according to steps 202-204, for the M keyboards, the total differential angle image S of each keyboard is determined. 1 DAI ,S 2 DAI ……S M DAI The total difference angle image S 1 DAI ,S 2 DAI ……S M DAIAs a training sample, it is used to train a neural network model to generate a keyboard classification model. In the example of the present invention, the neural network used to learn the texture features of the keyboard can be implemented by any neural network engine known in the art. For example, convolutional neural network CNN and recurrent neural network RNN ​​can be used to learn these texture features and realize classification through activation function. As for how to design the structure of neural network CNN and RNN and each unit in each RNN, the personnel in this field can determine it according to the actual situation. For example, in a more preferred embodiment, RNN adopts a bidirectional recurrent neural network (BRNN) and each node can be implemented by different units such as long short-term memory neural network (LSTM) or gated recurrent neural network (GRU). It should be pointed out here that the target identification model TAM is not limited to the neural network model, but can also be any big data machine learning model expressed by other algorithms or mathematical representations, as long as such algorithms or mathematical representations can learn texture scattering features. So after the target identification model TAM is trained, the architecture and parameters of the model are stored so that it can be used to identify the keyboard to be identified as described above.

[0032] The target identification method of the present invention can be implemented using any electronic device that integrates a light source and an image sensor. In addition to mobile phones, such electronic devices also include tablet computers, etc., in which the light source can be always kept turned on during the shooting process, thereby providing constant light for the shooting of each frame of the image. In addition, in addition to integrating the light source and the image sensor in the electronic device, a target identification module in the form of a software module, for example, can be installed to implement the target identification method described in the present invention.

[0033] The present invention is shown and described in detail above through the drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Those skilled in the art can make any modifications based on the above detailed disclosure, including the combination, replacement, addition and deletion of features. Figure 2 In the method example shown, not all steps are necessary, and those skilled in the art may make modifications based on this. i The step of assigning weights to the image angle difference ΔP between image frames may be omitted in another embodiment, and such a solution is also considered as part of the present disclosure, and the protection scope of the present invention is therefore defined by the appended claims.

Claims

1. A target identification method, comprising: Using a light source with a constant light intensity to illuminate the target to be identified and using a camera to capture an image of the target at the same time, so as to generate a plurality of image frames at different shooting angles, wherein the angle difference between the shooting angles of two consecutive image frames is less than a predetermined value, wherein the light source and the camera maintain the same relative position relationship at the different shooting angles; The plurality of image frames are registered using homography transformation, including: Registering a current frame image among the multiple image frames to an adjacent previous frame image serving as a reference image; or Taking one of the multiple image frames as a reference image, aligning the remaining image frames of the multiple image frames to the reference image; and Calculating a differential angle image of the registered current frame image relative to the reference image, wherein the differential angle image is the difference between the registered image frame and the reference image and represents the difference in scattering of the surface texture of the identification target to the irradiated light at different shooting angles, including a grayscale difference image or an intensity difference image; The multi-difference angle images generated based on the plurality of image frames are processed to generate a unique identification code representing the texture features of the target to be identified.

2. The target identification method as claimed in claim 1, wherein the light source and the camera are integrated on the same electronic device.

3. The method of claim 1 or 2, further comprising: selecting a same reference object on the plurality of image frames and extracting a feature representation of the reference object; The feature representation is processed to generate a transformation matrix for the homography transformation.

4. The method of claim 1 or 2, wherein processing the plurality of differential angle images to generate an identification code characterizing the target to be identified comprises: Summing the plurality of differential angle images to generate a total differential angle image representing the target to be identified at different photographing angles; The total differential angle image is processed using a trained object identification model to output the identification code.

5. The method of claim 4, wherein summing the plurality of differential angle images comprises: Assigning a weight to each differential angle image based on a photographing angle difference corresponding to the differential angle image; The weighted plurality of differential angle images are summed to calculate the total differential angle image.

6. The method of claim 4, wherein the target identification model is a trained neural network model.

7. An electronic device having an image sensor and a light source for providing auxiliary light for photography integrated thereon, the electronic device further comprising a target identification module for executing the method of any one of claims 1-6.

Images