Target verification method and equipment

Abstract

The present disclosure relates to a method and server for implementing target verification using an electronic device, wherein the electronic device includes a built-in image sensor and an auxiliary light source in a relatively fixed positional relationship. The method includes activating the auxiliary light source while simultaneously capturing an image of a target using the image sensor; extracting feature point information from the image; processing the image using a trained image recognition model to generate a feature vector for the image; and storing the feature point information and the feature vector as registration information for verifying the target.

Description

Technical Field

[0001] The present invention relates to target product verification, and in particular to target verification based on target surface texture. Background Art

[0002] Currently, image recognition technology is commonly used to identify and verify different objects or targets, such as human faces. Because different people's faces differ significantly, such as when viewed from a flat or three-dimensional perspective, macroscopic or stereoscopic imaging and analysis techniques can be used to achieve face recognition. However, in some applications, the objects being distinguished are very similar in appearance, such as batches of products of the same model, such as computer keyboards. Because their macroscopic features are essentially the same, the identification and verification of each object is achieved by analyzing their microscopic features, such as texture features. However, due to factors such as changes in ambient light, the extracted texture features are highly random, which poses a problem for achieving target verification. Summary of the Invention

[0003] The present invention provides an improved target verification technology that uses a light source with a fixed illumination intensity as the dominant light source to illuminate the target surface, thereby ensuring ambient light stability. This technology also ensures that the illumination source and the image sensor remain relatively fixed. This ensures that the captured image truly reflects the light scattering characteristics of the target surface texture, effectively reflecting the surface texture characteristics. This allows for target verification based on texture features.

[0004] According to one aspect of the present invention, a method for implementing target verification using an electronic device is provided, wherein the electronic device includes a built-in image sensor and an auxiliary light source having a relatively fixed positional relationship. The method includes: turning on the auxiliary light source and simultaneously capturing an image of the target using the image sensor; extracting feature point information of the image; processing the image using a trained image identification model to generate a feature vector of the image; and storing the feature point information and the feature vector as registration information for verifying the target.

[0005] According to another aspect of the present invention, a target verification method is provided, which includes: receiving an image of a target to be verified from a user electronic device, wherein the electronic device includes a built-in image sensor and an auxiliary light source having a relatively fixed positional relationship, and the image is captured using the image sensor while the auxiliary light source is turned on and illuminates the target surface; extracting feature point information of a predetermined area on the image; processing the image using a trained image identification model to generate a feature vector of the image; and matching the feature point information and the feature vector with pre-stored registration information to verify the target.

[0006] According to another aspect of the present invention, a method for implementing target verification using a user electronic device is provided, wherein the user electronic device includes a built-in image sensor and an auxiliary light source having a relatively fixed positional relationship, and the method includes: turning on the auxiliary light source to illuminate the target, and simultaneously capturing an image of the target using the image sensor; extracting feature point information of the image; processing the image using a trained image identification model to generate a feature vector of the image; transmitting the feature point information and feature vector to a remote server; and receiving a response from the remote server as to whether the target has been verified, wherein the server makes the response based on the degree of matching between the received feature point information and feature vector and a pre-stored reference identity.

[0007] According to another aspect of the present invention, an electronic device is provided, on which an image sensor and an auxiliary light source for providing auxiliary light for photography are integrated. The electronic device also includes a target identification module for use in the method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 The target imaging shooting process is schematically shown;

[0009] Figure 2 A flow chart of a target registration method according to an example of the present invention is shown;

[0010] Figure 3 A flow chart of a target registration method according to another example of the present invention is shown;

[0011] Figure 4 A flow chart of a target verification method according to an example of the present invention is shown;

[0012] Figure 5 A flow chart of a target verification method according to another example of the present invention is shown; DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only for the purpose of illustrating the present invention and are not restrictive.

[0014] To verify an object, such as a product, it is necessary to extract target features and determine unique representations of these features, thereby verifying the object based on these features. According to an embodiment of the present invention, both macroscopic and microscopic feature attributes of the target surface are comprehensively considered. In this example, macroscopic features can be characteristic regions or feature point information on the target surface. For example, feature points or regions can be position coordinates and feature description vectors within the target image; while microscopic features can be surface texture features.

[0015] According to the present invention, when reading the texture features of a target surface, a light source with constant light intensity is used to illuminate the target surface while simultaneously capturing the image using an image sensor. To accurately capture these texture features and avoid interference caused by variations in the relative position of the light source and image sensor, embodiments of the present invention require that the light source and image sensor maintain a fixed relative position. For example, the light source and image sensor are fixed within an electronic device, such as a mobile phone flash and image sensor. When capturing the target image, the flash is kept on at all times, ensuring that the auxiliary light emitted by the flash serves as the dominant light for image capture.

[0016] Figure 1 The schematic diagram shows a method of photographing the surface of a target 300 using a flashlight and an image sensor located inside a mobile phone. In the figure, the triangular symbol 100 represents the image sensor, and the circular symbol 200 represents the flashlight. Since the flashlight 200 and the sensor 100 are fixed inside the mobile phone frame, they have fixed positions and orientations. For example, Figure 1 As shown, the incident light from flash 200 directed at the surface of target 300 and the reflected light from the target surface to image sensor 100 maintain a relatively fixed angle β. However, the angle β between the incident and reflected light may vary between different electronic devices, such as different mobile phones, but it is generally no greater than 5 degrees. Therefore, using a fixed relative position between the light source and image sensor can avoid the influence of the scattering characteristics of the texture representation caused by the changing position, such as the angle, between the image sensor and the light source, which may interfere with the accurate reflection of the texture surface characteristics.

[0017] According to an embodiment of the present invention, in order to implement verification of an object, such as a product, identity authentication information of the object is first pre-registered at a verification center, such as a server. Figure 2 The flowchart of the target registration method for target verification is exemplified, and in this embodiment, a keyboard is used as an example for explanation.

[0018] like Figure 2 As shown, in step 201, the keyboard is kept stationary, the mobile phone flash 200 is turned on, and a frame image P of the keyboard is captured by the image sensor 100.

[0019] In step 202, a plurality of feature points or at least one feature area defined by a plurality of feature points are extracted from the image P. Here, a visual feature extraction algorithm commonly used in the prior art can be adopted, such as the ORB (Oriented FAST and Rotated BRIEF) algorithm, the scale-invariant feature transform (SIFT) algorithm, etc. For example, for the SIFT algorithm, it determines a plurality of feature points by detecting and describing local features in the image, such as corner points, or bright spots in dark areas. As an example, M feature points can be extracted around the right shift key on the keyboard, including the position coordinates [x, y] of the feature points and the description vector DV, which are hereinafter referred to as (L1, DV1), (L2, DV2), ... (L M ,DV M ), where L = [x, y]. The position coordinates here are coordinates established according to a predetermined coordinate system within the entire keyboard image area, for example, coordinates in a rectangular coordinate system established with the lower left corner of the keyboard in normal use as the coordinate origin. It should be noted that before extracting multiple feature points from image P, further image adjustment processing can be performed, such as cropping the image to eliminate environmental interference imaging of the keyboard image.

[0020] In step 203, the image P acquired in step 201 is processed using a target identification model TCM to generate a feature vector of the image. The feature vector may be a classification indication output of the identification model TCM. For example, the classification indication output may be a confidence probability, which may be defined as the identification code SN of the current keyboard, i.e., the feature vector. In one example of the present invention, the target identification model TCM may be a neural network model that implements binary classification, for example, using an activation function sigmoid or a tanh function to output a classification probability value, which may be used as the identification code SN. In the present invention, the target identification model TCM may be a neural network model for image classification obtained by training using samples collected from a target to be identified, such as a keyboard, or may be implemented using other algorithms based on big data processing or machine learning.

[0021] In step 204, the feature point information (L1, DV1), (L2, DV2), ... (L M ,DV M ) and the identity identifier SN calculated in step 203 are stored in the verification server as the registration feature points and registration code of the current keyboard for future use in verifying the keyboard.

[0022] In step 203 of the above embodiment, the keyboard registration code SN is calculated based on the processing of a single frame of image. In another embodiment of the present invention, to more accurately reflect the visual differences of each keyboard's texture surface at different angles, thereby further highlighting the differences between each keyboard, the identity code SN is calculated by processing multiple frames of imagery acquired at different camera angles.

[0023] The following combination Figure 3 The target registration method flow of another embodiment of the present invention is described. In this example, keyboard identification is taken as an example to illustrate the method flow of the present invention.

[0024] In step 301, the keyboard is kept stationary on the operating table, the flashlight of the mobile phone is turned on, and a series of image frames P1, P2 of the target 300 at different shooting angles are obtained by moving or rotating the mobile phone in a certain rotation direction (for example, clockwise). 2, P3…P N At the same time, the shooting angles of each frame of image are obtained, θ1, θ2, ..., θ N According to an example, the photographing angles of N frames of images are θ1, θ2, ..., θ N The angle sensor built into the mobile phone can be used for direct measurement. For example, the continuous shooting function of the mobile phone can be used to continuously shoot the target surface with the mobile phone. While the image sensor is taking pictures to obtain multiple frames of images, the angle change information of the three axes in the angle sensor can be read, thereby obtaining multiple image frames P and the angle θ in each frame. It should be noted here that the threshold θ can be used T , filter out the images whose angle difference between adjacent images is greater than the threshold θ T Thus, N frames of images P1, P that meet the threshold requirements are extracted. 2, P3…P N Threshold θ T Usually depends on the focal length of the image sensor and the distance between the image sensor and the flash. Based on the statistics of mobile phones available on the market, the threshold θ can be T The threshold value θ is set to 5 degrees, for example; however, the present invention is not limited thereto, and the threshold value θ may be set according to other practical requirements. T In addition, it should be noted that the N frames of images used for subsequent processing can be from the same continuous shooting operation or different continuous shooting operations, as long as the multiple image frames obtained in each continuous shooting operation meet the above threshold requirements.

[0025] In step 302, any frame image among the N frames of images, for example, P1, is selected and compared with Figure 2As in step 202, multiple feature points or at least one feature region defined by multiple features are extracted from the image P1. For example, M feature points can be extracted around the right shift key on the keyboard, including the position coordinates [x, y] of the feature points and the description vector DV, which are hereinafter referred to as (L1, DV1), (L2, DV2), ... (L M ,DV M ).

[0026] In step 303, the differential angle image DAI between every two adjacent images in the N frames of images is calculated. To this end, according to an example of the present invention, any frame image in the N frames of images, such as P1, is designated as a reference image, and the remaining N-1 frames of images are geometrically aligned to the image P1. To achieve image alignment, first, the same feature points on the two images to be aligned are determined. As an example, feature points can be extracted around specific keys on the keyboard, including the position coordinates [x, y] and description vectors of the feature points. Feature points that match each other on different images are determined by matching the description vectors of feature points at different positions. Here, the matching of description vectors can be represented by calculating the relative distance between the description vectors of different feature points. Assuming that the feature point [x, y] on image P1 matches the feature point [x′, y′] on image P2, then

[0027]

[0028] in This is the homography transformation matrix. Therefore, by finding multiple pairs of [x′, y′] and [x, y], we can solve the coefficients h in the homography matrix H.

[0029] Based on the established homography transformation matrix H, the entire P2 is aligned to P1, thereby forming the first transformed image frame P2'. Similarly, the homography transformation matrix between P1 and P3 is established and the entire P3 is aligned to P1, thereby forming the second transformed image frame P3'. In this way, the 1st to Nth frame images (P1, P2, P3...P i ,…P N ) of the converted image frames (P1', P2', P3'...P i ',…P N '), where the first image frame P1 remains unchanged before and after the conversion, that is, P1'=P1. The differential angle image ΔP between the N converted image frames and the first image frame P1 can be calculated according to the following formula: i =P i '-P1, where 1≤i≤N, thus N differential angle images ΔP1~ΔP can be obtained N , where ΔP1 = 0. As mentioned above, when the image P is represented by intensity, ΔP represents the intensity of two image frames P i+1 With Pi The intensity difference between the two frames P is represented by grayscale. i+1 With P i The grayscale difference between them.

[0030] In step 304, the number of adjacent pairs of image frames (P i-1 ,P i ) between the shooting angle difference Δθ i =θ i -θ i-1 , based on the shooting angle difference Δθ i , is the differential angle image ΔP i Assign weight γ i As an example, the roughness of the target surface can be taken into account when assigning weights. For example, when the surface roughness is low, a relatively large angle change can only be observed to show a difference. If the surface roughness is high, a relatively small angle change can better describe the difference. Therefore, the weights of the different shooting angles can be set based on different situations.

[0031] In step 305, the plurality of differential angle images ΔP are multiplied by their respective assigned weights to calculate a total differential angle image S. DAI ,Right now

[0032] In step 306, a trained target recognition model TCM is used to process the total difference angle image S obtained in step 305. DAI , to generate an output indicating the classification of the image, and to define this output value as the feature vector or identification code SN of the current keyboard to be identified. In one example of the present invention, the target identification model TCM can be a neural network model that implements binary classification, for example, using the sigmoid or tanh activation function to output a classification probability value, which can serve as the identification code SN.

[0033] The target identification model TCM can be implemented using a convolutional neural network (CNN) or a recurrent neural network (RNN), but the target identification model TCM is not limited to a neural network model. It can also be any big data machine learning model expressed by other algorithms or mathematical representations, as long as the algorithm or mathematical representation can learn texture scattering features.

[0034] In step 307, the feature point information determined in step 302 is stored in the verification server as the registered feature point of the current keyboard, for example, including the position coordinates and description vectors of the feature points, i.e. (L1, DV1), (L2, DV2), ... (L M ,DV MIn addition, the verification code SN calculated in step 306 is associated with the feature point information and stored in the verification server as the registration code of the current keyboard. The stored registration feature points and registration code are used to verify the keyboard.

[0035] Figure 4 FIG. 1 shows a flow chart of a method for verifying a target to be verified according to an example of the present invention. Figure 4 As shown, in step 401, the verification server receives an image P′ of a target to be verified, such as a keyboard, from a user electronic device. The electronic device includes a built-in image sensor and an auxiliary light source in a relatively fixed positional relationship, and the image P′ is captured by the image sensor while the auxiliary light source is turned on and illuminates the target surface. For example, the electronic device here can be a mobile phone or a tablet computer.

[0036] In step 402, the verification server extracts feature point information of a predetermined area on the image P'. For example, according to the requirements during registration, the M predetermined positions L1, L2, ... L of the right shift key on the image P' can be extracted. M Description vectors DV′1, DV′2, … DV′ M .

[0037] In step 403, the verification server extracts the description vectors DV′1, DV′2, ... DV′ M The description vectors (DV1, DV2, ... DV M ) for matching, for example, calculating the relative distance D between the corresponding description vectors of the same position L. If M is used to describe the relative distance (DV′) between each pair of vectors in the vector i -DV i ) are both less than the distance threshold D T , then it is considered that the feature points on the image P′ match the registered feature points of the registered image P, and the process proceeds to step 404, otherwise returns to step 401. In another embodiment, it is not necessary to require M pairs of relative distances (DV′) between description vectors. i -DV i ) are all less than the distance threshold D T For example, as long as more than 80% of the relative distances are less than the distance threshold D T , it can be considered that the feature points on the image P′ match the registered feature points of the registered image P.

[0038] In step 404, the verification server processes image P' using the pre-stored image recognition model (TCM) used during image registration to generate a feature vector for the image, such as a verification code SN'. Then, in step 405, the verification server determines whether the verification code SN' matches the registration code SN. If they are identical or within a predetermined tolerance, for example, the target keyboard is authenticated. Otherwise, the target keyboard is determined to be unregistered, authentication fails, and the server returns to step 401 to await the next authentication attempt.

[0039] In this embodiment, the verification server receives a frame of image P' from the user electronic device to verify the keyboard. In another embodiment, the received image may also be a sequence of image frames. Figure 3 Steps 303-305 generate a total differential image S′ of a sequence of image frames. DAI , and use the image discrimination model TCM trained for the differential angle image to process the total differential image S DAI To generate a verification code SN′.

[0040] In the above embodiment, the user electronic device uploads the captured image to the verification server for subsequent verification processing. However, in another embodiment of the present invention, after the user electronic device captures image P', instead of uploading image P', the user electronic device can fully utilize the powerful processing capabilities of the current user electronic device itself to process image P' to obtain feature point information (L, DV') and generate a verification code SN'. The user electronic device then sends the information (L, DV') and verification code SN' to the verification server for verification. Figure 5 A flow chart of the verification method according to this embodiment is shown. The method can be implemented by a user electronic device, such as a mobile phone, which includes a built-in image sensor and an auxiliary light source in a relatively fixed positional relationship. The mobile phone also includes a processor with computing capabilities and a memory storing instructions. The processor executes the instructions in the memory to implement the following method steps. Here, the instructions are stored in the memory in the form of a software module as a target verification module.

[0041] In step 501, the auxiliary light source in the mobile phone is turned on to illuminate the target to be verified, and the image P' of the target is captured by the image sensor in the electronic device. In step 502, the feature point information of the predetermined area on the image P' is extracted. In step 503, the image P' is processed using a trained image recognition model TCM to generate a feature vector of the image, such as a verification code SN'. In step 504, the feature point information and the verification code SN' are transmitted to a remote server and the processing result of the verification server is waited for. Here, the server makes a response decision on whether the verification is passed based on the degree of matching between the received feature point information and the verification code and the pre-stored registration feature points and registration codes. In step 505, a response is received from the remote server as to whether the target is verified, so that the user can verify the product based on the mobile phone.

[0042] The target verification method of the present invention can be implemented using any electronic device that integrates a light source and an image sensor. In addition to mobile phones, such electronic devices also include tablet computers, etc., in which the light source can be kept in an on state at all times during the shooting process, thereby providing constant light for the shooting of each frame of the image. According to the present invention, since electronic devices such as mobile phones are used to capture images during both target registration and target verification, and the flash is kept on throughout the image capture process to dominate the ambient light for image capture, it can be ensured that the images captured by both the registration end and the user end are obtained under the same or similar environmental conditions and shooting methods. Therefore, it can effectively avoid interference with the product image caused by external factors, thereby hindering the accurate verification of the product.

[0043] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Those skilled in the art can make any modifications based on the above detailed disclosure, including the combination, replacement, addition and deletion of features. For example, Figure 3 In the example of the method shown, not all steps are necessary, and those skilled in the art may make modifications based on this. For example, in step 304, based on the shooting angle difference Δθ i The step of assigning weights to the differential angle images ΔP between image frames can be omitted in another embodiment, and such a solution is also considered as part of the present disclosure. The scope of protection of the present invention is defined by the appended claims.

Claims

1. A method for implementing target verification using an electronic device, the electronic device comprising a built-in image sensor and an auxiliary light source in a relatively fixed positional relationship, the method comprising: Turning on the auxiliary light source and simultaneously capturing an image of the target using the image sensor, including a series of image frames captured at different shooting angles; Extracting feature point information of the image; Processing the image using a trained image recognition model to generate a feature vector of the image, including: registering the series of image frames using a homography transformation and calculating a differential angle image of a subsequent registered image relative to a previous image, the differential angle image representing a difference in light scattering due to the surface texture of the target at different shooting angles; and processing the differential angle image to generate the feature vector representing the target; The feature point information and the feature vector are stored as registration information for verifying the target.

2. The method of claim 1, wherein the feature point information comprises: The position information of the feature point on the target and the description vector of the feature point.

3. The method of claim 1 , further comprising: selecting a same reference object on the plurality of image frames and extracting reference feature point information of the reference object; The reference feature point information is processed to generate a transformation matrix for the homography transformation.

4. The method of claim 1 , wherein processing the differential angle image comprises: summing the differential angle images to generate a total differential angle image representing the target at different photographing angles; The total differential angle image is processed using the object authentication model to output an authentication code.

5. The method of claim 4 , wherein summing the plurality of differential angle images comprises: assigning a weight to each differential angle image based on a photographing angle difference corresponding to the differential angle image; The weighted plurality of differential angle images are summed to calculate the total differential angle image.

6. The method of claim 4 , wherein registering the plurality of image frames comprises: Registering a subsequent image frame among the plurality of image frames to an adjacent previous image frame; or Taking one image frame of the multiple image frames as a reference, the remaining image frames of the multiple image frames are aligned to the one image frame.

7. The method of claim 1 or 2, wherein the target identification model is a trained neural network model.

8. A target verification method, the method comprising: Receiving an image of an object to be verified from a user electronic device, wherein the electronic device includes a built-in image sensor and an auxiliary light source having a relatively fixed positional relationship, and the image is captured by the image sensor while the auxiliary light source is turned on and illuminates the surface of the object to be verified, wherein the received image of the object to be verified is a series of image frames captured at different shooting angles; Extracting feature point information of a predetermined area on the image; Processing the image using a trained image identification model to generate a feature vector of the image, including: registering the series of image frames using a homography transformation and calculating a differential angle image of a subsequent registered image relative to a previous image, the differential angle image representing light scattering differences of the surface texture of the target to be verified at different shooting angles; and processing the differential angle image to generate the feature vector representing the target to be verified; The feature point information and the feature vector are matched with pre-stored registration information to verify the target to be verified.

9. The method of claim 8, wherein the feature point information comprises: The feature point includes position information on the target to be verified and a description vector of the feature point, and the registration information includes registration feature point information and registration feature vector.

10. The method of claim 8 or 9, further comprising: selecting a same reference object on the series of image frames and extracting feature point information of the reference object; The feature point information is processed to generate a transformation matrix for the homography transformation.

11. The method of claim 8, wherein processing the differential angle image comprises: Summing the differential angle images to generate a total differential angle image representing the target to be verified at different photographing angles; The total differential angle image is processed using the object authentication model to output an authentication code.

12. The method of claim 11 , wherein summing the plurality of differential angle images comprises: assigning a weight to each differential angle image based on a photographing angle difference corresponding to the differential angle image; The weighted plurality of differential angle images are summed to calculate the total differential angle image.

13. The method of claim 8, wherein registering the plurality of image frames comprises: Registering a subsequent image frame among the plurality of image frames to an adjacent previous image frame; or Taking one image frame of the multiple image frames as a reference, the remaining image frames of the multiple image frames are aligned to the one image frame.

14. The method of claim 8 or 9, wherein the target identification model is a trained neural network model.

15. The method of claim 9, wherein Matching the feature point information and the feature vector with the pre-stored registration information includes: The matching degree of feature points is expressed by calculating the relative distance between the description vectors of different feature points.

16. A method for implementing target verification using a user electronic device, the user electronic device comprising a built-in image sensor and an auxiliary light source in a relatively fixed positional relationship, the method comprising: Turning on the auxiliary light source to illuminate the target, and simultaneously capturing an image of the target using the image sensor, including a series of image frames captured at different shooting angles; Extracting feature point information of the image; Processing the image using a trained image recognition model to generate a feature vector of the image, including: registering the series of image frames using a homography transformation and calculating a differential angle image of a subsequent registered image relative to a previous image, the differential angle image representing a difference in light scattering of the surface texture of the target at different shooting angles; and processing the differential angle image to generate the feature vector representing the target; Transmitting the feature point information and feature vector to a remote server; A response is received from the remote server regarding whether the target is authenticated, wherein the server makes the response based on a degree of matching between the received feature point information and feature vector and a pre-stored reference identity.

17. The method of claim 16, wherein the feature point information comprises: The position information of the feature point on the target and the description vector of the feature point.

18. The method of claim 16 or 17, further comprising: selecting a same reference object on the series of image frames and extracting feature point information of the reference object; The feature point information is processed to generate a transformation matrix for the homography transformation.

19. The method of claim 16, wherein processing the differential angle image comprises: summing the differential angle images to generate a total differential angle image representing the target at different photographing angles; The total differential angle image is processed using the identification model to output an authentication code.

20. The method of claim 19, wherein summing the plurality of differential angle images comprises: assigning a weight to each differential angle image based on a photographing angle difference corresponding to the differential angle image; The weighted plurality of differential angle images are summed to calculate the total differential angle image.

21. The method of claim 19, wherein registering the plurality of image frames comprises: Registering a subsequent image frame among the plurality of image frames to an adjacent previous image frame; or Taking one image frame of the multiple image frames as a reference, the remaining image frames of the multiple image frames are aligned to the one image frame.

22. The method of claim 16 or 17, wherein the target identification model is a trained neural network model.

23. An electronic device having an image sensor and an auxiliary light source for providing auxiliary light for photography integrated thereon, the electronic device further comprising a target identification module for executing the method according to any one of claims 16 to 22.

Images