A face image traceability method based on a detection algorithm

Abstract

The present invention discloses a face picture traceability method based on a detection algorithm. For any face photo, the face detection algorithm can locate the size and coordinates of the face area, and at the same time, it can also locate the coordinates of the eyes, nose, and mouth in the face picture. For any different photos, their coordinate values are different. After passing through the same face detection algorithm for each photo, the calculated coordinates of the eyes, nose, and mouth are unique. After further processing the coordinate values, a unique HASH value label corresponding to the face picture is generated. Using this algorithm, a unique label can be assigned to each face photo. When a photo is illegally leaked, by using the detection algorithm to calculate the HASH value of the photo, and then in the system database, finding the time when the photo was generated and the camera number, the traceability of the face picture can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a method for tracing the source of face images based on a detection algorithm. Background Art

[0002] With the increasing maturity of face recognition technology, face recognition technology is required in many scenarios in daily life, and face recognition technology is inseparable from the acquisition of face images. How to ensure that the acquired face images are not illegally used is a very crucial issue. Currently, image encryption methods are generally used to avoid the illegal use of face images, but this often requires a lot of computing resources and there is also a risk of being decrypted. The leakage of a large number of face photos indicates that there are many security vulnerabilities in network data storage.

[0003] Therefore, how to avoid the illegal use of the acquired face images and improve the security of the use of face images is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a method for tracing the source of face images based on a detection algorithm. For any face photo, the size and coordinates of the face area can be located through the face detection algorithm, and at the same time, the coordinates of the eyes, nose, and mouth in the face image can also be located. For any different photos, the coordinate values are different. After each photo passes through the same face detection algorithm, the calculated coordinates of the eyes, nose, and mouth are unique. After further processing the coordinate values, a unique HASH value label corresponding to the face image is generated. Using this algorithm, a unique label can be assigned to each face photo. When a photo is illegally leaked, the HASH value of the photo is calculated through the detection algorithm, and then the generation time and camera number of the photo are found in the system database, and the IP address of the illegal leakage of the photo can be judged, and the camera that collected the image can be traced, so that engineers can repair the possible leakage paths and avoid the recurrence of leakage incidents.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for tracing the source of face images based on a detection algorithm includes the following specific steps:

[0007] Step 1: Collect face images;

[0008] Step 2: Use a face detection algorithm to extract the key point coordinates and the corner coordinates of the face frame of the face image;

[0009] Step 3: Perform coordinate conversion on the key point coordinates to obtain the relative coordinates of the face key points;

[0010] Step 4: Perform a transformation hashing operation on the relative coordinates of the facial key points to obtain fingerprint data;

[0011] Step 5: Save the fingerprint data and its corresponding traceability information to the facial database, and save the corresponding facial photo to an image file;

[0012] Step 6: Use the loop in Steps 2 - 4 for the face image to be detected to obtain a hash string;

[0013] Step 7: Search in the facial database to see if there is fingerprint data that meets the consistent condition when compared with the hash string. If it exists, return the searched fingerprint data and its corresponding traceability information; otherwise, return a non - system image prompt message.

[0014] Preferably, in Step 2, the collected key point coordinates and corner coordinates are uniformly represented in the (x, y) coordinate system, and the upper - left corner point coordinates in the corner coordinates are defined as the origin coordinates; the key point coordinates include five - point coordinates.

[0015] Preferably, in Step 3, the key point coordinates are subtracted from the origin coordinates to generate the relative coordinates of the key points.

[0016] Preferably, in Step 4, when performing a transformation hashing operation on the relative coordinates of the key points, the numpy and hex calculation modules in the standard scientific computing library are used to generate a 32 - bit string. The specific process is as follows:

[0017] Step 41: Perform a one - time dimensionality reduction on the relative coordinates; select the relative coordinates corresponding to one of the key point coordinates as the origin, and subtract the relative coordinates corresponding to the origin from the relative coordinates of the remaining key point coordinates to convert the five - point coordinates into four - point coordinates;

[0018] Step 42: Perform a secondary dimensionality reduction; add the coordinate values in the four - point coordinates in pairs to obtain two sets of intermediate values;

[0019] Step 43: Perform integer operations on the two sets of intermediate values to obtain four sets of hash values;

[0020] Step 44: Perform radix conversion and combination on the four sets of hash values to obtain a 32 - bit string.

[0021] Preferably, the consistent condition includes the same condition and the fuzzy condition; the same condition means that all characters are exactly the same as the hash string; the fuzzy condition means that some characters are the same as the hash string.

[0022] Preferably, the traceability information includes the device IP, device address, device number, picture generation time, database operation records, etc. of the device that captures the face picture; the database operation records include the operator's account, login time, operation type, etc. The camera that leaked the picture can be traced through the device IP, device geology, and device number. Engineering technicians can repair the possible paths of external leakage of relevant devices to avoid recurrence of external leakage incidents; further investigation can be carried out by locking the operator whose operation type is copy to complete the traceability and accountability of the illegal use of face pictures.

[0023] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method for tracing face pictures based on a detection algorithm. Aiming at the risk that the captured face pictures may be illegally used, the method traces the face pictures, searches for the acquisition devices and storage databases of the leaked face pictures, so as to ensure the source security of the face pictures, ensure that all captured face pictures have traceability mark stamps, and make the acquisition and use of face pictures traceable. Therefore, once a face picture is illegally used, effective liability investigation can be carried out through the traceability path, providing technical guarantee for the legal use of face pictures. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0025] Figure 1 The attached drawing is a flowchart of the method for tracing face pictures based on a detection algorithm provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0027] The embodiments of the present invention disclose a method for tracing face pictures based on a detection algorithm. Coordinates of key points of face pictures are extracted based on a face detection algorithm, coordinate transformation and hash operation are performed on the coordinates to obtain a unique string corresponding to the face picture as fingerprint data for comparison with the face pictures to be detected to determine whether they are leaked pictures. At the same time, relevant traceability information is retrieved to realize the traceability and tracking of leaked face pictures.

[0028] Embodiment

[0029] S1: Use a camera to capture face images;

[0030] S2: Detect face key points. For any face photo, use face detection technology (face_detect module) to generate the coordinates of 5 points including eyes (left and right), nose, and mouth (left and right corners), and at the same time generate the coordinates of 4 points of the face bounding box, as Figure 1 shown;

[0031] boxes,landmarks=face_detect(face_img)

[0032] Each point uses the (x,y) coordinate system. The coordinate values of the 5 face key points are as follows in the array: landmarks=[[31.77565,56.387833],[70.75577,58.247246],[50.722,78.12365],[32.20663,94.886894],[64.11586,96.276794]]. The coordinate values of the 4 face bounding boxes are as follows in the array: boxes=[10.411129,22.1849,89.29885,121.02415], representing the x_left, y_top, x_right, y_bottom coordinate system. The top-left vertex coordinate is [10.411129,22.1849], defined as the origin coordinate o[10.411129,22.1849];

[0033] S3: Generate the relative coordinates of the face key points through coordinate transformation;

[0034] Subtract the origin coordinate from the coordinate values of the 5 face key points respectively to generate the relative coordinates of the 5 face key points, P=landmarks o:

[0035] P=[[21.364520999999996,34.202933],[60.344640999999996,36.062346],[40.310871,55.93875],[21.795500999999994,72.701994],[53.704730999999995,74.091894]];

[0036] At this time, P is the relative coordinates of the 5 points of eyes, nose, and mouth with the vertex coordinate o as the origin. The advantage of using relative coordinates is to reduce the interference of image data outside the face bounding box and ensure the absolute uniqueness of the relative coordinates of each face image;

[0037] S4: Perform a transformation hashing operation on the relative coordinate P value to generate a 32-bit string, which is also the fingerprint data of the picture; this unique 32-bit string is saved in the database as the traceability identifier of the face photo;

[0038] The specific transformation hashing operation is as follows:

[0039] The following operations require the numpy and hex calculation modules in the standard scientific computing library. The process is as follows:

[0040] The coordinate value of point P is:

[0041] P = array([[21.364521, 34.202934],

[0042] [60.344643, 36.062347],

[0043] [40.31087, 55.93875],

[0044] [21.795502, 72.701996],

[0045] [53.70473, 74.091896]], dtype = float32)

[0046] Perform dimensionality reduction on point P twice. For the first dimensionality reduction, use the nose coordinate as the origin, R = P - P[2], and the calculation results are as follows:

[0047] R = array([[-18.94635, -21.735817],

[0048] [20.033768, -19.876404],

[0049] [0., 0.],

[0050] [-18.51537, 16.763245],

[0051] [13.39386, 18.153145]], dtype = float32)

[0052] Coordinates with a value of 0 do not need to be concerned. The above results convert the 5-point coordinates into 4-point coordinates;

[0053] The second dimensionality reduction is to add the two eye coordinates and add the two mouth coordinates, and perform the following operations respectively:

[0054] ep = R[0] + R[1]

[0055] ep = array([1.0874214, -41.61222], dtype=float32)

[0056] mp = R[3] + R[4]

[0057] mp = array([-5.1215096, 34.91639], dtype=float32)

[0058] After the second dimensionality reduction, the 8 values are changed to 4 values, namely ep[0] is 1.0874214, ep[1] is -41.61222, mp[0] is -5.1215096, and mp[1] is 34.91639;

[0059] Perform integer operations on the two values of ep and mp, and take the uint32 data type:

[0060] ep = numpy.frombuffer(ep, dtype=numpy.uint32)

[0061] ep = array([1066086560, 3257299690], dtype=uint32)

[0062] mp = numpy.frombuffer(mp, dtype=numpy.uint32)

[0063] mp = array([3231966056, 1108060770], dtype=uint32)

[0064] After the above operations, assign the two coordinate data 1066086560, 3257299690, 3231966056, 1108060770 to hash0, hash1, hash2, hash3 respectively, as follows:

[0065] hash0 = ep[0]

[0066] hash1 = ep[1]

[0067] hash2 = mp[0]

[0068] hash3 = mp[1]

[0069] Then perform hexadecimal conversion on hash0, hash1, hash2, hash3, and concatenate the four values in string form (removing the leading 0x) to generate a 32-bit string:

[0070] hash_str = hex(hash0)[2:] + hex(hash1)[2:] + hex(hash2)[2:] + hex(hash3)[2:]

[0071] hash_str = '3f8b30a0c22672eac0a3e368420baa62'

[0072] This string is saved in the database as the unique hash value of this face photo;

[0073] Define the hash_strs() function as the function to generate the photo hash value;

[0074] As shown by the above calculation process, this algorithm is fast, accurate, does not require too much computing resources, and the generated hash value is unique. Since the four values are relatively independent, if two of the values of the photo are damaged, the original photo can still be obtained through fuzzy search using the other two values;

[0075] S5: Store the calculation result in the database, SQL instruction:

[0076] Insert into faceDB(hashid,ip,time_tag,address,image_path)

[0077] Values

[0078] ('3f8b30a0c22672eac0a3e368420baa62','192.168.1.2','1641540723.650545','Chengdu Tianfu Second Street',' / face_img / 3f8b30a0c22672eac0a3e368420baa62.jpg')

[0079] Save the hash_str = '3f8b30a0c22672eac0a3e368420baa62' of this photo into the faceDB library through the above instruction, and save the photo to the JPG image file named with hash_str (' / face_img / 3f8b30a0c22672eac0a3e368420baa62.jpg');

[0080] S6: Once it is suspected that the face photo has been leaked, calculate the traceability identifier of the 32-bit string of this face photo;

[0081] S7: Accurately find the leaked database by comparing the traceability identifier, and confirm the IP and address of the collected camera.

[0082] Example 2

[0083] Based on Example 1, if there is a photo named image1.jpg on the market and you want to know if this important photo has been leaked by a face collection or monitoring system, the following operations are used for traceability identification:

[0084] 1) Use the detection module to first perform face detection on the photo image1.jpg:

[0085] boxes,landmarks=face_detect(image1.jpg)

[0086] 2) Calculate the difference between the five-point coordinates and the vertex coordinates of the BOXES:

[0087] landmarks=landmarks–boxes[:2]

[0088] Landmarks=[[21.364520999999996,34.202933],[60.344640999999996,36.062346],[40.310871,55.93875],[21.795500999999994,72.701994],[53.704730999999995,74.091894]]

[0089] 3) Calculate the 32-bit hash string:

[0090] landmarks=numpy.array(landmarks,dtype=numpy.float32)

[0091] hash_str=hash_strs(landmarks)

[0092] hash_str='3f8b30a0c22672eac0a3e368420baa62'

[0093] 4) Use the SQL instruction to query whether the string '3f8b30a0c22672eac0a3e368420baa62' exists in the faceDB database:

[0094] Select*from faceDB where hashid=

[0095] '3f8b30a0c22672eac0a3e368420baa62';

[0096] If the result returns a record:

[0097] '3f8b30a0c22672eac0a3e368420baa62', '192.168.1.2', '1641540723.650545', 'Tianfu Second Street, Chengdu', ' / face_img / 3f8b30a0c22672eac0a3e368420baa62.jpg'

[0098] It indicates that the photo was captured and generated at 'Tianfu Second Street, Chengdu'. The original photo is ' / face_img / 3f8b30a0c22672eac0a3e368420baa62.jpg', and the IP of the capturing device is '192.168.1.2'. It is necessary to conduct a security check on this device to identify the photo leakage path of the device;

[0099] If no record is returned from the database: It indicates that this photo was not leaked by this system.

[0100] Specifically, by operating the database records (operator account, login time, operation type), lock the operator with the operation type of copy, complete the traceability and accountability of the illegally used face pictures, provide an effective technical means guarantee for the safe use of face pictures, and also effectively avoid the recurrence of leakage incidents.

[0101] Embodiment 3

[0102] Based on Embodiment 2, if two values of the photo are damaged, it is still possible to perform a fuzzy search through the other two values to obtain the original picture of the photo. For example, through face recognition calculation, the photo hash string is obtained as hash_str = '3f8b30a0c22672eac0a3e368420baa62', a total of 32 bits; this string is divided into 4 parts, each part has 8 strings, corresponding to the hash values of the left eye, right eye, left mouth corner, and right mouth corner respectively. If the hash value of the right mouth corner is missing, then the remaining three parts of the characters can be used for fuzzy search with fuzzy conditions, that is, use the 16 - bit string '3f8b30a0c22672eac0a3e368' to search in the database:

[0103] Select * from faceDB where hashid like '%3f8b30a0c22672eac0a3e368%'

[0104] Similarly, the generation path of this photo can be traced.

[0105] Similarly, if two values are damaged, a 16 - bit string can also be used to search in the database:

[0106] Select*from faceDB where hashid like'%3f8b30a0c22672ea%'.

[0107] Advantages of the present invention:

[0108] 1) In the database scenario, in the picture database, in order to accurately retrieve and store, a unique ID needs to be assigned to each face photo. Using the method of the present invention, the 32-bit fingerprint data ID of md5strs can be directly used as the storage ID.

[0109] 2) In the security scenario, using the method of the present invention, it is possible to trace the leaked pictures in a timely manner, track the source of the pictures, find out the leakage channels, and check the potential hazard points, thus ensuring the security of face data.

[0110] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, refer to the description of the method part.

[0111] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A face picture traceability method based on a detection algorithm, characterized in that, It includes the following specific steps: Step 1: Collect face images; Step 2: Use a face detection algorithm to extract the key point coordinates of the face image and the corner coordinates of the face frame; Step 3: Perform coordinate transformation on the key point coordinates to obtain the relative coordinates of the face key points; Step 4: Perform transform hashing operation on the relative coordinates of the face key points to obtain fingerprint data; Step 5: Save the fingerprint data and its corresponding traceability information to the face database, and save the corresponding face image to an image file; Step 6: Use the loop steps 2-4 for the face image to be detected to obtain a hash string; Step 7: Search in the face database to see if there is fingerprint data that meets the consistent condition when compared with the hash string. If it exists, return the searched fingerprint data and its corresponding traceability information; otherwise, return a non-system image prompt message; In step 4, when performing transform hashing operation on the relative coordinates of the key points, use the numpy and hex calculation modules in the standard scientific computing library to generate a 32-bit string. The specific process is as follows: Step 41: Perform a dimensionality reduction on the relative coordinates; select the relative coordinate corresponding to one of the key point coordinates as the origin, and subtract the relative coordinates of the remaining key point coordinates from the relative coordinate corresponding to the origin to convert the five-point coordinates into four-point coordinates; Step 42: Perform a second dimensionality reduction; add the coordinate values of the four-point coordinates in pairs to obtain two sets of intermediate values; Step 43: Perform integer operations on the two sets of intermediate values to obtain four sets of hash values; Step 44: Perform base conversion and combination on the four sets of hash values to obtain a 32-bit string.

2. The face picture tracing method based on a detection algorithm according to claim 1, characterized in that, In step 2, the collected key point coordinates and corner coordinates are uniformly represented in the (x,y) coordinate system, and the upper left corner point coordinates in the corner coordinates are defined as the origin coordinates; the key point coordinates include five-point coordinates.

3. The face image traceability method based on a detection algorithm according to claim 2, characterized in that, In step 3, subtract the origin coordinates from the key point coordinates to generate the relative coordinates of the key points.

4. A face image traceability method based on a detection algorithm according to claim 1, characterized in that The consistent conditions include the same condition and the fuzzy condition; the same condition is the fingerprint data that is exactly the same as the hash string; the fuzzy condition is the fingerprint data that is partially the same as the hash string.

5. A face picture traceability method based on a detection algorithm according to claim 1, characterized in that The traceability information includes the device IP, device address, device number, image generation time of the collected face image, and database operation records; the database operation records include the operator account, login time, and operation type.

Images