Mirror face detection method, device and equipment

Abstract

The application discloses a mirror face detection method, device and equipment. The method comprises the following steps: acquiring an image set collected by a target image collection device, determining similar face position information according to face information of multiple reference images in the image set; clustering the similar face position information to obtain at least one candidate reflector region in a collection field of view of the target image collection device; determining a target reflector region from the at least one candidate reflector region; and determining a mirror face in a target image collected by the target image collection device according to the target reflector region. Through identification and determination of a reflector region in a collection field of view of an image collection device, a false portrait generated by a reflector can be avoided, the problems of misrecognition and missed recognition caused by extraction of a mirror face are solved, and the accuracy and applicability of real portrait recognition are improved.

Description

Technical Field

[0001] This invention relates to the field of face recognition, and more particularly to a method, apparatus, and device for detecting mirrored faces. Background Technology

[0002] With the increasing demand for social informatization and the development of the intelligent security industry, the requirements for the accuracy of facial recognition are gradually increasing. Facial recognition technology relies on the accurate identification and extraction of facial features in images.

[0003] Among numerous camera locations, some are affected by reflective objects, resulting in two similar human figures appearing in the captured images. However, due to factors such as refractive index and lighting conditions, the features of the mirrored face still differ from those of the actual face. If the mirrored face is incorrectly selected for subsequent processing, it can easily lead to the loss of some of the true features corresponding to that face, resulting in the loss of some trajectory information of the person, and causing misidentification and missed identification. Summary of the Invention

[0004] This invention provides a method, apparatus, and device for detecting mirrored faces, in order to avoid misidentification and missed identification caused by the presence of reflective objects when extracting mirrored faces.

[0005] According to one aspect of the present invention, a method for detecting a mirrored face is provided, comprising:

[0006] The system acquires an image set captured by a target image acquisition device, and determines the location information of similar faces based on the facial information of multiple reference images in the image set; wherein, the reference images include at least two faces with a similarity greater than a preset similarity threshold;

[0007] Clustering the location information of similar faces yields at least one candidate reflector region in the field of view of the target image acquisition device;

[0008] Determine the target reflector region from the at least one candidate reflector region;

[0009] The mirrored face in the target image acquired by the target image acquisition device is determined based on the target reflector region.

[0010] According to another aspect of the present invention, a face mirror detection device is provided, comprising:

[0011] A similar face location determination module is used to acquire an image set acquired by a target image acquisition device, and determine similar face location information based on face information of multiple reference images in the image set; wherein, the reference images include at least two faces with a similarity greater than a preset similarity threshold;

[0012] The candidate region determination module is used to cluster the similar face location information to obtain at least one candidate reflector region in the field of view of the target image acquisition device;

[0013] A target region determination module is used to determine a target reflector region from the at least one candidate reflector region;

[0014] The mirror face determination module is used to determine the mirror face in the target image acquired by the target image acquisition device based on the target reflector region.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the mirror face detection method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the mirror face detection method according to any embodiment of the present invention.

[0020] The technical solution of this invention, by identifying and determining the reflective area within the field of view acquired by the image acquisition device, avoids identifying the human image generated by the reflective object, and does not require additional equipment. This solves the problems of misidentification and missed identification caused by extracting mirrored faces, and improves the accuracy and applicability of real human image recognition.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1This is a flowchart of a method for detecting a mirrored face according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of a method for detecting a mirrored face according to Embodiment 2 of the present invention;

[0025] Figure 3 This is a flowchart of a method for detecting a mirrored face according to Embodiment 3 of the present invention;

[0026] Figure 4 These are schematic diagrams showing four possible positions between the image acquisition device, the reflector, and the human figure.

[0027] Figure 5 This is a flowchart of a method for detecting a mirrored face according to Embodiment 4 of the present invention;

[0028] Figure 6 This is a schematic diagram of the structure of a mirror face detection device provided in Embodiment 5 of the present invention;

[0029] Figure 7 This is a schematic diagram of the structure of an electronic device that implements the mirror face detection method of this invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "candidate," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] Figure 1This is a flowchart illustrating a method for detecting mirrored faces according to Embodiment 1 of the present invention. This embodiment is applicable to situations where mirrored images are excluded from camera-captured images when reflective objects are present. This method can be executed by a mirrored face detection device, which can be implemented in hardware and / or software and can be configured on a server with computing power. Figure 1 As shown, the method includes:

[0034] S110. Obtain the image set acquired by the target image acquisition device, and determine the location information of similar faces based on the facial information of multiple reference images in the image set.

[0035] The target image acquisition device refers to the camera location within the field of view that may include reflective objects. Reflective objects are objects such as mirrors, water surfaces, or glass that can form a mirror image through light reflection. The image set refers to the collection of image data acquired by the target image acquisition device over a period of time. A reference image is any image in the image set that includes at least two faces with a similarity greater than a preset similarity threshold. Similar face location information refers to the set of location information of similar faces appearing in all reference images.

[0036] Specifically, if a reflective object exists within the field of view of the target image acquisition device, a single image acquired by the device will contain both a real face and a mirrored face. Therefore, a dataset of images acquired by the target image acquisition device over a period of time is compiled into an image set. Face similarity recognition is performed on each image in the image set. If any image contains two faces with a similarity greater than a preset similarity threshold, that image is determined as a reference image that may contain both a mirrored face and a real face. The location information of faces with a similarity greater than the preset similarity threshold in all reference images is obtained, and a set of location information is determined based on this information, serving as the location information of similar faces.

[0037] Since the similarity results of different people in the same image are usually low, a similarity threshold is set to compare the similarity results of all faces in the image to determine whether there are mirrored faces caused by reflective objects. For example, for each image in the image set, all face images within it are acquired, features are extracted, and they are compared. If a face feature with a similarity greater than a preset similarity threshold is found, the image is determined to be a reference image that may contain mirrored faces caused by reflective objects. The preset similarity threshold can be set according to the specific scenario and is not limited here; for example, the preset similarity threshold can be 80%. Any face image pair in all reference images in the image set that exceeds the preset similarity threshold is identified, and the relative position information of any face image pair in each reference image is obtained. The location information of similar faces is determined based on the set of all relative position information.

[0038] Optionally, the presence of reflective objects within the field of view of the target image acquisition device can be pre-screened based on the proportion of reference images in the image set. Specifically, the proportion of reference images in the image set is statistically analyzed; if the proportion exceeds a preset threshold, it can be preliminarily determined that a reflective object exists at the device location. This threshold can be adjusted according to actual conditions, and no specific value is limited here.

[0039] S120. Cluster the location information of similar faces to obtain at least one candidate reflector region in the field of view of the target image acquisition device.

[0040] In image acquisition devices with a fixed field of view, the position of the reflector is also relatively fixed. Therefore, in images containing both actual and mirrored faces, there will be two regions: one where mirrored faces are concentrated, and the other where actual faces appear. Specifically, clustering the positional information of similar faces allows for the preliminary determination of the mirrored face regions formed by the reflector. Clustering can preliminarily identify the regions where similar faces are concentrated. Since the reflector is fixed, the mirrored faces formed by it are also concentrated in a certain area. By clustering the positional information of similar faces, candidate locations for the reflector regions can be accurately obtained.

[0041] This embodiment applies to situations where there is a reflector within the field of view of the target image acquisition device. When a reflector exists, at most two candidate reflector regions can appear: one for a mirrored face and one for the actual face. Therefore, clustering by category can yield accurate clustering results for the candidate reflector regions. For example, a K-Means clustering algorithm with two categories is used to cluster similar face location information, generating two candidate reflector regions: one for the actual face and the other for the mirrored face. For example, since an actual face can potentially form a mirror image within a reflector region from any location, the actual face location information may be relatively dispersed. However, because the reflector's position is fixed, the mirrored face will only appear within the reflector region, resulting in a more concentrated mirrored face location information. Therefore, clustering may yield only one candidate reflector region.

[0042] S130. Determine the target reflector region from at least one candidate reflector region.

[0043] If a candidate reflector region is obtained based on the clustering results, this candidate reflector region is determined as the final reflector region and used as the target reflector region. If two candidate reflector regions are obtained based on the clustering results, one is a mirror face region and the other is a real face region. Further filtering is performed based on the distinguishing features between the mirror face and the real face to determine the target reflector region from the two candidate reflector regions. The distinguishing features between the mirror face and the real face include the mirror face being smaller than the real face.

[0044] In one feasible embodiment, S130 includes:

[0045] If the number of candidate reflector regions is 2, then the target reflector region is determined from the candidate reflector regions based on the face size information in the candidate reflector regions.

[0046] During the imaging process of an image acquisition device, the farther away an object is from the image acquisition device, the smaller its image appears in the image. Furthermore, since the imaging of a reflective object is the reflection of light from the human image, the distance between the mirrored human image and the image acquisition device must be greater than the distance between the actual human image and the image acquisition device. This is reflected in the image as a smaller size of the mirrored human image than the actual human image.

[0047] Specifically, if two candidate reflector regions exist, the one with the smaller face size is selected as the target reflector region. For example, based on the relative positions of the candidate reflector regions, they are divided into a left reflector region and a right reflector region. The heights of the face bounding boxes in the left and right reflector regions are extracted using facial features, and the face size information is obtained based on these heights. Alternatively, the area, width, or aspect ratio of the face bounding box can be used for judgment; the criteria for determining face size information are not limited here.

[0048] Optionally, the face size information of two candidate reflector regions in all reference images in the image set is statistically analyzed. It is determined whether the proportion of reference images showing a face size greater than that of the left reflector region is greater than that of the right reflector region exceeds a preset threshold. If so, the right reflector region is determined as the target reflector region. Alternatively, it is determined whether the proportion of reference images showing a face size smaller than that of the right reflector region is greater than a preset threshold. If so, the left reflector region is determined as the target reflector region. If neither of these conditions is met, a new image set is acquired and the detection is performed again.

[0049] S140. Determine the mirrored face in the target image acquired by the target image acquisition device based on the target reflector area.

[0050] If two faces with a similarity greater than a preset similarity threshold are detected in the target image, the location information of the face is determined. If the face information is located in the target reflector area, it is determined that the face is a mirror face formed by the reflector.

[0051] The technical solution of this invention, by identifying and determining the reflective area within the field of view acquired by the image acquisition device, avoids identifying the human image generated by the reflective object, and does not require additional equipment. This solves the problems of misidentification and missed identification caused by extracting mirrored faces, and improves the accuracy and applicability of real human image recognition.

[0052] Example 2

[0053] Figure 2 This is a flowchart of a mirrored face detection method provided in Embodiment 2 of the present invention. This embodiment further optimizes the clustering method in the above embodiments, wherein at least two clustering methods are included, namely a first clustering method and a second clustering method; the first clustering method is a clustering method based on the number of categories, and the second clustering method is a density clustering method; at least two clustering results are included, namely the first clustering result and the second clustering result. For example... Figure 2 As shown, the method includes:

[0054] S210. Obtain the image set acquired by the target image acquisition device, and determine the location information of similar faces based on the facial information of multiple reference images in the image set.

[0055] S220. Determine at least two clustering results based on at least two clustering methods to identify the location information of similar faces.

[0056] Using only one clustering method to determine candidate reflector regions can easily lead to inaccurate region determination. For example, using only clustering based on the number of categories will result in relatively blurry reflector region boundaries, causing recognition errors between mirrored and real faces. Using only density-based clustering will result in inaccurate localization of candidate reflector regions, leading to the determination of multiple candidate reflector regions and making it difficult to determine the final target reflector region. Therefore, this embodiment uses two clustering methods to cluster similar face location information. Specifically, clustering based on the number of categories is used to cluster similar face location information to obtain the first clustering result; density-based clustering is used to cluster similar face location information to obtain the second clustering result. For example, the K-Means clustering algorithm with K=2 categories is used for similar face location information, and the clustering result is two regions: one is the candidate region for the real face image, and the other is the candidate region for the mirrored face image. The density-based clustering algorithm, such as DBSCAN, is used for similar face location information. Initially, a small density parameter is set to obtain multiple result clusters, and candidate regions for the result clusters are generated. Optionally, in this embodiment, density clustering and number-of-classes clustering are used as examples to illustrate two clustering methods. Other clustering methods are also applicable to the method of this invention, such as fuzzy clustering or model-based clustering. This invention does not limit the type of clustering method. Preferably, in this embodiment, it is applicable to scenarios with one reflector. Since there is one reflector, the number of corresponding candidate reflector regions is fixed, i.e., one real human image candidate region and one mirrored human image candidate region. Therefore, using number-of-classes clustering as one of the clustering methods can improve the accuracy of the clustering results. Similarly, the other clustering method is not limited.

[0057] S230. Determine candidate reflector regions based on the overlap of at least two clustering results.

[0058] The first clustering result includes two clustering regions, and the second clustering result includes at least one clustering region. Candidate reflector regions are determined based on the overlap of at least two clustering results.

[0059] In one feasible embodiment, determining candidate reflector regions based on the overlap of at least two clustering results includes:

[0060] If the overlap between any cluster region in the first clustering result and any cluster region in the second clustering result exceeds the overlap threshold, then candidate reflector regions are determined based on the overlapping regions.

[0061] If a cluster region in the second clustering result has an overlap degree greater than the overlap degree threshold with any cluster region in the first clustering result, then the overlapping region between the corresponding two cluster regions in the two clustering results is determined as a candidate reflector region. This ensures both the accuracy of the number of cluster regions and the clarity of the cluster boundaries. Optionally, the detection method of the present invention is not limited to determining the candidate reflector region based on the overlap degree exceeding the overlap degree threshold in the above embodiment. Other methods can also be used to determine the candidate reflector region based on the overlap degree, for example, determining the candidate reflector region based on the comparison result of the overlap degree of at least two clustering results.

[0062] S240. Determine the target reflector region from at least one candidate reflector region.

[0063] Specifically, the candidate reflector regions determined based on the above two clustering results include three cases: First, if the number of candidate reflector regions is 0, the distance parameter of the density clustering algorithm is reset, and the second clustering result is re-determined after changing the parameter; Second, if the number of candidate reflector regions is 1, it indicates that the reflector area is small, the distribution of the mirrored human image is relatively concentrated, while the distribution of the actual human image is relatively dispersed, so this candidate reflector region is determined as the target reflector region; Third, if the number of candidate reflector regions is 2, it indicates that the distribution of both the real human image and the mirrored human image is relatively dispersed, so the target reflector region is determined from the candidate reflector regions based on the face size information in the candidate reflector regions.

[0064] S250. Determine the mirrored face in the target image acquired by the target image acquisition device based on the target reflector area.

[0065] The embodiments of the present invention determine candidate reflector regions through two clustering methods, thereby improving the accuracy of candidate reflector region determination and ensuring the accuracy of mirror face detection.

[0066] Example 3

[0067] Figure 3 This is a flowchart of a method for detecting a mirrored face according to Embodiment 2 of the present invention. This embodiment further optimizes the method for determining the target reflector region in the above embodiments, wherein the number of candidate reflector regions is 2. Figure 3 As shown, the method includes:

[0068] S310. Obtain the image set acquired by the target image acquisition device, and determine the location information of similar faces based on the facial information of multiple reference images in the image set.

[0069] S320. Cluster the location information of similar faces to obtain at least one candidate reflector region in the field of view of the target image acquisition device.

[0070] S330. Determine the target reflector region from the candidate reflector region based on the face size information in the candidate reflector region.

[0071] During the imaging process of an image acquisition device, the farther away an object is from the image acquisition device, the smaller its image appears in the image. Furthermore, since the imaging of a reflective object is the reflection of light from the human image, the distance between the mirrored human image and the image acquisition device must be greater than the distance between the actual human image and the image acquisition device. This is reflected in the image as a smaller size of the mirrored human image than the actual human image.

[0072] Specifically, if there are two candidate reflector regions, the candidate reflector region with the smaller face size is determined as the target reflector region.

[0073] In one feasible embodiment, S330 includes:

[0074] The candidate reflector region with smaller face size information among the two candidate reflector regions in the reference image is identified as the target reflector region.

[0075] For example, based on the position information of the two candidate reflector regions in the reference image, the two candidate reflector regions are divided into a left reflector region and a right reflector region. The size of the left face in the left reflector region and the size of the right face in the right reflector region are determined. The target reflector region is determined based on the ratio of the left face size to the right face size. If the ratio is greater than 1, the right reflector region is determined as the target reflector region. If the ratio is less than 1, the left reflector region is determined as the target reflector region.

[0076] In one feasible embodiment, S330 includes:

[0077] Determine the neighboring images corresponding to the reference image, as well as the face orientation information in the reference image and the neighboring images; wherein the time interval between the reference image and the corresponding neighboring image is less than a preset time interval; and the neighboring images include at least two faces with a similarity greater than a preset similarity threshold;

[0078] Based on face orientation information, the target reflector region is determined from the candidate reflector regions according to the changes in face size in the two candidate reflector regions in the reference image and the adjacent image.

[0079] When the distance between the mirrored image and the actual image is close (e.g., a pedestrian walking close to a mirror), the size ratio difference is not significant. Furthermore, due to slight errors in the image framing technique used in feature extraction, simply judging based on face size can lead to misjudgment. Therefore, in this embodiment, the judgment is based on the trend of face size changes to improve the accuracy of determining the target reflective area.

[0080] In this context, adjacent images are subsequent images that are close to the time interval of the reference image, and each adjacent image contains at least two faces with a similarity greater than a preset similarity threshold. Due to the short time interval, similar faces in the reference image and their corresponding adjacent images are considered to be the same person. Optionally, to ensure the accuracy of adjacent image determination, the similarity between at least two faces with a similarity greater than the preset similarity threshold in the adjacent image and at least two faces with a similarity greater than the preset similarity threshold in the reference image must also satisfy the condition of being greater than the preset similarity threshold.

[0081] Because the appearance of a real face and its mirror image differs during walking, using the size change information of the same person's face in reference and adjacent images can ensure the accuracy of target reflector region determination. For example, during walking, the actual face size changes at a greater rate than the mirror image size; therefore, this feature can be used to determine the target reflector region.

[0082] In one feasible embodiment, based on face orientation information, and according to the change information of face size in two candidate reflector regions in a reference image and an adjacent image, the target reflector region is determined from the candidate reflector regions, including:

[0083] The first candidate reflector region and the second candidate reflector region are determined based on their relative positions in the reference image and adjacent images.

[0084] Determine the ratio of the first face size to the second candidate reflector region in the reference image;

[0085] Determine the ratio of the second face size between the first candidate reflector region and the second candidate reflector region in adjacent images;

[0086] Based on face orientation information, the target reflector region is determined from the candidate reflector region according to the ratio of the first face size to the ratio of the second face size.

[0087] Specifically, the size features of the mirrored face and the actual face in the reference image are determined by the first face size ratio, the size features of the mirrored face and the actual face in the adjacent image are determined by the second face size ratio, and the change information of face size in the two candidate reflector regions can be determined by the change between the first face size ratio and the second face size ratio.

[0088] For example, the first candidate reflector region is the left candidate reflector region, and the second candidate reflector region is the right candidate reflector region. If the ratio of the first face size to the second face size is less than 1, the first candidate reflector region is initially determined to be a mirror region. If the ratio of the first face size to the second face size is greater than 1, the second candidate reflector region is initially determined to be a mirror region. Otherwise, the reference image and the corresponding adjacent images are re-determined. Further, if the face orientation information indicates facing the image acquisition device, it means that the actual face in the adjacent image is larger than the actual face in the reference image. If the face orientation information indicates facing away from the image acquisition device, it means that the actual face in the adjacent image is smaller than the actual face in the reference image. Combining this with the fact that the rate of change of the actual face is greater than the rate of change of the mirror face, the target reflector region can be determined by combining the changes in the ratio of the first face size to the second face size.

[0089] In one feasible embodiment, based on face orientation information, a target reflector region is determined from the candidate reflector region according to a first face size ratio and a second face size ratio, including:

[0090] If the face orientation information is facing the image acquisition device, and the ratio of the first face size is less than 1, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the first candidate reflector region is determined as the target reflector region.

[0091] If the face orientation information is facing the image acquisition device, and the ratio of the first face size is greater than 1, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the second candidate reflector region is determined as the target reflector region.

[0092] If the face orientation information is that the face is facing away from the image acquisition device, and the ratio of the first face size is less than 1, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the first candidate reflector region is determined as the target reflector region.

[0093] If the face orientation information is that the face is facing away from the image acquisition device, and the ratio of the first face size is greater than 1, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the second candidate reflector region is determined as the target reflector region.

[0094] Specifically, such as Figure 4The diagram illustrates four possible positions of the image acquisition device, the reflector, and the human image: Case 1: The image acquisition device is located between the reflector and the actual human image, with the reflector to the left of the image acquisition device; Case 2: The image acquisition device is located to the right of the actual human image, with the reflector to the left of the actual human image; Case 3: The image acquisition device is located between the actual human image and the reflector, with the reflector to the right of the image acquisition device; Case 4: The image acquisition device is located to the left of the actual human image, with the reflector to the right of the actual human image. The following analysis examines the changes in face size in the two candidate reflector regions of the reference image and adjacent images for each of these four cases. Other cases can be referred to in the proof process of this embodiment, and are not all listed here.

[0095] Figure 4 In the diagram, 'a' represents the vertical distance between the actual human image and the image acquisition device, 'b' represents the horizontal distance between the actual human image and the camera, 'c' represents the vertical distance between the actual human image and the reflector, 'Dleft' represents the square distance between the left-side human image and the image acquisition device, and 'Dright' represents the square distance between the right-side human image and the image acquisition device. In case 1, 'Dleft' = 'a'. 2 +(2c-b) 2 ,Dright=a 2 +b 2 First face size ratio Take the partial derivative of ratio_A with respect to a: The second face size ratio is ratio_B. Therefore, as 'a' increases, the first face size ratio ratio_A decreases. Correspondingly, in the image, when the face is facing the camera, the actual image gradually increases from time 1 to time 2, while 'a' gradually decreases. The rate of change of the actual image is greater than the rate of change of the mirrored image, so ratio_A is less than ratio_B, and the ratio of the first face size ratio to the second face size ratio is... When facing away from the camera, the actual human figure gradually decreases in size, while 'a' gradually increases. Case 2: D left = a 2 +(2c+b) 2 ,Dright=a 2 +b 2 , The partial derivative of ratio_A with respect to a: Therefore, as 'a' increases, 'ratio_A' decreases. Correspondingly, in an image, when the face is facing the camera, 'a' gradually decreases, and the actual rate of change of the human face is greater than the rate of change of the mirrored human face. When facing away from the camera, 'a' gradually decreases. In case 3, D_left = a 2 +b 2 ,Dright=a 2+(2c-b) 2 , The partial derivative of ratio_A with respect to a: Therefore, as 'a' increases, 'ratio_A' increases. Correspondingly, in an image, when the face is facing the camera... When facing away from the camera, In case 4, D_left = a 2 +b 2 ,Dright=a 2 +(2c+b) 2 , The partial derivative of ratio_A with respect to a: Therefore, as 'a' increases, 'ratio_A' increases. Correspondingly, in an image, when the face is facing the camera... When facing away from the camera,

[0096] As can be seen from the above, the changes in the ratios of the first face size, the second face size, and the ratio of the first face size to the second face size are related to the mirror area. Therefore, when facing the image acquisition device: if ratio_A, If both ratio_A and ratio_A are less than 1, then the first candidate reflector region is a mirror region, which is the target reflector region; if ratio_A and ratio_A are less than 1, then the first candidate reflector region is a mirror region, which is the target reflector region. If both ratio_A and ratio_A are greater than 1, then the second candidate reflector region is a mirror region, which is the target reflector region; if ratio_A and ratio_A are both greater than 1, then the second candidate reflector region is a mirror region, which is the target reflector region. If one image is greater than 1 and the other is less than 1, the verification fails, and the image pair cannot be used to determine the mirror region. A new reference image and its corresponding adjacent image must be selected. When facing away from the image acquisition device: if ratio_A is less than 1, If the ratio_A is greater than 1, then the first candidate reflector region is the mirror region, which is the target reflector region; if ratio_A is greater than 1, If the ratio_A is less than 1, then the second candidate reflector region is a mirror region, which is the target reflector region; if the ratio_A is less than 1, then the second candidate reflector region is a mirror region, which is the target reflector region. If all values ​​are greater than or less than 1, the verification fails, and the image pair cannot be used to determine the mirror region. A new reference image and its corresponding adjacent image should be selected.

[0097] S340. Determine the mirrored face in the target image acquired by the target image acquisition device based on the target reflector area.

[0098] This invention improves the accuracy of determining mirror regions from candidate reflector regions by determining the face size change information in candidate reflector regions in two images with short time intervals, thereby improving the accuracy of determining mirror faces.

[0099] Example 4

[0100] Figure 5 This is a flowchart of a method for detecting mirrored faces provided in Embodiment 4 of the present invention. This embodiment is a preferred embodiment of the above embodiments and specifically includes:

[0101] Step 1: Determine the image set of the camera location as the image set acquired by the target image acquisition device. Perform a Neural Network (NN) comparison on the images in the image set to determine whether a reflective object exists at that location. Specifically, count the proportion of reference images in the image set. If the proportion is greater than a preset proportion threshold, it can be preliminarily determined that a reflective object exists at that camera location.

[0102] Step 2: Extract the location information of similar face pairs in the reference image, and use the number of clusters algorithm and density clustering algorithm to cluster the unknown information of similar face pairs to obtain clustering result A and clustering result B. Determine the candidate reflector region based on the overlap between clustering result A and clustering result B. If the overlap between any clustering region in clustering result A and any clustering region in clustering result B exceeds the overlap threshold, then the candidate reflector region is determined based on the overlapping region.

[0103] If the number of candidate reflector regions is 0, the distance parameter of the density clustering algorithm is reset, and the second clustering result is re-determined after the parameter is changed; if the number of candidate reflector regions is 1, it means that the reflector area is small, the distribution of the mirrored human image is relatively concentrated, while the distribution of the actual human image is relatively dispersed, so the candidate reflector region is determined as the target reflector region; if the number of candidate reflector regions is 2, proceed to step 3.

[0104] Step 3: Select consecutive captured images from the point camera and determine the reference image and its corresponding neighboring images. The reference image and its corresponding neighboring images must both contain faces with a similarity greater than a threshold, and the similarity of similar faces in the reference image and its corresponding neighboring images must also be greater than the similarity threshold. The reflector region is determined based on the size ratio of the left and right faces in the reference image as a first conclusion. For example, if the first face size ratio is less than 1, the first candidate reflector region on the left is determined as the target reflector region; if the first face size ratio is greater than 1, the second candidate reflector region on the right is determined as the target reflector region. Simultaneously, the ratio of the first face size ratio to the second face size ratio is determined. Based on this ratio, the reflector region is further determined. For example, if the face orientation information indicates facing the image acquisition device, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the first candidate reflector region is determined as the target reflector region. If the face orientation information indicates facing the image acquisition device, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the second candidate reflector region is determined as the target reflector region. If the face orientation information indicates facing away from the image acquisition device, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the first candidate reflector region is determined as the target reflector region. If the face orientation information indicates facing away from the image acquisition device, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the second candidate reflector region is determined as the target reflector region. If the determination results of the two target reflector regions are consistent, then the final actual reflector region is determined.

[0105] Step 4: If two faces with a similarity greater than the preset similarity threshold are detected in the target image, the location information of the face is determined. If the face information is located in the target reflector area, it is determined that the face is a mirror face formed by the reflector.

[0106] Example 5

[0107] Figure 6 This is a schematic diagram of the structure of a mirrored face detection device provided in Embodiment 5 of the present invention. Figure 6 As shown, the device includes:

[0108] The similar face location determination module 610 is used to acquire an image set acquired by the target image acquisition device and determine similar face location information based on the face information of multiple reference images in the image set; wherein, the reference images include at least two faces with a similarity greater than a preset similarity threshold;

[0109] The candidate region determination module 620 is used to cluster the similar face location information to obtain at least one candidate reflector region in the acquisition field of view of the target image acquisition device.

[0110] The target region determination module 630 is used to determine a target reflector region from at least one candidate reflector region;

[0111] The mirror face determination module 640 is used to determine the mirror face in the target image acquired by the target image acquisition device based on the target reflector region.

[0112] Optionally, the candidate region determination module includes:

[0113] The clustering result determination unit is used to determine at least two clustering results of the similar face location information based on at least two clustering methods;

[0114] A candidate region determination unit is used to determine the candidate reflector region based on the overlap of the at least two clustering results.

[0115] Optionally, the at least two clustering methods include a first clustering method and a second clustering method; the first clustering method is a clustering method based on the number of categories, and the second clustering method is a density clustering method; the at least two clustering results include the first clustering result and the second clustering result.

[0116] The candidate region determination unit is specifically used for:

[0117] If the overlap between any cluster region in the first clustering result and any cluster region in the second clustering result exceeds the overlap threshold, then candidate reflector regions are determined based on the overlapping regions.

[0118] Optional, the target area determination module is specifically used for:

[0119] If the number of candidate reflector regions is 2, then the target reflector region is determined from the candidate reflector regions based on the face size information in the candidate reflector regions.

[0120] Optional, the target area determination module is specifically used for:

[0121] The candidate reflector region with smaller face size information among the two candidate reflector regions in the reference image is determined as the target reflector region.

[0122] Optional, the target area determination module includes:

[0123] An image determination unit is used to determine neighboring images corresponding to the reference image, as well as face orientation information in the reference image and the neighboring images; wherein the time interval between the reference image and the corresponding neighboring image is less than a preset time interval; and the neighboring images include at least two faces with a similarity greater than a preset similarity threshold;

[0124] The target region determination unit is used to determine the target reflector region from the candidate reflector regions based on the face orientation information and the change information of face size in two candidate reflector regions in the reference image and the adjacent image.

[0125] Optionally, the target area determination unit includes:

[0126] A region determination subunit is used to determine a first candidate reflector region and a second candidate reflector region based on the relative positions of the candidate reflector regions in the reference image and the adjacent images;

[0127] The first ratio determination subunit is used to determine the first face size ratio between the first candidate reflector region and the second candidate reflector region in the reference image;

[0128] The second ratio determination subunit is used to determine the second face size ratio between the first candidate reflector region and the second candidate reflector region in the adjacent images;

[0129] The target region determination subunit is used to determine the target reflector region from the candidate reflector regions based on the face orientation information and according to the first face size ratio and the second face size ratio.

[0130] Optionally, the target region determines sub-units, specifically used for:

[0131] If the face orientation information indicates that the face is facing the image acquisition device, and the first face size ratio is less than 1, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the first candidate reflector region is determined to be the target reflector region.

[0132] If the face orientation information indicates that the face is facing the image acquisition device, and the first face size ratio is greater than 1, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the second candidate reflector region is determined as the target reflector region.

[0133] If the face orientation information indicates that the face is facing away from the image acquisition device, and the first face size ratio is less than 1, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the first candidate reflector region is determined to be the target reflector region.

[0134] If the face orientation information indicates that the face is facing away from the image acquisition device, and the first face size ratio is greater than 1, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the second candidate reflector region is determined as the target reflector region.

[0135] The mirror face detection device provided in this embodiment of the invention can execute the mirror face detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0136] The acquisition, storage, use, and processing of data in this application comply with relevant national laws and regulations and do not violate public order and good morals.

[0137] Example 6

[0138] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0139] Figure 7 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0140] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0141] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0142] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the method of detecting mirrored faces.

[0143] In some embodiments, the detection of a method-mirrored face can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method-mirrored face detection described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform method-mirrored face detection by any other suitable means (e.g., by means of firmware).

[0144] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific reference products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0145] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0146] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0147] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0148] The systems and technologies described herein can be implemented in computing systems that include back-end components (e.g., as data servers), or computing systems that include switching components (e.g., application servers), or computing systems that include front-end components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such back-end, switching, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0149] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0150] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0151] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting mirrored faces, characterized in that, The method includes: The system acquires an image set captured by a target image acquisition device, and determines the location information of similar faces based on the facial information of multiple reference images in the image set; wherein, the reference images include at least two faces with a similarity greater than a preset similarity threshold; Clustering the location information of similar faces yields at least one candidate reflector region in the field of view of the target image acquisition device; Determine the target reflector region from the at least one candidate reflector region; The mirrored face in the target image acquired by the target image acquisition device is determined based on the target reflector region. Wherein, determining the target reflector region from the at least one candidate reflector region includes: If the number of candidate reflector regions is 2, then the target reflector region is determined from the candidate reflector regions based on the face size information in the candidate reflector regions.

2. The method according to claim 1, characterized in that, Clustering the location information of similar faces to obtain at least one candidate reflector region in the field of view of the target image acquisition device includes: At least two clustering results are determined based on at least two clustering methods to identify the location information of the similar faces; The candidate reflector region is determined based on the degree of overlap between the at least two clustering results.

3. The method according to claim 2, characterized in that, in, The at least two clustering methods include a first clustering method and a second clustering method; the first clustering method is a clustering method based on the number of categories, and the second clustering method is a density-based clustering method; The at least two clustering results include a first clustering result and a second clustering result; Determining the candidate reflector region based on the overlap of the at least two clustering results includes: If the overlap between any cluster region in the first clustering result and any cluster region in the second clustering result exceeds the overlap threshold, then candidate reflector regions are determined based on the overlapping regions.

4. The method according to claim 1, characterized in that, Determining the target reflector region from the candidate reflector region based on the face size information in the candidate reflector region includes: The candidate reflector region with smaller face size information among the two candidate reflector regions in the reference image is determined as the target reflector region.

5. The method according to claim 1, characterized in that, When the distance between the mirrored image and the actual image is relatively close, the target reflector region is determined from the candidate reflector region based on the face size information in the candidate reflector region, including: The neighboring images corresponding to the reference image are determined, as well as the face orientation information in the reference image and the neighboring images; wherein the time interval between the reference image and the corresponding neighboring image is less than a preset time interval; and the neighboring images include at least two faces with a similarity greater than a preset similarity threshold; Based on the face orientation information, and according to the change information of face size in the two candidate reflector regions in the reference image and the adjacent image, the target reflector region is determined from the candidate reflector regions.

6. The method according to claim 5, characterized in that, Based on the face orientation information, and according to the change information of face size in two candidate reflector regions in the reference image and the adjacent image, the target reflector region is determined from the candidate reflector regions, including: The first candidate reflector region and the second candidate reflector region are determined based on the relative positions of the candidate reflector regions in the reference image and the adjacent image; Determine the first face size ratio between the first candidate reflector region and the second candidate reflector region in the reference image; Determine the ratio of the second face size between the first candidate reflector region and the second candidate reflector region in the adjacent images; Based on the face orientation information, the target reflector region is determined from the candidate reflector region according to the first face size ratio and the second face size ratio.

7. The method according to claim 6, characterized in that, Based on the face orientation information, the target reflector region is determined from the candidate reflector region according to the first face size ratio and the second face size ratio, including: If the face orientation information indicates that the face is facing the image acquisition device, and the first face size ratio is less than 1, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the first candidate reflector region is determined to be the target reflector region. If the face orientation information indicates that the face is facing the image acquisition device, and the first face size ratio is greater than 1, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the second candidate reflector region is determined as the target reflector region. If the face orientation information indicates that the face is facing away from the image acquisition device, and the first face size ratio is less than 1, and the ratio of the first face size ratio to the second face size ratio is greater than 1, then the first candidate reflector region is determined to be the target reflector region. If the face orientation information indicates that the face is facing away from the image acquisition device, and the first face size ratio is greater than 1, and the ratio of the first face size ratio to the second face size ratio is less than 1, then the second candidate reflector region is determined as the target reflector region.

8. A device for detecting mirrored faces, characterized in that, include: A similar face location determination module is used to acquire an image set acquired by a target image acquisition device, and determine similar face location information based on face information of multiple reference images in the image set; wherein, the reference images include at least two faces with a similarity greater than a preset similarity threshold; The candidate region determination module is used to cluster the similar face location information to obtain at least one candidate reflector region in the field of view of the target image acquisition device; A target region determination module is used to determine a target reflector region from the at least one candidate reflector region; A mirror face determination module is used to determine a mirror face in a target image acquired by the target image acquisition device based on the target reflector region. The target region determination module is specifically used for: If the number of candidate reflector regions is 2, then the target reflector region is determined from the candidate reflector regions based on the face size information in the candidate reflector regions.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the mirror face detection method according to any one of claims 1-7.

Images