Facial recognition methods for security checks and multi-dimensional security gates

Abstract

This invention provides a method for face matching of passersby during security checks and a multi-dimensional security gate. The method includes: acquiring a first image captured by a visible light camera and a second image captured by a depth camera, wherein the first and second images are captured within a preset time difference range; both the first and second images include at least one identical passerby; acquiring the coordinates of the first pixel point of the center point of each passerby's face in the first image; determining the coordinates of a second pixel point in the second image corresponding to the first pixel point coordinates based on a first mapping function; acquiring the depth value corresponding to each second pixel point coordinate in the second image; determining a target depth value based on the preset depth value and the depth values, and identifying the face in the second image corresponding to the target depth value as the target face; the target face is the face of a passerby currently passing through the security gate. This invention improves the accuracy of matching.

Description

Technical Field

[0001] This invention relates to the field of security inspection technology, and in particular to a method for matching the faces of passersby and a multi-dimensional security gate for security inspection. Background Technology

[0002] Security checks on personnel typically involve the use of equipment such as metal detectors. In some security scenarios, multi-dimensional metal detectors are used. These detectors have multiple detection devices, enabling them to perform multi-dimensional checks and output various types of security results, such as the amount of metal carried and the individual's temperature.

[0003] When a multi-dimensional security gate outputs its current security check result, it matches this result with the image of the person to be checked captured by the gate, identifying the person most likely to be currently passing through the gate. However, the image of the person to be checked captured by the multi-dimensional security gate may include not only those currently passing through, but also those about to pass through, and those who have already passed through, as well as other individuals not currently undergoing security checks. Therefore, identifying the person currently passing through the multi-dimensional security gate from the image of the person to be checked becomes a pressing problem to solve.

[0004] In related technologies, the proportion of pixel area occupied by each face in the image of the person to be inspected is used to determine the person passing through the multi-dimensional security gate. Specifically, the person corresponding to the face with the largest pixel area is identified as the person passing through the multi-dimensional security gate. However, factors such as differences in face size or the distance between the face and the camera lens can affect the proportion of pixel area occupied by the face in the image. Therefore, the accuracy of determining the person passing through the multi-dimensional security gate based on the proportion of pixel area occupied by the face in the image of the person to be inspected is relatively low. Summary of the Invention

[0005] This invention provides a method for face matching of passersby for security checks and a multi-dimensional security gate, which solves the problem of low accuracy in determining the results of people passing through multi-dimensional security gates in the prior art, and achieves the goal of improving the accuracy of the determination results.

[0006] This invention provides a method for facial matching of passersby for security checks, applied to a multi-dimensional security gate. The method includes:

[0007] The system acquires a first image captured by a visible light camera and a second image captured by a depth camera. The first and second images are captured within a preset time difference. Both the first and second images include the same at least one person passing through. The at least one person passing through includes at least one of the following: a person about to pass through the multi-dimensional security gate, a person currently passing through the gate, and a person who has already passed through the gate.

[0008] Obtain the coordinates of the first pixel of the center point of the face of each person passing through in the first image;

[0009] For each first pixel coordinate, the coordinates of the second pixel in the second image corresponding to the first pixel coordinates are determined based on the first mapping function.

[0010] Obtain the depth value corresponding to the coordinates of each second pixel in the second image;

[0011] The target depth value is determined based on the preset depth value and each depth value, and the face in the second image corresponding to the target depth value is identified as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person being detected in the gate and the depth camera device; the target face is the face of the person passing through the gate.

[0012] According to the present invention, a method for matching faces of passersby for security checks determines a target depth value based on a preset depth value and various depth values, including: comparing the absolute difference between each depth value and the preset depth value, and determining the depth value corresponding to the smallest absolute difference as the target depth value.

[0013] According to the present invention, a method for matching the faces of passersby for security checks compares the absolute difference between each depth value and a preset depth value, including: obtaining the target height information of the person to be detected located inside the detection gate by the distance detection module; determining the target preset depth value corresponding to the target height information based on the correspondence between the height information and the preset depth value; and comparing the absolute difference between each depth value and the target preset depth value.

[0014] According to the present invention, a method for matching the faces of passersby for security checks further includes: acquiring a third image captured by an infrared temperature measuring camera; the third image and the first image are captured within a preset time difference range; both the third image and the first image include at least one identical passerby; determining the coordinates of a third pixel point in the third image corresponding to the coordinates of a target pixel point based on a second mapping function; the coordinates of the target pixel point are the pixel coordinates of the center point of the target face in the first image; acquiring the temperature value corresponding to the coordinates of the third pixel point in the third image; and associating the temperature value with the identifier of the target face.

[0015] According to the present invention, a method for matching the faces of passersby for security checks further includes: acquiring the amount of metal carried by passersby passing through the detection gate as detected by a metal quantity detection module; and associating temperature values ​​with the identifier of the target face, including associating both temperature values ​​and metal quantity with the identifier of the target face.

[0016] According to the present invention, a method for face matching of passersby for security checks is provided, which identifies the face in a second image corresponding to a target depth value as the target face. The method includes: for each depth value, subtracting a preset depth value from the depth value to obtain a first difference corresponding to the depth value; if each first difference includes a first difference less than zero, removing the first differences less than zero from each first difference to obtain a set of differences; determining the first face in the second image corresponding to each first difference in the set of differences; determining the pixel area occupied by each first face in the second image; and if the face with the largest pixel area is determined to be the face in the second image corresponding to the target depth value, then identifying the face in the second image corresponding to the target depth value as the target face.

[0017] According to the present invention, a method for matching faces of persons passing through security checks further includes: if it is determined that the face with the largest pixel area is not the face in the second image corresponding to the target depth value, displaying the face in the second image corresponding to the target depth value and the face with the largest pixel area; determining the target face based on the selection instruction input by the person passing through; the selection instruction is used to instruct the person passing through to select the face from the face in the second image corresponding to the target depth value and the face with the largest pixel area.

[0018] According to the present invention, a method for matching the faces of passersby for security checks is provided. The first mapping function includes a first horizontal mapping function and a first vertical mapping function. The first horizontal mapping function is represented by formula (1), and the first vertical mapping function is represented by formula (2).

[0019] f(x) = x × r x +d x (1)

[0020] f(y) = y × r y +d y (2)

[0021] Where f(x) represents the first horizontal mapping function, f(y) represents the first vertical mapping function, x represents the horizontal coordinate value of any pixel in the first image, y represents the vertical coordinate value of any pixel in the first image, and r x r represents the horizontal ratio between the first and second images. y d represents the vertical ratio between the first and second images. xIndicates the horizontal offset value, d y This represents the vertical offset value.

[0022] According to the present invention, a method for face matching of passersby for security checks is provided, which acquires a first image captured by a visible light camera and a second image captured by a depth camera, comprising: upon receiving first information sent by a human perception detection module, controlling the visible light camera and the depth camera to capture images to obtain the first image captured by the visible light camera and the second image captured by the depth camera; the first information is used to characterize the detected passersby.

[0023] This invention also provides a multi-dimensional security gate, comprising: a detection gate body, including two oppositely arranged side door panels and a top panel connected to the two side door panels; a connecting rod, one end of which is connected to the top panel, and the other end of which is equipped with a visible light camera, a depth camera, and an infrared temperature measurement camera; a metal detection module, disposed within the two side door panels; a human body detection module, disposed within the top panel; and a processing module, connected to the visible light camera, the depth camera, the infrared temperature measurement camera, the metal detection module, and the human body detection module; and a human body detection... The detection module sends first information to the processing module when a person to be detected is detected in the detection gate; the first information indicates that a person to be detected has been detected. The metal quantity detection module detects the amount of metal carried by the person passing through the detection gate and sends the amount of metal carried to the processing module. The processing module, upon receiving the first information from the human perception detection module, controls a visible light camera, a depth camera, and an infrared thermometer to take pictures, obtaining a first image taken by the visible light camera, a second image taken by the depth camera, and a third image taken by the infrared thermometer. The first, second, and third images are all taken within a preset time difference range. Each of the first, second, and third images includes at least one identical person. The processing module also obtains the first pixel coordinates of the center point of the face of each person in the first image, and determines the second pixel coordinates corresponding to the first pixel coordinates in the second image based on a first mapping function. It obtains the depth value corresponding to each second pixel coordinate in the second image, determines the target depth value based on the preset depth value and each depth value, and sets the target depth value accordingly. The face corresponding to the target depth value in the second image is identified as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person being detected in the gate and the depth camera device; the target face is the face of the person passing through the gate; the processing module is also used to determine the coordinates of the third pixel point corresponding to the target pixel point coordinates in the third image based on the second mapping function, obtain the temperature value corresponding to the third pixel point coordinates in the third image, and associate the temperature value and the amount of metal carried with the identifier of the target face; the target pixel point coordinates are the pixel coordinates of the center point of the target face in the first image.

[0024] The present invention also provides a face matching device for security checks, the device comprising: an acquisition unit for acquiring a first image captured by a visible light camera and a second image captured by a depth camera, the first image and the second image being acquired within a preset time difference range; both the first image and the second image include the same at least one person passing through; the at least one person passing through includes at least one of: a person about to pass through the detection gate of a multi-dimensional security gate, a person currently passing through the detection gate, and a person who has already passed through the detection gate; the acquisition unit is further configured to acquire the face center of each person passing through in the first image. The system includes: a first pixel coordinate unit; a determining unit, configured to determine the second pixel coordinates in the second image corresponding to the first pixel coordinates based on a first mapping function; an acquisition unit, further configured to acquire the depth value corresponding to each second pixel coordinate in the second image; and a determining unit, further configured to determine a target depth value based on a preset depth value and each depth value, and to identify the face in the second image corresponding to the target depth value as the target face. The preset depth value is used to characterize the distance between the center point of the face of the person being detected in the gate and the depth camera device. The target face is the face of the person passing through the gate.

[0025] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described methods for face matching of persons for security checks.

[0026] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above-described methods for face matching of persons for security checks.

[0027] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described methods for face matching of persons for security checks.

[0028] This invention provides a method for face matching of passersby for security checks and a multi-dimensional security gate. The method, based on a first mapping function, determines the second pixel coordinates in a second image captured by a depth camera, corresponding to the first pixel coordinates of the center point of each passerby's face in a first image captured by a visible light camera. In the second image, the depth value corresponding to each second pixel coordinate is obtained. A target depth value is determined based on a preset depth value and the depth values ​​corresponding to each second pixel coordinate. The face in the second image corresponding to the target depth value is then identified as the target face, which is the face of the passerby passing through the multi-dimensional security gate. The depth value corresponding to the second pixel coordinate reflects the distance between the center point of each passerby's face and the depth camera; therefore, the distance between the center point of each passerby's face and the depth camera can be determined using the depth value corresponding to each second pixel coordinate. The target depth value is determined based on a preset depth value and the depth values ​​corresponding to the coordinates of each second pixel. Specifically, the face of a person passing through the multi-dimensional security gate is determined based on a preset distance value and the distance between the center point of each person's face and the depth camera device. This invention utilizes the depth values ​​of each second pixel to determine the distance between the face of a person passing through the multi-dimensional security gate and the depth camera device, thus improving the accuracy of the target face determined by the distance value. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the face matching method for security checks provided in an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of a security inspection scenario provided in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram illustrating the determination of a preset depth value provided in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram illustrating the mapping relationship between the first image and the second image provided in an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the structure of the multi-dimensional security gate provided in an embodiment of the present invention;

[0035] Figure 6 This is a schematic diagram of the structure of the face matching device for security checks provided in an embodiment of the present invention;

[0036] Figure 7 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. It should be noted that the serial numbers assigned to the objects described in this invention, such as "first," "second," etc., are only used to distinguish the described objects and have no sequential or technical meaning.

[0038] In recent years, with the vigorous development of urban rail transit, security gates, as important security inspection equipment for entering subway and other rail transit venues, have been used more and more widely. At the same time, with the rapid development of technology, intelligent security gates integrating multiple detection functions are becoming more and more common. In particular, multi-dimensional security gates that integrate multiple data sources such as facial recognition, expression recognition, and infrared temperature measurement are becoming more and more common. However, in scenarios with low cooperation and large passenger flow, the data such as metal detection results and facial recognition results are either independent of each other or cannot be accurately fused and correlated, resulting in false alarms or missed alarms by security gates. This makes it impossible to detect potential dangerous people as early as possible and implement corresponding passage control.

[0039] In scenarios with low cooperation and high passenger flow, the queuing of people waiting to be inspected through security gates can lead to multiple faces appearing simultaneously in a single image frame at the gate location. Facial recognition of this frame will then produce multiple results, potentially preventing a one-to-one correspondence between the face and the multiple data sources, such as metal detection results, facial expression recognition results, and body temperature detection results. Therefore, achieving a unique and accurate match between metal detection results, facial expression recognition results, body temperature detection results, and multiple facial images is crucial for accurate security checks.

[0040] Current security gates typically select the face image with the largest pixel area from multiple captured facial images and correlate it with the actual person passing through the gate at that moment. However, in reality, factors such as height, age, and face size can cause the selected facial image to not match the person passing through the gate at that exact moment. This can lead to errors in the correlation of data such as metal detection results, captured facial images, expression recognition results, and body temperature detection results. Consequently, when the security gate alarms, the facial image displayed may not be the person actually passing through the gate, but rather someone who passed before or after them, resulting in a false alarm.

[0041] To address this problem, embodiments of the present invention provide a method for face matching of persons passing through security checks. This method utilizes the depth values ​​contained in each pixel of a depth image, and determines the target face based on a preset depth value and the depth values. This target face is the face of the person passing through the security gate, which can improve the accuracy of the determination result.

[0042] The following is combined with Figures 1 to 4 This invention describes a method for facial recognition of persons undergoing security checks, provided in an embodiment of the invention. The execution subject of this method can be an electronic device such as a computer, server, or server cluster, or a specially designed intelligent device, or a facial recognition device installed within such an electronic or intelligent device. This facial recognition device can be implemented through software, hardware, or a combination of both. This method can be applied to any scenario requiring facial recognition of persons undergoing security checks, such as security checks at subway entrances, airport entrances, or building access points. Using this method can improve the accuracy and efficiency of personnel security checks.

[0043] Figure 1 This is a flowchart illustrating a face matching method for security checks provided in an embodiment of the present invention. This method is applied to a multi-dimensional security gate. Figure 1 As shown, the method includes steps 110 to 150 as follows.

[0044] Step 110: Acquire a first image captured by a visible light camera and a second image captured by a depth camera. The first and second images are captured within a preset time difference range. Both the first and second images include the same at least one person passing through. The at least one person passing through includes at least one of the following: a person about to pass through the multi-dimensional security gate, a person passing through the gate, and a person who has already passed through the gate.

[0045] For example, a multi-dimensional security gate may include a visible light camera and a depth camera. The visible light camera may be, for example, a visible light camera, and the depth camera may be, for example, a 3D structured light depth camera. By capturing images of a person passing through using the visible light camera, a first image can be obtained; this first image is a visible light frame image. By capturing images of the person passing through using the depth camera, a second image can be obtained; this second image is a depth image containing depth information.

[0046] The first and second images are images captured within a preset time difference range, where the preset time difference range is a predetermined interval. When capturing images of the same person passing by, the time interval between the capture time of the visible light camera and the capture time of the depth camera must be within this preset time difference range, which can be, for example, from 0 milliseconds (ms) to 100 ms.

[0047] Both the first and second images captured within a preset time difference range include the same at least one person passing through. This at least one person may include someone currently passing through the multi-dimensional security gate, meaning someone undergoing security screening inside the gate at the current moment; someone about to pass through the gate, meaning someone who has not yet undergone security screening; and someone who has already passed through the gate, meaning someone who passed through and underwent security screening at a time prior to the current moment. After steps 110 to 150 of this embodiment, the person currently passing through the multi-dimensional security gate can be identified from among the people in the first and second images.

[0048] Figure 2 This is a schematic diagram of a security inspection scenario provided by an embodiment of the present invention. In a scenario with low cooperation and large passenger flow, there are many people passing through. When each person passes through the multi-dimensional security gate, they need to queue for security inspection. During the queuing security inspection, the image captured by the camera may show multiple people passing through at the same time. Figure 2As shown, individuals A, B, C, and D line up sequentially to pass through a multi-dimensional security gate for security checks. Individual A has already passed through gate 1, individual B is currently passing through gate 1, and individuals C and D are about to pass through gate 1. The multi-dimensional security gate is connected to a visible light camera and a depth camera via a connecting rod 3. These two cameras can be integrated into a camera assembly 2, which is then connected to the connecting rod 3, or they can be two independent cameras. The visible light camera and the depth camera capture images to obtain a first image and a second image, both of which include at least one of individuals A, B, C, and D.

[0049] Step 120: Obtain the coordinates of the first pixel of the center point of the face of each person passing through in the first image.

[0050] For example, a pixel coordinate system is established in the first image, with pixels as the unit. The coordinates of the first pixel of the center point of each person's face can be determined by analyzing the facial pixels of each person in the first image. For instance, the average of the maximum and minimum values ​​of the x-coordinates of the pixels in each person's face is calculated to obtain the average x-coordinate. The average of the maximum and minimum values ​​of the y-coordinates of the pixels in each person's face is calculated to obtain the average y-coordinate. The center point coordinates are then formed using the average x-coordinate and the average y-coordinate, and these center point coordinates are determined as the coordinates of the first pixel of the center point of that person's face.

[0051] Step 130: For each first pixel coordinate, determine the coordinates of the second pixel in the second image corresponding to the first pixel coordinates based on the first mapping function.

[0052] For example, a pixel coordinate system is established in the second image, with each pixel representing its position. The first mapping function is used to map the first pixel coordinates to the second pixel coordinates. After obtaining the first pixel coordinates of the center point of each person's face in the first image, the first mapping function can be used to obtain the corresponding pixel coordinates in the second image, i.e., the coordinates of each second pixel.

[0053] Step 140: Obtain the depth value corresponding to the coordinates of each second pixel point in the second image.

[0054] For example, the second image is an image obtained by capturing a person passing by using a depth camera. Therefore, each pixel in the second image has depth information, i.e., a depth value. After determining the coordinates of the second pixel corresponding to the coordinates of the first pixel in the second image using the first mapping function, the depth information of each second pixel can be directly obtained, thus obtaining the depth value corresponding to the coordinates of each second pixel. This depth value is the distance between the center point of the person's face and the depth camera.

[0055] Step 150: Determine the target depth value based on the preset depth value and each depth value, and identify the face in the second image corresponding to the target depth value as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person being detected in the gate and the depth camera device; the target face is the face of the person passing through the gate.

[0056] For example, the preset depth value is a preset benchmark used to determine whether the depth value corresponding to the coordinates of each second pixel point meets preset conditions. Based on the preset depth value and the depth value corresponding to the coordinates of each second pixel point, the depth value that meets the preset conditions can be determined from the depth values ​​corresponding to the coordinates of each second pixel point, and the depth value that meets the preset conditions is determined as the target depth value. Further, after determining the target depth value, the face corresponding to the second pixel coordinates of the target depth value is determined as the target face, which is the face of the person passing through the detection gate. The face of the person passing through the detection gate is used as the target face when it is electronically captured in the first image and the second image.

[0057] The preset conditions can be pre-defined conditions for determining the target depth value using a preset depth value and other depth values. For example, the preset conditions can be to determine the depth value that is equal to the preset depth value among all depth values ​​as the target depth value; or, the preset conditions can be to calculate the difference between each depth value and the preset depth value, and determine the depth value whose difference is less than a set threshold as the target depth value.

[0058] The preset depth value can be determined based on specific application scenarios or parameters of multidimensional security gates.

[0059] Alternatively, the preset depth value can be determined by measuring the distance between a calibrated person and the depth camera. The calibrated person can be a test subject of known height. Figure 3 This is a schematic diagram illustrating the determination of a preset depth value provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the calibrated person stands in a normal upright posture inside the detection gate 1 of the multi-dimensional security gate. The distance d between the depth camera in the camera assembly 2 and the center point of the calibrated person's face is measured. s This will allow you to measure the distance d. sThe preset depth value has been determined.

[0060] like Figure 3 As shown, the camera assembly 2 can be fixedly installed on the top of the detection gate 1 via the connecting rod 3. After the depth camera in the camera assembly 2 is installed and fixed, the distance d between the depth camera and the center point of the face of the person being calibrated is... s The distance d is related to the height of the person being marked for passage; the taller the person being marked for passage, the greater the distance d. s The smaller the value, the lower the height of the person being calibrated, and the greater the distance d. s The larger the value, the better. The height of the person being calibrated can be selected from any height range, such as between 165 cm and 175 cm, so that the corresponding preset depth value or the corresponding preset depth value range can be obtained.

[0061] Optionally, any one of the preset depth values ​​in the preset depth value range can be determined as the preset depth value, or the average value of all preset depth values ​​in the preset depth value range can be determined as the preset depth value.

[0062] For example, a multi-dimensional security gate is fixedly connected to a camera assembly 2 via a connecting rod 3. The camera assembly 2 includes a 3D structured light depth camera. The distance between the 3D structured light depth camera and the multi-dimensional security gate can be arbitrarily set, for example, by selecting any value between 100cm and 200cm. Multiple calibrated individuals within a height range are selected to determine the preset depth value range. For example, multiple calibrated individuals with heights between 165cm and 175cm are selected, and values ​​within the preset depth value range are measured. The selected calibrated individuals stand in a normal upright posture inside the multi-dimensional security gate, with the center point of their feet coinciding with the center points of their feet along both the length and width directions. When the calibrated individuals are facing the 3D structured light depth camera, the straight-line distance between the center point of their face and the 3D structured light depth camera is determined as a value within the preset depth value range. The preset depth value can be selected from the values ​​within this range, or the average of all values ​​within the preset depth value range can be used as the preset depth value.

[0063] This invention provides a method for matching faces of people passing through security checks, applied to multi-dimensional security gates. Based on a first mapping function, the method determines the second pixel coordinates in a second image captured by a depth camera, corresponding to the first pixel coordinates of the center point of each person's face in a first image captured by a visible light camera. In the second image, the depth value corresponding to each second pixel coordinate is obtained. A target depth value is determined based on the preset depth value and the depth values ​​corresponding to each second pixel coordinate. The face in the second image corresponding to the target depth value is then identified as the target face, which is the face of the person passing through the multi-dimensional security gate. The depth value corresponding to the second pixel coordinate reflects the distance between the center point of each person's face and the depth camera; therefore, the distance between the center point of each person's face and the depth camera can be determined using the depth value corresponding to each second pixel coordinate. The target depth value is determined based on a preset depth value and the depth values ​​corresponding to the coordinates of each second pixel. Specifically, the face of a person passing through the multi-dimensional security gate is determined based on a preset distance value and the distance between the center point of each person's face and the depth camera device. This invention utilizes the depth values ​​of each second pixel to determine the distance between the face of a person passing through the multi-dimensional security gate and the depth camera device, thus improving the accuracy of the target face determined by the distance value.

[0064] In one example embodiment, determining a target depth value based on a preset depth value and various depth values ​​includes: comparing the absolute difference between each depth value and the preset depth value, and determining the depth value corresponding to the smallest absolute difference as the target depth value.

[0065] For example, after obtaining the depth value corresponding to the coordinates of each second pixel, the difference between the preset depth value and each depth value is calculated, and the absolute value of each difference is taken to obtain the absolute difference between each depth value and the preset depth value. The minimum value is determined from each absolute difference, and the depth value corresponding to the minimum absolute difference is determined as the target depth value.

[0066] For example, a visible light camera and a depth camera can capture images to obtain a first image and a second image, respectively. Based on a first mapping function, the coordinates of a second pixel in the second image corresponding to the coordinates of the first pixel are determined. Then, the depth value corresponding to each second pixel coordinate is obtained in the second image. Each depth value represents the distance between the center point of each person's face in the first image and the depth camera. Figure 2 As shown in the image, the distances d1, d2, d3, and d4 between the center points of the faces of persons A, B, C, and D and the depth camera device can be obtained using the second image. If the preset depth value is d... sThe absolute difference between each depth value and the preset depth value can be determined using the following formula (3).

[0067] D(n)=|d s -d n | (3)

[0068] Where, d n This represents the depth value corresponding to the coordinates of each second pixel, i.e., the distance between the center point of each person's face and the depth camera device; d s D(n) represents the preset depth value, and D(n) represents the absolute difference between the current depth value and the preset depth value.

[0069] Substitute the distances d1, d2, d3, and d4 between the center points of the faces of persons A, B, C, and D and the depth camera into formula (3) and iterate through the data to obtain the corresponding absolute differences D(1), D(2), D(3), and D(4). From D(1), D(2), D(3), and D(4), the depth value corresponding to the smallest absolute difference is determined as the target depth value.

[0070] In this embodiment, by comparing the absolute difference between each depth value and the preset depth value, the depth value corresponding to the smallest absolute difference is determined as the target depth value. That is, among the distances between the center point of each person's face and the depth camera device, the depth value closest to the preset depth value is determined as the target depth value, which can improve the accuracy of the determined target depth value, and thus improve the accuracy of the determined target face.

[0071] Since the distance between the depth camera and the center point of the face of the person passing by is related to the height of the person passing by, the taller the person is, the closer the face is to the depth camera, and the shorter the person is, the farther the face is from the depth camera. Therefore, determining the target depth value based on a fixed preset depth value may affect the accuracy of the determined target depth value. Therefore, the accuracy of the determined target depth value can be improved by adjusting the preset depth value in real time.

[0072] In one example embodiment, comparing the absolute difference between each depth value and a preset depth value may include: obtaining the target height information of the person to be detected located inside the detection gate, detected by the distance detection module; determining the target preset depth value corresponding to the target height information based on the correspondence between the height information and the preset depth value; and comparing the absolute difference between each depth value and the target preset depth value.

[0073] For example, a distance detection module is set up to detect the height information of a person being detected inside the detection gate. A pre-defined correspondence between height information and pre-defined depth values ​​can be established; for example, multiple height values ​​can be matched one-to-one with multiple pre-defined depth values. After obtaining the height information of the person being detected inside the detection gate detected by the distance detection module, this height information is the target height information. The target pre-defined depth value corresponding to this target height information can be determined through the correspondence. When comparing the absolute differences between each depth value and the pre-defined depth value, the target pre-defined value is determined as the pre-defined depth value and compared with each depth value. This achieves the goal of adjusting the pre-defined depth value in real time according to the different height information of the person being detected. Here, the person being detected can be understood as someone passing through the detection gate of the multi-dimensional security gate.

[0074] Optionally, the distance detection module can be installed on the top inside of the multidimensional security gate or in another location that facilitates the detection of the height information of the person to be detected.

[0075] In this embodiment, the target height information of the person to be detected is detected by the distance detection module, and the target preset depth value corresponding to the target height information is determined based on the correspondence between the height information and the preset depth value. By comparing the target preset depth value with the absolute difference of each depth value, the preset depth value can be adjusted according to the height information of the person to be detected, reducing the error caused by the height difference of the people passing by, making the determined target depth value more accurate, and further improving the accuracy of the determined target face.

[0076] In some scenarios, it is necessary to determine the temperature of people passing through. However, in related technologies, due to factors such as a large number of people, low cooperation, or inaccurate matching methods, the determined temperature value cannot be correctly matched with the person passing through the multi-dimensional security gate, resulting in false alarms. The method provided in this invention uses an infrared temperature measurement camera to capture images, and based on the third image captured by the infrared temperature measurement camera, the temperature value is correctly matched with the person passing through the multi-dimensional security gate.

[0077] In one example embodiment, a third image captured by an infrared temperature measurement camera is acquired; the third image and the first image are captured within a preset time difference range; both the third image and the first image include the same at least one person passing by; the coordinates of a third pixel in the third image corresponding to the coordinates of a target pixel are determined based on a second mapping function; the coordinates of the target pixel are the pixel coordinates of the center point of the target face in the first image; the temperature value corresponding to the coordinates of the third pixel is acquired in the third image; and the temperature value is associated with the identifier of the target face.

[0078] For example, an infrared temperature measurement camera can be an infrared thermal imaging device, which captures an infrared temperature measurement image. Each pixel in the infrared temperature measurement image contains temperature information corresponding to a point on the imaged object. In other words, the temperature value of the imaged object corresponding to each pixel in the image can be determined through the image captured by the infrared temperature measurement camera. A multi-dimensional security gate can include an infrared temperature measurement camera. By capturing images of people passing through the gate, a third image can be obtained; this third image is an infrared temperature measurement image.

[0079] The third image and the first image are images captured within a preset time difference range, which is the same as the preset time difference range of the first image and the second image mentioned above. Both the third image and the first image captured within the preset time difference range include the same at least one person passing through. This at least one person includes at least one of the following: a person about to pass through the multi-dimensional security gate, a person currently passing through the gate, and a person who has already passed through the gate.

[0080] For example, in conjunction with the above embodiments, when the preset time difference is 0ms, it can be understood that the visible light camera, the depth camera, and the infrared temperature measurement camera simultaneously capture images, obtaining corresponding first, second, and third images. Each of the first, second, and third images includes at least one identical person passing through. When the face of a person passing through the detection gate is electronically captured in the first, second, and third images, it is considered the target face.

[0081] Once the target face is identified in the second image, the coordinates of the second pixel corresponding to the target face can be determined, and the coordinates of the first pixel corresponding to the second pixel are the coordinates of the target pixel.

[0082] Similarly, a pixel coordinate system is established in the third image, using pixels as the unit. This coordinate system allows us to determine the position of each pixel in the third image. The second mapping function maps the coordinates of the first pixel (the target pixel) in the first image to the corresponding coordinates of the third pixel in the third image.

[0083] The third image is an infrared thermographic image. Each pixel in the third image contains the temperature value of the corresponding point on the imaged object. This temperature value is the temperature value corresponding to the pixel, or the temperature value corresponding to the coordinates of that pixel. After determining the coordinates of the third pixel in the third image, the temperature value corresponding to those coordinates can be determined. This temperature value can characterize the temperature of the person corresponding to the target face in the second image; that is, this temperature value can characterize the temperature of the person passing through the detection gate of the multi-dimensional security gate.

[0084] The temperature value is associated with the identifier of the target face. This identifier can be any information representing the target face, such as a face ID or name. This identifier helps distinguish between different individuals passing through the security gate. Associating the temperature value with the target face identifier ensures accurate matching between the determined temperature and the person passing through the multi-dimensional security gate, preventing false alarms.

[0085] In this embodiment, a third image captured by an infrared temperature measurement camera is acquired, the temperature value of the target face is determined in the third image, and the temperature value is associated with the identifier of the target face. This can intuitively represent the temperature value of the person passing through the multi-dimensional security gate, and accurately associate the person's temperature information currently output by the multi-dimensional security gate with the person currently undergoing security detection through the multi-dimensional security gate.

[0086] In some scenarios, it is necessary to determine the amount of metal carried by individuals. However, in related technologies, due to factors such as a large number of individuals, low cooperation levels, or inaccurate matching methods, the determined metal amount may not be correctly matched with the individual passing through the multi-dimensional security gate, resulting in false alarms. The method provided in this invention uses a metal detection module to detect the metal amount and accurately match it with the individual passing through the multi-dimensional security gate.

[0087] In one example embodiment, the amount of metal carried by a person passing through the detection gate is obtained by the metal quantity detection module; associating the temperature value with the identifier of the target face includes associating both the temperature value and the amount of metal carried with the identifier of the target face.

[0088] For example, a metal detection module is used to detect the amount of metal carried by a person passing through the detection gate. The metal detection module can be installed on the multi-dimensional security gate, for example, inside the gate's detection area. When a person is inside the gate undergoing security screening, the amount of metal carried by that person can be obtained. By associating this amount with the identifier of the target face, the determined amount of metal carried can be correctly matched with the person passing through the multi-dimensional security gate, avoiding false alarms.

[0089] In this embodiment, the temperature value and metal carrying amount of the person passing through are associated with the identifier of the target face. This can correctly match the temperature value and metal carrying amount with the person passing through the multi-dimensional security gate, thereby improving the accuracy of the association of multi-dimensional data results.

[0090] Based on the above embodiments, in some scenarios, it is also necessary to associate at least one of the face recognition results, expression recognition results, and face occlusion recognition results with the identifier of the target face to achieve more-dimensional data results and accurate matching with the people passing through the multi-dimensional security gate.

[0091] In one example embodiment, the method further includes at least one of the following: inputting the target face into an identity recognition module to obtain a face recognition result output by the identity recognition module; inputting the target face into an expression recognition module to obtain an expression recognition result output by the expression recognition module; inputting the target face into a face occlusion recognition module to obtain a face occlusion recognition result output by the face occlusion recognition module; and associating both the temperature value and the amount of metal carried with the identifier of the target face, including: associating at least one of the face recognition result, the expression recognition result, and the face occlusion recognition result, the temperature value, and the amount of metal carried with the identifier of the target face.

[0092] The process involves associating at least one of the following: face recognition results, expression recognition results, and facial occlusion recognition results, along with temperature and metal carrying capacity, with the target face identifier. This includes the following scenarios: First, associating the face recognition result, temperature, and metal carrying capacity with the target face identifier. Second, associating the face recognition result, expression recognition result, temperature, and metal carrying capacity with the target face identifier. Third, associating the face recognition result, expression recognition result, facial occlusion recognition result, temperature, and metal carrying capacity with the target face identifier.

[0093] For example, the identity recognition module can be any module that outputs face recognition results, such as being composed of at least one of a face recognition device, a face recognition program, or a face recognition network model. The expression recognition module can be any module that outputs expression recognition results, such as being composed of at least one of an expression recognition device, an expression recognition program, or an expression recognition network model. The face occlusion recognition module can be any module that outputs face occlusion recognition results, such as being composed of at least one of a face occlusion recognition device, a face occlusion recognition program, or a face occlusion recognition network model. The face occlusion can be, for example, an object such as a mask, face shield, or glasses.

[0094] For example, F-shaped target face X By inputting the data into the identity recognition module, expression recognition module, and facial occlusion recognition module respectively, the face recognition result R can be obtained. F Facial expression recognition results R E Face occlusion recognition results R mask The determined temperature value is R. T The metal carrying capacity is Rmetal These data can form a dataset R, R = (R... F R E R mask R T R metal By associating the dataset R with the identifiers of the target faces, the fusion and association of multidimensional data can be achieved.

[0095] In this embodiment, at least one of the following—face recognition results, expression recognition results, and facial occlusion recognition results—along with the temperature value and the amount of metal carried, is associated with the identifier of the target face. This allows for accurate matching of the data with the person passing through the multi-dimensional security gate, improving the accuracy of the association between the multi-dimensional data results.

[0096] To further improve the accuracy of target face identification, the face in the second image corresponding to the target depth value can be identified as the target face by comparing the pixel area occupied by the first face in the second image under certain conditions.

[0097] In one example embodiment, determining the face in the second image corresponding to the target depth value as the target face includes: for each depth value, subtracting a preset depth value from the depth value to obtain a first difference corresponding to the depth value; if each first difference includes a first difference less than zero, removing the first differences less than zero from each first difference to obtain a set of differences; determining the first face in the second image corresponding to each first difference in the set of differences; determining the pixel area occupied by each first face in the second image; and if the face with the largest pixel area is determined to be the face in the second image corresponding to the target depth value, determining the face in the second image corresponding to the target depth value as the target face.

[0098] For example, the first difference is the value obtained by subtracting a preset depth value from the depth value, that is, the distance between the center point of each person's face and the depth camera device, minus the preset depth value. Each first difference reflects the relationship between the distance between the center point of each person's face and the depth camera device and the preset depth value. Each first difference corresponds one-to-one with each depth value.

[0099] If the first difference is less than zero, it indicates that the depth value corresponding to the first difference is less than a preset depth value, reflecting that the distance between the center point of the person's face and the depth camera device is less than a preset distance. In this case, the person has passed through the detection gate. If each first difference includes a value less than zero, the first differences less than zero are removed from the set of differences, resulting in a set of differences. The depth values ​​corresponding to each first difference in this set are the depth values ​​corresponding to the person currently passing through the detection gate and the person about to pass through the multi-dimensional security gate. Identifying the target face in this set will improve the accuracy of the determination result.

[0100] For example, such as Figure 2 As shown, person A has passed through the detection gate. The distance d1 between person A's face center point and the depth camera is smaller than the distance d2 between person B's face center point and the depth camera. If the distance d2 between person B, who is currently passing through the detection gate, and the depth camera is equal to the preset depth value d... s When the distance between the center point of person A's face and the depth camera is d1 minus the preset depth value d, the distance between them is calculated. s The first difference obtained is less than zero. Therefore, whether the first difference is less than zero can be used to determine whether the person passing through has passed through the gate.

[0101] The first face is the face of each person passing through the scene corresponding to the first difference in the difference set. In the second image, the pixel area occupied by each first face is determined. Determining the pixel area of ​​each first face can be done by: for each first face, first determining the number of pixels in the first face, and then multiplying the unit area of ​​each pixel by the number of pixels in the first face to obtain the pixel area of ​​the first face. The pixel area occupied by the first face can reflect, to some extent, the distance between the person's face and the depth camera. For example, if two people passing through have the same face area, the person closer to the depth camera will have a relatively larger pixel area in the image.

[0102] If the face with the largest pixel area is determined to be the face in the second image corresponding to the target depth value, then the face in the second image corresponding to the target depth value is identified as the target face. This can be understood as follows: after obtaining the pixel area of ​​each first face, the pixel areas of each first face are compared; when the depth value corresponding to the first face with the largest pixel area is also the target depth value, then that first face can be identified as the target face.

[0103] In this embodiment, the first difference can be used to exclude people who have passed through the detection gate. Then, by using the pixel area size and target depth value of each first face in the difference set, the target face can be determined. The accuracy of the target face determined by this method will be further improved.

[0104] Based on the above embodiments, in order to eliminate erroneous identification results caused by differences in the size of the face body among the various passers-by, the face corresponding to the target depth value and the face with the largest pixel area can be displayed. The selection command input by the security personnel can further determine the target face, thereby eliminating erroneous identification results and improving accuracy.

[0105] In one example embodiment, the method further includes: if it is determined that the face with the largest pixel area is not the face in the second image corresponding to the target depth value, displaying the face in the second image corresponding to the target depth value and the face with the largest pixel area; determining the target face based on the selection instruction input by the person passing through; the selection instruction is used to instruct the person passing through to select the face from the face in the second image corresponding to the target depth value and the face with the largest pixel area.

[0106] For example, in the case where the face with the largest pixel area is not the same face in the second image corresponding to the target depth value, that is, the face with the largest pixel area and the face corresponding to the target depth value in the second image are not the same face, this situation is very likely due to the difference in the size of the faces of the people passing by. For example, it can be understood as the case where a person standing outside the preset depth value has a larger face, and this person's face occupies the largest pixel area in the second image. In this case, the face corresponding to the target depth value and the face with the largest pixel area in the second image are displayed. For example, the images of these two faces can be displayed simultaneously.

[0107] The selection command can be entered by security personnel to select between the face corresponding to the target depth value and the face with the largest pixel area in the second image. For example, the selection command could be clicking to select the two displayed face images mentioned above.

[0108] In this embodiment, security personnel select the displayed faces and input a selection command. Based on this selection command, the target face can be determined with extremely high accuracy, which can greatly improve the accuracy of the determination result.

[0109] In one example embodiment, the first mapping function includes a first horizontal mapping function and a first vertical mapping function; the first horizontal mapping function is represented by formula (1), and the first vertical mapping function is represented by formula (2);

[0110] f(x) = x × r x +d x (1)

[0111] f(y) = y × r y +d y (2)

[0112] Where f(x) represents the first horizontal mapping function, f(y) represents the first vertical mapping function, x represents the horizontal coordinate value of any pixel in the first image, y represents the vertical coordinate value of any pixel in the first image, and r x r represents the horizontal ratio between the first and second images. y d represents the vertical ratio between the first and second images. x Indicates the horizontal offset value, d y This represents the vertical offset value.

[0113] Figure 4 This is a schematic diagram illustrating the mapping relationship between the first image and the second image provided in an embodiment of the present invention, such as... Figure 4 As shown, the first image P v With the second image P d These are different images captured within a preset time difference range. Although visible light cameras and depth cameras can be adjusted to the same image-capturing angle, such as... Figure 3 The angle α in the image is different because the installation positions of the two images cannot coincide, resulting in a horizontal gap. Therefore, the actual fields of view captured by the two images differ, and the edges of the fields of view do not completely overlap. Furthermore, the resolutions of the output images are usually inconsistent, so the first and second images cannot be aligned at the pixel level. Therefore, to obtain the correspondence between the pixels of the first and second images, it is necessary to determine the mapping relationship between each pixel, i.e., to determine the first mapping function. Similarly, when an infrared thermography camera is installed near a visible light camera and a depth camera, the problem of pixel-level alignment between the first and third images also exists. Therefore, it is also necessary to establish the correspondence between pixels by determining a second mapping function. The following description uses the determination of the first mapping function as an example; the method for determining the second mapping function is similar and will not be repeated here.

[0114] like Figure 4 As shown, the first image P v Second image P d These are frame images of their respective fields of view captured by a visible light camera and a depth camera at the same time or within a preset time difference range (0ms to 100ms).

[0115] First image P v The width is W vThe height is H v Second image P d The width is W d The height is H d Reference point A is used in the first image P. v The imaging center point is A(x1, y1), and the reference point B is in the first image P. v The imaging center point in the second image is B(x2, y2); the reference point A is in the second image P. d The imaging center point is A'(x1', y1'), and the reference point B is in the second image P. d The imaging center point is B'(x2', y2'). Optionally, reference point A and / or reference point B can be the face center point. Figure 4 It can be seen that: the first image P v Second image P d The proportion r in the horizontal direction x The first image P can be determined using the following formula (4). v Second image P d The proportion r in the vertical direction y It can be determined using the following formula (5).

[0116]

[0117]

[0118] Because of the first image P v Second image P d The field of view varies depending on the size of the image being captured, therefore for the first image P... v After transforming the coordinates of any point (x, y) according to formulas (4) and (5), it still needs to be offset to become the second image P. d The corresponding points are then mapped. Assume the horizontal offset value is d. x Then d can be determined using the following formula (6). x Further transformations yield formula (7).

[0119] x×r x +d x =x ′ (6)

[0120] d x =x ′ -x×r x (7)

[0121] Similarly, assuming the vertical offset value is d y Then d can be determined using the following formula (8). y Further transformations yield formula (9).

[0122] y×r y +d y =y ′ (8)

[0123] d y =y ′ -y×r y (9)

[0124] Therefore, in the practical application of the algorithm, it is only necessary to consider the first image P. v Second image P d Find two center points of the same object's image, obtain their coordinate values, and then substitute them into formulas (4) and (5) to obtain the horizontal and vertical proportions respectively. Then, take a point in the first image P. v The coordinates of that point in the second image P, and the coordinates of that point in the second image P. d The coordinates in the equations are used to substitute the values ​​into formulas (7) and (9) to obtain the horizontal and vertical offset values, respectively. Based on this, the first mapping function includes a first horizontal mapping function and a first vertical mapping function. The first horizontal mapping function can be represented by the following formula (1), and the first vertical mapping function can be represented by the following formula (2).

[0125] f(x) = x × r x +d x (1)

[0126] f(y) = y × r y +d y (2)

[0127] Where f(x) represents the first horizontal mapping function, f(y) represents the first vertical mapping function, x represents the horizontal coordinate value of any pixel in the first image, y represents the vertical coordinate value of any pixel in the first image, and r x r represents the horizontal ratio between the first and second images. y d represents the vertical ratio between the first and second images. x Indicates the horizontal offset value, d y This represents the vertical offset value.

[0128] For the first image P v The coordinates (x, y) of any first pixel point in the image can be mapped to the second image P using the first mapping function described above. d The coordinates of the second pixel in the second image are then determined. Furthermore, after determining the coordinates of the second pixel, the depth value corresponding to the second pixel in the second image can be used to determine the target face.

[0129] In one example embodiment, acquiring a first image captured by a visible light camera and a second image captured by a depth camera includes: upon receiving first information sent by a human perception detection module, controlling the visible light camera and the depth camera to capture images to obtain the first image captured by the visible light camera and the second image captured by the depth camera; the first information is used to characterize the detected person passing by.

[0130] For example, a human body detection module can be used to detect the position and state of a human body. For instance, the human body detection module can be an infrared beam sensor. The human body detection module can be installed inside the detection gate of a multi-dimensional security gate or in other locations convenient for human body detection. When a person to be detected is inside the detection gate, the human body detection module determines that the person to be detected is present. At this time, the human body detection module can send first information to the processing module, which indicates that the person to be detected has been detected. Upon receiving the first information sent by the human body detection module, the visible light camera and depth camera are controlled to take pictures, obtaining a first image captured by the visible light camera and a second image captured by the depth camera. Based on this, the visible light camera and depth camera can be used to take pictures of the person to be detected after the person to be detected triggers the human body detection module.

[0131] This invention also provides a multi-dimensional security gate, which includes: a detection gate body, a connecting rod, a metal quantity detection module, a human body sensing detection module, and a processing module.

[0132] The detection gate includes two oppositely arranged side panels and a top panel connected to the two side panels. A connecting rod has one end connected to the top panel and the other end equipped with a visible light camera, a depth camera, and an infrared thermometer. A metal detection module is located within the two side panels. A human body detection module is located within the top panel. A processing module is connected to the visible light camera, depth camera, infrared thermometer, metal detection module, and human body detection module. The human body detection module sends first information to the processing module when it detects a person passing through the detection gate; the first information indicates that a person has been detected. The metal detection module detects the amount of metal carried by the person passing through the detection gate and sends the metal amount to the processing module. The processing module, upon receiving first information from the human body perception detection module, controls a visible light camera, a depth camera, and an infrared temperature measurement camera to capture images, obtaining a first image captured by the visible light camera, a second image captured by the depth camera, and a third image captured by the infrared temperature measurement camera. The first, second, and third images are all captured within a preset time difference range; each of the first, second, and third images includes at least one identical person passing through. The processing module is also used to obtain the first pixel coordinates of the center point of each person's face in the first image; for each first pixel coordinate, it determines the corresponding second pixel coordinates in the second image based on a first mapping function; it obtains the depth value corresponding to each second pixel coordinate in the second image; it determines a target depth value based on a preset depth value and each depth value, and identifies the face in the second image corresponding to the target depth value as the target face; the preset depth value characterizes the distance between the center point of the face of the person being detected in the gate and the depth camera; the target face is the face of the person passing through the gate. The processing module is also used to determine the coordinates of the third pixel point corresponding to the coordinates of the target pixel point in the third image based on the second mapping function, obtain the temperature value corresponding to the coordinates of the third pixel point in the third image, and associate the temperature value and the amount of metal carried with the identifier of the target face; the coordinates of the target pixel point are the pixel coordinates of the center point of the target face in the first image.

[0133] Furthermore, the multidimensional security gate also includes a distance detection module connected to the processing module. The distance detection module is located on the top inner side of the gate body. The distance detection module is used to send the target height information of the person to be detected inside the gate body to the processing module. Based on the correspondence between the height information and the preset depth value, the processing module determines the target preset depth value corresponding to the target height information. The absolute difference between each depth value and the target preset depth value is compared, and the depth value corresponding to the smallest absolute difference is determined as the target depth value.

[0134] The face matching method for security checks provided in this embodiment of the invention can be applied to the multi-dimensional security gate described above and can achieve the technical effects described in the above embodiments. To avoid repetition, it will not be repeated here.

[0135] Figure 5 This is a schematic diagram of the structure of the multi-dimensional security gate provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the multi-dimensional security gate includes a gate body, a connecting rod, a metal detection module, a human body detection module, a visible light camera, a depth camera, and an infrared thermometer. The visible light camera, depth camera, and infrared thermometer can form a camera assembly, which is connected to one end of the connecting rod, and the other end of the connecting rod is connected to the top of the gate body. Both the metal detection module and the human body detection module are located inside the gate body. The metal detection module can detect the amount of metal carried by the person being checked, and the detection result can include whether metal is being carried, and the amount of metal carried. When the person being checked triggers the human body detection module, the visible light camera, depth camera, and infrared thermometer can capture an image of them.

[0136] The following describes the face matching device for security checks provided by embodiments of the present invention. The face matching device for security checks described below can be referred to in correspondence with the face matching method for security checks described above.

[0137] Figure 6This is a schematic diagram of the structure of a face matching device for security checks provided in an embodiment of the present invention. Applied to a multi-dimensional security gate, the face matching device 600 for security checks includes: an acquisition unit 610, used to acquire a first image captured by a visible light camera and a second image captured by a depth camera, wherein the first and second images are captured within a preset time difference range; both the first and second images include the same at least one person; the at least one person includes at least one of: a person about to pass through the multi-dimensional security gate, a person currently passing through the gate, and a person who has already passed through the gate; the acquisition unit 610 is further used to acquire... The system retrieves the coordinates of the first pixel of the center point of the face of each person passing through in the first image; the determining unit 620 is used to determine the coordinates of the second pixel in the second image corresponding to the coordinates of the first pixel based on a first mapping function; the acquiring unit 610 is also used to acquire the depth value corresponding to the coordinates of the second pixel in the second image; the determining unit 620 is also used to determine the target depth value based on the preset depth value and each depth value, and to determine the face in the second image corresponding to the target depth value as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person being detected in the gate and the depth camera device; the target face is the face of the person passing through the gate.

[0138] In one example embodiment, the determining unit 620 is specifically used to: compare the absolute difference between each depth value and a preset depth value, and determine the depth value corresponding to the smallest absolute difference as the target depth value.

[0139] In one example embodiment, the determining unit 620 is specifically used to: acquire the target height information of the person to be detected located inside the detection gate detected by the distance detection module; determine the target preset depth value corresponding to the target height information based on the correspondence between the height information and the preset depth value; and compare the absolute difference between each depth value and the target preset depth value.

[0140] In one example embodiment, the face matching device 600 for security checks further includes an association unit; an acquisition unit 610, which is further configured to acquire a third image captured by an infrared temperature measuring camera; the third image and the first image are captured within a preset time difference range; both the third image and the first image include the same at least one person passing through; a determination unit 620, which is further configured to determine the coordinates of a third pixel point in the third image corresponding to the coordinates of a target pixel point based on a second mapping function; the coordinates of the target pixel point are the pixel coordinates of the center point of the target face in the first image; the acquisition unit 610, which is further configured to acquire the temperature value corresponding to the coordinates of the third pixel point in the third image; and an association unit, which is configured to associate the temperature value with the identifier of the target face.

[0141] In one example embodiment, the acquisition unit 610 is further configured to acquire the amount of metal carried by a person passing through the detection gate as detected by the metal quantity detection module; the association unit is specifically configured to associate both the temperature value and the amount of metal carried with the identifier of the target face.

[0142] In one example embodiment, the determining unit 620 is specifically configured to: subtract a preset depth value from each depth value to obtain a first difference corresponding to the depth value; if each first difference includes a first difference less than zero, remove the first difference less than zero from each first difference to obtain a set of differences; determine the first face corresponding to each first difference in the second image; determine the pixel area occupied by each first face in the second image; and if the face with the largest pixel area is determined to be the face corresponding to the target depth value in the second image, determine the face corresponding to the target depth value in the second image as the target face.

[0143] In one example embodiment, the determining unit 620 is specifically configured to: display the face corresponding to the target depth value and the face with the largest pixel area in the second image when it is determined that the face with the largest pixel area is not the face corresponding to the target depth value in the second image; determine the target face based on the selection instruction input by the person passing through; the selection instruction is used to instruct the person passing through to select the face from the face corresponding to the target depth value and the face with the largest pixel area in the second image.

[0144] In one example embodiment, the first mapping function includes a first horizontal mapping function and a first vertical mapping function; the first horizontal mapping function is represented by formula (1), and the first vertical mapping function is represented by formula (2);

[0145] f(x) = x × r x +d x (1)

[0146] f(y) = y × r y +d y (2)

[0147] Where f(x) represents the first horizontal mapping function, f(y) represents the first vertical mapping function, x represents the horizontal coordinate value of any pixel in the first image, y represents the vertical coordinate value of any pixel in the first image, and r x r represents the horizontal ratio between the first and second images. y d represents the vertical ratio between the first and second images. x Indicates the horizontal offset value, d y This represents the vertical offset value.

[0148] In one example embodiment, the acquisition unit 610 is further configured to: upon receiving first information sent by the human body perception detection module, control the visible light camera and the depth camera to take pictures, thereby obtaining a first image captured by the visible light camera and a second image captured by the depth camera; the first information is used to characterize the detected person passing by.

[0149] The apparatus of this embodiment can be used to execute the method of any embodiment in the side embodiment of the method for matching faces of people for security checks. Its specific implementation process and technical effects are similar to those in the side embodiment of the method for matching faces of people for security checks. For details, please refer to the detailed description in the side embodiment of the method for matching faces of people for security checks, which will not be repeated here.

[0150] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 7 As shown, the electronic device 700 may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a face matching method for security checks. This method is applied to a multi-dimensional security gate, and includes: acquiring a first image captured by a visible light camera and a second image captured by a depth camera, wherein the first and second images are captured within a preset time difference range; both the first and second images include the same at least one person; the at least one person includes: a person about to pass through the multi-dimensional security gate, a person currently passing through the gate, and a person who has already passed through the gate. The process involves: obtaining the coordinates of the first pixel of the face center point of each person passing through the first image; determining the coordinates of the second pixel in the second image corresponding to the first pixel coordinates based on a first mapping function; obtaining the depth value corresponding to each second pixel coordinate in the second image; determining the target depth value based on the preset depth value and each depth value, and identifying the face in the second image corresponding to the target depth value as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person passing through the detection gate and the depth camera device; the target face is the face of the person passing through the detection gate.

[0151] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the face matching method for security checks provided by the above methods. This method is applied to a multi-dimensional security gate. The method includes: acquiring a first image captured by a visible light camera and a second image captured by a depth camera, wherein the first image and the second image are captured within a preset time difference range; both the first image and the second image include the same at least one person passing through; the at least one person passing through includes: a person about to pass through the detection gate of the multi-dimensional security gate. The system identifies at least one of the following: personnel, personnel passing through the detection gate, and personnel who have already passed through the detection gate; it acquires the first pixel coordinates of the center point of the face of each person in the first image; for each first pixel coordinate, it determines the second pixel coordinates corresponding to the first pixel coordinates in the second image based on a first mapping function; it acquires the depth value corresponding to each second pixel coordinate in the second image; it determines a target depth value based on a preset depth value and each depth value, and identifies the face in the second image corresponding to the target depth value as the target face; the preset depth value is used to characterize the distance between the center point of the face of the identified person in the detection gate and the depth camera device; the target face is the face of the person passing through the detection gate.

[0153] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the face matching method for security checks provided by the methods described above. This method is applied to a multi-dimensional security gate and includes: acquiring a first image captured by a visible light camera and a second image captured by a depth camera, wherein the first and second images are acquired within a preset time difference range; both the first and second images include the same at least one person passing through; the at least one person passing through includes: a person about to pass through the multi-dimensional security gate's detection gate, and a person currently passing through the detection gate. The system detects at least one of the personnel and those who have passed through the detection gate; obtains the coordinates of the first pixel of the center point of the face of each person in the first image; determines the coordinates of the second pixel in the second image corresponding to the coordinates of the first pixel based on a first mapping function; obtains the depth value corresponding to the coordinates of the second pixel in the second image; determines the target depth value based on the preset depth value and the depth values, and identifies the face in the second image corresponding to the target depth value as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person being detected in the detection gate and the depth camera device; the target face is the face of the person passing through the detection gate.

[0154] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for facial matching of passersby for security checks, characterized in that, Applied to multi-dimensional security gates, the method includes: The system acquires a first image captured by a visible light camera and a second image captured by a depth camera, wherein the first image and the second image are captured within a preset time difference range; both the first image and the second image include the same at least one person passing through; the at least one person passing through includes: a person about to pass through the detection gate of the multi-dimensional security gate, a person passing through the detection gate, and a person who has already passed through the detection gate; Obtain the first pixel coordinates of the center point of the face of each of the passers-by in the first image; For each of the first pixel coordinates, the coordinates of the second pixel in the second image corresponding to the first pixel coordinates are determined based on the first mapping function; In the second image, obtain the depth value corresponding to the coordinates of each second pixel point; A target depth value is determined based on a preset depth value and each of the aforementioned depth values, and the face in the second image corresponding to the target depth value is identified as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person being detected in the gate and the depth camera device; the target face is the face of the person passing through the gate. Determining the face in the second image corresponding to the target depth value as the target face includes: For each of the aforementioned depth values, the preset depth value is subtracted from the depth value to obtain the first difference corresponding to the depth value; If each of the first differences includes a first difference less than zero, remove the first difference less than zero from each of the first differences to obtain a set of differences; Determine the first face in the second image corresponding to each first difference in the difference set; Determine the pixel area occupied by each of the first faces in the second image; If the face with the largest pixel area is determined to be the face in the second image corresponding to the target depth value, then the face in the second image corresponding to the target depth value is determined as the target face.

2. The face matching method for security checks of passersby according to claim 1, characterized in that, The process of determining the target depth value based on the preset depth value and each of the aforementioned depth values ​​includes: The absolute difference between each depth value and the preset depth value is compared, and the depth value corresponding to the smallest absolute difference is determined as the target depth value.

3. The face matching method for security checks of passersby according to claim 2, characterized in that, The step of comparing the absolute difference between each depth value and a preset depth value includes: The target height information of the person to be detected located inside the detection gate is obtained by the distance detection module. Based on the correspondence between height information and preset depth value, the target preset depth value corresponding to the target height information is determined; The absolute difference between each depth value and the target preset depth value is compared.

4. The face matching method for security checks of passersby according to any one of claims 1-3, characterized in that, Also includes: A third image is acquired by an infrared temperature measurement camera; the third image and the first image are acquired within the preset time difference range; both the third image and the first image include the same at least one person passing by; The coordinates of the third pixel in the third image corresponding to the coordinates of the target pixel are determined based on the second mapping function; the coordinates of the target pixel are the coordinates of the center point of the target face in the first image; Obtain the temperature value corresponding to the coordinates of the third pixel in the third image; The temperature value is associated with the identifier of the target face.

5. The face matching method for security checks of passersby according to claim 4, characterized in that, Also includes: The metal quantity detection module detects the amount of metal carried by a person passing through the detection gate. Associating the temperature value with the identifier of the target face includes: The temperature value and the amount of metal carried are both associated with the identifier of the target face.

6. The face matching method for security checks of passersby according to claim 1, characterized in that, The method further includes: If it is determined that the face with the largest pixel area is not the face in the second image corresponding to the target depth value, then the face in the second image corresponding to the target depth value and the face with the largest pixel area are displayed. Based on the selection command input by the person passing through, the target face is determined; the selection command is used to instruct the person passing through to select a face from the face corresponding to the target depth value and the face with the largest pixel area in the second image.

7. The face matching method for security checks of passersby according to any one of claims 1-3, characterized in that, The first mapping function includes a first horizontal mapping function and a first vertical mapping function; the first horizontal mapping function is represented by formula (1), and the first vertical mapping function is represented by formula (2); f(x)=x×r x +d x (1) f(y)=y×r y +d y (2) Where f(x) represents the first horizontal mapping function, f(y) represents the first vertical mapping function, x represents the horizontal coordinate value of any pixel in the first image, y represents the vertical coordinate value of any pixel in the first image, and r x r represents the horizontal ratio between the first image and the second image. y d represents the vertical ratio between the first image and the second image. x Indicates the horizontal offset value, d y This represents the vertical offset value.

8. The face matching method for security checks of passersby according to any one of claims 1-3, characterized in that, The acquisition of the first image captured by the visible light camera and the second image captured by the depth camera includes: Upon receiving the first information sent by the human perception detection module, the visible light camera and the depth camera are controlled to take pictures, resulting in a first image captured by the visible light camera and a second image captured by the depth camera; the first information is used to indicate that a person to be detected has been detected.

9. A multi-dimensional security gate, characterized in that, The multi-dimensional security gate includes: The detection door includes two side door panels arranged opposite each other and a top panel connected to the two side door panels; A connecting rod, one end of which is connected to the top plate, and the other end of which is equipped with a visible light camera, a depth camera, and an infrared temperature measurement camera; A metal quantity detection module is installed inside the two side door panels; The human body sensing and detection module is installed inside the top panel; The processing module is connected to the visible light camera, the depth camera, the infrared temperature measurement camera, the metal content detection module, and the human body perception detection module. The human body sensing and detection module is used to send first information to the processing module when it detects a person to be detected passing through the detection gate; the first information is used to indicate that a person to be detected has been detected. The metal detection module is used to detect the amount of metal carried by the person to be detected passing through the detection gate, and send the amount of metal carried to the processing module. The processing module is configured to, upon receiving first information from the human body perception detection module, control the visible light camera, the depth camera, and the infrared temperature measurement camera to capture images, thereby obtaining a first image captured by the visible light camera, a second image captured by the depth camera, and a third image captured by the infrared temperature measurement camera; the first image, the second image, and the third image are all captured within a preset time difference range; and each of the first image, the second image, and the third image includes at least one identical person passing through. The processing module is further configured to obtain the first pixel coordinates of the center point of the face of each of the persons passing through in the first image; for each first pixel coordinate, determine the second pixel coordinates corresponding to the first pixel coordinates in the second image based on a first mapping function; obtain the depth value corresponding to each second pixel coordinate in the second image; determine a target depth value based on a preset depth value and each of the depth values; and determine the face in the second image corresponding to the target depth value as the target face; the preset depth value is used to characterize the distance between the center point of the face of the person passing through the detection gate and the depth camera device; the target face is the face of the person passing through the detection gate. The processing module is further configured to determine the coordinates of a third pixel in the third image corresponding to the coordinates of the target pixel based on the second mapping function, obtain the temperature value corresponding to the coordinates of the third pixel in the third image, and associate the temperature value and the amount of metal carried with the identifier of the target face; the coordinates of the target pixel are the pixel coordinates of the center point of the target face in the first image; The processing module is specifically configured to subtract a preset depth value from each depth value to obtain a first difference corresponding to the depth value; if each first difference includes a first difference less than zero, remove the first difference less than zero from each first difference to obtain a set of differences; determine the first face corresponding to each first difference in the second image; determine the pixel area occupied by each first face in the second image; and if the face with the largest pixel area is determined to be the face in the second image corresponding to the target depth value, determine the face in the second image corresponding to the target depth value as the target face.

Images