Verification method and system for self-service security

Abstract

The application discloses a kind of verification method and system for self-service security check, including the following steps: collecting the front face image and side face image of sample object as sample image, after sample image is preprocessed, then after shallow feature extraction, replacement and integration, sample fusion face image shallow feature matrix is obtained;A certain scale of sample fusion face image shallow feature matrix is constructed as training data set, the training data set is input into MobileNet network model, and a deep feature extraction model is obtained;Second fusion face image deep feature matrix is compared with the similarity of each first fusion face image deep feature matrix of the current boarding flight in passenger feature library, and the similarity threshold is set, if the similarity is higher than the preset similarity threshold, then it is verified, if the similarity is lower than the preset similarity threshold, then it is not verified.

Description

TECHNICAL FIELD

[0001] The present application relates to the security check technical field, specifically relates to a verification method and system for self-service security check. BACKGROUND

[0002] Airport security is a necessary link for passengers to take civil aviation aircraft, mainly through airport management personnel to verify the identity of the passengers, and then check the personal belongings of the passengers, but with the continuous acceleration of China's urbanization process, the population density of cities increases sharply, and the passenger flow of large and medium-sized airports, high-speed rail stations and other public travel places also increases year by year, which also brings new challenges to airport security, and forces airlines and airports to improve the efficiency of each link, so it is particularly important to simplify the process after the passengers arrive at the airport and reduce the queuing time for security check, and ensure the authenticity and safety of the passengers boarding.

[0003] In order to solve the problems of low efficiency of artificial identity verification and long queuing time for security check, the airport currently introduces face recognition into the verification of the identity of the airport security, such as the existing patent CN109117727A - a method for verifying the authenticity of passengers at the boarding gate based on face recognition technology, which discloses that when the passenger handles the check-in, the passenger's face is automatically collected by the camera, the authenticity of the identity card is automatically judged by the identity card identification instrument, and the identity card number is automatically identified, and when the passenger passes through the airport security, the barcode scanner scans the departure date and flight number on the passenger's boarding pass associated with the passenger, which compares the face photo of the passenger's identity card with the face scanned on the spot by the passenger, that is, the verification of the identity of the passenger, and then scans the boarding pass to complete the verification of the ticket.

[0004] The existing problems are: the verification of the identity of the passenger is a system that requires the active cooperation of the passenger, which requires the passenger to face the face collection device (camera) and collect the front face, and the passenger needs to carry the identity card or boarding pass and other valid identity documents for verification, and the passenger needs to take the valid identity documents out of the bag and actively cooperate with the face collection during the airport security check, which consumes a long time and cannot meet the needs of the airport with a large passenger flow, and similar faces have certain common characteristics, which can easily cause identification errors, resulting in low recognition accuracy, only 97%, and the information authenticity cannot be guaranteed, which can cause illegal persons to use other people's identity cards to enter the entrance channel and board the plane, which poses a great threat to the safety of people's lives and property. SUMMARY

[0005] The first object of the present application is to provide a verification method for self-service security check, and the second object of the present application is to provide a verification system for self-service security check.

[0006] The first object of the present application is implemented by the following technical scheme: a verification method for self-service security check, comprising the following steps:

[0007] S1, establish a passenger feature library, and extract a first fused face image deep feature matrix:

[0008] The identity information of the passenger's identity document is used as registration information, and the front face image and the side face image of the passenger are collected as registration images. After the registration images are preprocessed, shallow feature extraction, replacement and integration are performed to obtain a first fused face image shallow feature matrix. The first fused face image shallow feature matrix is subjected to deep feature extraction by using a deep feature extraction model to obtain a first fused face image deep feature matrix. The registration information and the first fused face image deep feature matrix are stored in the passenger feature library.

[0009] S2, the passenger feature library is associated with an aviation information platform.

[0010] S3, extract a second fused face image deep feature matrix:

[0011] The front face image and the side face image of the passenger at the security check site are collected as real-time images. After the real-time images are preprocessed, shallow feature extraction, replacement and integration are performed to obtain a second fused face image shallow feature matrix. The second fused face image shallow feature matrix is subjected to deep feature extraction by using a deep feature extraction model to obtain a second fused face image deep feature matrix.

[0012] S4, similarity comparison:

[0013] According to the boarding pass information of the current flight, the corresponding first fused face image deep feature matrix in the passenger feature library is retrieved. The second fused face image deep feature matrix is compared with the retrieved first fused face image deep feature matrix of the passenger feature library. A comparison similarity threshold is set. If the similarity is higher than the preset similarity threshold, the verification is passed. If the similarity is lower than the preset similarity threshold, the verification is not passed.

[0014] Further, the deep feature extraction model is constructed, and the specific steps include: collecting the front face image and the side face image of a sample object as sample images, and after the sample images are preprocessed, shallow feature extraction, replacement and integration are performed to obtain a sample fused face image shallow feature matrix.

[0015] A certain scale of the sample fused face image shallow feature matrix is constructed as a training data set. The training data set is input into a MobileNet network model to complete the training of the MobileNet network model, and a deep feature extraction model is obtained.

[0016] Further, the picture preprocessing includes the following specific steps:

[0017] Face detection is performed on the frontal face image and the side face image to obtain a frontal face region image and a side face region image;

[0018] The frontal face region image and the side face region image are expanded outward by a certain multiple, and then the expanded frontal face region image and the expanded side face region image are intercepted to obtain a cropped frontal face region image and a cropped side face region image;

[0019] Face key point detection is performed on the cropped side face region image to obtain three key points of the ear, and a rectangular region is constructed with the two key points as diagonal anchor points to obtain an ear region image. The ear region image is expanded outward by a certain multiple, and then the expanded ear region image is intercepted to obtain a cropped ear region image;

[0020] The cropped frontal face region image, the cropped side face region image and the cropped ear region image are scaled to a fixed size to obtain a scaled frontal face region image, a scaled side face region image and a scaled ear region image.

[0021] Further, the frontal face region image, the side face region image and the ear region image are expanded outward by a certain multiple, and the specific multiple is 1.2.

[0022] Further, the cropped frontal face region image, the cropped side face region image and the cropped ear region image are scaled to a fixed size, and the specific size of the scaled frontal face region image (WxH) retains three color channels (WxHx3) of the color image. The size of the scaled side face region image (WxH) is converted from a color image to a grayscale image, and only one color channel (WxHx1) is retained. According to the scaling ratio of the side face region image, the size of the scaled ear region image (w x h) is converted from a color image to a grayscale image, and only one color channel (w x h x 1) is retained.

[0023] Further, the shallow feature extraction, replacement and integration include the following steps:

[0024] The three pre-trained full convolutional neural networks are used to extract shallow features from the scaled frontal face region image, the scaled side face region image and the scaled ear region image respectively, while maintaining the same dimension of the output feature matrix and the input image. The size of the frontal face shallow feature matrix is (WxHx3), the size of the side face shallow feature matrix is (WxHx1), and the size of the ear shallow feature matrix is (w x h x 1).

[0025] The elements in the corresponding ear region in the side face shallow feature matrix are replaced by the elements in the ear shallow feature matrix one by one, and the integrated side face image shallow feature matrix is obtained.

[0026] The front face shallow feature matrix and the integrated side face image shallow feature matrix are combined and stacked in the color channel to obtain a fused face image shallow feature matrix with a size of (WxHx4).

[0027] Further, the passenger feature library is associated with an aviation information platform, and the registration information of the passenger feature library passenger, the first fused face image deep feature matrix and the boarding pass information of the aviation information platform are bound.

[0028] Further, the front face image and the side face image of the passenger at the security check site are collected by a front camera and a side camera respectively.

[0029] The second object of the application is implemented by the following technical scheme: a verification system for self-service security check, comprising:

[0030] A registration module is configured to collect the identification information of the passenger, the front face image and the side face image of the passenger, and obtain registration information and registration images;

[0031] A passenger feature library is configured to store the first fused face image deep feature matrix obtained by the image processing and feature extraction module and the registration information associated therewith, and the passenger feature library binds the registration information of the passenger, the first fused face image deep feature matrix and the boarding pass information of the aviation information platform.

[0032] A collection module includes a front camera and a side camera, and is configured to collect the front face image and the side face image of the passenger at the security check site, and obtain real-time images;

[0033] An image processing and feature extraction module is configured to expand, cut and scale the collected front face image, side face image and constructed ear region image, extract shallow features of the scaled front face region image, scaled side face region image and scaled ear region image through a convolutional neural network, and obtain a fused face image shallow feature matrix through shallow feature extraction, replacement and integration, and extract deep features of the fused face image shallow feature matrix through a deep feature extraction model to obtain a fused face image deep feature matrix.

[0034] The verification module is configured to compare the second fused face image deep feature matrix with each first fused face image deep feature matrix of the current boarding flight in the passenger feature library, set a comparison similarity threshold, and send an instruction to the control module according to a comparison similarity result.

[0035] The control module is configured to control a gate or an alarm, and if the similarity is higher than the preset similarity threshold, the control module controls the gate to open, and if the similarity is lower than the preset similarity threshold, the control module controls the alarm to send an alarm.

[0036] The output end of the registration module is communicatively connected to the input end of the image processing and feature extraction module, the output end of the image processing and feature extraction module is communicatively connected to the input end of the passenger feature library, the output end of the collection module is communicatively connected to the input end of the image processing and feature extraction module, the output end of the image processing and feature extraction module and the output end of the passenger feature library are communicatively connected to the input end of the verification module, the output end of the verification module is communicatively connected to the input end of the control module, and the output end of the control module is communicatively connected to the input end of the gate and the alarm.

[0037] Advantages of the present application:

[0038] The second fused face image deep feature matrix is compared with each first fused face image deep feature matrix of the current boarding flight in the passenger feature library to verify the identity of the passenger, the passenger feature library is associated with the aviation information platform, the registration information of the passenger, the first fused face image deep feature matrix and the boarding pass information of the aviation information platform are bound, the corresponding first fused face image deep feature matrix in the passenger feature library is retrieved according to the boarding pass information of the current flight of the aviation information platform, the second fused face image deep feature matrix is compared with the retrieved first fused face image deep feature matrix of the passenger feature library, the identity verification is completed, the security check time and the boarding time of the passenger are shortened, the security check efficiency is improved, the identity information does not need to be repeatedly verified, only the passengers meeting the current flight conditions can pass the security check, and the problem of incorrect boarding of the passenger is avoided.

[0039] The front face and the side face of the passenger in the security check site are collected, and the pictures are preprocessed to obtain a scaled front face region image, a scaled side face region image and a scaled ear region image, then the second fusion face image shallow feature matrix is obtained through shallow feature extraction, replacement and integration, the deep feature extraction operation is performed on the second fusion face image shallow feature matrix to obtain the second fusion face image deep feature matrix, the second fusion face image deep feature matrix is obtained based on the ear information fusion of the front face and the side face, the fusion of the overall and local face features is realized, the accuracy of verification is improved by comparing the second fusion face image deep feature matrix with the first fusion face image deep feature matrix, the common features of similar faces are effectively reduced, the incidence of identification errors is reduced, the recognition accuracy of security check is improved, the recognition accuracy is more than 99.5%, and the safety of people's life and property is ensured.

[0040] By using the system, the comparison between the second fusion face image deep feature matrix of the passenger in the security check site and the first fusion face image deep feature matrix in the passenger feature library is realized, the accuracy of security check is improved; meanwhile, under the association of the passenger feature library and the aviation information platform, the registration information, the first fusion face image deep feature matrix and the boarding pass information are associated, when the second fusion face image deep feature matrix matches the first fusion face image deep feature matrix, it is determined that the person-certificate-ticket verification is passed, the security check is completed, the security check speed is accelerated, the security check process is optimized, and the airport security manpower is saved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description only show some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 The flowchart of the verification method for self-service security check of embodiment one;

[0043] Figure 2 The flowchart of the deep feature extraction model construction of embodiment one;

[0044] Figure 3 The flowchart of the verification method for self-service security check of embodiment one;

[0045] Figure 4 The principle block diagram of the verification system for self-service security check of embodiment two; DETAILED DESCRIPTION

[0046] With reference to the accompanying drawings: clear and complete description of the technical solutions in the embodiments of the application will be described below. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments. Based on the embodiments of the application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of the application.

[0047] Embodiment one:

[0048] As shown in the figure, the verification method for self-service security check comprises the following steps: Figures 1-3

[0049] S1, establishing a passenger feature library and extracting a first fused face image deep feature matrix:

[0050] The passenger feature library is established, the certificate information of the passenger's identity certificate is taken as the registration information, the front face image and the side face image of the passenger are collected as the registration image, after the registration image is preprocessed, the first fused face image shallow feature matrix is obtained through shallow feature extraction, replacement and integration, the first fused face image shallow feature matrix is subjected to deep feature extraction by using a deep feature extraction model to obtain a first fused face image deep feature matrix, and the registration information is associated with the first fused face image deep feature matrix and stored in the passenger feature library.

[0051] S2, associating the passenger feature library with an aviation information platform;

[0052] The registration information of the passenger in the passenger feature library, the first fused face image deep feature matrix and the boarding pass information of the aviation information platform are bound. The registration information of the passenger, the first fused face image deep feature matrix and the flight information are bound, and the corresponding first fused face image deep feature matrix in the passenger feature library is called according to the boarding pass information of the current flight of the aviation information platform.

[0053] S3, extracting a second fused face image deep feature matrix:

[0054] The front face image and the side face image of the passenger in the security check site are collected as real-time images, the front face image and the side face image are collected by a front camera and a side camera respectively, after the real-time images are preprocessed, the second fused face image shallow feature matrix is obtained through shallow feature extraction, replacement and integration, the second fused face image shallow feature matrix is subjected to deep feature extraction by using a deep feature extraction model to obtain a second fused face image deep feature matrix.

[0055] S4, similarity comparison:

[0056] ​According to the boarding pass information of the current flight, the corresponding first fused face image deep feature matrix in the passenger feature library is retrieved, the second fused face image deep feature matrix is compared with the first fused face image deep feature matrix retrieved from the passenger feature library, a comparison similarity threshold is set, if the similarity is higher than the preset similarity threshold, the verification is passed, if the similarity is lower than the preset similarity threshold, the verification is not passed, and the comparison is performed on the basis of the final fused deep feature, so as to realize the passenger identity verification. The first fused face image deep feature matrix and the second fused face image deep feature matrix are one-dimensional vectors with fixed length. After normalizing all feature vectors (the feature vectors are converted into unit vectors with invariant direction by dividing by their own length), the cosine similarity (the radian value of the angle between two unit vectors) between each other is calculated for comparison.

[0057] The deep feature extraction model is constructed, and the specific steps include: collecting the front face image and the side face image of the sample object as sample images, after the sample images are preprocessed, the sample images are subjected to shallow feature extraction, replacement and integration to obtain a sample fused face image shallow feature matrix. A sample fused face image shallow feature matrix of a certain scale is constructed as a training data set, the training data set is input into the MobileNet network model, the training of the MobileNet network model is completed, and the deep feature extraction model is obtained.

[0058] The above picture preprocessing includes the following specific steps: performing face detection on the front face image and the side face image to obtain a front face region image and a side face region image.

[0059] The front face region image and the side face region image are expanded outward by a certain multiple, the front face region image, the side face region image and the ear region image are expanded outward by a certain multiple, and the specific multiple is 1.2. Expanding the face region image outward by a part can include more comprehensive face edge information (such as ears, hair, accessories, etc.) on the basis of the face information, which is helpful to obtain more accurate recognition results.

[0060] Then, the expanded front face region image and the expanded side face region image are intercepted to obtain a cropped front face region image and a cropped side face region image, and the corresponding images are cropped according to the obtained region to exclude irrelevant redundant information, so as to facilitate subsequent use.

[0061] In the cropped side face region image, face key point detection (35 points) is performed to obtain three key points of the upper, middle and lower ears. A rectangular region is constructed with the upper and lower two key points as diagonal anchor points. If the width of the rectangular region is too narrow, the width of the rectangular region is expanded by at least half of the height to obtain an ear region image. The face key points are obtained for ear region positioning. In the 35 face key points, a single ear region includes the upper and lower two key points (if the two points are approximately on a vertical line, the width of the positioned ear region may be too narrow, and the width needs to be appropriately expanded to include a more complete ear region). The ear region image is expanded outward by a certain multiple, and then the expanded ear region image is cropped to obtain a cropped ear region image;

[0062] The cropped front face region image, the cropped side face region image and the cropped ear region image are scaled to a fixed size. The purpose is to facilitate the input of the subsequent neural network model. Because the input image size of the neural network model is fixed, the image must be scaled to the corresponding size to be input. The "fixed size" here corresponds to the input image size of the neural network model. Specifically, the size of the scaled front face region image (WxH) is retained. The RGB color channel (WxHx3) is retained. Retaining the RGB color channel can retain more original input information. That is, in the subsequent recognition process, the front face image is the main information.

[0063] The size of the scaled side face region image (WxH) is converted from a color image to a grayscale image, and only one color channel (WxHx1) is retained. The image is scaled to a fixed size to facilitate the input of the subsequent neural network model. Retaining only a single grayscale channel can compress the input information to a certain extent. That is, in the subsequent recognition process, the side face image is the secondary information.

[0064] According to the scaling ratio of the side face region image, the face image and the ear region image are scaled at the same ratio. The purpose is to ensure that the images are not misaligned before and after scaling. The ear region in the face region can correspond to a region with the same relative position, so as to facilitate subsequent processing in the same size dimension. The size of the scaled ear region image (w x h) is converted from a color image to a grayscale image, and only one color channel (w x h x 1) is retained. The scaled front face region image, the scaled side face region image and the scaled ear region image are obtained.

[0065] The specific steps of the above shallow feature extraction, replacement and integration include: extracting shallow features from the scaled front face region image, the scaled side face region image and the scaled ear region image respectively through the three pre-trained full convolutional neural networks, while keeping the output feature matrix the same as the input image in dimension and keeping the output feature the same as the input image in size, so as to facilitate the subsequent replacement processing of elements in the shallow feature matrix, and obtaining a front face shallow feature matrix with a size of (WxHx3), a side face shallow feature matrix with a size of (WxHx1) and an ear shallow feature matrix with a size of (w x h x 1) respectively; the pre-training of FCN on ImageNet can be trained by adding a full connection layer at the back of the network through the label of the existing image, and the full connection layer can be discarded after training, and the reason for using FCN is that the input and output of the model are matrix images, which can keep the size consistent before and after extraction of shallow features, facilitating subsequent operations.

[0066] The elements in the side face shallow feature matrix corresponding to the relative position of the ear region are replaced with the elements in the ear shallow feature matrix one by one, and the elements in the side face shallow feature corresponding to the relative position of the ear region shallow feature are replaced, which is equivalent to a more refined calculation and processing of this part of the shallow feature, and on this basis, further extraction of deep features can obtain more accurate recognition results, and after integration, a fused side face image shallow feature matrix is obtained.

[0067] The front face shallow feature matrix extracted and the fused side face image shallow feature matrix obtained by integration are merged and stacked together to obtain a fused face image shallow feature matrix with a size of (WxHx4), and the front face shallow feature matrix and the fused side face image shallow feature matrix are merged to obtain the final integrated face shallow feature with a total of 3 (front) + 1 (side) = 4 channels.

[0068] The computer program is stored in the computer readable storage medium and is executed by the processor to realize the steps of the above-mentioned verification method for self-service security check, the computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media, which can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), optical medium (e.g., DVD) or solid state disk.

[0069] The front face and the side face of the passenger in the security check site are collected, and the pictures are preprocessed to obtain a scaled front face region image, a scaled side face region image and a scaled ear region image, and then the second fusion face image shallow feature matrix is obtained through shallow feature extraction, replacement and integration, the second fusion face image shallow feature matrix is subjected to deep feature extraction to obtain the second fusion face image shallow feature matrix, and the second fusion face image deep feature matrix is obtained based on the ear information fusion of the front face and the side face, so that the fusion of the overall and local face features is realized, the verification accuracy is improved, the common features of similar faces are effectively reduced, the identification error rate is reduced, the recognition precision is improved, and the safety of people's life and property is ensured.

[0070] The first fusion face image deep feature matrix in the passenger feature library is retrieved according to the boarding pass information of the current flight, and the second fusion face image deep feature matrix is compared with the retrieved first fusion face image deep feature matrix in the passenger feature library to realize the verification of the passenger's identity; since the passenger feature library is associated with the aviation information platform, the passenger feature library binds the registration information, the first fusion face image deep feature matrix and the boarding pass information of the aviation information platform, retrieves the first fusion face image deep feature matrix in the passenger feature library according to the boarding pass information of the current flight of the aviation information platform, and compares the first fusion face image deep feature matrix of the current flight to complete the verification of the paper ticket, so that the passenger does not need to carry an ID card in the security check process, the security check time and the boarding time of the passenger are shortened, the security check efficiency is improved, the repeated verification of the identity information is not needed, only the passengers who meet the conditions of the current flight can pass the security check, and the problem of incorrect boarding of the passenger is avoided.

[0071] Embodiment two

[0072] As shown in Figure 4 , based on the same inventive concept of the verification method for self-service security check provided in embodiment one, the present embodiment two further provides a verification system for self-service security check, which comprises:

[0073] The registration module is configured to collect the certificate information of the passenger, the front face image and the side face image of the passenger, and obtain the registration information and the registration image;

[0074] The passenger feature library is configured to store the first fusion face image deep feature matrix extracted by the image processing and feature extraction module from the registration image and the registration information associated therewith, and bind the registration information, the first fusion face image deep feature matrix and the boarding pass information of the aviation information platform.

[0075] The collection module includes a front camera and a side camera, and is configured to collect a front face image and a side face image of a passenger in a security field to obtain a real-time image.

[0076] The image processing and feature extraction module is configured to extract shallow features of a scaled front face region image, a scaled side face region image and a scaled ear region image through a convolutional neural network after expansion, interception and scaling of the collected front face image, side face image and constructed ear region image, and to obtain a shallow feature matrix of a fused face image through shallow feature extraction, replacement and integration, and to obtain a deep feature matrix of the fused face image through deep feature extraction of the shallow feature matrix of the fused face image by using a deep feature extraction model. 、 The deep feature extraction model.

[0077] The verification module is configured to compare the second deep feature matrix of the fused face image with each first deep feature matrix of the fused face image of a current boarding flight in a passenger feature library in terms of similarity, set a comparison similarity threshold, and send an instruction to the control module according to a similarity comparison result.

[0078] The control module is configured to control a gate or an alarm, and if the similarity is higher than a preset similarity threshold, the control module controls the gate to open, and if the similarity is lower than the preset similarity threshold, the control module controls the alarm to issue an alarm.

[0079] An output end of the registration module is communicatively connected to an input end of the image processing and feature extraction module, an output end of the image processing and feature extraction module is communicatively connected to an input end of the passenger feature library, an output end of the collection module is communicatively connected to an input end of the image processing and feature extraction module, an output end of the image processing and feature extraction module and an output end of the passenger feature library are communicatively connected to an input end of the verification module, an output end of the verification module is communicatively connected to an input end of the control module, and an output end of the control module is communicatively connected to input ends of the gate and the alarm.

[0080] The embodiment specifically includes the following operation process:

[0081] 1. Preliminary preparation

[0082] Before purchasing an airplane ticket, the passenger needs to register a personal account through a user terminal, and the registration module collects certificate information of an identity certificate of the passenger as registration information, and collects a front face image and a side face image of the passenger as registration images.

[0083] The output end of the registration module is connected with the input end of the image processing and feature extraction module, after the registration image is processed through the image processing and feature extraction module, a first fused face image deep feature matrix is obtained, then the first fused face image deep feature matrix and the registration information associated therewith are stored into the passenger feature library, the passenger feature library is associated with the aviation information platform, and the passenger feature library binds the registration information of the passenger, the first fused face image deep feature matrix and the boarding pass information of the aviation information platform;

[0084] 2. On-site security check:

[0085] The front face camera and the side face camera of the acquisition module acquire the front face image and the side face image of the passenger on the security check site, and a real-time image is obtained;

[0086] The output end of the acquisition module is connected with the input end of the image processing and feature extraction module, after the real-time image is processed through the image processing and feature extraction module, a second fused face image deep feature matrix is obtained;

[0087] 3. Similarity verification:

[0088] The second fused face image deep feature matrix is sent to the verification module, the second fused face image deep feature matrix is compared with each first fused face image deep feature matrix of the current boarding flight in similarity through the verification module, if the similarity is higher than a preset similarity threshold, the control module controls the gate to be opened, and if the similarity is lower than the preset similarity threshold, the control module controls the alarm to issue an alarm.

[0089] Through the cooperative work among the above various component modules, the comparison between the second fused face image deep feature matrix of the passenger on the security check site and the first fused face image deep feature matrix in the passenger feature library is realized, and the accuracy of the security check is improved; meanwhile, under the association between the passenger feature library and the aviation information platform, the registration information, the first fused face image deep feature matrix and the boarding pass information are associated, when the second fused face image deep feature matrix matches the first fused face image deep feature matrix, it is defaulted that the person-credential-ticket verification is passed, the security check is completed, the security check speed is accelerated, the security check process is optimized, and the airport security check manpower is saved.

[0090] The above merely describes the preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of verification for self-service security screening, characterized in that, The method comprises the following steps: S1, establishing a passenger feature library, and extracting a first fused face image deep feature matrix: Taking the certificate information of the passenger's identity certificate as registration information, collecting the front face image and the side face image of the passenger as registration images, after the registration images are preprocessed, shallow feature extraction, replacement and integration are performed to obtain a first fused face image shallow feature matrix, the first fused face image shallow feature matrix is subjected to deep feature extraction by using a deep feature extraction model to obtain a first fused face image deep feature matrix, and the registration information and the first fused face image deep feature matrix are stored in the passenger feature library; S2, the passenger feature library is associated with an aviation information platform; S3, extracting a second fused face image deep feature matrix: Collecting the front face image and the side face image of the passenger at the security check site as real-time images, after the real-time images are preprocessed, shallow feature extraction, replacement and integration are performed to obtain a second fused face image shallow feature matrix, the second fused face image shallow feature matrix is subjected to deep feature extraction by using a deep feature extraction model to obtain a second fused face image deep feature matrix; S4, similarity comparison: According to the boarding pass information of the current flight, the corresponding first fused face image deep feature matrix in the passenger feature library is retrieved, the second fused face image deep feature matrix is compared with the first fused face image deep feature matrix retrieved from the passenger feature library, a comparison similarity threshold is set, if the similarity is higher than the preset similarity threshold, the verification is passed, and if the similarity is lower than the preset similarity threshold, the verification is failed; Wherein, the shallow feature extraction, replacement and integration comprise the following specific steps: The three pre-trained full convolutional neural networks are used to extract shallow features from the scaled front face region image, the scaled side face region image and the scaled ear region image respectively, while keeping the output feature matrix and the input image dimension the same, to obtain a front face shallow feature matrix with a size of (WxHx3), a side face shallow feature matrix with a size of (WxHx1) and an ear shallow feature matrix with a size of (w x h x 1); The elements in the relative position corresponding to the ear region in the side face shallow feature matrix are replaced with the elements in the ear shallow feature matrix one by one, and the integrated fused side face image shallow feature matrix is obtained; The front face shallow feature matrix and the integrated fused side face image shallow feature matrix are combined and stacked together in the color channel to obtain a fused face image shallow feature matrix with a size of (WxHx4).

2. The method of claim 1, wherein: The deep feature extraction model is constructed, and the specific steps comprise: collecting the front face image and the side face image of a sample object as sample images, pre-processing the sample images, and then performing shallow feature extraction, replacement and integration to obtain a sample fused face image shallow feature matrix. The shallow feature matrix of the sample fused human face image of a certain scale is constructed as a training data set, the training data set is input into a MobileNet network model, training of the MobileNet network model is completed, and a deep feature extraction model is obtained.

3. The method of claim 2, wherein: The picture preprocessing specifically comprises the following steps: Face detection is performed on the front face image and the side face image to obtain a front face region image and a side face region image; The front face region image and the side face region image are expanded outward by a certain multiple, and then the expanded front face region image and the expanded side face region image are intercepted to obtain a cropped front face region image and a cropped side face region image; Face key point detection is performed on the cropped side face region image to obtain three key points of the upper, middle and lower ears, a rectangular region is constructed with the upper and lower key points as diagonal anchor points to obtain an ear region image, the ear region image is expanded outward by a certain multiple, and then the expanded ear region image is intercepted to obtain a cropped ear region image; The cropped front face region image, the cropped side face region image and the cropped ear region image are scaled to a fixed size to obtain a scaled front face region image, a scaled side face region image and a scaled ear region image.

4. The method of claim 3, wherein: The front face region image, the side face region image and the ear region image are expanded outward by a certain multiple, and the specific multiple is 1.

2.

5. The method of claim 4, wherein: The cropped front face region image, the cropped side face region image and the cropped ear region image are scaled to a fixed size, and the specific fixed size is that the size of the scaled front face region image is (W×H), and three color channels (W×H×3) of a color image are retained; the size of the scaled side face region image is (W×H), and the scaled side face region image is converted from a color image to a grayscale image, and only one color channel (W×H×1) is retained; according to the scaling ratio of the side face region image, the size of the scaled ear region image is (w×h), and the scaled ear region image is converted from a color image to a grayscale image, and only one color channel (w×h×1) is retained.

6. The method of claim 1, wherein: The passenger feature library is associated with an aviation information platform, and the registration information of the passenger in the passenger feature library is bound with boarding pass information of the aviation information platform.

7. The method of claim 1, wherein: The front face image and the side face image of the passenger in the security check site are collected by a front camera and a side camera respectively.

8. A verification system for self-service security screening, characterized by: It comprises: A registration module is configured to collect the identification information of the passenger, the front face image and the side face image of the passenger, and obtain registration information and registration images; A passenger feature library is configured to store a first fused human face deep feature matrix extracted from the registration images by an image processing and feature extraction module and registration information associated with the first fused human face deep feature matrix, and the passenger feature library binds the registration information of the passenger, the first fused human face deep feature matrix and boarding pass information of the aviation information platform; A collection module comprises a front camera and a side camera, and is configured to collect the front face image and the side face image of the passenger in the security check site to obtain real-time images. The image processing and feature extraction module is configured to extract shallow features from the scaled front face region image, the scaled side face region image and the scaled ear region image through a convolutional neural network after the collected front face image, the side face image and the constructed ear region image are expanded, intercepted and scaled, and then extract, replace and integrate the shallow features, specifically including: extracting shallow features from the scaled front face region image, the scaled side face region image and the scaled ear region image through three pre-trained full convolutional neural networks, while keeping the output feature matrix and the input image having the same dimension, and obtaining a front face shallow feature matrix with a size of (WxHx3), a side face shallow feature matrix with a size of (WxHx1) and an ear shallow feature matrix with a size of (w x h x 1); The elements in the side face shallow feature matrix corresponding to the relative position of the ear region are replaced with the elements in the ear shallow feature matrix one by one, and the integrated side face image shallow feature matrix is obtained; The front face shallow feature matrix and the integrated side face image shallow feature matrix are combined and stacked together to obtain a fused face image shallow feature matrix, and a deep feature extraction model is used to extract deep features from the fused face image shallow feature matrix to obtain a fused face image deep feature matrix; The verification module is configured to compare the second fused face image deep feature matrix with each first fused face image deep feature matrix in the passenger feature library of the current boarding flight in terms of similarity, set a comparison similarity threshold, and send an instruction to the control module according to the similarity comparison result; The control module is configured to control a gate or an alarm, if the similarity is higher than the preset similarity threshold, the control module controls the gate to open, and if the similarity is lower than the preset similarity threshold, the control module controls the alarm to issue an alarm; The output end of the registration module is in communication connection with the input end of the image processing and feature extraction module, the output end of the image processing and feature extraction module is in communication connection with the input end of the passenger feature library, the output end of the image processing and feature extraction module is in communication connection with the input end of the passenger feature library, the output end of the image processing and feature extraction module is in communication connection with the input end of the verification module, the output end of the verification module is in communication connection with the input end of the control module, and the output end of the control module is in communication connection with the input end of the gate and the alarm.

Images