Methods and electronic devices for identifying specific personnel in airports

Abstract

This disclosure provides a method and electronic device for identifying specific personnel in airports. It aims to improve the accuracy of identifying specific personnel in airports. The method includes: for target video images in a monitored video of a target area within a specified time period, performing target detection on the target video images using a target detection algorithm to obtain the bounding box coordinates of each first target object in the target video image and the target feature vector of the first target object, whereby the target feature vector includes height features, weight features, human posture features, and facial features; for any first target object; if, based on the bounding box coordinates and target feature vector of the first target object in the target video image, a target specific person meeting a first preset condition is found in a pre-set airport specific personnel database, then the first target object is determined to be a specific person, wherein the airport specific personnel database is obtained based on historical monitoring videos.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and electronic device for identifying specific individuals in an airport. Background Technology

[0002] In airport hub operation and management scenarios, the involvement of certain individuals (such as scalpers or unlicensed taxi drivers) has become a common phenomenon, forming a mature illegal profit chain. This illegal operation poses significant safety hazards, easily infringes upon passengers' property, disrupts the passenger transport market order, and in severe cases, can lead to public security incidents.

[0003] Currently, there are two main methods for identifying specific individuals in airports: one is through intervention by airport security and order maintenance personnel via video surveillance or on-site manual intervention. This method lacks effective technological support and suffers from low efficiency and accuracy. The other method is based on facial recognition analysis to report individuals who linger in the airport for extended periods as specific individuals. However, in recent years, most travelers have been wearing masks, making traditional facial recognition methods for identifying specific individuals technically limited and difficult to effectively detect faces. Furthermore, simply identifying those who linger for extended periods as specific individuals can lead to a large number of normal passengers being identified and reported as such, resulting in low accuracy for identifying specific individuals in airports. Summary of the Invention

[0004] The exemplary embodiments of this disclosure provide a method and electronic device for identifying specific personnel in an airport, which can improve the accuracy of identifying specific personnel in an airport.

[0005] The first aspect of this disclosure provides a method for identifying specific personnel in an airport, the method comprising:

[0006] At specified intervals, acquire surveillance video of multiple target areas in the airport within the specified interval;

[0007] A target detection algorithm is used to detect targets in the acquired surveillance video images, obtaining detection parameters for each first target object in the target video image. These detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, body shape features, human posture features, and facial features.

[0008] For any given first target object, if, based on the bounding box coordinates of the first target object in the target video image and the target feature vector, a target specific person whose similarity to the first target object meets a first preset condition is found in a pre-stored airport specific personnel database, then the first target object is determined to be an airport characteristic personnel; wherein, the airport specific personnel database is obtained in the following manner:

[0009] The target detection algorithm is used to perform target detection on historical video images of each target region within a specified historical time period. Detection parameters of each second target object in each historical video image are obtained. Based on the detection parameters of each second target object in each historical video image, the similarity between each second target object in any two historical video images is obtained. Based on the similarity between each second target object in any two historical video images, airport travel information of each second target object within the specified historical time period is obtained. The target detection parameters of each second target object whose airport travel information meets a second preset condition are stored to obtain the airport characteristic personnel database. The target detection parameters are the detection parameters obtained from the historical video image of the target region with the smallest difference between the shooting time and the current time.

[0010] In this embodiment, surveillance videos of multiple target areas within an airport are acquired over a specified time period. Then, a target detection algorithm is used to detect any target video image in the acquired surveillance videos, obtaining the bounding box coordinates and target feature vectors of each first target object in the video image. For any first target object, based on the bounding box coordinates and target feature vectors of the first target object in the target video image, a search is performed in a pre-stored database of specific personnel in the airport to determine if a specific person with a similarity to the first target object meets preset conditions exists. If such a person exists, the first target object is identified as a specific person. Therefore, in this embodiment, the bounding box position, height, weight, posture, and facial features of the first target object are used to perform spatiotemporal tracking and association with specific personnel in the pre-established database of specific personnel to identify specific personnel. This improves the accuracy of identifying specific personnel in the airport. Furthermore, in this embodiment, the airport travel information of each second target object is determined based on detection parameters obtained by target detection of historical surveillance videos, and the airport travel information of each target object is used to establish an airport database of specific personnel, ensuring the accuracy of the airport feature personnel database and further improving the accuracy of identifying specific personnel in the airport.

[0011] In one embodiment, after determining that the first target object is a specific airport personnel, the method further includes:

[0012] For any given first target being a specific person at the airport, using a pre-defined hierarchical relationship between target areas, adjacent target areas are determined corresponding to the target area where the surveillance video of the first target is located; wherein, the adjacent target areas include target areas one level above the target area and target areas one level below the target area in the hierarchical relationship; and,

[0013] For any adjacent target area, after determining the monitoring video of the adjacent target area within the specified duration as the acquired monitoring video, the step of performing target detection on the target video image in the acquired monitoring video using a target detection algorithm is returned. The duration of the monitoring video acquired in the adjacent target area is the same as the duration of the monitoring video acquired in the target area, and the time period of the monitoring video acquired in the adjacent target area is different from the time period of the monitoring video acquired in the target area.

[0014] If the first target object exists in the surveillance video of the adjacent target area within the specified time period, then after determining the adjacent target area as the target area, the process returns to the step of determining the adjacent target area corresponding to the target area where the surveillance video of the identified target object is located, using the pre-set hierarchical relationship between each target area, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and,

[0015] Based on the hierarchical relationship between each target region, the trajectory of the first target object is obtained;

[0016] Airport-specific personnel warnings are issued based on the trajectory of the first target object and its image in surveillance video.

[0017] This embodiment achieves process evidence collection for specific personnel by determining the trajectory of the first target object after identifying it as a specific person, and then issuing a warning for the specific person. Furthermore, it sends the trajectory and image of the first target object to the management personnel for warning purposes, so that the management personnel can implement control measures.

[0018] In one embodiment, it is determined whether a target specific person exists in the airport specific personnel database whose similarity to the first target object meets a first preset condition by means of the following method:

[0019] Based on the detection parameters of each airport-specific person in the airport-specific personnel database and the detection parameters of the first target object, the similarity between each airport-specific person and the first target object is obtained.

[0020] If the similarity satisfies the first preset condition, then the airport-specific personnel are determined to be the target-specific personnel corresponding to the first target object.

[0021] In one embodiment, the similarity between any specific airport personnel and the first target object is obtained in the following manner:

[0022] Based on the location coordinates of the bounding box of the specific personnel at the airport and the location coordinates of the bounding box of the first target object, a first similarity index is obtained; and,

[0023] The second similarity index is obtained by using the target feature vector of the specific personnel at the airport and the target feature vector of the first target object;

[0024] Based on the first similarity index and the second similarity index, the similarity between the specific personnel at the airport and the first target object is obtained.

[0025] In this embodiment, the similarity between the first target object and a specific person in the specific personnel database is determined by two parameters: the position of the bounding box and the target feature vector. This ensures the accuracy of the determined similarity.

[0026] In one embodiment, obtaining the first similarity index based on the bounding box coordinates of the airport-specific personnel and the bounding box coordinates of the first target object includes:

[0027] Based on the location coordinates of the bounding box of the airport-specific personnel and the location coordinates of the bounding box of the first target object, the cosine distance between the airport-specific personnel and the first target object is obtained, and the first similarity index is determined based on the cosine distance.

[0028] The second similarity index is obtained by using the target feature vectors of specific personnel at the airport and the target feature vectors of the first target object, including:

[0029] Based on the target feature vector of the airport-specific personnel and the target feature vector of the first target object, the Mahalanobis distance between the airport-specific personnel and the first target object is determined, and the second similarity index is determined based on the Mahalanobis distance.

[0030] In one embodiment, obtaining the airport travel information of each second target object within a specified historical time period based on the similarity between each second target object in any two target historical video images includes:

[0031] Based on the similarity between the second target objects in any two target historical video images, the target historical video images containing the same second target object are obtained.

[0032] Based on historical video images of the same second target object, determine the frequency of the second target object's appearance and travel time within each specified time period, wherein the historical time period includes each specified time period;

[0033] By utilizing the frequency of occurrence and travel time of each of the second target objects within each specified time period, the airport travel information of each of the second target objects is obtained.

[0034] In this embodiment, the airport travel information of the second target object is determined by the frequency of its appearance in each specified historical time period and the corresponding target time period when it appears, thus ensuring the accuracy of the airport-specific personnel database.

[0035] A second aspect of this disclosure provides an electronic device, including a processor and a memory, wherein the processor and the memory are connected via a bus;

[0036] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0037] At specified intervals, acquire surveillance video of multiple target areas in the airport within the specified interval;

[0038] A target detection algorithm is used to detect targets in the acquired surveillance video images, obtaining detection parameters for each first target object in the target video image. These detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, body shape features, human posture features, and facial features.

[0039] For any given first target object, if, based on the bounding box coordinates of the first target object in the target video image and the target feature vector, a target specific person whose similarity to the first target object meets a first preset condition is found in a pre-stored airport specific personnel database, then the first target object is determined to be an airport characteristic personnel; wherein, the airport specific personnel database is obtained in the following manner:

[0040] The target detection algorithm is used to perform target detection on historical video images of each target region within a specified historical time period. Detection parameters of each second target object in each historical video image are obtained. Based on the detection parameters of each second target object in each historical video image, the similarity between each second target object in any two historical video images is obtained. Based on the similarity between each second target object in any two historical video images, airport travel information of each second target object within the specified historical time period is obtained. The target detection parameters of each second target object whose airport travel information meets a second preset condition are stored to obtain the airport characteristic personnel database. The target detection parameters are the detection parameters obtained from the historical video image of the target region with the smallest difference between the shooting time and the current time.

[0041] In one embodiment, the processor is further configured to:

[0042] After determining that the first target object is a specific airport personnel, for any first target object that is a specific airport personnel, using a pre-set hierarchical relationship between target areas, adjacent target areas corresponding to the target area where the surveillance video of the first target object is located are determined; wherein, the adjacent target areas include target areas at the one-level above the target area and target areas at the one-level below the target area in the hierarchical relationship; and,

[0043] For any adjacent target area, after determining the monitoring video of the adjacent target area within the specified duration as the acquired monitoring video, the step of performing target detection on the target video image in the acquired monitoring video using a target detection algorithm is returned. The duration of the monitoring video acquired in the adjacent target area is the same as the duration of the monitoring video acquired in the target area, and the time period of the monitoring video acquired in the adjacent target area is different from the time period of the monitoring video acquired in the target area.

[0044] If the first target object exists in the surveillance video of the adjacent target area within the specified time period, then after determining the adjacent target area as the target area, the process returns to the step of determining the adjacent target area corresponding to the target area where the surveillance video of the identified target object is located, using the pre-set hierarchical relationship between each target area, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and,

[0045] Based on the hierarchical relationship between each target region, the trajectory of the first target object is obtained;

[0046] Airport-specific personnel warnings are issued based on the trajectory of the first target object and its image in surveillance video.

[0047] In one embodiment, the processor is further configured to:

[0048] The following method is used to determine whether a target specific person exists in the airport specific personnel database whose similarity to the first target object meets a first preset condition:

[0049] Based on the detection parameters of each airport-specific person in the airport-specific personnel database and the detection parameters of the first target object, the similarity between each airport-specific person and the first target object is obtained.

[0050] If the similarity satisfies the first preset condition, then the airport-specific personnel are determined to be the target-specific personnel corresponding to the first target object.

[0051] In one embodiment, the processor is further configured to:

[0052] The similarity between any specific person at the airport and the first target object is obtained in the following way:

[0053] Based on the location coordinates of the bounding box of the specific personnel at the airport and the location coordinates of the bounding box of the first target object, a first similarity index is obtained; and,

[0054] The second similarity index is obtained by using the target feature vector of the specific personnel at the airport and the target feature vector of the first target object;

[0055] Based on the first similarity index and the second similarity index, the similarity between the specific personnel at the airport and the first target object is obtained.

[0056] In one embodiment, the processor executes the step of obtaining a first similarity index based on the location coordinates of the bounding box of the airport-specific personnel and the location coordinates of the bounding box of the first target object, specifically configured as follows:

[0057] Based on the location coordinates of the bounding box of the airport-specific personnel and the location coordinates of the bounding box of the first target object, the cosine distance between the airport-specific personnel and the first target object is obtained, and the first similarity index is determined based on the cosine distance.

[0058] The processor executes the target feature vectors of the specific personnel at the airport and the target feature vectors of the first target object to obtain a second similarity index, which is specifically configured as follows:

[0059] Based on the target feature vector of the airport-specific personnel and the target feature vector of the first target object, the Mahalanobis distance between the airport-specific personnel and the first target object is determined, and the second similarity index is determined based on the Mahalanobis distance.

[0060] In one embodiment, the processor's execution of obtaining airport travel information of each second target object within a specified historical time period based on the similarity between each second target object in any two target historical video images is specifically configured as follows:

[0061] Based on the similarity between the second target objects in any two target historical video images, the target historical video images containing the same second target object are obtained.

[0062] Based on historical video images of the same second target object, determine the frequency of the second target object's appearance and travel time within each specified time period, wherein the historical time period includes each specified time period;

[0063] By utilizing the frequency of occurrence and travel time of each of the second target objects within each specified time period, the airport travel information of each of the second target objects is obtained.

[0064] According to a third aspect provided in the embodiments of this disclosure, a computer storage medium is provided, the computer storage medium storing a computer program for performing the method as described in the first aspect. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic diagram illustrating one of the applicable scenarios according to an embodiment of the present disclosure;

[0067] Figure 2 This is a second schematic diagram illustrating an applicable scenario according to one embodiment of the present disclosure;

[0068] Figure 3 This is one of the flowcharts illustrating a method for identifying specific individuals in an airport according to an embodiment of the present disclosure;

[0069] Figure 4 This is a schematic diagram of the structure of an ITMFF model according to an embodiment of the present disclosure;

[0070] Figure 5 This is a schematic diagram of a target image according to an embodiment of the present disclosure;

[0071] Figure 6 This is a flowchart illustrating an embodiment of the present disclosure;

[0072] Figure 7 This is a flowchart illustrating the process of determining the trajectory of a specific person according to an embodiment of the present disclosure.

[0073] Figure 8 This is a schematic diagram illustrating the hierarchical relationship between target regions according to an embodiment of the present disclosure;

[0074] Figure 9 This is a schematic diagram of a trajectory according to an embodiment of the present disclosure;

[0075] Figure 10 This is a schematic diagram of a process for determining a specific personnel database according to an embodiment of the present disclosure;

[0076] Figure 11 An identification device for specific persons in an airport according to one embodiment of the present disclosure;

[0077] Figure 12 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0079] In this disclosure, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0080] The application scenarios described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided in this disclosure. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided in this disclosure are also applicable to similar technical problems. In the description of this disclosure, unless otherwise stated, "multiple" means two or more.

[0081] Currently, there are two main methods for identifying specific individuals in airports: one is through intervention by airport security and order maintenance personnel via video surveillance or on-site manual intervention. This method lacks effective technological support and suffers from low efficiency and accuracy. The other method is based on facial recognition analysis to report individuals who linger in the airport for extended periods as specific individuals. However, in recent years, most travelers have been wearing masks, making traditional facial recognition methods for identifying specific individuals technically limited and difficult to effectively detect faces. Furthermore, simply identifying those who linger for extended periods as specific individuals can lead to a large number of normal passengers being identified and reported as such, resulting in low accuracy for identifying specific individuals in airports.

[0082] Therefore, this disclosure provides a method for identifying specific personnel in an airport. It involves acquiring surveillance videos of multiple target areas within a specified time period in the airport, and then using a target detection algorithm to detect any target video image in the acquired surveillance videos. This yields the position coordinates of the bounding boxes of each first target object in the video image and the target feature vector of the first target object. For any first target object, based on the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, a pre-stored database of specific personnel in the airport is searched to determine if a specific person with a similarity to the first target object meets a preset condition exists. If such a person exists, the first target object is identified as a specific person. Thus, in this embodiment, the position of the bounding box of the first target object, its height, weight, posture, and facial features are used to perform spatiotemporal tracking and association with specific personnel in a pre-established database of specific personnel to identify specific personnel. This improves the accuracy of identifying specific personnel in the airport. Furthermore, in this embodiment, the airport travel information of each second target object is determined based on the detection parameters obtained by target detection of historical surveillance videos, and an airport-specific personnel database is established using the airport travel information of each target object, ensuring the accuracy of the airport characteristic personnel database and further improving the accuracy of identifying specific personnel in the airport. The solution of this disclosure will be described in detail below with reference to the accompanying drawings.

[0083] like Figure 1 As shown, an application scenario for a method of identifying specific personnel in an airport is illustrated. This application scenario includes multiple cameras 110, a server 120, and terminal devices 130, such as... Figure 1As shown, this application scenario uses an electronic device as a server as an example. However, this application embodiment does not limit the electronic device; the electronic device can be a server or a terminal device, and can be configured according to actual conditions. Server 120 can be implemented using a single server or multiple servers. Server 120 can be implemented using a physical server or a virtual server.

[0084] In one possible application scenario, camera 110 captures images of various target areas of the airport in real time. At specified intervals, server 120 acquires surveillance videos of multiple target areas within the specified intervals. Server 120 uses a target detection algorithm to perform target detection on the acquired surveillance video images, obtaining detection parameters for each first target object in the target video images. These detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, weight features, human posture features, and facial features. For any given first target object, if server 120 finds a target specific person in a pre-stored airport specific personnel database whose similarity to the first target object meets a first preset condition based on the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, then the target target object is determined to be a target specific person. The first target object is airport-specific personnel; wherein, server 120 obtains the airport-specific personnel database in the following manner: using the target detection algorithm to perform target detection on each target historical video image in the historical surveillance videos of the multiple target areas within a specified historical time period, obtaining the detection parameters of each second target object in each target historical video image, and based on the detection parameters of each second target object in each target historical video image, obtaining the similarity between each second target object in any two target historical video images, obtaining the airport travel information of each second target object within the specified historical time period based on the similarity between each second target object in any two target historical video images, storing the target detection parameters of each second target object whose airport travel information meets the second preset condition, thereby obtaining the airport-specific personnel database, wherein the target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.

[0085] In another possible scenario, such as Figure 2As shown, the system includes a camera 110 and a terminal device 130. The camera 110 captures images of various target areas of the airport in real time. Every specified time interval, the terminal device 130 acquires surveillance videos of multiple target areas within the specified time interval. The terminal device 130 uses a target detection algorithm to perform target detection on the acquired surveillance video images, obtaining detection parameters for each first target object in the target video images. The detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, weight features, human posture features, and facial features. For any first target object, if the terminal device 130 finds a target specific person in a pre-stored airport specific personnel database whose similarity to the first target object meets a first preset condition based on the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, the system will be able to detect the target specific person. If the target object is identified as an airport-specific personnel, then the terminal device 130 obtains the airport-specific personnel database in the following manner: It uses the target detection algorithm to perform target detection on each target historical video image in the historical surveillance videos of the multiple target areas within a specified historical time period, obtaining the detection parameters of each second target object in each target historical video image. Based on the detection parameters of each second target object in each target historical video image, it obtains the similarity between each second target object in any two target historical video images. Based on the similarity between each second target object in any two target historical video images, it obtains the airport travel information of each second target object within the specified historical time period. It stores the target detection parameters of each second target object whose airport travel information meets the second preset condition, thus obtaining the airport-specific personnel database. The target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.

[0086] in, Figure 1 The server 120 and the terminal device 130 can exchange information through a communication network. The communication network can be either wireless or wired.

[0087] For example, server 120 can access the network via cellular mobile communication technology and communicate with terminal device 130, wherein the cellular mobile communication technology includes, for example, 5th Generation Mobile Networks (5G) technology.

[0088] Optionally, the server 120 can access the network and communicate with the terminal device 130 via short-range wireless communication, wherein the short-range wireless communication method includes, for example, Wireless Fidelity (Wi-Fi) technology.

[0089] The description in this application focuses on a single server camera 110, a single server 120, and a single terminal device 130. However, those skilled in the art should understand that the illustrated camera 110, server 120, and terminal device 130 are intended to illustrate the operation of the camera 110, server 120, and terminal device 130 involved in the technical solution of this application, and do not imply any limitation on the number, type, or location of the camera 110, server 120, and terminal device 130. It should be noted that adding additional modules to or removing individual modules from the illustrated environment will not change the underlying concept of the exemplary embodiments of this application.

[0090] It should be noted that the method for identifying specific personnel in airports proposed in this application is not only applicable to... Figure 1 and Figure 2 The application scenarios shown can also be applied to any identification device for specific individuals in an airport.

[0091] The following describes an exemplary embodiment of the method for identifying specific personnel in an airport, in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the methods and principles of this application, and the implementation of this application is not limited in any way in this respect.

[0092] like Figure 3 The diagram shown is a flowchart illustrating the method for identifying specific individuals in an airport according to this disclosure, which may include the following steps:

[0093] Step 301: At specified intervals, acquire surveillance videos of multiple target areas in the airport within the specified intervals;

[0094] In this application, the specified duration in the real-time example can be 1 minute, 10 minutes, or 30 minutes, etc., and can be set according to the actual situation. This application embodiment does not impose any limitation on this. Furthermore, the target area in this application embodiment is pre-set and can be set according to the actual situation. This application embodiment does not impose any limitation on this.

[0095] Step 302: Use a target detection algorithm to perform target detection on the target video image in the acquired surveillance video to obtain the detection parameters of each first target object in the target video image. The detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, weight features, human posture features and face features.

[0096] In this embodiment, the target video image in each surveillance video is obtained by frame extraction, meaning that video images at specified frame intervals are determined as target video images. For example, if the specified frame interval is 5 frames, and the video has a total of 30 frames, then the video images corresponding to the 6th, 12th, 18th, and 24th frames are determined as target video images. However, this embodiment does not limit the method for determining the target video image and can be set according to actual conditions. Furthermore, the target video images in this embodiment include RGB target video images and depth target video images.

[0097] The target detection algorithm used in this embodiment is ITMFF. However, this embodiment does not limit the target detection algorithm; the target detection algorithm in this embodiment can be set according to the actual situation. ITMFF will be described below. Figure 4 The diagram shows the structure of ITMFF, which includes residual module 401, residual module 402, ASPP (atrous spatial pyramid pooling) layer 403, atrous spatial pyramid pooling layer 404, five fusion layers 405, four feature concatenation layers 406, and four upsampling layers 407. Residual module 401 contains five consecutive convolutional layers: convolutional layer 4011, convolutional layer 4012, convolutional layer 4013, convolutional layer 4014, and convolutional layer 4015. Similarly, residual module 402 contains five consecutive convolutional layers: convolutional layer 4021, convolutional layer 4012, convolutional layer 4013, convolutional layer 4014, and convolutional layer 4015.

[0098] in:

[0099] The RGB target video image is input into the residual module 401. The convolutional layer 4011 in the residual module 401 extracts features from the RGB target video image to obtain a first RGB feature map. Then, the convolutional layer 4012 extracts features from the first RGB feature map to obtain a second RGB feature map. The second RGB feature map is then input into the convolutional layer 4013 and the fusion layer 405. The convolutional layer 4013 extracts features from the second RGB feature map to obtain a third RGB feature map. The third RGB feature map is then input into the convolutional layer 4014 and the fusion layer 405. The convolutional layer 4014 extracts features from the third RGB feature map to obtain a fourth RGB feature map. Finally, the fourth RGB feature map is input into the convolutional layer 4015 and the fusion layer 405. The convolutional layer 4015 extracts features from the fourth RGB feature map to obtain a fifth RGB feature map.

[0100] Similarly, the depth target video image is input into the residual module 402. The convolutional layer 4021 in the residual module 402 extracts features from the depth target video image to obtain a first depth feature map. Then, the convolutional layer 4022 extracts features from the first depth feature map to obtain a second depth feature map. The second depth feature map is then input into the convolutional layer 4023 and the fusion layer 405. The convolutional layer 4023 extracts features from the second depth feature map to obtain a third depth feature map. The third depth feature map is then input into the convolutional layer 4024 and the fusion layer 405. The convolutional layer 4024 extracts features from the third depth feature map to obtain a fourth depth feature map. Finally, the fourth depth feature map is input into the convolutional layer 4025 and the fusion layer 405. The convolutional layer 4025 extracts features from the fourth depth feature map to obtain a fifth depth feature map.

[0101] Convolutional layer 4015 inputs the fifth RGB feature map into the dilated spatial pyramid pooling layer 403 for feature extraction, obtaining the sixth RGB feature map. Similarly, convolutional layer 4025 inputs the fifth depth feature map into the dilated spatial pyramid pooling layer 404, obtaining the sixth depth feature map. Then, the sixth RGB feature map and the sixth depth feature map are fused by fusion layer 405 to obtain the first fused feature map. The fifth RGB feature map and the fifth depth feature map are fused by fusion module 405 to obtain the second fused feature map. The fourth RGB feature map and the fourth depth feature map are fused by fusion module 405 to obtain the third fused feature map. The third RGB feature map and the third depth feature map are fused by fusion module 405 to obtain the fourth fused feature map. Finally, the second RGB feature map and the second depth feature map are fused by fusion module 405 to obtain the fifth fused feature map. Then, the first and second fused features are concatenated by the feature concatenation layer 406 and upsampled by the upsampling layer 407 to obtain the first intermediate feature. The first intermediate feature and the third fused feature are then concatenated by the feature concatenation layer 406 and upsampled by the upsampling layer 407 to obtain the second intermediate feature. The second intermediate feature and the fourth fused feature are then concatenated by the feature concatenation layer 406 and upsampled by the upsampling layer 407 to obtain the third intermediate feature. Finally, the third intermediate feature and the fifth fused feature are concatenated by the feature concatenation layer 406 and upsampled by the upsampling layer 407 to obtain the target image. The target image contains the bounding box position of the target object in the target video image and the corresponding target feature vector.

[0102] For example, such as Figure 5 As shown, this is the obtained target image. It can be seen from the image that the bounding box of the target object contains the position coordinates and the corresponding target feature vector.

[0103] Step 303: If, based on the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, a target specific person whose similarity to the first target object meets the first preset condition is found in the pre-stored airport specific person database, then the first target object is determined to be an airport characteristic person.

[0104] In one embodiment, step 303 may be specifically implemented as follows: based on the detection parameters of each airport-specific person in the airport-specific personnel database and the detection parameters of the first target object, obtain the similarity between each airport-specific person and the first target object; if the similarity satisfies the first preset condition, then determine that the airport-specific person is the target-specific person corresponding to the first target object.

[0105] The first preset condition can be that the similarity is greater than a first specified similarity. In this embodiment, the first specified similarity can be 0.8. However, this embodiment does not limit the first preset condition and the first specified similarity. The first preset condition and the first specified similarity can be set according to the specific actual situation.

[0106] The following describes the method for determining the similarity between a specific person at any airport and the first target object, such as... Figure 6 The diagram shown illustrates the process for determining similarity, which may include the following steps:

[0107] Step 601: Obtain the first similarity index based on the location coordinates of the bounding box of the specific personnel at the airport and the location coordinates of the bounding box of the first target object;

[0108] In one embodiment, step 601 can be specifically implemented as follows: Based on the location coordinates of the bounding box of the airport-specific personnel and the location coordinates of the bounding box of the first target object, obtain the cosine distance between the airport-specific personnel and the first target object, and determine the first similarity index based on the cosine distance. The cosine distance can be obtained using formula (1):

[0109]

[0110] Where d1 is the cosine distance, x1 is the horizontal coordinate of the center point of the bounding box of the airport-specific personnel, y1 is the vertical coordinate of the center point of the bounding box of the airport-specific personnel, x2 is the horizontal coordinate of the center point of the bounding box of the first target object, and y2 is the vertical coordinate of the center point of the bounding box of the first target object.

[0111] Step 602: Obtain a second similarity index using the target feature vector of the specific personnel at the airport and the target feature vector of the first target object;

[0112] In one embodiment, step 602 can be specifically implemented as follows: determining the Mahalanobis distance between the airport-specific personnel and the first target object based on the target feature vector of the airport-specific personnel and the target feature vector of the first target object, and determining the second similarity index based on the Mahalanobis distance. The Mahalanobis distance between the airport-specific personnel and the first target object can be obtained using formula (2):

[0113]

[0114] Where d2 is the Mahalanobis distance, x i Let x be the i-th vector in the target feature vector of the first target object.j Let be the j-th vector in the target feature vector of a specific person at the airport, and n be the total number of vectors in the target feature vector.

[0115] Step 603: Based on the first similarity index and the second similarity index, obtain the similarity between the specific personnel at the airport and the first target object.

[0116] In this embodiment, the method for determining the first similarity index is the same as the method for determining the second similarity index. The following explanation uses the method for determining the first similarity index as an example:

[0117] Method 1: Determine the cosine distance as the first similarity index.

[0118] Method 2: Multiply the cosine distance by the preset weight to obtain the first similarity index.

[0119] It should be noted that the method for determining the second similarity index is the same as that for determining the first similarity index, except that the cosine distance is replaced by the Mahalanobis distance. This will not be described in detail in the embodiments of this application.

[0120] In one embodiment, step 603 can be specifically implemented as follows: if the first similarity index and the second similarity index are obtained using the method described above, then the first similarity index and the second similarity index are added together to obtain the total similarity index between the first target object and the specific personnel at the airport, and the total similarity index is normalized to obtain the similarity between the specific personnel and the first target object. If the first similarity index and the second similarity index are obtained using the method described above, then the first similarity index and the second similarity index are weighted and summed to obtain the total similarity index between the first target object and the specific personnel, and the total similarity index is normalized to obtain the similarity between the specific personnel and the first target object.

[0121] In this embodiment, the normalization method may be to divide the total similarity index by a normalization parameter, where the normalization parameter is either the total similarity index with the highest value between the first target object and specific personnel at each airport, or the sum of the total similarity indices between the first target object and specific personnel at each airport. The specific normalization method is not limited in this embodiment and can be set according to actual circumstances.

[0122] To facilitate the implementation of control measures by management personnel and to collect evidence from specific personnel at the airport, in one embodiment, such as Figure 7 The diagram shown illustrates a process for determining the trajectory of specific individuals at an airport, which may include the following steps:

[0123] Step 701: For any first target object that is a specific person at the airport, using the pre-set hierarchical relationship between target areas, determine and identify whether there is a corresponding adjacent target area in the target area where the surveillance video of the first target object is located. If yes, proceed to step 702; if no, proceed to step 705. The adjacent target area includes the target area at the next higher level and the target area at the next lower level in the hierarchical relationship.

[0124] In this embodiment, the target area where the surveillance video is located is obtained using a pre-set relationship between the camera and the target area. As shown in Table 1:

[0125] target area Camera Target Area A Camera a, Camera b, Camera c Target area B Camera d, Camera e, Camera f, Camera g Target area C Camera m, camera n, camera r … …

[0126] Table 1 shows the cameras corresponding to each target area. For example, target area A contains cameras a, b, and c.

[0127] like Figure 8 The diagram shows the hierarchical relationship between the target regions. Figure 8 The representation uses a multi-way tree. For example, if the target region C corresponds to the target region A at the upper level and the target region F at the lower level, then the representation is possible.

[0128] Step 702: For any adjacent target area, after determining the monitoring video of the adjacent target area within the specified duration as the acquired monitoring video, return to step 302. The duration of the monitoring video acquired in the adjacent target area is the same as the duration of the monitoring video acquired in the target area, and the time period of the monitoring video acquired in the adjacent target area is different from the time period of the monitoring video acquired in the target area.

[0129] For example, if the time period for acquiring surveillance video of the target area is 10:00-10:30 on June 12, 2022, then the time period for the target area at the next higher level is 9:30-10:00 on June 12, 2022, and the time period for the target area at the next lower level is 10:30-11:00 on June 12, 2022.

[0130] Step 703: Determine whether the first target object exists in the surveillance video of the adjacent target area within the specified time period. If yes, proceed to step 704; otherwise, proceed to step 706.

[0131] Step 704: After determining the adjacent target area as the target area, return to step 701;

[0132] Step 705: Based on the hierarchical relationship between each target region, obtain the trajectory of the first target object;

[0133] For example, if the target regions are obtained in order of hierarchy from front to back as: target region A, target region C, target region F, and target region H, then the trajectory of the first target object is: target region A → target region C → target region F → target region H. Specifically, as shown below... Figure 9 As shown, Figure 9 The middle part marks the trajectories on the map in chronological order. However, Figure 9 The examples provided are for illustrative purposes only. The specific trajectory representation can be set according to the actual situation, and the embodiments in this application are not limited herein.

[0134] Step 706: Issue a warning for specific personnel based on the trajectory of the first target object and the image of the first target object in the surveillance video.

[0135] In this embodiment of the application, the method for issuing a warning for a specific person is to send the trajectory of the first target object and the image of the first target object in the monitoring video to the administrator's terminal device. However, this embodiment of the application does not limit the method of issuing a warning for a specific person. The method of issuing a warning for a specific person in this embodiment of the application can be set according to the actual situation.

[0136] The following describes the methods for establishing a database of specific personnel within an airport, such as... Figure 10 The diagram shown illustrates the process for establishing a database of specific individuals within an airport, including the following steps:

[0137] Step 1001: Obtain historical surveillance videos of the multiple target areas in the airport within a specified historical time period;

[0138] The specified historical time period in this application embodiment can be set according to the actual situation, and this application embodiment does not limit it.

[0139] Step 1002: Use the target detection algorithm to perform target detection on each target historical video image in the historical surveillance video to obtain the detection parameters of each second target object in each target historical video image. The detection parameters include the position coordinates of the bounding box of the second target object in the target video image and the target feature vector of the second target object. The target feature vector includes height features, weight features, human posture features and face features.

[0140] The target detection algorithm in this embodiment is the ITMFF algorithm described above, and will not be repeated here.

[0141] Step 1003: Based on the detection parameters of each second target object in each target historical video image, obtain the similarity between each second target object in any two target historical video images, and based on the similarity, obtain each target historical video image where the same second target object is located;

[0142] In this embodiment, two second target objects with a similarity greater than a second specified similarity are determined to be the same second target object. For example, if the similarity between second target object A in target historical video image 1 and second target object B in target historical video image 2 is greater than the second specified similarity, then second target object A and second target object B are determined to be the same second target object.

[0143] If all of the target historical video images 1, 4, 5, 6, and 8 contain a second target object whose similarity to the second target object B in target historical video image 2 is greater than the second specified similarity, then the target historical video images containing the second target object B are determined to be: target historical video image 1, target historical video image 4, target historical video image 5, target historical video image 6, and target historical video image 8.

[0144] The method for determining the similarity between two second target objects in step 1003 is the same as the method for determining the similarity between the first target object and any specific person at an airport, as described above, and will not be repeated here. Furthermore, the values ​​of the second specified similarity and the second specified similarity in this embodiment can be the same or different, and can be set according to the actual situation. This embodiment does not limit the value of the second specified similarity.

[0145] Step 1004: Based on the historical video images of the same second target object, determine the frequency of the second target object's appearance and the travel time within each specified time period, wherein the historical time period includes each specified time period;

[0146] In one embodiment, step 1004 may be specifically implemented as follows: based on the shooting time corresponding to each target historical video image where the second target object is located, determine the target historical video images whose shooting time is within a specified time period, and based on the target historical video images whose shooting time is within the specified time period, count the frequency of the second target object appearing in each specified time period and the travel time period.

[0147] The frequency can be determined by dividing the specified time period into multiple sub-time periods, determining the sub-time periods corresponding to each target historical video image based on the shooting time of each target historical video image, counting the target sub-time periods in each target historical video image where the second target object is located in each target historical video image corresponding to each sub-time period, and determining the number of the target sub-time periods as the frequency of occurrence in the specified segment, and determining the travel time period based on the shooting time of each target historical video image where the second target object is located.

[0148] For example, if the specified time period is 2022.01.01.00:00-2022.01.07.24:00, then the multiple sub-time periods are: 2022.01.01.00:00-2022.01.01.24:00, 2022.01.02.00:00-2022.01.02.24:00, and 2022.01.03.00:00-2022. The time periods are defined as follows: 2022.01.03.24:00, 2022.01.04.00:00-2022.01.04.24:00, 2022.01.05.00:00-2022.01.05.24:00, 2022.01.06.00:00-2022.01.06.24:00, and 2022.01.07.00:00-2022.01.07.24:00. If it is determined that the target historical video images corresponding to each target exist in each target historical video image within each sub-time period, then the frequency of the second target object A appearing in the specified time period 2022.01.01.00:00-2022.01.07.24:00 is determined to be 7.

[0149] In this embodiment, the length of the specified time period can be one week, one month, half a month, etc., and can be set according to the actual situation. This embodiment does not limit this. Furthermore, the length of the sub-time periods in this embodiment can be set according to the actual situation. This embodiment does not limit this.

[0150] Step 1005: Using the frequency of occurrence and travel time of each of the second target objects in each specified time period, obtain the airport travel information of each of the second target objects;

[0151] In one embodiment, the frequency of occurrence of each second target object within a specified time period and the travel time period are determined as the airport travel information.

[0152] Step 1006: Store the target detection parameters of each second target object whose airport travel information meets the second preset condition to obtain the airport characteristic personnel database, wherein the target detection parameters are the detection parameters obtained from the historical surveillance video of the target with the smallest difference between the shooting time and the current time.

[0153] In this embodiment, the second preset condition may be that the frequency is greater than a specified frequency and / or the number of time intervals containing the travel time period is greater than a specified number. The second preset condition in this embodiment can be set according to actual circumstances, and this embodiment does not limit the second preset condition. The time intervals in this embodiment are pre-set and can be set according to actual circumstances.

[0154] Based on the same disclosed concept, the method for identifying specific personnel in an airport as described above can also be implemented by an identification device for specific personnel in an airport. The effect of this identification device is similar to that of the aforementioned method, and will not be described again here.

[0155] Figure 11 This is a schematic diagram of the structure of an identification device for specific personnel in an airport according to an embodiment of the present disclosure.

[0156] like Figure 11 As shown, the airport specific personnel identification device 1100 disclosed herein may include an airport surveillance video acquisition module 1110, a target object detection module 1120, and a specific personnel detection module 1130.

[0157] The airport surveillance video acquisition module 1110 is used to acquire surveillance videos of multiple target areas in the airport within the specified time intervals.

[0158] The target object detection module 1120 is used to perform target detection on target video images in the acquired surveillance video using a target detection algorithm, and obtain detection parameters for each first target object in the target video image. The detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, weight features, human posture features, and facial features.

[0159] The specific personnel detection module 1130 is used to determine that, for any first target object, if a specific target object whose similarity to the first target object meets a first preset condition is found in a pre-stored airport specific personnel database based on the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, the first target object is identified as an airport specific personnel. The airport specific personnel database is obtained as follows: The target detection algorithm is used to perform target detection on each historical video image of the multiple target areas within a specified historical time period, obtaining detection parameters for each second target object in each historical video image. Based on the detection parameters of each second target object in each historical video image, the similarity between each second target object in any two historical video images is obtained. Based on the similarity between each second target object in any two historical video images, airport travel information of each second target object within the specified historical time period is obtained. The target detection parameters of each second target object whose airport travel information meets the second preset condition are stored to obtain the airport specific personnel database. The target detection parameters are the detection parameters obtained from the historical video image with the smallest difference between the shooting time and the current time.

[0160] In one embodiment, the apparatus further includes:

[0161] The trajectory determination module 1140 is used, after determining that the first target object is an airport-specific personnel, to determine, for any first target object that is an airport-specific personnel, adjacent target areas corresponding to the target area where the surveillance video of the first target object is located, using a pre-set hierarchical relationship between target areas; wherein, the adjacent target areas include target areas one level above the target area and target areas one level below the target area in the hierarchical relationship; and,

[0162] For any adjacent target area, after determining the monitoring video of the adjacent target area within the specified duration as the acquired monitoring video, the step of performing target detection on the target video image in the acquired monitoring video using a target detection algorithm is returned. The duration of the monitoring video acquired in the adjacent target area is the same as the duration of the monitoring video acquired in the target area, and the time period of the monitoring video acquired in the adjacent target area is different from the time period of the monitoring video acquired in the target area.

[0163] If the first target object exists in the surveillance video of the adjacent target area within the specified time period, then after determining the adjacent target area as the target area, the process returns to the step of determining the adjacent target area corresponding to the target area where the surveillance video of the identified target object is located, using the pre-set hierarchical relationship between each target area, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and,

[0164] Based on the hierarchical relationship between each target region, the trajectory of the first target object is obtained;

[0165] Airport-specific personnel warnings are issued based on the trajectory of the first target object and its image in surveillance video.

[0166] In one embodiment, the apparatus further includes:

[0167] The comparison module 1150 is used to determine whether there is a target specific person in the airport specific personnel database whose similarity to the first target object meets a first preset condition through the following methods:

[0168] For any specific person in the database of specific persons in the airport, the similarity between the specific person and the first target object is obtained based on the detection parameters of the specific person and the detection parameters of the first target object;

[0169] If the similarity meets the preset condition, then the specific person is determined to be the target specific person corresponding to the first target object.

[0170] In one embodiment, the apparatus further includes:

[0171] The similarity determination module 1160 is used to obtain the similarity between any specific person at the airport and the first target object in the following manner:

[0172] Based on the location coordinates of the bounding box of the specific personnel at the airport and the location coordinates of the bounding box of the first target object, a first similarity index is obtained; and,

[0173] The second similarity index is obtained by using the target feature vector of the specific personnel at the airport and the target feature vector of the first target object;

[0174] Based on the first similarity index and the second similarity index, the similarity between the specific personnel at the airport and the first target object is obtained.

[0175] In one embodiment, the similarity determination module 1160 is specifically used for:

[0176] Based on the location coordinates of the bounding box of the airport-specific personnel and the location coordinates of the bounding box of the first target object, the cosine distance between the airport-specific personnel and the first target object is obtained, and the first similarity index is determined based on the cosine distance.

[0177] Based on the target feature vector of the airport-specific personnel and the target feature vector of the first target object, the Mahalanobis distance between the airport-specific personnel and the first target object is determined, and the second similarity index is determined based on the Mahalanobis distance.

[0178] In one embodiment, the specific person detection module 1130 is further configured to:

[0179] Based on the similarity between the second target objects in any two target historical video images, the target historical video images containing the same second target object are obtained.

[0180] Based on historical video images of the same second target object, determine the frequency of the second target object's appearance and travel time within each specified time period, wherein the historical time period includes each specified time period;

[0181] By utilizing the frequency of occurrence and travel time of each of the second target objects within each specified time period, the airport travel information of each of the second target objects is obtained.

[0182] Having introduced a method and electronic device for identifying specific persons in an airport according to an exemplary embodiment of the present disclosure, the following describes an electronic device according to another exemplary embodiment of the present disclosure.

[0183] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0184] In some possible implementations, the electronic device according to this disclosure may include at least one processor and at least one computer storage medium. The computer storage medium stores program code that, when executed by the processor, causes the processor to perform steps in the airport identification method according to various exemplary embodiments of this disclosure described above. For example, the processor may perform actions such as... Figure 3 Steps 301-303 are shown in the diagram.

[0185] The following reference Figure 12 To describe an electronic device 1200 according to such an embodiment of the present disclosure. Figure 12 The electronic device 1200 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0186] like Figure 12 As shown, the electronic device 1200 is manifested in the form of a general electronic device. The components of the electronic device 1200 may include, but are not limited to: at least one processor 1201, at least one computer storage medium 1202, and a bus 1203 connecting different system components (including the computer storage medium 1202 and the processor 1201).

[0187] Bus 1203 represents one or more of several bus structures, including a computer storage media bus or computer storage media controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0188] Computer storage medium 1202 may include readable media in the form of volatile computer storage media, such as random access computer storage medium (RAM) 1221 and / or cache storage medium 1222, and may further include read-only computer storage medium (ROM) 1223.

[0189] The computer storage medium 1202 may also include a program / utility 1225 having a set (at least one) of program modules 1224, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0190] Electronic device 1200 can also communicate with one or more external devices 1204 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 1200, and / or with any device that enables electronic device 1200 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1205. Furthermore, electronic device 1200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1206. As shown, network adapter 1206 communicates with other modules used in electronic device 1200 via bus 1203. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0191] In some possible implementations, various aspects of the method for identifying specific persons in an airport provided by this disclosure can also be implemented in the form of a program product, which includes program code that, when the program product is run on a computer device, causes the computer device to perform the steps in the method for identifying specific persons in an airport according to various exemplary embodiments of this disclosure as described above.

[0192] The program product may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access computer storage media (RAM), read-only computer storage media (ROM), erasable programmable read-only computer storage media (EPROM or flash memory), optical fibers, portable compact disk read-only computer storage media (CD-ROM), optical computer storage media, magnetic computer storage media, or any suitable combination thereof.

[0193] The program product for identifying specific personnel in an airport according to embodiments of this disclosure can be a portable compact disc read-only computer storage medium (CD-ROM) and include program code, and can run on an electronic device. However, the program product of this disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0194] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0195] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0196] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's electronic device, partially on the user's device, as a standalone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user's electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0197] It should be noted that although several modules of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0198] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0199] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk computer storage media, CD-ROMs, optical computer storage media, etc.) containing computer-usable program code.

[0200] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0201] These computer program instructions may also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable computer storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0202] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0203] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for identifying specific personnel in an airport, characterized in that, The method includes: At specified intervals, acquire surveillance videos of multiple target areas within the airport within the specified intervals; A target detection algorithm is used to detect targets in the acquired surveillance video images, obtaining detection parameters for each first target object in the target video image. These detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, body shape features, human posture features, and facial features. For any given first target object, if, based on the bounding box coordinates of the first target object in the target video image and the target feature vector, a target specific person whose similarity to the first target object meets a first preset condition is found in a pre-stored airport specific personnel database, then the first target object is determined to be an airport characteristic personnel; wherein, the airport specific personnel database is obtained in the following manner: The target detection algorithm is used to perform target detection on historical video images of each target region within a specified historical time period. Detection parameters of each second target object in each historical video image are obtained. Based on the detection parameters of each second target object in each historical video image, the similarity between each second target object in any two historical video images is obtained. Based on the similarity between each second target object in any two historical video images, airport travel information of each second target object within the specified historical time period is obtained. The target detection parameters of each second target object whose airport travel information meets a second preset condition are stored to obtain the airport characteristic personnel database. The target detection parameters are the detection parameters obtained from the historical video image of the target region with the smallest difference between the shooting time and the current time.

2. The method according to claim 1, characterized in that, After determining that the first target is a specific person at the airport, the method further includes: For any given first target being a specific person at the airport, using a pre-defined hierarchical relationship between target areas, adjacent target areas are determined corresponding to the target area where the surveillance video of the first target is located; wherein, the adjacent target areas include target areas one level above the target area and target areas one level below the target area in the hierarchical relationship; and, For any adjacent target area, after determining the monitoring video of the adjacent target area within the specified duration as the acquired monitoring video, the step of performing target detection on the target video image in the acquired monitoring video using a target detection algorithm is returned. The duration of the monitoring video acquired in the adjacent target area is the same as the duration of the monitoring video acquired in the target area, and the time period of the monitoring video acquired in the adjacent target area is different from the time period of the monitoring video acquired in the target area. If the first target object exists in the surveillance video of the adjacent target area within the specified time period, then after determining the adjacent target area as the target area, the process returns to the step of determining the adjacent target area corresponding to the target area where the surveillance video of the identified target object is located, using the pre-set hierarchical relationship between each target area, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and, Based on the hierarchical relationship between each target region, the trajectory of the first target object is obtained; Airport-specific personnel warnings are issued based on the trajectory of the first target object and its image in surveillance video.

3. The method according to claim 1, characterized in that, The following method is used to determine whether a target specific person exists in the airport specific personnel database whose similarity to the first target object meets a first preset condition: Based on the detection parameters of each airport-specific person in the airport-specific personnel database and the detection parameters of the first target object, the similarity between each airport-specific person and the first target object is obtained. If the similarity satisfies the first preset condition, then the airport-specific personnel are determined to be the target-specific personnel corresponding to the first target object.

4. The method according to claim 1 or 3, characterized in that, The similarity between any specific person at the airport and the first target object is obtained in the following way: Based on the location coordinates of the bounding box of the specific personnel at the airport and the location coordinates of the bounding box of the first target object, a first similarity index is obtained; and, The second similarity index is obtained by using the target feature vector of the specific personnel at the airport and the target feature vector of the first target object; Based on the first similarity index and the second similarity index, the similarity between the specific personnel at the airport and the first target object is obtained.

5. The method according to claim 4, characterized in that, The step of obtaining a first similarity index based on the bounding box coordinates of the specific airport personnel and the bounding box coordinates of the first target object includes: Based on the location coordinates of the bounding box of the airport-specific personnel and the location coordinates of the bounding box of the first target object, the cosine distance between the airport-specific personnel and the first target object is obtained, and the first similarity index is determined based on the cosine distance. The second similarity index is obtained by using the target feature vectors of specific personnel at the airport and the target feature vectors of the first target object, including: Based on the target feature vector of the airport-specific personnel and the target feature vector of the first target object, the Mahalanobis distance between the airport-specific personnel and the first target object is determined, and the second similarity index is determined based on the Mahalanobis distance.

6. The method according to claim 1, characterized in that, The step of obtaining airport travel information for each second target object within a specified historical time period based on the similarity between each second target object in any two target historical video images includes: Based on the similarity between the second target objects in any two target historical video images, the target historical video images containing the same second target object are obtained. Based on historical video images of the same second target object, determine the frequency of the second target object's appearance and travel time within each specified time period, wherein the historical time period includes each specified time period; By utilizing the frequency of occurrence and travel time of each of the second target objects within each specified time period, the airport travel information of each of the second target objects is obtained.

7. An electronic device, characterized in that, It includes a processor and a memory, which are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: At specified intervals, acquire surveillance video of multiple target areas in the airport within the specified interval; A target detection algorithm is used to detect targets in the acquired surveillance video images, obtaining detection parameters for each first target object in the target video image. These detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object. The target feature vector includes height features, body shape features, human posture features, and facial features. For any given first target object, if, based on the bounding box coordinates of the first target object in the target video image and the target feature vector, a target specific person whose similarity to the first target object meets a first preset condition is found in a pre-stored airport specific personnel database, then the first target object is determined to be an airport characteristic personnel; wherein, the airport specific personnel database is obtained in the following manner: The target detection algorithm is used to perform target detection on historical video images of each target region within a specified historical time period. Detection parameters of each second target object in each historical video image are obtained. Based on the detection parameters of each second target object in each historical video image, the similarity between each second target object in any two historical video images is obtained. Based on the similarity between each second target object in any two historical video images, airport travel information of each second target object within the specified historical time period is obtained. The target detection parameters of each second target object whose airport travel information meets a second preset condition are stored to obtain the airport characteristic personnel database. The target detection parameters are the detection parameters obtained from the historical video image of the target region with the smallest difference between the shooting time and the current time.

8. The electronic device according to claim 7, characterized in that, The processor is also configured to: After determining that the first target object is a specific airport personnel, for any first target object that is a specific airport personnel, using a pre-set hierarchical relationship between target areas, adjacent target areas corresponding to the target area where the surveillance video of the first target object is located are determined; wherein, the adjacent target areas include target areas at the one-level above the target area and target areas at the one-level below the target area in the hierarchical relationship; and, For any adjacent target area, after determining the monitoring video of the adjacent target area within the specified duration as the acquired monitoring video, the step of performing target detection on the target video image in the acquired monitoring video using a target detection algorithm is returned. The duration of the monitoring video acquired in the adjacent target area is the same as the duration of the monitoring video acquired in the target area, and the time period of the monitoring video acquired in the adjacent target area is different from the time period of the monitoring video acquired in the target area. If the first target object exists in the surveillance video of the adjacent target area within the specified time period, then after determining the adjacent target area as the target area, the process returns to the step of determining the adjacent target area corresponding to the target area where the surveillance video of the identified target object is located, using the pre-set hierarchical relationship between each target area, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and, Based on the hierarchical relationship between each target region, the trajectory of the first target object is obtained; Airport-specific personnel warnings are issued based on the trajectory of the first target object and its image in surveillance video.

9. The electronic device according to claim 7, characterized in that, The processor is also configured to: The following method is used to determine whether a target specific person exists in the airport specific personnel database whose similarity to the first target object meets a first preset condition: Based on the detection parameters of each airport-specific person in the airport-specific personnel database and the detection parameters of the first target object, the similarity between each airport-specific person and the first target object is obtained. If the similarity satisfies the first preset condition, then the airport-specific personnel are determined to be the target-specific personnel corresponding to the first target object.

10. The electronic device according to claim 7 or 9, characterized in that, The processor is also configured to: The similarity between any specific person at the airport and the first target object is obtained in the following way: Based on the location coordinates of the bounding box of the specific personnel at the airport and the location coordinates of the bounding box of the first target object, a first similarity index is obtained; and, The second similarity index is obtained by using the target feature vector of the specific personnel at the airport and the target feature vector of the first target object; Based on the first similarity index and the second similarity index, the similarity between the specific personnel at the airport and the first target object is obtained.

Images