Method for identifying specific personnel in airport and electronic equipment
By using object detection algorithms and feature vector alignment technology in airports, identifying and tracking specific people, the problem of limited traditional face recognition technology is solved, the recognition accuracy and efficiency are improved, and misrecognition is reduced.
Patent Information
- Application Number
- CN202310576102.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-05-19
AI Technical Summary
The prior art has low efficiency and accuracy when identifying specific personnel in airports, especially because traditional facial recognition technology is affected by masks, resulting in high misidentification rates and inability to effectively detect specific personnel.
By obtaining surveillance videos of multiple target areas of the airport, using the object detection algorithm for object detection, obtaining the location and feature vectors of the bounding box, including height, fatness, posture and face features, and comparing them with the pre-stored specific personnel database. Establish space-time relationships and improve recognition accuracy.
It improves the accuracy and efficiency of identification of specific personnel in the airport, reduces misidentification, ensures the accuracy of the database of specific personnel, and realizes effective identification and early warning of specific personnel.
Smart Images

Figure CN119007057A_ABST
Abstract
Description
Background Art
[0002] In the application scenario of airport hub operation management, specific personnel (such as scalpers or illegal taxi drivers, etc.) have become a common phenomenon and have formed a mature illegal profit chain. This illegal operation behavior has great safety hazards, easily infringes on passengers' property, damages the operating order of the passenger transport market, and in serious cases can cause social security cases.
[0003] In the prior art, there are two main ways to identify specific personnel in airports: one is to intervene and manage through video surveillance or on-site manual methods by airport security and order maintenance personnel. This method lacks effective technical auxiliary management methods, and has low efficiency and accuracy. The other is to report people who stay in the airport for a long time as specific personnel based on face comparison analysis. However, in recent years, users who travel generally wear masks, and there are technical limitations in the comparison and analysis of specific personnel based on traditional face recognition methods, making it difficult to effectively detect faces. Moreover, simply identifying people who stay for a long time as specific personnel will cause a large number of normal passengers to be identified as specific personnel and reported, so the accuracy of identifying specific personnel in airports is low. Summary of the invention
[0004] In an exemplary embodiment of the present disclosure, a method and electronic device for identifying a specific person in an airport are provided, which are used to improve the recognition accuracy of the specific person in the airport.
[0005] A first aspect of the present disclosure provides a method for identifying a specific person in an airport, the method comprising:
[0006] At specified time intervals, obtaining surveillance videos of multiple target areas in the airport within the specified time interval;
[0007] Performing target detection on a target video image in the acquired surveillance video using a target detection algorithm to obtain detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of a bounding box of the first target object in the target video image and a target feature vector of the first target object, and the target feature vector includes a height feature, a fat feature, a human body posture feature, and a human face feature; and,
[0008] For any first target object, if a target specific person whose similarity with the first target object meets a first preset condition is found in a pre-stored airport specific person database according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, the first target object is determined to be an airport feature person; wherein the airport specific person database is obtained by:
[0009] The target detection algorithm is used to perform target detection on each target historical video image in the historical surveillance video of the multiple target areas within the specified historical time period, and the detection parameters of each second target object in each target historical video image are obtained. Based on the detection parameters of each second target object in each target historical video image, the similarity between each second target object in any two target historical video images is obtained. According to the similarity between each second target object in any two target historical video images, the airport travel information of each second target object within the specified historical time period is obtained, and the target detection parameters of each second target object whose airport travel information meets the second preset condition are stored to obtain the airport feature personnel database, wherein the target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.
[0010] In this embodiment, by acquiring surveillance videos of multiple target areas in the airport within a specified time, and then using the target detection algorithm to perform target detection on any target video image in the acquired surveillance video, the position coordinates of the bounding box in each first target object in the video image and the target feature vector of the first target object are obtained. For any first target object, according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, a search is performed in the pre-stored database of specific personnel in the airport to find whether there is a target specific person whose similarity with the first target object meets the preset conditions; if there is, the first target object is determined to be a specific person. Therefore, in the embodiment of the present application, the position, height feature, fat and thin feature, human posture feature and face feature of the bounding box of the first target object are used to perform spatiotemporal tracking association with the specific person in the pre-established database of specific personnel, so as to identify the specific person. The accuracy of identifying specific personnel in the airport is improved. In addition, in the embodiment of the present application, the airport travel information of each second target object is determined according to the detection parameters obtained by performing target detection on the historical surveillance video, and the airport specific personnel database is established through the airport travel information of each target object, which ensures the accuracy of the airport feature personnel database and further improves the accuracy of identifying specific personnel in the airport.
[0011] In one embodiment, after determining that the first target object is a specific person at the airport, the method further includes:
[0012] For any first target object that is a specific person at the airport, using the pre-set hierarchical relationship between the target areas, determine an adjacent target area corresponding to the target area where the surveillance video that identifies the first target object is located; wherein the adjacent target area includes a target area located one level above the target area and a target area located one level below the target area in the hierarchical relationship; and,
[0013] For any adjacent target area, after determining the surveillance video of the adjacent target area within the specified time length as the acquired surveillance video, return to the step of performing target detection on the target video image in the acquired surveillance video using the target detection algorithm, wherein the time length of the surveillance video acquired in the adjacent target area is the same as the time length of the surveillance video acquired in the target area, and the time period of the surveillance video acquired in the adjacent target area is different from the time period of the surveillance video acquired in the target area;
[0014] If the first target object exists in the surveillance video within the specified time period in the adjacent target area, after determining the adjacent target area as the target area, returning to the step of using the pre-set hierarchical relationship between the target areas to determine the adjacent target area corresponding to the target area where the surveillance video identifying the target object is located, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and,
[0015] Based on the hierarchical relationship between the target areas, obtaining the trajectory of the first target object;
[0016] An airport specific personnel warning is performed based on the trajectory of the first target object and the image of the first target object in the surveillance video.
[0017] This embodiment achieves process evidence collection on a specific person by determining the trajectory of the first target object after determining that the first target object is a specific person and then issuing an early warning to the specific person, and sends the trajectory of the first target object and its image to the management personnel for early warning, so that the management personnel can implement control.
[0018] In one embodiment, it is determined whether there is a target specific person in the airport specific person database whose similarity with the first target object meets a first preset condition by:
[0019] Obtaining the similarity between each airport specific person and the first target object according to the detection parameters of each airport specific person in the airport specific person database and the detection parameters of the first target object;
[0020] If the similarity satisfies the first preset condition, the airport specific person is determined to be the target specific person corresponding to the first target object.
[0021] In one embodiment, the similarity between any airport specific person and the first target object is obtained in the following manner:
[0022] Obtaining a first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object; and
[0023] Obtaining a second similarity index through the target feature vector of the airport specific person and the target feature vector of the first target object;
[0024] Based on the first similarity index and the second similarity index, the similarity between the airport specific person and the first target object is obtained.
[0025] In the embodiment of the present application, the similarity between the first target object and the specific person in the specific person database is determined by using the parameters of two dimensions, namely, the position of the bounding box and the target feature vector, thereby ensuring the accuracy of the determined similarity.
[0026] In one embodiment, obtaining the first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object includes:
[0027] Obtaining a cosine distance between the airport specific person and the first target object according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object, and determining the first similarity index based on the cosine distance;
[0028] The obtaining of a second similarity index by using the target feature vector of the airport specific person and the target feature vector of the first target object includes:
[0029] The Mahalanobis distance between the airport-specific person and the first target object is determined according to the target feature vector of the airport-specific person and the target feature vector of the first target object, and the second similarity index is determined based on the Mahalanobis distance.
[0030] In one embodiment, obtaining the airport travel information of each second target object in a specified historical time period according to the similarity between each second target object in any two target historical video images includes:
[0031] Based on the similarity between the second target objects in any two target historical video images, obtaining the target historical video images where the same second target object is located;
[0032] Determine the frequency of occurrence and travel time period of the second target object in each specified time period according to each target historical video image where the same second target object is located, wherein the historical time period includes each specified time period;
[0033] The airport travel information of each second target object is obtained by using the frequency of occurrence of each second target object in each specified time period and the travel time period.
[0034] In the embodiment of the present application, the airport travel information of the second target object is determined by the frequency of occurrence of each second target object in each specified historical time period and the corresponding target time period when it appears, thereby ensuring the accuracy of the airport specific personnel database.
[0035] A second aspect of the present disclosure provides an electronic device, including a processor and a memory, wherein the processor and the memory are connected via a bus;
[0036] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:
[0037] At specified time intervals, obtaining surveillance videos of multiple target areas in the airport within the specified time interval;
[0038] Performing target detection on a target video image in the acquired surveillance video using a target detection algorithm to obtain detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of a bounding box of the first target object in the target video image and a target feature vector of the first target object, and the target feature vector includes a height feature, a fat feature, a human body posture feature, and a human face feature; and,
[0039] For any first target object, if a target specific person whose similarity with the first target object meets a first preset condition is found in a pre-stored airport specific person database according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, the first target object is determined to be an airport feature person; wherein the airport specific person database is obtained by:
[0040] The target detection algorithm is used to perform target detection on each target historical video image in the historical surveillance video of the multiple target areas within the specified historical time period, and the detection parameters of each second target object in each target historical video image are obtained. Based on the detection parameters of each second target object in each target historical video image, the similarity between each second target object in any two target historical video images is obtained. According to the similarity between each second target object in any two target historical video images, the airport travel information of each second target object within the specified historical time period is obtained, and the target detection parameters of each second target object whose airport travel information meets the second preset condition are stored to obtain the airport feature personnel database, wherein the target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.
[0041] In one embodiment, the processor is further configured to:
[0042] After determining that the first target object is a specific person at the airport, for any first target object that is a specific person at the airport, using the pre-set hierarchical relationship between the target areas, determine an adjacent target area corresponding to the target area where the surveillance video identifying the first target object is located; wherein the adjacent target area includes a target area located one level above the target area and a target area located one level below the target area in the hierarchical relationship; and,
[0043] For any adjacent target area, after determining the surveillance video of the adjacent target area within the specified time length as the acquired surveillance video, return to the step of performing target detection on the target video image in the acquired surveillance video using the target detection algorithm, wherein the time length of the surveillance video acquired in the adjacent target area is the same as the time length of the surveillance video acquired in the target area, and the time period of the surveillance video acquired in the adjacent target area is different from the time period of the surveillance video acquired in the target area;
[0044] If the first target object exists in the surveillance video within the specified time period in the adjacent target area, after determining the adjacent target area as the target area, returning to the step of using the pre-set hierarchical relationship between the target areas to determine the adjacent target area corresponding to the target area where the surveillance video identifying the target object is located, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and,
[0045] Based on the hierarchical relationship between the target areas, obtaining the trajectory of the first target object;
[0046] An airport specific personnel warning is performed based on the trajectory of the first target object and the image of the first target object in the surveillance video.
[0047] In one embodiment, the processor is further configured to:
[0048] Determine whether there is a target specific person in the airport specific person database whose similarity with the first target object meets a first preset condition by:
[0049] Obtaining the similarity between each airport specific person and the first target object according to the detection parameters of each airport specific person in the airport specific person database and the detection parameters of the first target object;
[0050] If the similarity satisfies the first preset condition, the airport specific person is determined to be the target specific person corresponding to the first target object.
[0051] In one embodiment, the processor is further configured to:
[0052] The similarity between any airport specific person and the first target object is obtained by:
[0053] Obtaining a first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object; and
[0054] Obtaining a second similarity index through the target feature vector of the airport specific person and the target feature vector of the first target object;
[0055] Based on the first similarity index and the second similarity index, the similarity between the airport specific person and the first target object is obtained.
[0056] In one embodiment, the processor executes the step of obtaining the first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object, and is specifically configured as follows:
[0057] Obtaining a cosine distance between the airport specific person and the first target object according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object, and determining the first similarity index based on the cosine distance;
[0058] The processor executes the target feature vector of the specific person passing through the airport and the target feature vector of the first target object to obtain a second similarity index, and is specifically configured as follows:
[0059] The Mahalanobis distance between the airport-specific person and the first target object is determined according to the target feature vector of the airport-specific person and the target feature vector of the first target object, and the second similarity index is determined based on the Mahalanobis distance.
[0060] In one embodiment, the processor performs the step of obtaining the airport travel information of each second target object within a specified historical time period according to the similarity between each second target object in any two target historical video images, and is specifically configured as follows:
[0061] Based on the similarity between the second target objects in any two target historical video images, obtaining the target historical video images where the same second target object is located;
[0062] Determine the frequency of occurrence and travel time period of the second target object in each specified time period according to each target historical video image where the same second target object is located, wherein the historical time period includes each specified time period;
[0063] The airport travel information of each second target object is obtained by using the frequency of occurrence of each second target object in each specified time period and the travel time period.
[0064] According to a third aspect provided by an embodiment of the present disclosure, a computer storage medium is provided, wherein the computer storage medium stores a computer program, and the computer program is used to execute the method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0066] Figure 1 This is one of the schematic diagrams of applicable scenarios in one embodiment of the present disclosure;
[0067] Figure 2 This is a second schematic diagram of an applicable scenario in an embodiment of the present disclosure;
[0068] Figure 3 This is one of the flowcharts of a method for identifying a specific person in an airport according to an embodiment of the present disclosure;
[0069] Figure 4 is a structural schematic diagram of an ITMFF model according to an embodiment of the present disclosure;
[0070] Figure 5 is a schematic diagram of a target image according to an embodiment of the present disclosure;
[0071] Figure 6 A schematic diagram of a process according to an embodiment of the present disclosure;
[0072] Figure 7 A schematic diagram of a process for determining a trajectory of a specific person according to an embodiment of the present disclosure;
[0073] Figure 8 is a schematic diagram of the hierarchical relationship between target areas according to an embodiment of the present disclosure;
[0074] Fig. 9 is a trajectory schematic diagram according to an embodiment of the present disclosure;
[0075] Fig.10 A schematic diagram of a process for determining a specific personnel database according to an embodiment of the present disclosure;
[0076] Fig.11 A device for identifying a specific person in an airport according to an embodiment of the present disclosure;
[0077] Fig.12 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0078] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0079] In the embodiments of the present disclosure, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0080] The application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems. In the description of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more.
[0081] In the prior art, there are two main ways to identify specific personnel in airports: one is to intervene and manage through video surveillance or on-site manual methods by airport security and order maintenance personnel. This method lacks effective technical auxiliary management methods, and has low efficiency and accuracy. The other is to report people who stay in the airport for a long time as specific personnel based on face comparison analysis. However, in recent years, users who travel generally wear masks, and there are technical limitations in the comparison and analysis of specific personnel based on traditional face recognition methods, making it difficult to effectively detect faces. Moreover, simply identifying people who stay for a long time as specific personnel will cause a large number of normal passengers to be identified as specific personnel and reported, so the accuracy of identifying specific personnel in airports is low.
[0082] Therefore, the present disclosure provides a method for identifying specific personnel in an airport, by acquiring surveillance videos of multiple target areas in an airport within a specified time length, and then using a target detection algorithm to perform target detection on any target video image in the acquired surveillance video, to obtain the position coordinates of the bounding box in each first target object in the video image and the target feature vector of the first target object. For any first target object, according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, a search is performed in a pre-stored database of specific personnel in the airport to determine whether there is a target specific person whose similarity with the first target object meets a preset condition; if so, the first target object is determined to be a specific person. Therefore, in the embodiment of the present application, the position, height features, fat and thin features, human body posture features, and facial features of the bounding box of the first target object are used to perform spatiotemporal tracking association with specific personnel in a pre-established database of specific personnel, so as to identify the specific person. The accuracy of identifying specific personnel in the airport is improved. Furthermore, in the embodiment of the present application, the airport travel information of each second target object is determined based on the detection parameters obtained by performing target detection on historical surveillance videos, and an airport specific personnel database is established based on the airport travel information of each target object, thereby ensuring the accuracy of the airport feature personnel database and further improving the accuracy of specific personnel identification in the airport. Below, the scheme of the present disclosure is introduced in detail with reference to the accompanying drawings.
[0083] like Figure 1 As shown, an application scenario of a method for identifying a specific person in an airport includes multiple cameras 110, a server 120 and a terminal device 130, such as Figure 1As shown, the application scenario is described by taking the electronic device as a server as an example. However, the embodiment of the present application does not limit the electronic device, and the electronic device can be a server and a terminal device, etc., which can be set according to actual conditions. The server 120 can be implemented by a single server or by multiple servers. The server 120 can be implemented by a physical server or by a virtual server.
[0084] In a possible application scenario, the camera 110 respectively shoots each target area of the airport in real time, and at every specified time interval, the server 120 obtains the surveillance video of multiple target areas in the airport within the specified time interval; the server 120 uses the target detection algorithm to perform target detection on the target video image in the acquired surveillance video to obtain the detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object, and the target feature vector includes height features, fatness features, human posture features and facial features; and for any first target object, if the server 120 finds a target specific person whose similarity with the first target object meets a first preset condition in the pre-stored airport specific person database according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, then it determines The first target object is a characteristic person at the airport; wherein the server 120 obtains the airport specific person database in the following manner: using the target detection algorithm to perform target detection on each target historical video image in the historical surveillance video of the multiple target areas within the specified historical time period, respectively, to obtain the detection parameters of each second target object in each target historical video image, and based on the detection parameters of each second target object in each target historical video image, respectively obtain the similarity between each second target object in any two target historical video images, and according to the similarity between each second target object in any two target historical video images, obtain the airport travel information of each second target object within the specified historical time period, store the target detection parameters of each second target object whose airport travel information meets the second preset condition, and obtain the airport characteristic person database, wherein the target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.
[0085] In another possible scenario, Figure 2As shown, it includes a camera 110 and a terminal device 130, the camera 110 respectively shoots each target area of the airport in real time, and at every specified time period, the terminal device 130 obtains the surveillance video of multiple target areas in the airport within the specified time period; the terminal device 130 uses a target detection algorithm to perform target detection on the target video image in the acquired surveillance video, and obtains the detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object, and the target feature vector includes height features, fat and thin features, human body posture features and face features; and for any first target object, if the terminal device 130 searches for a target specific person whose similarity with the first target object meets a first preset condition in a pre-stored airport specific person database according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector. personnel, then the first target object is determined to be an airport characteristic person; wherein the terminal device 130 obtains the airport specific personnel database in the following manner: using the target detection algorithm to perform target detection on each target historical video image in the historical surveillance video of the multiple target areas within the specified historical time period, respectively, to obtain the detection parameters of each second target object in each target historical video image, and based on the detection parameters of each second target object in each target historical video image, respectively obtain the similarity between each second target object in any two target historical video images, and according to the similarity between each second target object in any two target historical video images, obtain the airport travel information of each second target object within the specified historical time period, store the target detection parameters of each second target object whose airport travel information meets the second preset condition, and obtain the airport characteristic personnel database, wherein the target detection parameter is the detection parameter obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.
[0086] in, Figure 1 The server 120 and the terminal device 130 can exchange information through a communication network, wherein the communication mode adopted by the communication network can be divided into a wireless communication mode or a wired communication mode.
[0087] Exemplarily, the server 120 can access the network through cellular mobile communication technology to communicate with the terminal device 130, wherein the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th Generation Mobile Networks, 5G) technology.
[0088] Optionally, the server 120 may access the network and communicate with the terminal device 130 via short-range wireless communication, wherein the short-range wireless communication method, for example, includes Wireless Fidelity (Wi-Fi) technology.
[0089] Among them, the description in this application only details a single server camera 110, a single server 120, and a single terminal device 130, but those skilled in the art should understand that the illustrated camera 110, server 120, and terminal device 130 are intended to represent the operations of the camera 110, server 120, and terminal device 130 involved in the technical solution of this application. It does not imply any limitation on the number, type, or location of the camera 110, server 120, and terminal device 130. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed therefrom, the underlying concepts of the exemplary embodiments of this application will not be changed.
[0090] It should be noted that the method for identifying specific persons in an airport proposed in this application is not only applicable to Figure 1 and Figure 2 The application scenario shown is also applicable to any device for identifying specific persons in an airport.
[0091] The following describes a method for identifying specific personnel in an airport according to an exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the methods and principles of the present application, and the implementation methods of the present application are not limited in this regard.
[0092] like Figure 3 FIG. 1 is a flow chart of a method for identifying a specific person in an airport according to the present disclosure, which may include the following steps:
[0093] Step 301: acquiring surveillance videos of multiple target areas in an airport within a specified time period at specified time intervals;
[0094] The specified duration in the real-time example of the present application can be 1 minute, 10 minutes, 30 minutes, etc., which can be set according to the actual situation, and the present application embodiment does not limit it here. In addition, the target area in the present application embodiment is pre-set, which can be set according to the actual situation, and the present application embodiment does not limit it here.
[0095] Step 302: Target detection is performed on the target video image in the acquired surveillance video using a target detection algorithm to obtain detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of the bounding box of the first target object in the target video image and the target feature vector of the first target object, and the target feature vector includes height features, fat and thin features, human body posture features, and face features;
[0096] Among them, the target video image in each surveillance video in the embodiment of the present application is obtained by frame extraction, that is, the video image of every specified number of frames is determined as the target video image. For example, if the specified number of frames is 5 frames, and the video has a total of 30 frames, the video images corresponding to the 6th frame, the 12th frame, the 18th frame, and the 24th frame are determined as the target video image. However, the method for determining the target video image is not limited in the embodiment of the present application, and can be set according to actual conditions. In addition, the target video image in the embodiment of the present application includes an RGB target video image and a depth target video image.
[0097] The target detection algorithm used in the embodiment of the present application is ITMFF. However, the target detection algorithm is not limited in the implementation of the present application. The target detection algorithm in the embodiment of the present application can be set according to the actual situation. Figure 4 As shown in FIG. 1 , it is a schematic diagram of the structure of ITMFF. It can be seen from the figure that it includes a residual module 401, a residual module 402, an ASPP (atrous spatial pyramid pooling, atrous spatial pyramid pooling layer) 403, an atrous spatial pyramid pooling layer 404, five fusion layers 405, four feature cascade layers 406, and four upsampling layers 407. Among them, the residual module 401 includes five consecutive convolutional layers, namely, convolutional layer 4011, convolutional layer 4012, convolutional layer 4013, convolutional layer 4014, and convolutional layer 4015. And the residual module 402 includes five consecutive convolutional layers, namely, convolutional layer 4021, convolutional layer 4012, convolutional layer 4013, convolutional layer 4014, and convolutional layer 4015.
[0098] in:
[0099] The RGB target video image is input into the residual module 401, and the convolution layer 4011 in the residual module 401 is used to extract features of the RGB target video image to obtain a first RGB feature map, and then the convolution layer 4012 is used to extract features of the first RGB feature map to obtain a second RGB feature map, and then the second RGB feature map is input into the convolution layer 4013 and the fusion layer 405, and the convolution layer 4013 extracts the second RGB feature map to obtain a third RGB feature map, and the third RGB feature map is respectively input into the convolution layer 4014 and the fusion layer 405, and the convolution layer 4014 extracts the third RGB feature map to obtain a fourth RGB feature map, and then the fourth RGB feature map is input into the convolution layer 4015 and the fusion layer 405, and the convolution layer 4015 extracts features of the fourth RGB feature map to obtain a fifth RGB feature map.
[0100] Similarly, the depth target video image is input into the residual module 402, and the convolution layer 4021 in the residual module 402 is used to extract features of the depth target video image to obtain a first depth feature map, and then the convolution layer 4022 is used to extract features of the first depth feature map to obtain a second depth feature map, and then the second depth feature map is input into the convolution layer 4023 and the fusion layer 405, and the convolution layer 4023 extracts the second depth feature map to obtain a third depth feature map, and the third depth feature map is respectively input into the convolution layer 4024 and the fusion layer 405, and the convolution layer 4024 extracts the third depth feature map to obtain a fourth depth feature map, and then the fourth depth feature map is input into the convolution layer 4025 and the fusion layer 405, and the convolution layer 4025 extracts features on the fourth depth feature map to obtain a fifth depth feature map.
[0101] The convolution layer 4015 inputs the fifth RGB feature map into the atrous space pyramid pooling layer 403 for feature extraction to obtain a sixth RGB feature map. Similarly, the convolution layer 4025 inputs the fifth depth feature map into the atrous space pyramid pooling layer 404 to obtain a sixth depth feature map. Then, the sixth RGB feature map and the sixth depth feature map are feature fused by the fusion layer 405 to obtain a first fused feature map. The fifth RGB feature map and the fifth depth feature map are feature fused by the fusion module 405 to obtain a second fused feature. The fourth RGB feature map and the fourth depth feature map are feature fused by the fusion module 405 to obtain a third fused feature. The third RGB feature map and the third depth feature map are feature fused by the fusion module 405 to obtain a fourth fused feature. The second RGB feature map and the second depth feature map are feature fused by the fusion module 405 to obtain a fifth fused feature. Then, the first fused feature and the second fused feature are cascaded by the feature cascade layer 406, and then upsampled by the upsampling layer 407 to obtain the first intermediate feature, and the first intermediate feature and the third fused feature are cascaded by the feature cascade layer 406, and then upsampled by the upsampling layer 407 to obtain the second intermediate feature. Then, the second intermediate feature and the fourth fused feature are cascaded by the feature cascade layer 406, and then upsampled by the upsampling layer 407 to obtain the third intermediate feature. Then, the third intermediate feature and the fifth fused feature are cascaded by the feature cascade layer 406, and then upsampled by the upsampling layer 407 to obtain the target image, which contains the position of the bounding box of the target object in the target video image and the corresponding target feature vector.
[0102] For example, Figure 5 As shown in FIG. 1 , the target image is obtained. It can be seen from the figure that the figure contains the position coordinates of the bounding box of the target object and the corresponding target feature vector.
[0103] Step 303: If a target specific person whose similarity with the first target object meets a first preset condition is found in a pre-stored airport specific person database based on the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, then the first target object is determined to be an airport feature person.
[0104] In one embodiment, step 303 can be specifically implemented as follows: based on the detection parameters of each airport-specific personnel in the airport-specific personnel database and the detection parameters of the first target object, the similarity between each airport-specific personnel and the first target object is obtained; if the similarity meets the first preset condition, the airport-specific personnel is determined to be the target specific personnel corresponding to the first target object.
[0105] Among them, the first preset condition may be that the similarity is greater than the first specified similarity, and the first specified similarity in the embodiment of the present application may be 0.8. However, the embodiment of the present application does not limit the first preset condition and the first specified similarity, and the first preset condition and the first specified similarity in the embodiment of the present application may be set according to specific actual conditions.
[0106] Next, the method of determining the similarity between any airport specific person and the first target object is described. Figure 6 As shown, it is a schematic diagram of a process for determining similarity, which may include the following steps:
[0107] Step 601: obtaining a first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object;
[0108] In one embodiment, step 601 may be specifically implemented as follows: according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object, a cosine distance between the airport specific person and the first target object is obtained, and the first similarity index is determined based on the cosine distance. The cosine distance may be obtained by formula (1):
[0109]
[0110] Among them, d1 is the cosine distance, x1 is the horizontal position coordinate of the center point of the bounding box of the specific person at the airport, y1 is the vertical position coordinate of the center point of the bounding box of the specific person at the airport, x2 is the horizontal position coordinate of the center point of the bounding box of the first target object, and y2 is the vertical position coordinate of the center point of the bounding box of the first target object.
[0111] Step 602: Obtain a second similarity index through the target feature vector of the airport specific person and the target feature vector of the first target object;
[0112] In one embodiment, step 602 may be specifically implemented as follows: determining the Mahalanobis distance between the airport specific person and the first target object according to the target feature vector of the airport specific person and the target feature vector of the first target object, and determining the second similarity index based on the Mahalanobis distance. The Mahalanobis distance between the airport specific person and the first target object may be obtained by formula (2):
[0113]
[0114] Wherein, d2 is the Mahalanobis distance, x i is the i-th vector in the target feature vector of the first target object, xj is the jth vector in the target feature vector for airport-specific personnel, and n is the total number of vectors in the target feature vector.
[0115] Step 603: Based on the first similarity index and the second similarity index, obtain the similarity between the airport specific person and the first target object.
[0116] In the embodiment of the present application, the method for determining the first similarity index is the same as the method for determining the second similarity index. The following is an example of the method for determining the first similarity index:
[0117] Method 1: Determine the cosine distance as the first similarity indicator.
[0118] Method 2: Multiply the cosine distance by a preset weight to obtain the first similarity index.
[0119] It should be noted that the method for determining the second similarity index is the same as the method for determining the first similarity index, except that the cosine distance is replaced by the Mahalanobis distance, which will not be described in detail in the embodiment of the present application.
[0120] In one embodiment, step 603 can be specifically implemented as follows: if the first similarity index and the second similarity index are obtained by using the method 1 described above, the first similarity index and the second similarity index are added together to obtain the total similarity index between the first target object and the specific person at the airport, and the total similarity index is normalized to obtain the similarity between the specific person and the first target object. If the first similarity index and the second similarity index are obtained by using the method 2 described above, the first similarity index and the second similarity index are weightedly summed to obtain the total similarity index between the first target object and the specific person, and the total similarity index is normalized to obtain the similarity between the specific person and the first target object.
[0121] Among them, the normalization method in the embodiment of the present application can be to divide the total similarity index by a normalization parameter, and the normalization parameter is the total similarity index with the largest value between the first target object and the specific personnel of each airport or the sum of the total similarity indexes between the first target object and the specific personnel of each airport. The specific normalization method is not limited in the embodiment of the present application and can be set according to actual conditions.
[0122] In order to facilitate management personnel to implement control and realize process evidence collection of specific airport personnel, in one embodiment, Figure 7 As shown, a schematic diagram of a process for determining the trajectory of a specific person at an airport may include the following steps:
[0123] Step 701: for any first target object that is a specific person at the airport, using the pre-set hierarchical relationship between the target areas, determine whether there is a corresponding adjacent target area to the target area where the surveillance video of the first target object is located, if yes, execute step 702, if no, execute step 705; wherein the adjacent target area includes a target area located one level above the target area and a target area located one level below the target area in the hierarchical relationship;
[0124] In the embodiment of the present application, the target area where the surveillance video is located is obtained by using the pre-set relationship between the camera and the target area. As shown in Table 1:
[0125] Target area Camera Target area A Camera A, Camera B, Camera C Target area B Camera d, camera e, camera f, camera g Target area C Camera m, Camera n, Camera r … …
[0126] It can be seen from Table 1 that each target area corresponds to a camera. For example, the cameras in target area A are camera a, camera b, and camera c.
[0127] like Figure 8 As shown in Figure 1, it is the hierarchical relationship between the target areas. Figure 8 A multi-branch tree is used for representation. For example, if the target area C corresponds to the target area A located at an upper level of the target area and the target area F located at a lower level of the target area.
[0128] Step 702: for any adjacent target area, after determining the surveillance video of the adjacent target area within the specified time length as the acquired surveillance video, return to step 302 for execution, wherein the time length of the surveillance video acquired in the adjacent target area is the same as the time length of the surveillance video acquired in the target area, and the time period of the surveillance video acquired in the adjacent target area is different from the time period of the surveillance video acquired in the target area;
[0129] For example, the time period for acquiring the surveillance video of the target area is 10:00-10:30 on June 12, 2022, then the time period for the target area at the upper level of the target area is 9:30-10:00 on June 12, 2022, and the time period for the target area at the lower level of the target area is 10:30-11:00 on June 12, 2022.
[0130] Step 703: Determine whether the first target object exists in the surveillance video of the adjacent target area within the specified time period, if yes, execute step 704, if no, execute step 706;
[0131] Step 704: after determining the adjacent target area as the target area, return to step 701;
[0132] Step 705: obtaining the trajectory of the first target object based on the hierarchical relationship between the target areas;
[0133] For example, the target areas are obtained in order from front to back according to the levels: target area A, target area C, target area F and target area H. Then the trajectory of the first target object is: target area A→target area C→target area F→target area H. Fig. 9 As shown, Fig. 9 In the example above, each track is marked on the map in chronological order. Fig. 9 It is only used for illustration, and the specific trajectory representation method can be set according to actual conditions, and the embodiments of the present application are not limited here.
[0134] Step 706: Issue a warning to a specific person based on the trajectory of the first target object and the image of the first target object in the surveillance video.
[0135] In the embodiment of the present application, the method for early warning of specific personnel is to send the trajectory of the first target object and the image of the first target object in the surveillance video to the administrator's terminal device. However, the embodiment of the present application does not limit the method for early warning of specific personnel. In the embodiment of the present application, the method for early warning of specific personnel can be set according to actual conditions.
[0136] Next, the method of determining the database of specific personnel in the airport is introduced, such as Fig.10 As shown, it is a schematic diagram of the process of determining a specific personnel database in an airport, including the following steps:
[0137] Step 1001: Acquire historical surveillance videos of the plurality of target areas in the airport within a historically specified time period;
[0138] The historical specified time period in the embodiment of the present application can be set according to actual conditions, and the embodiment of the present application is not limited here.
[0139] Step 1002: Utilizing the target detection algorithm to perform target detection on each target historical video image in the historical surveillance video, respectively, to obtain detection parameters of each second target object in each target historical video image, wherein the detection parameters include the position coordinates of the bounding box of the second target object in the target video image and the target feature vector of the second target object, and the target feature vector includes height features, fat and thin features, human body posture features, and face features;
[0140] The target detection algorithm in the embodiment of the present application is the ITMFF algorithm described above, and the embodiment of the present application will not be described in detail here.
[0141] Step 1003: based on the detection parameters of each second target object in each target historical video image, respectively obtain the similarity between each second target object in any two target historical video images, and based on the similarity, obtain each target historical video image where the same second target object is located;
[0142] In the embodiment of the present application, two second target objects whose similarity is greater than the second specified similarity are determined to be the same second target object. For example, if the similarity between the second target object A in the target historical video image 1 and the second target object B in the target historical video image 2 is greater than the second specified similarity, the second target object A and the second target object B are determined to be the same second target object.
[0143] If there is a second target object in target historical video image 1, target historical video image 4, target historical video image 5, target historical video image 6 and target historical video image 8, whose similarity with the second target object B in target historical video image 2 is greater than the second specified similarity, then the target historical video images where the second target object B is located are determined to be: target historical video image 1, target historical video image 4, target historical video image 5, target historical video image 6 and target historical video image 8 respectively.
[0144] Among them, the method of determining the similarity between the two second target objects in step 1003 is the same as the method of determining the similarity between the first target object and any specific person at the airport described above, and the embodiment of the present application will not be repeated here. And the second specified similarity and the value of the second specified similarity in the embodiment of the present application can be the same or different, and can be set according to actual conditions. The embodiment of the present application does not limit the value of the second specified similarity.
[0145] Step 1004: determining the frequency of occurrence and travel time period of the second target object in each specified time period according to each target historical video image where the same second target object is located, wherein the historical time period includes each specified time period;
[0146] In one embodiment, step 1004 can be specifically implemented as follows: based on the shooting time corresponding to each target historical video image where the second target object is located, determine the target historical video image with a shooting time within a specified time period; and based on the target historical video image with a shooting time within the specified time period, count the frequency of appearance of the second target object in each specified time period and the travel time period.
[0147] Among them, the frequency can be determined in the following manner: dividing the designated time period into multiple sub-time periods, determining the sub-time period corresponding to each target historical video image based on the shooting time of each target historical video image, counting the target sub-time periods of each target historical video image where the second target object is located in each target historical video image corresponding to each sub-time period, and determining the number of the target sub-time periods as the frequency of occurrence in the designated segment and determining the travel time period according to the shooting time corresponding to each target historical video image where the second target object is located.
[0148] For example, if the specified time period is 2022.01.01.00:00-2022.01.07.24:00, the multiple sub-time periods after division are: 2022.01.01.00:00-2022.01.01.24:00, 2022.01.02.00:00-2022.01.02.24:00, 2022.01.03.00:00-2022. 01.03.24:00, 2022.01.04.00:00-2022.01.04.24:00, 2022.01.05.00:00-2022.01.05.24:00, 2022.01.06.00:00-2022.01.06.24:00, 2022.01.07.00:00-2022.01.07.24:00. If it is determined that each target historical video image corresponding to each sub-time period contains each target historical video image where the second target object A is located, then it is determined that the frequency of the second target object A appearing in the specified time period 2022.01.01.00:00-2022.01.07.24:00 is 7.
[0149] The length of the specified time period in the embodiment of the present application can be one week, one month, half a month, etc., which can be set according to the actual situation, and the embodiment of the present application does not limit it here. In addition, the length of the sub-time period in the embodiment of the present application can be set according to the actual situation, and the embodiment of the present application does not limit it here.
[0150] Step 1005: using the frequency of occurrence of each second target object in each designated time period and the travel time period, obtaining the airport travel information of each second target object;
[0151] In one embodiment, in the embodiment of the present application, the frequency of occurrence of each second target object in each specified time period and the travel time period are determined as the airport travel information.
[0152] Step 1006: Store the target detection parameters of each second target object whose airport travel information meets the second preset condition to obtain the airport feature personnel database, wherein the target detection parameters are the detection parameters obtained from the target historical monitoring video with the smallest difference between the shooting time and the current time.
[0153] Among them, the second preset condition in the embodiment of the present application may be that the frequency is greater than the specified frequency and / or the number of time intervals in which the travel time period is located is greater than the specified number. The second preset condition in the embodiment of the present application can be set according to actual conditions, and the embodiment of the present application does not limit the second preset condition here. In the embodiment of the present application, the time interval is pre-set. It can be set according to actual conditions.
[0154] Based on the same disclosure concept, the method for identifying specific persons in an airport as described above can also be implemented by a device for identifying specific persons in an airport. The effect of the device for identifying specific persons in an airport is similar to that of the aforementioned method, and will not be described in detail here.
[0155] Fig.11 The figure is a schematic diagram of the structure of a device for identifying specific personnel in an airport according to an embodiment of the present disclosure.
[0156] like Fig.11 As shown, the device 1100 for identifying specific personnel in an airport of the present disclosure may include an airport surveillance video acquisition module 1110 , a target object detection module 1120 and a specific personnel detection module 1130 .
[0157] The airport surveillance video acquisition module 1110 is used to acquire surveillance videos of multiple target areas in the airport within the specified time period at specified time intervals;
[0158] A target object detection module 1120 is used to perform target detection on a target video image in the acquired surveillance video using a target detection algorithm to obtain detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of a bounding box of the first target object in the target video image and a target feature vector of the first target object, and the target feature vector includes a height feature, a fat feature, a human body posture feature, and a human face feature; and
[0159] The specific person detection module 1130 is used to determine that, for any first target object, the first target object is an airport characteristic person if a target specific person whose similarity with the first target object meets a first preset condition is found in a pre-stored airport specific person database according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector; wherein the airport specific person database is obtained in the following manner: using the target detection algorithm to perform target detection on each target historical video image in the historical surveillance video of the multiple target areas within a specified historical time period, respectively, to obtain detection parameters of each second target object in each target historical video image, and based on the detection parameters of each second target object in the target historical video images, respectively obtain the similarity between each second target object in any two target historical video images, obtain airport travel information of each second target object within the specified historical time period according to the similarity between each second target object in any two target historical video images, store the target detection parameters of each second target object whose airport travel information meets the second preset condition, and obtain the airport characteristic person database, wherein the target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.
[0160] In one embodiment, the apparatus further comprises:
[0161] The trajectory determination module 1140 is used to determine, after determining that the first target object is a specific person at the airport, for any first target object that is a specific person at the airport, an adjacent target area corresponding to the target area where the surveillance video that identifies the first target object is located by using the pre-set hierarchical relationship between the target areas; wherein the adjacent target area includes a target area located one level above the target area and a target area located one level below the target area in the hierarchical relationship; and,
[0162] For any adjacent target area, after determining the surveillance video of the adjacent target area within the specified time length as the acquired surveillance video, return to the step of performing target detection on the target video image in the acquired surveillance video using the target detection algorithm, wherein the time length of the surveillance video acquired in the adjacent target area is the same as the time length of the surveillance video acquired in the target area, and the time period of the surveillance video acquired in the adjacent target area is different from the time period of the surveillance video acquired in the target area;
[0163] If the first target object exists in the surveillance video within the specified time period in the adjacent target area, after determining the adjacent target area as the target area, returning to the step of using the pre-set hierarchical relationship between the target areas to determine the adjacent target area corresponding to the target area where the surveillance video identifying the target object is located, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and,
[0164] Based on the hierarchical relationship between the target areas, obtaining the trajectory of the first target object;
[0165] An airport specific personnel warning is performed based on the trajectory of the first target object and the image of the first target object in the surveillance video.
[0166] In one embodiment, the apparatus further comprises:
[0167] The comparison module 1150 is used to determine whether there is a target specific person in the airport specific person database whose similarity with the first target object meets a first preset condition by:
[0168] For any specific person in the specific person database in the airport, obtaining the similarity between the specific person and the first target object according to the detection parameters of the specific person and the detection parameters of the first target object;
[0169] If the similarity satisfies the preset condition, the specific person is determined to be the target specific person corresponding to the first target object.
[0170] In one embodiment, the apparatus further comprises:
[0171] The similarity determination module 1160 is used to obtain the similarity between any airport specific person and the first target object by:
[0172] Obtaining a first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object; and
[0173] Obtaining a second similarity index through the target feature vector of the airport specific person and the target feature vector of the first target object;
[0174] Based on the first similarity index and the second similarity index, the similarity between the airport specific person and the first target object is obtained.
[0175] In one embodiment, the similarity determination module 1160 is specifically configured to:
[0176] Obtaining a cosine distance between the airport specific person and the first target object according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object, and determining the first similarity index based on the cosine distance;
[0177] The Mahalanobis distance between the airport-specific person and the first target object is determined according to the target feature vector of the airport-specific person and the target feature vector of the first target object, and the second similarity index is determined based on the Mahalanobis distance.
[0178] In one embodiment, the specific person detection module 1130 is further used to:
[0179] Based on the similarity between the second target objects in any two target historical video images, obtaining the target historical video images where the same second target object is located;
[0180] Determine the frequency of occurrence and travel time period of the second target object in each specified time period according to each target historical video image where the same second target object is located, wherein the historical time period includes each specified time period;
[0181] The airport travel information of each second target object is obtained by using the frequency of occurrence of each second target object in each specified time period and the travel time period.
[0182] After introducing a method for identifying a specific person in an airport and an electronic device according to an exemplary embodiment of the present disclosure, next, an electronic device according to another exemplary embodiment of the present disclosure is introduced.
[0183] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".
[0184] In some possible implementations, the electronic device according to the present disclosure may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the method for identifying a specific person in an airport according to various exemplary embodiments of the present disclosure described above in this specification. For example, the processor may execute the following steps: Figure 3 Steps 301-303 shown in .
[0185] Refer to the following Fig.12 1200 according to this embodiment of the present disclosure is described. Fig.12 The electronic device 1200 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0186] like Fig.12 As shown, the electronic device 1200 is in the form of a general electronic device. The components of the electronic device 1200 may include but are not limited to: the at least one processor 1201, the at least one computer storage medium 1202, and a bus 1203 connecting different system components (including the computer storage medium 1202 and the processor 1201).
[0187] Bus 1203 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.
[0188] Computer storage media 1202 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1221 and / or cache storage media 1222 , and may further include read-only computer storage media (ROM) 1223 .
[0189] The computer storage medium 1202 may also include a program / utility 1225 having a set (at least one) of program modules 1224, such program modules 1224 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0190] The electronic device 1200 may also communicate with one or more external devices 1204 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1200, and / or communicate with any device that enables the electronic device 1200 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1205. Furthermore, the electronic device 1200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1206. As shown, the network adapter 1206 communicates with other modules for the electronic device 1200 via a bus 1203. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0191] In some possible implementations, various aspects of a method for identifying specific personnel in an airport provided by the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method for identifying specific personnel in an airport according to various exemplary embodiments of the present disclosure described above in this specification.
[0192] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access computer storage medium (RAM), a read-only computer storage medium (ROM), an erasable programmable read-only computer storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only computer storage medium (CD-ROM), an optical computer storage medium, a magnetic computer storage medium, or any suitable combination of the above.
[0193] The program product for identifying specific persons in an airport of an embodiment of the present disclosure may adopt a portable compact disk read-only computer storage medium (CD-ROM) and include program code, and may be run on an electronic device. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.
[0194] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0195] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0196] Program code for performing the disclosed operations may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In the case of a remote electronic device, the remote electronic device may be connected to the user electronic device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).
[0197] It should be noted that although several modules of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be embodied.
[0198] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
[0199] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk computer storage media, CD-ROM, optical computer storage media, etc.) containing computer-usable program codes.
[0200] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0201] These computer program instructions may also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable computer storage medium produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0202] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0203] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.
Claims
1. A method for identifying a specific person in an airport, characterized in that: The method comprises: At specified time intervals, obtaining surveillance videos of multiple target areas in the airport within the specified time interval; Performing target detection on a target video image in the acquired surveillance video using a target detection algorithm to obtain detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of a bounding box of the first target object in the target video image and a target feature vector of the first target object, and the target feature vector includes a height feature, a fat feature, a human body posture feature, and a human face feature; and, For any first target object, if a target specific person whose similarity with the first target object meets a first preset condition is found in a pre-stored airport specific person database according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, the first target object is determined to be an airport feature person; wherein the airport specific person database is obtained by: The target detection algorithm is used to perform target detection on each target historical video image in the historical surveillance video of the multiple target areas within the specified historical time period, and the detection parameters of each second target object in each target historical video image are obtained. Based on the detection parameters of each second target object in each target historical video image, the similarity between each second target object in any two target historical video images is obtained. According to the similarity between each second target object in any two target historical video images, the airport travel information of each second target object within the specified historical time period is obtained, and the target detection parameters of each second target object whose airport travel information meets the second preset condition are stored to obtain the airport feature personnel database, wherein the target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.
2. The method according to claim 1, characterized in that After determining that the first target object is a specific person at the airport, the method further includes: For any first target object that is a specific person at the airport, using the pre-set hierarchical relationship between the target areas, determine an adjacent target area corresponding to the target area where the surveillance video that identifies the first target object is located; wherein the adjacent target area includes a target area located one level above the target area and a target area located one level below the target area in the hierarchical relationship; and, For any adjacent target area, after determining the surveillance video of the adjacent target area within the specified time length as the acquired surveillance video, return to the step of performing target detection on the target video image in the acquired surveillance video using the target detection algorithm, wherein the time length of the surveillance video acquired in the adjacent target area is the same as the time length of the surveillance video acquired in the target area, and the time period of the surveillance video acquired in the adjacent target area is different from the time period of the surveillance video acquired in the target area; If the first target object exists in the surveillance video within the specified time period in the adjacent target area, after determining the adjacent target area as the target area, returning to the step of using the pre-set hierarchical relationship between the target areas to determine the adjacent target area corresponding to the target area where the surveillance video identifying the target object is located, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and, Based on the hierarchical relationship between the target areas, obtaining the trajectory of the first target object; An airport specific personnel warning is performed based on the trajectory of the first target object and the image of the first target object in the surveillance video.
3. The method according to claim 1, characterized in that Determine whether there is a target specific person in the airport specific person database whose similarity with the first target object meets a first preset condition by: Obtaining the similarity between each airport specific person and the first target object according to the detection parameters of each airport specific person in the airport specific person database and the detection parameters of the first target object; If the similarity satisfies the first preset condition, the airport specific person is determined to be the target specific person corresponding to the first target object.
4. The method according to claim 1 or 3, characterized in that: The similarity between any airport specific person and the first target object is obtained by: Obtaining a first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object; and Obtaining a second similarity index through the target feature vector of the airport specific person and the target feature vector of the first target object; Based on the first similarity index and the second similarity index, the similarity between the airport specific person and the first target object is obtained.
5. The method according to claim 4, characterized in that The obtaining of a first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object includes: Obtaining a cosine distance between the airport specific person and the first target object according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object, and determining the first similarity index based on the cosine distance; The obtaining of a second similarity index by using the target feature vector of the airport specific person and the target feature vector of the first target object includes: The Mahalanobis distance between the airport-specific person and the first target object is determined according to the target feature vector of the airport-specific person and the target feature vector of the first target object, and the second similarity index is determined based on the Mahalanobis distance.
6. The method according to claim 1, characterized in that The obtaining, according to the similarity between each second target object in any two target historical video images, the airport travel information of each second target object within a specified historical time period includes: Based on the similarity between the second target objects in any two target historical video images, obtaining the target historical video images where the same second target object is located; Determine the frequency of occurrence and travel time period of the second target object in each specified time period according to each target historical video image where the same second target object is located, wherein the historical time period includes each specified time period; The airport travel information of each second target object is obtained by using the frequency of occurrence of each second target object in each specified time period and the travel time period.
7. An electronic device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: At specified time intervals, obtaining surveillance videos of multiple target areas in the airport within the specified time interval; Performing target detection on a target video image in the acquired surveillance video using a target detection algorithm to obtain detection parameters of each first target object in the target video image, wherein the detection parameters include the position coordinates of a bounding box of the first target object in the target video image and a target feature vector of the first target object, and the target feature vector includes a height feature, a fat feature, a human body posture feature, and a human face feature; and, For any first target object, if a target specific person whose similarity with the first target object meets a first preset condition is found in a pre-stored airport specific person database according to the position coordinates of the bounding box of the first target object in the target video image and the target feature vector, the first target object is determined to be an airport feature person; wherein the airport specific person database is obtained by: The target detection algorithm is used to perform target detection on each target historical video image in the historical surveillance video of the multiple target areas within the specified historical time period, and the detection parameters of each second target object in each target historical video image are obtained. Based on the detection parameters of each second target object in each target historical video image, the similarity between each second target object in any two target historical video images is obtained. According to the similarity between each second target object in any two target historical video images, the airport travel information of each second target object within the specified historical time period is obtained, and the target detection parameters of each second target object whose airport travel information meets the second preset condition are stored to obtain the airport feature personnel database, wherein the target detection parameters are the detection parameters obtained from the target historical surveillance video with the smallest difference between the shooting time and the current time.
8. The electronic device according to claim 7, characterized in that: The processor is further configured to: After determining that the first target object is a specific person at the airport, for any first target object that is a specific person at the airport, using the pre-set hierarchical relationship between the target areas, determine an adjacent target area corresponding to the target area where the surveillance video identifying the first target object is located; wherein the adjacent target area includes a target area located one level above the target area and a target area located one level below the target area in the hierarchical relationship; and, For any adjacent target area, after determining the surveillance video of the adjacent target area within the specified time length as the acquired surveillance video, return to the step of performing target detection on the target video image in the acquired surveillance video using the target detection algorithm, wherein the time length of the surveillance video acquired in the adjacent target area is the same as the time length of the surveillance video acquired in the target area, and the time period of the surveillance video acquired in the adjacent target area is different from the time period of the surveillance video acquired in the target area; If the first target object exists in the surveillance video within the specified time period in the adjacent target area, after determining the adjacent target area as the target area, returning to the step of using the pre-set hierarchical relationship between the target areas to determine the adjacent target area corresponding to the target area where the surveillance video identifying the target object is located, until there is no corresponding adjacent target area or the first target object does not exist in the adjacent target area; and, Based on the hierarchical relationship between the target areas, obtaining the trajectory of the first target object; An airport specific personnel warning is performed based on the trajectory of the first target object and the image of the first target object in the surveillance video.
9. The electronic device according to claim 7, characterized in that: The processor is further configured to: Determine whether there is a target specific person in the airport specific person database whose similarity with the first target object meets a first preset condition by: Obtaining the similarity between each airport specific person and the first target object according to the detection parameters of each airport specific person in the airport specific person database and the detection parameters of the first target object; If the similarity satisfies the first preset condition, the airport specific person is determined to be the target specific person corresponding to the first target object.
10. The electronic device according to claim 7 or 9, characterized in that: The processor is further configured to: The similarity between any airport specific person and the first target object is obtained by: Obtaining a first similarity index according to the position coordinates of the bounding box of the airport specific person and the position coordinates of the bounding box of the first target object; and Obtaining a second similarity index through the target feature vector of the airport specific person and the target feature vector of the first target object; Based on the first similarity index and the second similarity index, the similarity between the airport specific person and the first target object is obtained.
Citation Information
Patent Citations
Identity recognition method, device and equipment and computer storage medium
CN111461047A
Subway pedestrian flow network fusion method based on video pedestrian identification, and pedestrian flow prediction method
CN112541440A
Personnel monitoring recognition method and device, electronic equipment and readable storage medium
CN114022841A
Airport passenger identification and location system based on target tracking technology
CN202218344U
Subway pedestrian flow network fusion method based on video pedestrian recognition, and pedestrian flow prediction method
WO2022126669A1
