Import-exit passenger group key feature fusion evaluation method and system and storage medium

Abstract

The invention discloses an entry-exit passenger group key feature fusion evaluation method and system and a storage medium. The method comprises the steps of extracting multi-dimensional features of entry-exit personnel according to a video stream; calculating a first similarity P (A1) between the portrait attribute features and historical key persons; when P (A1) is greater than or equal to M, performing increase processing on P (A1) according to the associated historical case information identifier and the entry-exit attribute feature to obtain a second similarity; n < = P (A1) lt; when M is greater than M, increasing the P (A1) according to a gait attribute feature, an entry-exit attribute feature, a high anti-pseudo portrait attribute feature, a luggage attribute feature and a human body attribute feature to obtain a third similarity; if P (A1) lt is satisfied; when N is greater than N, calculating a fourth similarity according to high anti-pseudo portrait attribute characteristics, luggage attribute characteristics and human body attribute characteristics of the entry and exit personnel; and according to the second similarity or the third similarity or the fourth similarity, evaluating the risk of the entry and exit personnel.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of feature recognition, in particular to an entry-exit passenger group key feature fusion evaluation method and system and a storage medium. BACKGROUND

[0002] Due to the rapid increase in the number of entry-exit personnel, the complexity of purposes and identities, and the increasing variety of carried luggage, the means of evasion of key personnel (groups) who commit illegal and criminal acts are constantly changing, and their anti-detection and disguise abilities are becoming stronger. In addition, there are factors such as large port traffic, variable motion target characteristics, interference, and intentional or unintentional disguises of entry-exit personnel. The entry-exit supervision task is becoming more and more difficult, the situation is becoming more and more complex, and existing recognition technologies are mostly only for single features such as faces, making it more and more difficult to identify key personnel (groups) including water passengers.

[0003] In related technologies, the invention patent application with publication number CN115273250A discloses a human feature recognition method based on a network camera, which identifies the target by recognizing human features (including face features, body features, facial expressions, and motion features), effectively improving the capture rate of fugitive suspects in public places and solving the problem that existing cameras cannot actively discover fugitive suspects. However, this scheme only proposes to use multiple features for judgment, but does not propose how to fuse and evaluate multi-dimensional features, and the features lack specificity and are not suitable for the identification of key personnel such as water passengers, nor for high-throughput entry-exit customs sites with occlusions. The invention patent application with publication number CN109509476A discloses a method for identifying criminal suspects, which identifies criminal suspects based on audio and identity information entered by the internet user. However, this scheme only proposes two features, and the features are less, and the internet user needs to enter audio and identity information, which is also not suitable for high-throughput entry-exit customs sites without stopping.

[0004] In summary, existing key personnel (group) feature evaluation and identification do not consider the illegal and criminal features of entry-exit key passengers and historical case information, or only use single portrait recognition methods, which leads to low accuracy in identifying entry-exit key personnel, especially when experienced key personnel with anti-detection capabilities are involved. The existing personnel identification in public places has the concept of multi-feature evaluation, but lacks specific fusion evaluation methods, does not effectively fuse multi-dimensional features, and is not suitable for entry-exit high-throughput personnel customs sites. Therefore, it is urgent to propose an entry-exit passenger group key feature evaluation method that considers multi-dimensional feature fusion and key personnel disguises. SUMMARY

[0005] One of the technical problems to be solved by the present application is that it is difficult to quickly and accurately find out key personnel of entry and exit at high-throughput passenger customs clearance sites.

[0006] The present application solves the above technical problems by the following technical means:

[0007] An entry and exit passenger group key feature fusion evaluation method is proposed, and the method comprises:

[0008] Multi-dimensional features of entry and exit personnel are extracted from video stream data of the customs clearance channel, and the multi-dimensional features include portrait attribute features, gait attribute features, entry and exit attribute features, high-anti-fake portrait attribute features, luggage attribute features and human body attribute features;

[0009] A first similarity P(A1) between the portrait attribute features and the portrait attribute features of historical key personnel is calculated;

[0010] When the first similarity satisfies P(A1)≥M, the first similarity is increased according to the historical case information identifier associated with the entry and exit personnel and the corresponding entry and exit attribute features to obtain a second similarity, and M is a first threshold value;

[0011] When the first similarity satisfies N≤P(A1)<M, the first similarity is increased according to the gait attribute features, entry and exit attribute features, high-anti-fake portrait attribute features, luggage attribute features and human body attribute features of the entry and exit personnel to obtain a third similarity, and N is a second threshold value;

[0012] When the first similarity satisfies P(A1)<N, a fourth similarity is calculated according to the high-anti-fake portrait attribute features, luggage attribute features and human body attribute features of the entry and exit personnel;

[0013] The danger of the entry and exit personnel is evaluated according to the second similarity or the third similarity or the fourth similarity.

[0014] Further, when the first similarity satisfies P(A1)≥M, the first similarity is adjusted according to the historical case information identifier associated with the entry and exit personnel and the corresponding entry and exit attribute features to obtain a second similarity, which comprises:

[0015] When the first similarity satisfies P(A1)≥M, it is judged whether the entry and exit personnel have associated historical case information identifier according to the identity attribute information of the entry and exit personnel;

[0016] If yes, the first similarity P(A1) is increased to obtain a first similarity correction value;

[0017] If no, the first similarity remains unchanged;

[0018] The first similarity or the first similarity correction value is increased based on the entry and exit attribute characteristics, and the second similarity is obtained.

[0019] Further, the historical case information identifier includes a key personnel relative identifier, a key personnel companion identifier, and a historical key personnel identifier of interest; accordingly, the determination of whether the entry and exit personnel is associated with the historical case information identifier based on the identity attribute information of the entry and exit personnel includes:

[0020] The determination of whether the entry and exit personnel is associated with at least one of the key personnel relative identifier, the key personnel companion identifier, and the historical key personnel identifier of interest based on the identity attribute information of the entry and exit personnel.

[0021] Further, the adjustment of the first similarity or the first similarity correction value based on the entry and exit attribute characteristics to obtain the second similarity includes:

[0022] The determination of whether the entry and exit attribute characteristics of the entry and exit personnel are abnormal;

[0023] If yes, the first similarity or the first similarity correction value is increased to obtain the second similarity;

[0024] If no, the first similarity or the first similarity correction value is directly used as the second similarity.

[0025] Further, when the first similarity satisfies N≤P(A1)<M, the first similarity is adjusted based on the gait attribute characteristics, the entry and exit attribute characteristics, the high-anti-fake portrait attribute characteristics, the luggage attribute characteristics, and the human body attribute characteristics of the entry and exit personnel to obtain a third similarity, including:

[0026] When the first similarity satisfies N≤P(A1)<M, a gait similarity between the gait attribute characteristics of the entry and exit personnel and the gait attribute characteristics of the historical key personnel is calculated;

[0027] When the gait similarity is greater than a third threshold value, a second similarity correction value is calculated based on the first similarity and the gait similarity;

[0028] When the gait similarity is less than or equal to the third threshold value, the gait similarity remains unchanged;

[0029] The second similarity correction value or the gait similarity is increased based on the abnormality of the entry and exit attribute characteristics, the luggage attribute characteristics, the high-anti-fake portrait attribute characteristics, and the human body attribute characteristics of the entry and exit personnel to obtain the third similarity.

[0030] Further, the second similarity correction value or the gait similarity is increased according to the abnormality of the entry-exit attribute feature, the luggage attribute feature, the high-anti-fake portrait attribute feature and the human body attribute feature of the entry-exit personnel to obtain the third similarity, including:

[0031] The second similarity correction value or the gait similarity is taken as a first to-be-corrected value;

[0032] It is judged whether the entry-exit attribute feature of the entry-exit personnel is abnormal;

[0033] If yes, the first to-be-corrected value is increased to be a second to-be-corrected value;

[0034] If no, the second similarity correction value or the gait similarity is taken as a second to-be-corrected value;

[0035] It is judged whether the luggage attribute feature or the high-anti-fake portrait attribute feature is abnormal;

[0036] If yes, the second to-be-corrected value is increased to be a third to-be-corrected value;

[0037] If no, the second to-be-corrected value is taken as a third to-be-corrected value;

[0038] It is judged whether the human body attribute feature is abnormal, and the third to-be-corrected value is processed according to the abnormality of the human body attribute feature to obtain the third similarity.

[0039] Further, the human body attribute feature includes a basic feature, a gesture feature, a decoration feature and a behavior feature; correspondingly, it is judged whether the human body attribute feature is abnormal, and the third to-be-corrected value is processed according to the abnormality of the human body attribute feature to obtain the third similarity, including:

[0040] It is judged whether the gesture feature or the decoration feature of multiple entry-exit personnel is convergent;

[0041] If yes, the third to-be-corrected value is increased to be a fourth to-be-corrected value;

[0042] If no, the third to-be-corrected value is taken as a fourth to-be-corrected value;

[0043] It is judged whether the basic feature is abnormal;

[0044] If yes, the fourth to-be-corrected value is increased to be a fifth to-be-corrected value;

[0045] If no, the fourth to-be-corrected value is taken as a fifth to-be-corrected value;

[0046] It is judged whether the behavior feature is abnormal;

[0047] If yes, the fifth to be corrected value is increased and taken as the third similarity;

[0048] If no, the fifth to be corrected value is taken as the third similarity.

[0049] Further, the human body attribute features include human body basic features, gesture features, decoration features and behavior features; accordingly, when the first similarity satisfies P(A1)<N, a fourth similarity is calculated according to the high anti-fake portrait attribute features, the luggage attribute features and the human body attribute features of the entry and exit personnel, including:

[0050] When the first similarity satisfies P(A1)<N, a pseudo-dressing similarity P1 is calculated by summing up the index system constructed by the high anti-fake portrait attribute features, the luggage attribute features and the basic features of the entry and exit personnel by using the analytic hierarchy process;

[0051] Based on the gesture features and / or the decoration features, it is judged whether there are group convergence behaviors of multiple entry and exit personnel;

[0052] If yes, the pseudo-dressing similarity P1 is increased to obtain a pseudo-dressing similarity correction value;

[0053] If no, the pseudo-dressing similarity remains unchanged;

[0054] Based on the behavior features, it is judged whether there are dangerous behaviors;

[0055] If yes, the pseudo-dressing similarity correction value or the pseudo-dressing similarity is increased and taken as the fourth similarity;

[0056] If no, the pseudo-dressing similarity correction value or the pseudo-dressing similarity is taken as the fourth similarity.

[0057] Further, the entry and exit personnel danger is evaluated according to the second similarity or the third similarity or the fourth similarity, including:

[0058] The second similarity is outputted to output the key personnel alarm information and the second similarity sorting result;

[0059] The third similarity is compared with a set first alarm threshold value to output the alarm information of the key personnel or the suspected key personnel;

[0060] The fourth similarity is compared with a set second alarm threshold value to output the alarm information of the suspected key personnel.

[0061] In addition, the application further provides an entry and exit passenger group key feature fusion evaluation system, which comprises a collection device and a backend server, the collection device is connected with the backend server through a switch, and the collection device is arranged in a passenger customs channel;

[0062] A collection device is configured to collect video stream data of the passenger channel and transmit the video stream data to the backend server via a switch;

[0063] The backend server is configured to execute the steps of the immigration passenger group key feature fusion evaluation method.

[0064] Further, the backend server comprises a database server, an image algorithm comparison server and an application server, wherein the image algorithm comparison server comprises a human feature extraction module, a human feature comparison module and a key feature evaluation module connected in sequence, and the database server is provided with a real-time feature database, a key personnel feature database and a historical case information database;

[0065] The human feature extraction module is configured to extract multi-dimensional features of the immigration personnel from the video stream data of the channel and output the multi-dimensional features to the real-time feature database for storage.

[0066] The human feature comparison module is configured to compare the portrait attribute features and the gait attribute features of the immigration personnel with the feature information stored in the key personnel feature database, and send the comparison results and the associated historical case information identifiers stored in the historical case information database to the key feature evaluation module, and output the comparison results to the real-time feature database for storage.

[0067] The key feature evaluation module is configured to perform feature fusion evaluation according to the comparison results, the associated historical case information identifiers and the multi-dimensional features to obtain a risk evaluation result of the immigration personnel.

[0068] The application server is configured to perform alarm based on the risk evaluation result of the immigration personnel.

[0069] Further, the information stored in the key personnel feature database comprises a portrait photo, a gait video stream and human attribute features.

[0070] In addition, the application further provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the immigration passenger group key feature fusion evaluation method.

[0071] The application has the following advantages:

[0072] The application extracts multi-dimensional features of entry and exit personnel according to the video stream data of the overpass channel, calculates the face similarity between the portrait attribute features and the confirmed historical key personnel face image, and when the face similarity is in different value range intervals, combines the gait, entry and exit, high anti-fake, luggage, human body, historical case and other multi-dimensional attribute features for fusion evaluation, when the calculated face similarity is large, it indicates that the disguise degree is not high, at this time the face similarity with low disguise degree is adjusted to improve the accuracy of the similarity calculation result, which can quickly and accurately identify the repeated illegal and criminal key personnel who are not disguised or have low disguise degree and have been captured in high-throughput passenger customs clearance places; when the calculated face similarity is small, it indicates that the entry and exit personnel may have high disguise degree or the face is blocked, at this time the personnel with high disguise degree or blocked face are improved through hierarchical analysis and similarity, for the case that there is serious disguise or blocking resulting in the inability to judge by face, the suspicious key personnel who may exist hidden and cause body posture or behavior change are judged by the fusion of high anti-fake, human body, luggage and other attribute features, therefore, the personnel with high disguise degree can also be quickly and accurately identified.

[0073] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following description and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 is a flowchart of a key feature fusion evaluation method for entry and exit passenger groups according to an embodiment of the application;

[0075] Figure 2 is a complete flowchart of key feature fusion evaluation according to an embodiment of the application;

[0076] Figure 3 is a structure diagram of a key feature fusion evaluation system for entry and exit passenger groups according to an embodiment of the application;

[0077] Figure 4 is a recognition and evaluation principle diagram of a key feature fusion evaluation system for entry and exit passenger groups according to an embodiment of the application. DETAILED DESCRIPTION

[0078] In order to make the objects, technical solutions and advantages of the embodiments of the application clearer, the technical solutions in the embodiments of the application will be described below in a clear and complete manner, obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0079] As Figure 1As shown, the first embodiment of the present application proposes a method for evaluating the key features of an entry-exit passenger group, which comprises the following steps:

[0080] S10, extracting multi-dimensional features of the entry-exit personnel from the video stream data of the customs channel, wherein the multi-dimensional features include portrait attribute features, gait attribute features, entry-exit attribute features, high-anti-fake portrait attribute features, luggage attribute features, and human body attribute features;

[0081] S20, calculating a first similarity P(A1) between the portrait attribute features and the portrait attribute features of historical key personnel;

[0082] It should be noted that the present embodiment can use a similarity calculation method based on geometric features, a similarity calculation method based on template matching, a similarity calculation method based on deep learning, a similarity calculation method based on Euclidean distance, a cosine similarity calculation method, etc. to calculate the face similarity, and the specific implementation process is a mature technology, which will not be described here.

[0083] S30, when the first similarity satisfies P(A1)≥M, increasing the first similarity according to the historical case information associated with the entry-exit personnel and the corresponding entry-exit attribute features to obtain a second similarity, and M is a first threshold value;

[0084] S40, when the first similarity satisfies N≤P(A1)<M, increasing the first similarity according to the gait attribute features, entry-exit attribute features, high-anti-fake portrait attribute features, luggage attribute features, and human body attribute features of the entry-exit personnel to obtain a third similarity, and N is a second threshold value;

[0085] S50, when the first similarity satisfies P(A1)<N, then calculating a disguised similarity according to the high-anti-fake portrait attribute features, luggage attribute features, and human body attribute features of the entry-exit personnel in combination with the analytic hierarchy process, and increasing the suspicious similarity through dangerous behaviors and group convergent behaviors to obtain a fourth similarity;

[0086] S60, evaluating the danger of the entry-exit personnel according to the second similarity or the third similarity or the fourth similarity.

[0087] It should be noted that the embodiment extracts the multi-dimensional features of the entry and exit personnel according to the video stream data of the customs clearance channel, compares the portrait attribute features of each entry and exit personnel with the face features of the confirmed historical key personnel, calculates the face similarity, and when the face similarity, i.e. the first similarity P(A1), is greater than or equal to M, it can be basically determined that the corresponding entry and exit personnel is the key personnel to be paid attention to and checked, and the first similarity is increased in combination with the historical case information and the corresponding entry and exit attribute features of the entry and exit personnel, so as to improve the final output similarity, facilitate the priority investigation of key personnel with larger similarity according to the size of the output similarity, and better control the key personnel; when the first similarity satisfies N≤P(A1)<M, it means that the entry and exit personnel can be judged as suspected key personnel with a large probability, so the multi-dimensional attribute features such as gait attribute features, entry and exit attribute features, high anti-fake portrait attribute features, luggage attribute features and human body attribute features of the entry and exit personnel are further combined for auxiliary judgment, and the first similarity is increased according to the abnormal conditions of these multi-dimensional attribute features, so as to improve the accuracy of key personnel judgment; when the first similarity satisfies P(A1)<N, it can be basically judged that the entry and exit personnel is a normal transit personnel, but there may be some experienced key personnel with anti-detection ability who pass through disguise or other means, or due to the existence of shielding in the flux entry and exit place, so that part of the key personnel cannot be identified, therefore, the first similarity is increased in combination with the high anti-fake portrait attribute features, luggage attribute features and human body attribute features of the entry and exit personnel, so as to improve the final output similarity, thereby improving the accuracy and comprehensiveness of key personnel judgment. In summary, the entry and exit passenger group key feature fusion evaluation method proposed in the embodiment can quickly and accurately identify entry and exit key personnel even in a high-flux passenger customs clearance place,

[0088] It should be noted that the value of the second threshold N is less than the value of the first threshold M, and the value of M is 90%, and the value of N is 75%; but it should be understood that the specific values of M and N can be adjusted according to the specific application place and actual management situation, and the embodiment does not limit the values of M and N.

[0089] As a further preferred technical solution, the multi-dimensional features proposed in step S10 include portrait attribute features, gait attribute features, entry and exit attribute features, high anti-fake portrait attribute features, luggage attribute features and human body attribute features, which are only examples, and those skilled in the art can select other dimensional attribute features according to actual conditions.

[0090] Specifically, as shown in Table 1, the key evaluation feature index system designed in this embodiment contains 103, which involves portrait attribute features, gait attribute features, entry and exit attribute features, high anti-fake portrait attribute features, luggage attribute features, human body attribute features and historical case information identification.

[0091] Table 1 Key Feature Fusion Evaluation Index System

[0092]

[0093]

[0094]

[0095] Among them, (1) the portrait attribute feature A1 is an important feature for distinguishing key personnel, which can be extracted from the video stream data by image processing method, and by comparing the extracted portrait attribute feature with the confirmed historical key personnel portrait, the face similarity is obtained to judge whether there is historical key personnel in the customs personnel.

[0096] (2) The gait attribute feature B1 can be extracted from the video stream data by image processing method, which is used to compare with the gait feature of the confirmed historical key personnel to obtain the gait similarity.

[0097] (3) The entry and exit attribute features C1-C16, i.e. the number of entry and exit of entry and exit personnel in the period and the selection of customs port, can be divided into the number of exit, the number of entry, the port drift, etc., which can be judged by the number of personnel, time, and the position of the camera. According to the investigation, some key personnel may have the characteristics of short-term multiple entry and exit, and some key personnel may have the phenomenon of port drift to avoid the pursuit of officials. Among them, the port drift means that the port supervision is loose or tight, and the key personnel will exist from A port to B port to avoid the pursuit of officials.

[0098] (4) The high anti-fake portrait attribute features D1-D4, i.e. the entry and exit personnel may intentionally or unintentionally disguise in the process of entry and exit, which increases the difficulty of initial portrait identification, so the index system adds high anti-fake portrait. In actual research, the high anti-fake portrait can be divided into whether there is serious obstruction, whether there is a mask, whether there are sunglasses, whether there are ordinary glasses, etc.

[0099] (5) Luggage attribute features F1-F4. The customs personnel may carry different luggage during the entry and exit process, or there may be behaviors such as entering with no luggage and exiting with full luggage, i.e. empty-in full-out, or entering with full luggage and exiting with no luggage, i.e. empty-in full-out, and behaviors such as carrying too much luggage during entry and exit, and these behaviors have a high risk of smuggling during entry and exit. In actual implementation, whether the luggage associated with the portrait attribute features has the above-mentioned luggage attribute features is determined.

[0100] (6) Human body attribute features include: basic features G1-G19, gesture convergence features H1-H18, luggage convergence features I1-I3, accessory convergence features I4, clothing convergence features I5, frequent looking back features I6, and dangerous behavior features J1-J6. The basic features of the human body are used for searching and searching for specific known personnel; the multi-person gesture convergence features, the multi-person luggage convergence features, the multi-person accessory convergence features, and the multi-person clothing convergence features are mainly used for searching for key personnel groups, such as key personnel gangs who may carry the same luggage and wear the same hats; the personnel frequent looking back features are mainly used for searching for the contact between key personnel and key personnel; and the personnel dangerous behavior features are mainly used for searching for key personnel who escape from pursuit. The human body attribute features can also be extracted from the video stream data by image processing methods.

[0101] (7) Historical case information identifiers K1-K28 can include key personnel relatives, key personnel traveling together, and historically concerned key personnel, which are various types of smuggling personnel. The purpose of setting the historically concerned key personnel is that the entry and exit site will have personnel types of key concern at certain time intervals, which is related to the domestic and foreign situation, such as the notification of the Ministry of Commerce that coral smuggling has occurred frequently in recent times, and the evaluation weight of key personnel needs to be adjusted to enhance the management of such key personnel.

[0102] Further, in the embodiment, a human body photo containing a face in the video stream data is collected, then the face attribute features in the human body photo containing the face are recognized and numbered in real time, and then the rest of the 102 attribute features (such as gait attribute features, customs attribute features, etc.) recognized from the human body photo thereof are associated. The multi-dimensional features after the association are stored in the real-time feature database.

[0103] Further, the historical case information identifiers are stored in the historical case information library, the identity information of the historical key personnel in the key personnel feature library is associated through the face attribute features, and the historical case information library is associated through the identity information of the historical key personnel, and the association result is stored in the real-time feature database.

[0104] As a further preferred technical solution, such as Figure 2As shown, the step S30: when the first similarity degree satisfies P≥M, the first similarity degree is increased according to the historical case information identifier associated with the entry and exit personnel and the corresponding entry and exit attribute feature to obtain a second similarity degree, which specifically includes the following steps:

[0105] S31, when the first similarity degree satisfies P≥M, whether the historical case information identifier is associated with the identity attribute information of the entry and exit personnel is determined, if yes, step S32 is executed, if not, step S33 is executed;

[0106] S32, the first similarity degree P(A1) is increased to obtain a first similarity degree correction value;

[0107] S33, the first similarity degree is kept unchanged;

[0108] S34, the first similarity degree or the first similarity degree correction value is increased based on the entry and exit attribute feature to obtain the second similarity degree.

[0109] It should be noted that when the first similarity degree P(A1)≥90%, it is basically determined that the entry and exit personnel is a key personnel, then whether the historical case information identifier exists is determined, if the historical case information identifier exists, the similarity degree is increased through the algorithm, if not, it is not increased; then the entry and exit attribute feature is determined, and finally the second similarity degree is output, the key personnel alarm is performed. This is because in the actual management process, the law enforcement officials will prioritize the personnel with larger similarity degree according to the similarity degree output by the system, and the purpose of increasing the similarity degree is to better control the key personnel.

[0110] As a further preferred technical solution, whether the historical case information identifier is associated with the identity attribute information of the entry and exit personnel includes:

[0111] According to the identity attribute information of the entry and exit personnel, whether at least one of the key personnel relative identifier, the key personnel same trip identifier and the historical attention key personnel identifier is associated is determined.

[0112] It should be noted that when the entry and exit personnel is associated with at least one historical case information identifier, i.e. at least one of the historical case information identifiers K1-K28 is associated, the first similarity degree P(A1) is increased to obtain a first similarity degree correction value P1=P(A1)×80%+20%; if there is no associated historical case information identifier, the first similarity degree P(A1) is not adjusted, and P1=P(A1).

[0113] As a further preferred technical solution, the step S34 of adjusting the first similarity or the first similarity correction value based on the entry and exit attribute features to obtain the second similarity specifically comprises the following steps:

[0114] S341, judging whether the entry and exit attribute features corresponding to the entry and exit personnel are abnormal, if yes, executing step S342, and if no, executing step S343;

[0115] S342, increasing the first similarity or the first similarity correction value to obtain the second similarity;

[0116] S343, directly taking the first similarity or the first similarity correction value as the second similarity.

[0117] Specifically, the embodiment judges the entry and exit attribute features C1-C16, and increases the first similarity P(A1) or the first similarity correction value P1 according to the abnormality of the entry and exit attribute features. The specific implementation mode comprises:

[0118] Judging the abnormality of the entry and exit attribute features C1-C16, such as the port drift feature, the number of exit features and the number of entry features, and increasing the first similarity P(A1) or the first similarity correction value P1 when the abnormality is judged.

[0119] Or judging in turn according to the set order. The specific judgment order is:

[0120] If the port drift C16 = yes, i.e. there is a port drift, then the second similarity P2 = P1 x 60% + 40%;

[0121] If C1>1 or C2>1 or C3>2 exists, then the second similarity P2 = P1 x 70% + 30%;

[0122] If C4>1 or C5>1 or C6>2 exists, then the second similarity P2 = P1 x 85% + 15%;

[0123] If C7>7 or C8>7 or C9>14 or C10>15 or C11>15 or C12>30 or C13>25 or C14>25 or C15>50 exists, then the second similarity P2 = P1 x 90% + 10%;

[0124] It should be noted that in the judgment process of the entry and exit attribute features, the judgment order is C16 first, and then C1 to C15 are judged one by one. If the previous entry and exit attribute feature is judged to be abnormal, the subsequent entry and exit attribute feature is not judged.

[0125] If the entry and exit attribute characteristics C1-C16 do not have the above abnormal conditions, the similarity P1 is unchanged, and the second similarity P2 is set as P1.

[0126] As a further preferred technical solution, the step S40: when the first similarity satisfies N≤P<M, the first similarity is increased according to the gait attribute characteristics, the entry and exit attribute characteristics, the high anti-fake portrait attribute characteristics, the luggage attribute characteristics and the human body attribute characteristics of the entry and exit personnel, to obtain a third similarity, specifically including the following steps:

[0127] S41, when the first similarity satisfies N≤P<M, the gait similarity between the gait attribute characteristics of the entry and exit personnel and the gait attribute characteristics of the historical key personnel is calculated;

[0128] S42, when the gait similarity is greater than a set third threshold value, a second similarity correction value is calculated based on the first similarity and the gait similarity;

[0129] S43, when the gait similarity is less than or equal to the set third threshold value, the gait similarity is kept unchanged;

[0130] S44, according to the abnormal conditions of the entry and exit attribute characteristics, the luggage attribute characteristics, the high anti-fake portrait attribute characteristics and the human body attribute characteristics of the entry and exit personnel, the second similarity correction value or the gait similarity is increased to obtain the third similarity.

[0131] Specifically, when the gait similarity P(B1) is greater than a set third threshold value (which can be 90%), the similarity P1 is adjusted based on the first similarity P(A1) and the gait similarity P(B1) to P1=P(A1)×60%+P(B1)×40%; when the gait similarity is less than or equal to the set third threshold value, the gait similarity does not need to be adjusted, and is kept unchanged, and P1=P(B1); then the gait attribute characteristics, the entry and exit attribute characteristics, the luggage attribute characteristics, the human body attribute characteristics and the like are sequentially used for auxiliary judgment, and the purpose is to confirm whether the recognized personnel is a key personnel through more dimensional characteristics.

[0132] As a further preferred technical solution, the step S44: according to the abnormal conditions of the entry and exit attribute characteristics, the luggage attribute characteristics, the high anti-fake portrait attribute characteristics and the human body attribute characteristics of the entry and exit personnel, the second similarity correction value or the gait similarity is increased to obtain the third similarity, specifically including the following steps:

[0133] S441, the second similarity correction value or the gait similarity is set as a first to-be-corrected value;

[0134] S442, judging whether the entry-exit attribute feature of the entry-exit personnel is abnormal, if yes, executing step S443, if not, executing step S444;

[0135] S443, increasing the first to-be-corrected value to obtain a second to-be-corrected value;

[0136] S444, taking the second similarity correction value or the gait similarity as the second to-be-corrected value;

[0137] It should be noted that the judgment process of whether the entry-exit attribute feature is abnormal in step S44 is similar to the specific implementation process of step S34, which will not be described here.

[0138] S445, judging whether the luggage attribute feature or the high-anti-fake portrait attribute feature is abnormal, if yes, executing step S446, if not, executing step S447;

[0139] S446, increasing the second to-be-corrected value to obtain a third to-be-corrected value;

[0140] S447, taking the second to-be-corrected value as the third to-be-corrected value;

[0141] S448, judging whether the human body attribute feature is abnormal, and processing the third to-be-corrected value according to the abnormality of the human body attribute feature to obtain the third similarity.

[0142] Specifically, as shown in Figure 2 the embodiment sequentially judges the abnormality of the luggage attribute features F1-F4, and the judgment order is F2, F4, F1, F3. If F2=“yes” or F4=“yes”, the third to-be-corrected value P3=P2x60%+40%; if F1=“yes”, the third to-be-corrected value P3=P2x80%+20%; if F3=“yes”, the third to-be-corrected value P3=P2x85%+15%; if there is no above abnormality judgment result, the similarity remains unchanged

[0143] Further, the embodiment can also sequentially judge the luggage attribute features F1-F4 and whether wearing sunglasses D3, and the order is F2, F4, F1, D3, F3. If F2=“yes” or F4=“yes”, the third to-be-corrected value P3=P2x60%+40%; if F1=“yes” or D3=“yes”, the third to-be-corrected value P3=P2x80%+20%; if F3=“yes”, the third to-be-corrected value P3=P2x85%+15%; if there is no above judgment, the similarity remains unchanged.

[0144] It should be noted that when the first similarity satisfies N≤P<M, the second similarity correction value or gait similarity is increased according to the abnormality of the entry and exit attribute characteristics, the luggage attribute characteristics, the high anti-fake portrait attribute characteristics and the human body attribute characteristics of the entry and exit personnel. This step is mainly used to determine whether the key personnel has engaged in smuggling behavior through certain characteristics. The luggage difference is a direct way to show it, and the sunglasses are a possible disguise behavior. Because the channel for passing through customs is indoors, wearing sunglasses indoors is an abnormal behavior itself (but it does not exclude that some people prefer to wear sunglasses), so the high anti-fake sunglasses are singled out (masks and the like have been normalized after the epidemic, so this embodiment is not used as a judgment factor), but it does not need to be judged separately. Therefore, this abnormal behavior is attributed to the luggage characteristics, and the luggage difference risk in the customs process is greater than the sunglasses according to experience.

[0145] It should be noted that the embodiment follows the set judgment order, and if the previous attribute characteristics are determined to be abnormal, the subsequent judgment of other luggage attribute characteristics is not performed.

[0146] As a further preferred technical solution, the step S448: judging whether the human body attribute characteristics are abnormal and processing the third to-be-corrected value according to the abnormality of the human body attribute characteristics to obtain the third similarity, specifically includes the following steps:

[0147] S4481, judging whether there are multiple entry and exit personnel gesture characteristics or decoration characteristics that tend to converge, if yes, executing step S4482, and if no, executing step S4483;

[0148] S4482, increasing the third to-be-corrected value to obtain a fourth to-be-corrected value;

[0149] S4483, taking the third to-be-corrected value as the fourth to-be-corrected value;

[0150] S4484, judging whether the basic characteristics are abnormal, if yes, executing step S4485, and if no, executing step S4486;

[0151] S4485, increasing the fourth to-be-corrected value to obtain a fifth to-be-corrected value;

[0152] S4486, taking the fourth to-be-corrected value as the fifth to-be-corrected value;

[0153] S4487, judging whether the behavior characteristics are abnormal, if yes, executing step S4488, and if no, executing step S4489;

[0154] S4488, increasing the fifth to-be-corrected value to obtain the third similarity;

[0155] S4489、take the fifth to be modified value as the third similarity.

[0156] Specifically, the embodiment judges the features H1-H18 and I1-I6 in the human attribute features, and if they exist, the similarity is adjusted, the fourth to be modified value P4=P3x90%+10%, and if not, P4=P3; Then the human attribute features G1-G19 and whether wearing glasses D4 are judged, and if the key personnel exist known features, the similarity is adjusted according to the number of known features, the fifth to be modified value P5=P4x(100-a)%+a%1, where a is the number of known features, otherwise it is not adjusted; Then the human attribute features J1-J6 personnel dangerous behavior are judged, and if the dangerous behavior exists, the similarity is adjusted, the similarity P6=P5x90%+10%, otherwise it is not adjusted.

[0157] It should be noted that when the first similarity satisfies N≤P<M, the gait attribute features, entry and exit attribute features, luggage attribute features, and human attribute features are sequentially judged to assist in judgment, the purpose is to confirm whether the recognized personnel is a key personnel through more dimensional features, and at the same time, the personnel dangerous behavior is judged, the dangerous behavior of the personnel will affect the customs site, and finally the third similarity is output to judge whether it is≥85%(the first alarm threshold), if yes, it is judged as a key personnel, and if not, it is judged as a suspected key personnel, the suspected key personnel at this place is a face recognition similarity, but cannot be completely confirmed as a key personnel, and needs to be manually checked.

[0158] As a further preferred technical solution, the step S50: when the first similarity satisfies P(A1)<N, the fourth similarity is calculated according to the high anti-false portrait attribute features, luggage attribute features and human attribute features of the entry and exit personnel, and specifically includes the following steps:

[0159] S51, when the first similarity satisfies P(A1)<N, the anti-false similarity P1 is calculated by using the analytic hierarchy process to sum the index system constructed by the high anti-false portrait attribute features, luggage attribute features and basic features of the entry and exit personnel;

[0160] S52, whether there are group convergence behaviors of multiple entry and exit personnel is judged based on the gesture features and / or decoration features, if yes, step S53 is executed, and if not, step S54 is executed;

[0161] S53, the anti-false similarity P1 is increased to obtain an anti-false similarity modified value;

[0162] S54, the anti-false similarity is kept unchanged;

[0163] S55, determine whether there is dangerous behavior based on the behavior characteristics, if yes, execute step S56, if no, execute step S57;

[0164] S56, increase the camouflage similarity correction value or the camouflage similarity and take it as the fourth similarity;

[0165] S57, take the camouflage similarity correction value or the camouflage similarity as the fourth similarity.

[0166] Specifically, when the first similarity satisfies P < N, the original human face similarity P(A1) is ignored, the camouflage similarity is calculated according to the high anti-pseudo portrait attribute characteristics, the luggage attribute characteristics and the human body attribute characteristics of the entry and exit personnel, the suspected similarity is increased by combining the analytic hierarchy process and the dangerous behavior and the group convergence behavior, and the fourth similarity is obtained. Specifically, when the first similarity satisfies P < N, the camouflage similarity is calculated in the suspected key personnel evaluation index system, the index system includes high anti-pseudo attribute characteristics D1-D4, luggage attribute characteristics F1-F4, and human body attribute characteristics G1-G19. The scores are determined, and the score value and the weight value of each item are multiplied and summed to obtain the suspected key personnel similarity P1. The specific evaluation model is shown in Table 2. Then the human body attribute characteristics H1-H18 and I1-I6 are judged. If there is "yes", the similarity is adjusted as P2 = P1 x 90% + 10%, and if there is no "yes", the similarity is not adjusted. Then the human body attribute characteristics J1-J6 are judged. If there is a dangerous behavior, the similarity is adjusted as P3 = P2 x 90% + 10%, and a dangerous behavior alarm is given. If there is no dangerous behavior, the similarity is not adjusted and no alarm is given. Figure 2 Adjust the similarity, P2 = P1 x 90% + 10%, if not, do not adjust; Then the human body attribute characteristics J1-J6 are judged. If there is a dangerous behavior, the similarity is adjusted as P3 = P2 x 90% + 10%, and a dangerous behavior alarm is given. If there is no dangerous behavior, the similarity is not adjusted and no alarm is given. Figure 2 Adjust the similarity, P3 = P2 x 90% + 10%, and give a dangerous behavior alarm at the same time, if there is no dangerous behavior, do not adjust the similarity and do not give an alarm.

[0167] Table 2 suspected key personnel evaluation

[0168]

[0169]

[0170] It should be noted that in the foregoing judgment, when the first similarity is less than the second alarm threshold of 75%, the suspected key personnel evaluation model with occlusion is first judged. The suspected key personnel at this place means that the entry and exit personnel may have a camouflage behavior, which leads to the failure to identify the human face. This evaluation determines the first level characteristics such as high anti-pseudo portrait attribute characteristics, luggage attribute characteristics and human body attribute characteristics by the analytic hierarchy process, calculates the final similarity according to the index weight, judges the convergence attribute characteristics and the dangerous behavior of the personnel, and finally outputs the possibility of suspected key personnel and whether to alarm.

[0171] As a further preferred technical solution, the step S60: evaluating the dangerousness of the entry and exit personnel according to the second similarity or the third similarity or the fourth similarity, comprises:

[0172] Output the key personnel alarm information according to the second similarity, and output the second similarity ranking result;

[0173] Compare the third similarity with the set first alarm threshold, and output the alarm information of the key personnel or suspected key personnel;

[0174] Compare the fourth similarity with the set second alarm threshold, and output the alarm information of the suspected key personnel.

[0175] It should be noted that the threshold or weight values set in the foregoing evaluation logic are initial values, such as the threshold in the face recognition similarity P(A1) judgment, the threshold set in the gait similarity P(B1) judgment, and the alarm threshold, etc. are initial design values, which can be adjusted by machine learning or manual adjustment according to the actual management situation in the later period.

[0176] In addition, as shown in Figure 3 The second embodiment of the present application also proposes a key feature fusion evaluation system for entry and exit passenger groups, which comprises a collection device and a backend server, the collection device is connected with the backend server through a switch, and the collection device is arranged in a passenger channel;

[0177] The collection device is used for collecting video stream data of the passenger channel and transmitting the video stream data to the backend server through the switch;

[0178] The backend server is used for executing the key feature fusion evaluation method for entry and exit passenger groups as described in the first embodiment.

[0179] Among them, the collection device specifically adopts a camera 1, the camera 1 is arranged in the passenger channel, including an exit channel and an entry channel, and requires good light to shoot human features, and there may be multiple personnel entry and exit channels in some areas; the switch 2 is used for video stream transmission between the backend server and the camera 1; the backend server is used for executing the steps in the method described in the first embodiment according to the video stream data.

[0180] As a further preferred technical solution, as shown in Figure 4 The backend server comprises a database server 3, an image algorithm comparison server 4 and an application server 5, wherein the image algorithm comparison server 4 comprises a human feature extraction module, a human feature comparison module and a key feature evaluation module connected in sequence, and the database server is provided with a real-time feature database, a key personnel feature library and a historical case information library;

[0181] The human feature extraction module 12 is configured to extract multi-dimensional features of the entry-exit personnel from the video stream data of the passage and output the multi-dimensional features to the real-time feature database for storage;

[0182] The human feature comparison module 10 is configured to compare the portrait attribute features and the gait attribute features of the entry-exit personnel with the feature information stored in the key personnel feature library, and send the comparison results and the associated historical case information identifiers stored in the historical case information library to the key feature evaluation module, and output the comparison results to the real-time feature database for storage;

[0183] The key feature evaluation module 11 is configured to perform feature fusion evaluation according to the comparison results, the associated historical case information identifiers and the multi-dimensional features to obtain a risk assessment result of the entry-exit personnel;

[0184] The application server 5 is configured to perform alarm based on the risk assessment result of the entry-exit personnel.

[0185] Specifically, the database server 3 is configured to store the feature data of the entry-exit personnel, including a real-time feature database 7, a key personnel feature library 8 and a historical case information library 9. The real-time feature database 7 is configured to store the human feature data obtained by processing the video stream, such as the face attribute features A1 and the gait attribute features B1. The key personnel feature library 8 is configured to store the multi-dimensional features of the key personnel, i.e. the personnel who have committed illegal and criminal behaviors in the passage. The multi-dimensional features of the key personnel include portrait photos, gait video streams, names, certificate information, etc. In addition, the key personnel feature library 8 can also set basic attribute features such as gender and age, which are detailed in the key feature evaluation index system table G1-G19, and the convergence features H1-H18 and I1-I6 for searching specific personnel. The historical case information library 9 is configured to store the key personnel found by the previous officials or the personnel who have other illegal and criminal behaviors and are concerned by the public security department. The feature data of the key personnel includes historical case information identifiers K1-K28.

[0186] Specifically, the image algorithm comparison server 4 is a server for human feature comparison and key feature evaluation, including a human feature extraction module 12, a human feature comparison module 10 and a key feature evaluation module 11.

[0187] The human feature extraction module 12 is configured to extract the face attribute features A1, the gait attribute features B1, the high anti-fake attribute features D1-D4, the luggage attribute features F1-F4, the human attribute features G1-G19, H1-H18, I1-I6 and J1-J6.

[0188] The human body feature comparison module 10 is used to compare the human face attribute features A1, the gait attribute features B1, the high anti-fake attribute features D1-D4, the basic features G1-G19, etc. with the key personnel feature library feature data 8, and associate the historical case information library 9, judge the entry and exit attribute features C1-C16, etc.

[0189] The key feature evaluation module 11 is used to calculate and judge the results output and associated by the human body feature comparison module 10 through the key feature fusion evaluation model, output the similarity or possibility, and output the early warning information.

[0190] Specifically, the attribute features extracted and compared by the image algorithm comparison server 4 are stored in the real-time feature database 7; the application server 5 is mainly used to display the results of key feature evaluation and key personnel and suspected key personnel early warning, as well as some data searching and statistical functions.

[0191] Specifically, as shown in Figure 4 The system recognition evaluation logic is: the entry and exit customs flow 6 of different ports passes through the port entry and exit camera 1, enters the switch 2, is stored in the real-time feature database 7 in the form of video stream, then the human body feature extraction module 12 in the image algorithm comparison server 4 extracts photos every certain time, and extracts the human face attribute features A1, the gait attribute features B1, the entry and exit attribute features, the high anti-fake portrait attribute features D1-D4, the luggage attribute features F1-F4, the human body attribute features G1-G19, H1-H18, I1-I6, J1-J6, etc. The extraction logic of multi-dimensional features is that the image algorithm comparison server 4 collects the human body photos containing faces in the video stream, then recognizes the human face attribute features A1 in the human body attribute photos containing faces, then performs real-time numbering, and associates the remaining 102 attribute features (such as the gait attribute features B1, the customs attribute features C1-C16, etc.) recognized by the human body photos, and the data in this part are stored in the real-time feature database 7

[0192] Wherein A1 is a facial attribute feature, compared with the key personnel feature library 8 to determine the similarity P(A1); B1 is a gait attribute feature, compared with the key personnel feature library 8 to determine the similarity P(B1); C1-C16 is the number of customs clearance, there is a port drift (i.e. in different ports, such as entering in a port, and exiting in another port, etc.) and the more the number of entry and exit, the greater the risk; D1-D4 is a high anti-fake attribute feature, i.e. the attribute feature that may have a disguise behavior on the face, obtained by on-site human feature recognition; F1-F4 is a luggage attribute feature, i.e. the key personnel entering and exiting the port may have changes in the entry and exit luggage, such as the behavior exists, the risk is greater, obtained by on-site recognition; human attribute features G1-G19 are basic attributes of entry and exit personnel, such as the basic attributes of key personnel under customs control are known, which can be identified and judged, obtained by on-site human feature recognition; human attribute features H1-H18 and I1-I4 are feature convergence judgments, used to judge key groups, according to previous research, illegal and criminal personnel entering and exiting the country may appear in the form of a gang, and it is easy to identify and communicate during the customs clearance process, there are certain gestures or other feature convergence, obtained by on-site human feature recognition; human attribute features J1-J4 are personnel dangerous behavior judgments, used to judge whether the passenger has dangerous behaviors such as pushing and punching during the customs clearance process, obtained by on-site human feature recognition; K1-K28 is historical case information identification, used to confirm whether the key personnel identified by face recognition has historical criminal behavior or related, obtained by historical case information library 9.

[0193] Then, the human feature comparison module 16 compares the above-mentioned facial attribute features, gait attribute characteristics, etc. with the key personnel feature library 8 for personnel feature and information comparison. For key personnel with high facial attribute feature similarity, the historical case information in the historical case information library 9 is associated through the key personnel identity attribute information, and the comparison result and the associated historical case information are evaluated by the key feature evaluation module 11. Specifically:

[0194] When the human feature comparison module 10 pushes the similarity data, the key feature evaluation module starts to run. First, the facial A1 similarity is judged, whether ≥ 90%, if yes, then the historical case information identification K1-K28 is judged, if there is historical case information, then the similarity is adjusted, similarity P1=P(A1)×80%+20%, if not, then it is not adjusted. Then the entry and exit feature attribute features C1-C16 are judged in turn, if the previous attribute feature judgment is successful, then the subsequent entry and exit feature judgment is not performed, and the order is C16 first, then C1 to C15 are judged one by one.

[0195] If C16="Yes", similarity P2=P1x60%+40%; if C1>1 or C2>1 or C3>2 exist, similarity P2=P1x70%+30%; if C4>1 or C5>1 or C6>2 exist, similarity P2=P1x85%+15%; if C7>7 or C8>7 or C9>14 or C10>15 or C11>15 or C12>30 or C13>25 or C14>25 or C15>50 exist, similarity P2=P1x90%+10%; if C1-C16 do not exist the above judgment, the similarity remains unchanged. At the same time, the personnel are alarmed as key personnel, and the final similarity and alarm data are pushed to the application server 5.

[0196] Face A1 similarity judgment, if <90%, then make a second judgment, whether <75%, if yes, enter the suspected key personnel evaluation index system, which includes high anti-false attribute characteristics D1-D4, luggage attribute characteristics F1-F4, and human body attribute characteristics G1-G19. The score is determined, and each score value is multiplied by the weight value to obtain the suspected key personnel similarity P1. The specific evaluation model is shown in Table 2. Then the human body attribute characteristics H1-H18, I1-I6 are judged, if "Yes" exists, the similarity is adjusted as P2=P1x90%+10%, if not, no adjustment; then the human body attribute characteristics J1-J6 are judged, if dangerous behavior exists, the similarity is adjusted as P3=P2x90%+10%, and the dangerous behavior alarm is performed, if no dangerous behavior exists, the similarity is adjusted without alarm. The dangerous behavior alarm is irrelevant to the key personnel or suspected key personnel alarm. Finally, whether the similarity is ≥70% is judged, if yes, the suspected key personnel alarm is performed, if not, no key personnel or suspected key personnel alarm is performed. Figure 2 Figure 2

[0197] If the face A1 similarity is ≥75% and <90%, the gait B1 similarity is judged whether >90%, if yes, the similarity is adjusted as P2=P1x90%+10%, if no, the similarity is not adjusted. At the same time, the dangerous behavior alarm is performed, if no dangerous behavior exists, the similarity is adjusted without alarm. The dangerous behavior alarm is irrelevant to the key personnel or suspected key personnel alarm. Finally, whether the similarity is ≥70% is judged, if yes, the suspected key personnel alarm is performed, if not, no key personnel or suspected key personnel alarm is performed. Figure 2 ​​Adjust the similarity, similarity adjustment, similarity P1 = P(A1) x 60% + P(B1) x 40%, otherwise not adjusted. Then in turn to the entry and exit characteristic attribute characteristics C1 ~ C16 are judged, such as the previous attribute characteristics are determined successfully, then no subsequent entry and exit characteristics are judged, the order is first C16, then C1 to C15 are judged one by one. If C16 = "yes", similarity P2 = P1 x 60% + 40%; If there is C1 > 1 or C2 > 1 or C3 > 2, similarity P2 = P1 x 70% + 30%; If there is C4 > 1 or C5 > 1 or C6 > 2, similarity P2 = P1 x 85% + 15%; If there is C7 > 7 or C8 > 7 or C9 > 14 or C10 > 15 or C11 > 15 or C12 > 30 or C13 > 25 or C14 > 25 or C15 > 50, similarity P2 = P1 x 90% + 10%; If C1 ~ C16 does not exist the above judgment, the similarity is unchanged.

[0198] Then the luggage attribute characteristics F1 ~ F4, and whether to wear sunglasses D3 are judged in turn, such as the previous attribute characteristics are determined successfully, then no subsequent entry and exit characteristics are judged, the order is F2, F4, F1, D3, F3. If F2 = "yes", or F4 = "yes", similarity P3 = P2 x 60% + 40%; If F1 = "yes", or D3 = "yes", similarity P3 = P2 x 80% + 20%; If F3 = "yes", similarity P3 = P2 x 85% + 15%; If there is no above judgment, the similarity is unchanged;

[0199] Then the human body attribute characteristics H1 ~ H18, I1 ~ I6 are judged, if there is, the similarity is adjusted, similarity P4 = P3 x 90% + 10%, if not, it is not adjusted; Then the human body attribute characteristics G1 ~ G19 and whether to wear glasses D4 are judged, if the key personnel exist known characteristics, the similarity is adjusted according to the number of known characteristics, similarity P5 = P4 x (100-a) % + a x 1%, where a is the number of known characteristics, otherwise not adjusted; Then the human body attribute characteristics J1 ~ J6 personnel dangerous behavior are judged, if there is dangerous behavior, the similarity is adjusted, similarity P6 = P5 x 90% + 10%, otherwise not adjusted, at the same time there is dangerous behavior, the dangerous behavior alarm is given, otherwise not alarm, the dangerous behavior alarm is irrelevant to the key personnel or suspected key personnel alarm; Finally, the similarity is judged whether ≥ 85%, yes, the key personnel alarm, otherwise the suspected key personnel alarm.

[0200] Finally, the personnel type evaluated includes: key personnel, suspected key personnel, other personnel, and the application server 5 alarms for key personnel and suspected key personnel. Subsequent field officials check the key personnel and suspected key personnel according to the warning information in the application server 5, and file the suspected key personnel who have illegal behavior and do not belong to the key personnel feature library 8 into the key personnel feature library 8, and the historical case information of the key personnel is stored into the historical case information library 9.

[0201] In addition, the third embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the entry and exit passenger group key feature fusion evaluation method according to the first embodiment.

[0202] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be specifically embodied in any computer readable medium for use by or in conjunction with an instruction execution system, device or apparatus, such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions, or in conjunction with these instruction execution systems, devices or apparatus. For the present specification, the "computer readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in conjunction with an instruction execution system, device or apparatus, or in conjunction with these instruction execution systems, devices or apparatus. More specific examples (non-exhaustive list) of computer readable medium include the following: electrical connections having one or more wires (electronic devices), portable computer disk boxes (magnetic devices), random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memories), fiber optic devices, and portable compact disc read-only memories (CDROMs). In addition, the computer readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or processing as necessary, or in other suitable manner, and then stored in a computer memory.

[0203] It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0204] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0205] In addition, the terms "first", "second", are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0206] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A method for integrating and evaluating key characteristics of inbound and outbound passenger groups, characterized in that, Comprising: Extracting multi-dimensional features of entry-exit personnel from the video stream data of the customs clearance channel, where the multi-dimensional features include portrait attribute features, gait attribute features, entry-exit attribute features, high anti-counterfeiting portrait attribute features, luggage attribute features, and human body attribute features; Calculating the first similarity P(A1) between the portrait attribute features of the entry-exit personnel and the portrait attribute features of historical key personnel; When the first similarity satisfies P(A1)≥M, increasing the first similarity according to the historical case information identifier associated with the entry-exit personnel and the corresponding entry-exit attribute features to obtain the second similarity, where M is a set first threshold; When the first similarity satisfies N≤P(A1)<M, increasing the first similarity according to the gait attribute features, entry-exit attribute features, high anti-counterfeiting portrait attribute features, luggage attribute features, and human body attribute features of the entry-exit personnel to obtain the third similarity, where N is a set second threshold; When the first similarity satisfies P(A1)<N, calculating the fourth similarity according to the high anti-counterfeiting portrait attribute features, luggage attribute features, and human body attribute features of the entry-exit personnel; Evaluating the risk of entry-exit personnel according to the second similarity or the third similarity or the fourth similarity.

2. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 1, characterized in that, The adjusting the first similarity according to the historical case information identifier associated with the entry-exit personnel and the corresponding entry-exit attribute features to obtain the second similarity when the first similarity satisfies P(A1)≥M includes: When the first similarity satisfies P(A1)≥M, judging whether there is a historical case information identifier associated according to the identity attribute information of the entry-exit personnel; If so, increasing the first similarity P(A1) to obtain a corrected value of the first similarity; If not, keeping the first similarity unchanged; Increasing the first similarity or the corrected value of the first similarity based on the entry-exit attribute features to obtain the second similarity.

3. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 2, characterized in that, The historical case information identifier includes a key personnel relative identifier, a key personnel traveling together identifier, and a historical key personnel identifier of concern; correspondingly, the judging whether there is a historical case information identifier associated according to the identity attribute information of the entry-exit personnel includes: Judging whether there is at least one of the key personnel relative identifier, the key personnel traveling together identifier, and the historical key personnel identifier of concern associated according to the identity attribute information of the entry-exit personnel.

4. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 2, characterized in that, The adjusting the first similarity or the corrected value of the first similarity based on the entry-exit attribute features to obtain the second similarity includes: Judging whether the entry-exit attribute features corresponding to the entry-exit personnel are abnormal; If so, increasing the first similarity or the corrected value of the first similarity to obtain the second similarity; If not, directly taking the first similarity or the corrected value of the first similarity as the second similarity.

5. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 1, characterized in that, The adjusting the first similarity according to the gait attribute features, entry-exit attribute features, high anti-counterfeiting portrait attribute features, luggage attribute features, and human body attribute features of the entry-exit personnel to obtain the third similarity when the first similarity satisfies N≤P(A1)<M includes: When the first similarity satisfies N ≤ P(A1) < M, calculate the gait similarity between the gait attribute features of the entry-exit personnel and the gait attribute features of the historical key personnel; When the gait similarity is greater than the set third threshold, calculate the second similarity correction value based on the first similarity and the gait similarity; When the gait similarity is less than or equal to the set third threshold, keep the gait similarity unchanged; According to the abnormal conditions of the entry-exit attribute features, luggage attribute features, high anti-counterfeiting portrait attribute features, and human body attribute features of the entry-exit personnel, increase the second similarity correction value or the gait similarity to obtain the third similarity.

6. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 5, characterized in that, The increasing the second similarity correction value or the gait similarity according to the abnormal conditions of the entry-exit attribute features, luggage attribute features, high anti-counterfeiting portrait attribute features, and human body attribute features of the entry-exit personnel to obtain the third similarity includes: Let the second similarity correction value or the gait similarity be the first value to be corrected; Judge whether the entry-exit attribute features of the entry-exit personnel are abnormal; If so, increase the first value to be corrected and use it as the second value to be corrected; If not, let the second similarity correction value or the gait similarity be the second value to be corrected; Judge whether there are abnormalities in the luggage attribute features or high anti-counterfeiting portrait attribute features; If so, increase the second value to be corrected and use it as the third value to be corrected; If not, let the second value to be corrected be the third value to be corrected; Judge whether the human body attribute features are abnormal, and process the third value to be corrected according to the abnormal conditions of the human body attribute features to obtain the third similarity.

7. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 6, characterized in that, The human body attribute features include human body basic features, gesture features, decoration features, and behavior features; correspondingly, the judging whether the human body attribute features are abnormal and processing the third value to be corrected according to the abnormal conditions of the human body attribute features to obtain the third similarity includes: Judge whether there are convergences in the gesture features or decoration features of multiple entry-exit personnel; If so, increase the third value to be corrected and use it as the fourth value to be corrected; If not, let the third value to be corrected be the fourth value to be corrected; Judge whether the basic features are abnormal; If so, increase the fourth value to be corrected and use it as the fifth value to be corrected; If not, let the fourth value to be corrected be the fifth value to be corrected; Judge whether the behavior features are abnormal; If so, increase the fifth value to be corrected and use it as the third similarity; If not, let the fifth value to be corrected be the third similarity.

8. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 1, characterized in that, The human body attribute features include human body basic features, gesture features, decoration features, and behavior features; correspondingly, when the first similarity satisfies P(A1) < N, then calculate the fourth similarity according to the high anti-counterfeiting portrait attribute features, luggage attribute features, and human body attribute features of the entry-exit personnel, including: When the first similarity satisfies P(A1) < N, use the analytic hierarchy process to sum and calculate the disguise similarity P1 for the index system constructed by the high anti-counterfeiting portrait attribute features, luggage attribute features, and basic features of the entry-exit personnel; Determine whether multiple inbound and outbound travelers exhibit similar group behaviors based on gesture characteristics and / or decorative characteristics; If so, the camouflage similarity P1 is increased to obtain the camouflage similarity correction value; If not, then the camouflage similarity remains unchanged; Determine whether dangerous behavior exists based on behavioral characteristics; If so, the disguised similarity correction value or the disguised similarity is increased and then used as the fourth similarity; If not, then let the spoofing similarity correction value or the spoofing similarity be used as the fourth similarity.

9. The method for integrating and evaluating key characteristics of inbound and outbound passenger groups as described in claim 1, characterized in that, The assessment of the risk level of inbound and outbound personnel based on a second, third, or fourth similarity score includes: Based on the second similarity score, output alarm information for key personnel and output the ranking result of the second similarity score; The third similarity is compared with the set first alarm threshold, and alarm information for key personnel or suspected key personnel is output. The fourth similarity is compared with the set second alarm threshold, and alarm information for suspected key personnel is output.

10. A system for integrating and evaluating key characteristics of inbound and outbound passenger groups, characterized in that, It includes data collection equipment and a back-end server. The data collection equipment is connected to the back-end server through a switch and is deployed in the passenger clearance channel. The acquisition equipment is used to collect video stream data of the passenger passageway and transmit it to the backend server via a switch; A backend server is used to execute the method for fusion evaluation of key characteristics of inbound and outbound passenger groups as described in any one of claims 1 to 9.

11. The inbound and outbound passenger group key characteristic fusion assessment system as described in claim 10, characterized in that, The backend server includes a database server, an image algorithm comparison server, and an application server. The image algorithm comparison server includes a human feature extraction module, a human feature comparison module, and a key feature evaluation module connected in sequence. The database service includes a real-time feature database, a key personnel feature database, and a historical case information database. The human feature extraction module is used to extract multi-dimensional features of inbound and outbound personnel based on the video stream data of the customs clearance channel and output them to the real-time feature database for storage. The human feature comparison module is used to compare the facial and gait attributes of inbound and outbound personnel with the feature information stored in the key personnel feature database, and send the comparison results and the historical case information identifiers stored in the associated historical case information database to the key feature evaluation module, and output the comparison results to the real-time feature database for storage. The key feature assessment module is used to perform feature fusion assessment based on comparison results, related historical case information, and multi-dimensional features to obtain the risk assessment results of inbound and outbound personnel. The application server is used to generate alerts based on the risk assessment results of inbound and outbound personnel.

12. The inbound and outbound passenger group key characteristic fusion assessment system as described in claim 11, characterized in that, The information stored in the key personnel feature database includes portrait photos, gait video streams, and human attribute features.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for fusion evaluation of key characteristics of inbound and outbound passenger groups as described in any one of claims 1-9.

Images