An airport security data intelligent analysis method and system

Abstract

The present application relates to the technical field of data processing, and more particularly to an airport security data intelligent analysis method and system. The method comprises the following steps: obtaining airport multi-source security data; performing preliminary analysis on the security data according to the airport multi-source security data to obtain security characteristic sequence data; performing behavior mutation feature correlation according to the security characteristic sequence data to obtain security mutation feature correlation data, and performing cross-modal feature correlation according to the security characteristic sequence data to obtain security cross-modal feature correlation data; performing mutation behavior threat level evaluation according to the security mutation feature correlation data to obtain first security danger data, and performing cross-modal threat level evaluation according to the security cross-modal feature correlation data to obtain second security danger data; and performing security decision auxiliary work according to the first security danger data and the second security danger data. The present application greatly improves the intelligent level and efficiency of airport security through quantitative threat and intelligent decision support.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an intelligent analysis method and system for airport security inspection data. Background Technology

[0002] With the increasing demand for airport security checks and the growing complexity of security threats, traditional manual security checks are no longer sufficient to meet the security requirements of modern air transport. Modern airport security systems not only need to inspect baggage, personnel, and their belongings, but also need to provide real-time warnings and threat assessments of potential security risks during the security process. Existing security systems typically rely on single-modal data inputs, such as X-ray scan images, video from behavioral surveillance cameras, or manual inspection records. The processing efficiency and accuracy of these single data sources have certain limitations.

[0003] Furthermore, the analysis of security inspection data and threat identification processes often rely on human experience, making it difficult to handle complex and diverse threat situations. With the rapid development of technologies such as deep learning, image recognition, and behavioral analysis, traditional security inspection methods are gradually failing to meet the demands for real-time processing of multi-source data and intelligent decision support. In particular, threat identification in the security inspection process involves the fusion and analysis of data from different modalities, placing higher demands on existing technologies.

[0004] While some research has focused on image recognition and behavior analysis, most existing methods still rely on processing single data sources and lack the ability to comprehensively analyze multimodal data. At the same time, there is still no effective intelligent analysis framework for accurately identifying and assessing the threat levels of different behaviors, or for combining multiple data sources to optimize security inspection decisions. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes an intelligent analysis method and system for airport security inspection data, thereby resolving at least one of the aforementioned technical issues.

[0006] This application provides an intelligent analysis method for airport security inspection data, including the following steps:

[0007] Step S1: Obtain multi-source security inspection data from the airport;

[0008] Step S2: Conduct preliminary analysis of security inspection data based on multi-source security inspection data at the airport to obtain security inspection characteristic sequence data;

[0009] Step S3: Perform behavioral mutation feature association based on security inspection characteristic sequence data to obtain security inspection mutation feature association data, and perform cross-modal feature association based on security inspection characteristic sequence data to obtain security inspection cross-modal feature association data;

[0010] Step S4: Based on the security inspection mutation feature correlation data, assess the threat level of mutation behavior to obtain the first security inspection hazard data, and based on the security inspection cross-modal feature correlation data, assess the cross-modal threat level to obtain the second security inspection hazard data;

[0011] Step S5: Conduct security inspection decision support operations based on the first security inspection hazard data and the second security inspection hazard data.

[0012] This invention utilizes behavioral mutation feature correlation analysis of security inspection data to detect potential dangerous behaviors at an early stage, such as sudden rapid movement or abnormal pauses. By detecting early warning signs of potential threats, the accuracy of threat identification is improved. By integrating data from different modalities (such as video surveillance, sensor data, and facial recognition), it captures changes in human behavior and emotions, enhancing the accuracy of cross-modal feature correlation analysis, avoiding blind spots from single data sources, and thus strengthening the ability to identify complex threat scenarios. Real-time threat level assessment of security inspection data provides precise decision support. Security personnel can receive real-time action suggestions when potential threats are identified, enabling them to take additional inspection measures or issue alarms in a timely manner, improving response speed and reducing risks caused by human error. The system can also optimize the decision-making process based on assessment results, achieving adaptive and intelligent adjustments to ensure a balance between security and efficiency. By integrating data from different sources (video, sensors, facial recognition, behavioral analysis, etc.), this method can comprehensively analyze the dynamic behavior of inspected items and personnel, revealing more potential danger signals and enhancing the comprehensiveness and reliability of security inspections.

[0013] Preferably, step S1 specifically includes:

[0014] Step S11: Collect real-time airport security data by accessing the API interface of security inspection equipment (such as baggage scanners, behavior monitoring cameras, and identity verification systems);

[0015] Step S12: Perform multimodal data edge processing on the airport's real-time security inspection data to obtain preprocessed security inspection data;

[0016] Step S13: Assign data tags based on the preprocessed security inspection data to obtain tagged security inspection data;

[0017] Step S14: Merge the tagged security check data into the cloud to generate multi-source security check data for the airport.

[0018] This invention employs edge processing to analyze data from various security screening devices (such as baggage scanners, cameras, and identity verification systems), enabling comprehensive analysis of the security screening process from multiple perspectives. For example, baggage scanners provide image data of items, behavioral monitoring cameras provide video data, and identity verification systems provide biometric data. Combining these different modalities of data helps to more comprehensively and accurately assess potential threats in security screening scenarios. Real-time data acquisition and multimodal data preprocessing effectively improve the accuracy and real-time performance of data processing. Multimodal data fusion and tagged data provide richer information for threat identification and decision analysis. Cloud storage and integration make data management more efficient, while supporting real-time analysis and decision-making for large-scale data.

[0019] Preferably, step S2 specifically includes:

[0020] Step S21: Perform multimodal data cleaning based on multi-source security inspection data from the airport to obtain cleaned security inspection data;

[0021] Step S22: Extract security inspection hazard features and spatial features from the security inspection cleaning data to obtain security inspection hazard feature data and security inspection spatial feature data, respectively;

[0022] Step S23: Perform feature fusion encoding on the security inspection hazard feature data and the security inspection spatial feature data to obtain security inspection characteristic sequence data.

[0023] This invention cleanses multi-source security inspection data (such as images, sensor data, video surveillance, etc.) to remove noise, redundant information, and incomplete data, making the remaining data more accurate, clean, and efficient. By extracting hazard features from the cleaned security inspection data, potential security threats can be effectively identified. For example, abnormal item shapes can be extracted from baggage scanning data, and abnormal behavioral features can be extracted from behavioral surveillance videos. The extracted hazard features provide more accurate information support for subsequent threat assessment and decision-making. Spatial feature extraction focuses on analyzing the spatial layout, movement trajectories, and positional relationships of people and items within the security inspection area. For example, identifying abnormal movement trajectories of security personnel, item stacking, and spatial relationships between people and items helps to further determine potential security risks. Especially in complex security inspection environments, spatial feature identification can effectively support behavioral pattern recognition and abnormal activity detection. By fusing and encoding security hazard features and spatial features, feature information from different sources and types can be integrated into a unified feature sequence. This not only optimizes the data expression but also effectively captures multi-dimensional information interaction, such as the relationship between the degree of danger of an item and its spatial location. Feature fusion coding combines hazard features and spatial features to extract more comprehensive and richer security inspection characteristics, which facilitates pattern recognition, behavior analysis and threat assessment.

[0024] Preferably, the extraction of security inspection hazard features specifically includes:

[0025] Visual features are extracted from security inspection image data in the security inspection cleaning data to obtain security inspection visual feature data;

[0026] A time-stamped map is constructed based on the security inspection image data in the security inspection and cleaning data to obtain the time-stamped map data;

[0027] Time feature fusion is performed on time-stamped map data and security visual feature data to obtain security visual feature time map data;

[0028] Hazard features are extracted from the time-map data of visual features in security checks to obtain hazard feature data.

[0029] This invention extracts visual features from security inspection image data to identify potential threatening items, such as shape, material, size, and density. These features may be overlooked or misjudged in traditional security inspection methods. Visual feature extraction, using deep learning or computer vision algorithms, can automatically extract fine-grained item features, effectively improving identification accuracy. Especially in complex images with diverse items, visual feature extraction provides stronger evidence for item hazard assessment. Time-stamped map construction tracks changes in items over time, capturing dynamic information during the security inspection process, such as movement trajectories, changes in scanned images, and abnormal pauses. Through temporal feature fusion, the system can capture item behavior patterns across time periods, thereby identifying time-series features related to dangerous items. For example, abnormal changes during scanning (such as sudden stops or unnatural shape changes) can serve as warning signals. Extracting hazard-related features from the fused visual and temporal features allows for the determination of whether an item is dangerous based on its spatiotemporal behavior patterns. To further improve the system's sensitivity and accuracy to potential threats, avoid false alarms and missed alarms, and enhance the reliability of the security inspection system.

[0030] Preferably, the construction of the time-stamped graph specifically involves:

[0031] Based on the security inspection image data in the security inspection and cleaning data, a time tag map of a single item is constructed to obtain the first time tag map data;

[0032] Based on the security inspection image data in the security inspection and cleaning data, a multi-item time-label map is constructed to obtain the second time-label map data;

[0033] The first time-labeled map data and the second time-labeled map data are fused into a layered map to obtain time-labeled map data;

[0034] The construction of the single-item time-stamped graph is specifically as follows:

[0035] Single-item target data is obtained by identifying and extracting single-item target data from the security inspection and cleaning data;

[0036] Extract time-series labels from single-item target data to obtain single-item time-series label data;

[0037] Nodes are constructed based on single-item time-series tag data and single-item target data to obtain single-item time-series node data.

[0038] Neighbor node relationship analysis is performed on the time-series node data of a single item to obtain neighbor node relationship data of the single item;

[0039] Graphs are constructed based on the neighbor node relationship data and time sequence node data of single items to obtain the first time label graph data.

[0040] The construction of the multi-item time-stamped graph is specifically as follows:

[0041] Multi-item node data is obtained by constructing multi-item nodes based on different single-item time-series node data corresponding to the same single-item time-series tag data.

[0042] Based on the multi-item node data, interaction relationship processing, functional relationship processing, and structural relationship processing are performed to obtain multi-item interaction relationship data, multi-item functional relationship data, and multi-item structural coupling relationship data.

[0043] Graphs are constructed on the multi-item node data based on the multi-item interaction relationship data, multi-item functional relationship data, and multi-item structural coupling relationship data, respectively, to obtain multi-item interaction relationship graph data, multi-item functional relationship graph data, and multi-item structural coupling relationship graph data;

[0044] Static multi-level graph fusion is performed based on multi-item interaction relationship graph data, multi-item functional relationship graph data, and multi-item structural coupling relationship graph data to obtain second time-labeled graph data.

[0045] This invention, through target identification and temporal tag extraction of individual items, can clearly capture the dynamic changes of items during security checks. For example, whether an item exhibits abnormal behavior (such as sudden pausing or abnormal shape changes) during security checks can be accurately recorded and analyzed using time-series data. When multiple items appear simultaneously in a security check image, the construction of a multi-item temporal tag graph allows for the simultaneous analysis of the relationships between different items and their changes over time, helping to identify the potential dangers of certain items, such as abnormal interactions between multiple items or combinations suspected of concealing dangerous items. By constructing and analyzing the relationships between multiple item nodes, complex interactions, functional relationships, and structural couplings between multiple items can be identified and modeled. By comprehensively analyzing the dynamic changes of multiple items over different time periods and their interactions, early warnings of abnormal items can be achieved. Especially in high-density security check environments, the combination or proximity of multiple items may pose a higher risk, and traditional methods may struggle to detect such complex interactions. Graph construction allows for better identification and handling of this complexity. For relationships between multiple items, simple item identification may not be sufficient to reveal their potential dangers. By constructing multi-item interaction diagrams, functional relationship diagrams, and structural coupling relationship diagrams, we can delve deeper into the interaction patterns and structural characteristics between items during security checks, thereby improving the ability to identify potentially dangerous items in complex situations. For example, by analyzing the interaction relationships of multiple items, we can detect that some items are being used as tools to conceal other items, thus uncovering dangerous signals that traditional security inspection methods cannot identify.

[0046] By fusing layered graphs of the first and second time-labeled graphs, and further through static multi-layered graph fusion, time-series data from different levels can be combined to form a more refined time-labeled graph. This not only improves the utilization efficiency of time-series data but also optimizes the modeling of dynamic relationships between multiple items, allowing data from different levels to complement and reinforce each other, thus providing a more comprehensive threat assessment. When security inspection equipment struggles to identify some potentially dangerous items, the fused data can provide more reliable information for decision-making. By combining static spatiotemporal information (such as the appearance features of items in images) and dynamic information (such as the movement trajectory and interaction behavior of items), more accurate behavior prediction and threat assessment can be performed at different time points, further improving the predictive ability for threatening items. For security inspection scenarios requiring long-term monitoring and high-density items, multi-layered graph fusion provides stronger support for threat assessment.

[0047] Preferably, the layered graph fusion specifically involves:

[0048] Based on the first time-label map data and the second time-label map data, a time change layer is constructed to obtain the time change layer data;

[0049] Spatial relationship layer data is constructed based on the first time-label map data and the second time-label map data.

[0050] The interaction layer is constructed based on the first time tag map data and the second time tag map data to obtain the interaction layer data;

[0051] Graph attention networks were used to extract feature data from the time-varying layer data, spatial relationship layer data, and interaction layer data, respectively.

[0052] Based on the time-varying layer feature data, spatial relationship layer feature data, and interaction layer feature data, inter-layer connections are made between the time-varying layer data, spatial relationship layer data, and interaction layer data to obtain layered graph connection data.

[0053] Graph fusion is performed based on the data connected by the hierarchical graph to obtain time-stamped graph data.

[0054] This invention, through independent modeling of the time-varying layer, spatial relationship layer, and interaction layer, effectively distinguishes different types of information, thereby enabling refined processing of time-series data, spatial relationships, and interactions between items. The data processing method for each layer can be independently optimized, resulting in more accurate information extraction. The time-varying layer focuses on the dynamic changes of items along the timeline, suitable for capturing trends during security checks and helping to identify abnormal fluctuations in item status. The spatial relationship layer focuses on the spatial positional relationships between items, effectively identifying patterns of concealed or potentially hazardous materials through spatial layout analysis. The interaction layer handles the interactions between multiple items, revealing mutual influences and potential threatening behaviors, which is particularly crucial in security checks involving multiple people or high-density items.

[0055] In feature extraction at each layer, a graph attention network effectively captures and weights the relationships between different layers, enhancing the ability to identify key features. Through the attention mechanism, the importance of each node and its neighbors can be adaptively adjusted, thus highlighting threat-related features. For example, some potentially threatening items exhibit abrupt changes in temporal sequence, and the graph attention network prioritizes these abrupt change patterns, thereby helping to improve the accuracy of dangerous item identification.

[0056] By connecting and fusing feature data at different levels, effective integration of multi-level information can be achieved. After fusion, the data from each layer not only retains its unique characteristics but also captures the relationships between layers, providing more comprehensive and accurate time-stamped graph data. By connecting time changes, spatial relationships, and interaction layers, the traditional single-level information structure can be broken down, enabling deep fusion of multi-dimensional features. Fusion of graphs from different levels not only preserves the information advantages of each level but also eliminates the limitations of single-level data analysis, ensuring the comprehensiveness and accuracy of judgments. For example, the combination of interaction and spatial layers can reveal potential threat patterns that may arise from changes in objects over time, while the combination of time change and interaction layers can help discover potential dynamic threat behaviors.

[0057] Preferably, the spatial feature extraction specifically includes:

[0058] Personnel detection data is obtained by performing personnel detection based on multi-source security screening data from the airport.

[0059] Perform character tracking on the character detection data to obtain character tracking data;

[0060] Facial detection is performed based on person tracking data to obtain facial detection data;

[0061] Facial expression data is obtained by performing facial label analysis based on facial detection data.

[0062] By combining facial tracking data and facial tag data to analyze the spatial relationships of facial expression changes, we can obtain facial expression trajectory correlation data.

[0063] Clustering calculations were performed based on person tracking data to obtain clustering feature data for person security check scenarios;

[0064] Based on the clustering feature data of security check scenarios, spatial behavior features of people are extracted to obtain preliminary security check spatial feature data;

[0065] The spatial feature data of the preliminary security check is processed to determine the spatial relationship between the security check equipment and the personnel security check data.

[0066] This invention enables dynamic monitoring and analysis of personnel behavior within security checkpoints through precise person detection and tracking. This is particularly important in high-traffic security scenarios, allowing for real-time tracking of personnel location, movement trajectories, and behavioral changes, and timely identification of potential abnormal behaviors and threats. Facial detection and expression analysis technologies allow the system to extract facial expression information, which is particularly significant in security scenarios. Changes in facial expressions, such as tension and anxiety, can serve as potential indicators of abnormal behavior, helping security personnel identify individuals with unusual psychological states. By correlating facial expression trajectories and extracting spatial behavioral features, the system can not only assess emotional changes but also track behavioral paths, revealing movement patterns in space. Combining facial expressions and spatial behavioral features helps the system determine whether a person's behavior poses a potential threat. Clustering calculations categorize and group individuals in security checkpoints, effectively distinguishing between normal and abnormal behavior patterns. For example, classifying individuals into "normal behavior" and "abnormal behavior" groups based on their behavior type further improves the efficiency of security personnel's judgment. By analyzing the spatial relationships of security screening equipment, we can reveal the relative position and interaction patterns between individuals and the equipment during the screening process. For example, if an individual lingers in front of the scanning device for too long or gets too close to it, it indicates a risk of concealed items.

[0067] Preferably, step S3 specifically includes:

[0068] Step S31: Detect behavioral mutations based on security inspection characteristic sequence data to obtain behavioral mutation point data;

[0069] Step S32: Correlate mutation behaviors based on the behavioral mutation point data to obtain security inspection mutation feature correlation data;

[0070] Step S33: Perform cross-modal feature matching based on the security inspection characteristic sequence data to obtain cross-modal feature matching data;

[0071] Step S34: Generate a multimodal data association graph based on the cross-modal feature matching data to obtain the security inspection cross-modal feature association data.

[0072] In this invention, single-modal analysis is often limited by the incompleteness of information. However, through cross-modal feature matching, the system can integrate information from multiple data sources, improving the accuracy of identification. For example, image recognition technology cannot fully identify a person's intentions, but by combining behavioral pattern data, audio data, etc., a comprehensive analysis of a person's behavior can be performed. By combining cross-modal feature matching data, security inspection cross-modal feature correlation data can be generated, providing an intuitive multi-dimensional data correlation diagram. This helps security personnel identify the relationships between various data sources and reveal the inherent connections between different modal data. Based on changes in multimodal data, the system can flexibly adjust threat assessment criteria and dynamically identify and predict security risks. For example, if a sudden change in the behavior of multiple people occurs in a certain area, the system can automatically adjust the focus of analysis and conduct focused monitoring of that area.

[0073] Preferably, step S4 specifically includes:

[0074] Step S41: Identify mutation behavior threats based on the security inspection mutation feature association data to obtain mutation behavior threat data;

[0075] Step S42: Quantify the mutation threat data of the mutation behavior to obtain the first security inspection hazard data;

[0076] Step S43: Perform cross-modal threat labeling based on the cross-modal feature association data of security inspection to obtain cross-modal threat labeling data;

[0077] Step S44: Perform cross-modal threat analysis on the cross-modal threat annotation data to obtain the second security inspection hazard data.

[0078] This invention analyzes the correlation data of security check mutation characteristics to accurately identify abnormal behaviors or potential threats during security checks. These behavioral changes are often associated with dangerous events, such as sudden abnormal behavior, sudden violence, or unusual behavior by an individual. Quantified threat data allows security personnel to assess risk levels based on specific numerical values ​​and model results, rather than relying on intuition or experience, thus improving the scientific rigor and accuracy of decision-making. By labeling and analyzing cross-modal feature correlation data of security checks, the system can fully utilize data from different sensors (such as video surveillance, behavioral trajectories, and voice recognition) to comprehensively assess threats. Real-time threat assessment helps security personnel determine the severity of potential threats in the shortest possible time and take appropriate action. For example, if a person's behavior is abnormal and their facial expression shows anxiety, the system can combine behavioral, facial data, and other features for real-time threat analysis, helping security personnel react more quickly.

[0079] Preferably, this application also provides an intelligent airport security data analysis system for performing the intelligent airport security data analysis method described above. The intelligent airport security data analysis system includes:

[0080] Airport multi-source security inspection data acquisition module, used to acquire airport multi-source security inspection data;

[0081] The preliminary security check data analysis module is used to perform preliminary security check data analysis based on multi-source airport security check data to obtain security check characteristic sequence data.

[0082] The security inspection feature association module is used to perform behavioral mutation feature association based on security inspection feature sequence data to obtain security inspection mutation feature association data, and to perform cross-modal feature association based on security inspection feature sequence data to obtain security inspection cross-modal feature association data.

[0083] The security inspection feature hazard assessment module is used to assess the threat level of mutation behavior based on the correlation data of security inspection mutation features to obtain the first security inspection hazard data, and to assess the cross-modal threat level based on the correlation data of security inspection cross-modal features to obtain the second security inspection hazard data;

[0084] The security inspection decision support module is used to assist in security inspection decision-making based on the first security inspection hazard data and the second security inspection hazard data.

[0085] The beneficial effects of this invention are as follows: The system can comprehensively analyze data from different devices and combine the advantages of each data source to fully capture potential threats during security checks. For example, baggage scanners can provide image data, behavior monitoring cameras can provide motion trajectory data, and identity verification systems can provide personal identification information. By combining these multi-source data, threats can be identified more accurately, reducing false positives and false negatives. Utilizing mutation feature correlation, the system can quickly identify abnormal behavior or sudden dangerous events. For example, if a passenger's behavior suddenly changes drastically, the system can promptly label and handle it, preventing the potential threat from spreading. Threats during security checks may manifest in various forms, such as behavior, images, sounds, and facial expressions. Through refined correlation of cross-modal features, the system can more comprehensively and meticulously evaluate multi-dimensional information, thereby improving the accuracy and comprehensiveness of threat identification. By quantifying threats, security personnel can make decisions based on scientific data, no longer relying on personal judgment, greatly improving the accuracy and efficiency of decision-making. The system can propose optimal countermeasures based on real-time data, such as personnel allocation and equipment deployment, thereby improving security check efficiency. By analyzing cross-modal data in real time, the system can immediately issue alerts and provide threat assessments when potential threats occur, providing security personnel with timely decision-making support. Attached Figure Description

[0086] Other features, objects, and advantages of this application will become more apparent from the following detailed description of the non-limiting embodiments, taken with reference to the accompanying drawings:

[0087] Figure 1 A flowchart illustrating the steps of an intelligent analysis method for airport security check data according to one embodiment is shown.

[0088] Figure 2 A flowchart illustrating the steps of an airport multi-source security check data acquisition method according to an embodiment is shown.

[0089] Figure 3 A flowchart illustrating the steps of a preliminary analysis method for security inspection data according to one embodiment is shown.

[0090] Figure 4 A flowchart illustrating the steps of a security inspection feature association method according to an embodiment is shown.

[0091] Figure 5 A flowchart illustrating the steps of a security inspection feature hazard assessment method according to one embodiment is shown. Detailed Implementation

[0092] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0093] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. Functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0094] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0095] Please see Figures 1 to 5 This application provides an intelligent analysis method for airport security inspection data, including the following steps:

[0096] Step S1: Obtain multi-source security inspection data from the airport;

[0097] Specifically, the process of acquiring multi-source security inspection data at airports includes collecting data from multiple security inspection subsystems, such as image data output by security scanners (e.g., X-ray machines, CT scanners), passenger information data, item identification information, video data collected by surveillance cameras, and voice recordings of manual inspections.

[0098] Step S2: Conduct preliminary analysis of security inspection data based on multi-source security inspection data at the airport to obtain security inspection characteristic sequence data;

[0099] Specifically, after data collection, preliminary processing of multi-source data is required to extract security inspection characteristic sequence data. Data is then categorized and archived according to its source (passengers, baggage, security equipment). Image data is processed to extract features such as object shape and density; passenger information data is processed to extract identity information (e.g., age, gender) and movement patterns (e.g., time spent passing through security, pace). Since data generated by different security equipment has varying formats, normalization processing is necessary to unify dimensions. For example, image features are converted into numerical matrices, and speech features are converted into text feature codes.

[0100] Step S3: Perform behavioral mutation feature association based on security inspection characteristic sequence data to obtain security inspection mutation feature association data, and perform cross-modal feature association based on security inspection characteristic sequence data to obtain security inspection cross-modal feature association data;

[0101] Specifically, monitoring behavioral trajectories involves analyzing changes in the path of passengers or baggage during the security check process, such as abnormally increased dwell time or changes in the number of items carried. The analysis of characteristic changes over each time period identifies significant abrupt changes (such as a change in the shape of an item from regular to irregular). These behavioral abrupt changes are then matched with predefined threat patterns (such as a database of known dangerous goods characteristics) to output correlation results.

[0102] Integrate multi-modal data (such as images, identity information, and voice) for correlation matching. For example, compare the shape of an object scanned by X-ray with information about the items carried by a passenger mentioned in their voice recording. Annotate the correlation strength and outliers to form cross-modal feature correlation data.

[0103] Step S4: Based on the security inspection mutation feature correlation data, assess the threat level of mutation behavior to obtain the first security inspection hazard data, and based on the security inspection cross-modal feature correlation data, assess the cross-modal threat level to obtain the second security inspection hazard data;

[0104] Specifically, the association data of behavioral mutation characteristics are analyzed to quantify the threat level of the mutated behavior. For example, if a passenger's stay is significantly longer than other passengers and their belongings have high-risk characteristics, they are assessed as a high threat. A grading rule is used to output the threat level result (e.g., low threat, medium threat, high threat).

[0105] Cross-modal feature correlation data is compared with historical hazard cases. For example, inconsistencies between the shapes of irregular objects in image data and passenger identities are used to determine the threat level. The assessment results are weighted and output as threat level data.

[0106] Step S5: Conduct security inspection decision support operations based on the first security inspection hazard data and the second security inspection hazard data.

[0107] Specifically, based on threat level data, the system assists security personnel in taking further action. It merges the risk data from the first and second security checks and generates a final threat level list based on comprehensive scoring rules. The system automatically generates suggestions for optimizing the security check process. For example, it prioritizes secondary manual checks for high-threat passengers and conducts open-bag checks on luggage carrying unusual items. The system automatically issues high-threat alerts to security personnel, including the source of the threat, suggested actions, and supporting evidence (such as video screenshots and behavioral records).

[0108] Preferably, step S1 specifically includes:

[0109] Step S11: Collect real-time airport security data by accessing the API interface of security inspection equipment (such as baggage scanners, behavior monitoring cameras, and identity verification systems);

[0110] Specifically, the main security screening equipment within the airport (such as baggage scanners, surveillance cameras, and identity verification systems) is connected to the data acquisition server via a local area network. Each device needs to be configured with a unique device ID to distinguish the data source. The API interfaces of different devices are configured to ensure real-time data acquisition. The baggage scanner API can output baggage X-ray image data and item attribute information; the surveillance camera API provides real-time video streams; and the identity verification system API outputs passenger identity information (such as ID card numbers and boarding pass information). A data acquisition program is written to periodically request real-time data from the API interfaces, for example, acquiring image data and video frames per second, and synchronizing passenger information in real time after each identity verification. All acquired data is timestamped to ensure synchronization for subsequent analysis.

[0111] Step S12: Perform multimodal data edge processing on the airport's real-time security inspection data to obtain preprocessed security inspection data;

[0112] Specifically, an edge computing module is deployed on the acquisition server to perform localized processing of multimodal data using a distributed computing framework, reducing transmission pressure. Image denoising algorithms are used to clean up artifacts and noise in scanned images and adjust image contrast; keyframes are extracted from the video stream, and intra-frame downsampling technology is used to reduce redundancy. Passenger information output by the identity verification system is normalized (e.g., removing spaces and duplicate fields). Timestamps are used to align data from different modalities, for example, ensuring that the time of a passenger's baggage image matches that of their identity information.

[0113] Step S13: Assign data tags based on the preprocessed security inspection data to obtain tagged security inspection data;

[0114] Specifically, labeling rules are defined based on different data types. For example, image data is automatically tagged with item categories based on item type (electronic devices, liquids, metals); video data is tagged with passenger behavior labels (fast passage, lingering, going against traffic) based on the monitored area; and identity verification data is automatically tagged with passenger identity labels (age group, gender, nationality). Based on image preprocessing, an item recognition algorithm is used to partition the image content, outputting item category information for each area and generating corresponding labels. Behavior recognition algorithms are used on keyframes of the video to detect abnormal passenger behavior and label the behavior. Based on passenger information fields, corresponding labels are generated by matching the defined labeling rules.

[0115] Step S14: Merge the tagged security check data into the cloud to generate multi-source security check data for the airport.

[0116] Specifically, encrypted transmission protocols (such as HTTPS or TLS) are used to synchronize tagged security check data from edge nodes to a cloud server. A data integration module is deployed on the cloud server to merge multimodal data of the same passenger into a unified data entry using a unique passenger identifier (such as a passport number or boarding pass number). An index is generated for all merged data and stored in a distributed database. Data is then categorized and stored by region, time, or passenger.

[0117] Preferably, step S2 specifically includes:

[0118] Step S21: Perform multimodal data cleaning based on multi-source security inspection data from the airport to obtain cleaned security inspection data;

[0119] Specifically, redundant, inconsistent, and abnormal data in airport multi-source security screening data should be removed to ensure the accuracy of the analysis. The data should be checked for null or missing values. For example, if the age field is missing in passenger identity data, it should be filled using the average age of passengers on the same flight; if some frames in the image data are missing, interpolation should be used to generate the missing frames. Reasonable ranges should be set for each data type to remove outliers. For example, negative values ​​or unreasonably large values ​​(exceeding the normal passage time range) in the passenger security screening time records should be marked as outliers and removed. Numerical data (such as passage time and item weight) should be normalized; textual data (such as passenger identity fields) should be standardized, such as unifying date formats and name case. Duplicate passenger information records should be merged; and filtering techniques should be used to reduce noise in frames with excessive noise in the image data.

[0120] Step S22: Extract security inspection hazard features and spatial features from the security inspection cleaning data to obtain security inspection hazard feature data and security inspection spatial feature data, respectively;

[0121] Specifically, the process involves identifying potential risk factors in the data, such as high-risk items, unusual behavior, and identity risks. This includes analyzing scanned images of cleaned baggage, segmenting the image into areas containing items, and extracting dangerous items (such as liquids and sharp objects) based on features like shape and density. Risk features are extracted based on identity information, such as comparing flight destinations with lists of high-risk areas and detecting the presence of risky passengers flagged by the security system (e.g., those on wanted lists). Recordings from behavioral surveillance cameras are analyzed to extract unusual behavioral features, such as repeated baggage checks and unusually long dwell times. All extracted risk features are labeled as structured data, including categories of dangerous items and types of dangerous behavior, and stored as security risk feature data.

[0122] Capture the spatial behavior characteristics of passengers and luggage within the security checkpoint area. Extract passenger spatial trajectories from surveillance camera data and record changes in their coordinate positions over time as a time series. Analyze the distribution of passenger dwell time in different security checkpoint areas, identifying abnormal clustering or reverse-movement behaviors. Combine X-ray image data to correlate the spatial location of items in luggage with the relative position of passengers during security checks. The extracted results form spatial feature data, including passenger location trajectories and dwell time distribution, which is stored as security checkpoint spatial feature data.

[0123] Step S23: Perform feature fusion encoding on the security inspection hazard feature data and the security inspection spatial feature data to obtain security inspection characteristic sequence data.

[0124] Specifically, security check hazard characteristic data and spatial characteristic data are uniformly fused and encoded to form time-series data. Based on the timestamps of the hazard and spatial characteristic data, a one-to-one correspondence between different modalities is ensured in the time dimension. For example, the time point of abnormal passenger behavior is aligned with the corresponding dangerous item detection results in their luggage. Hazard and spatial characteristics are converted into numerical coding formats. For example, 0 represents no danger, 1 represents low risk, 2 represents medium risk, and 3 represents high risk; the spatial location of passengers is encoded using three-dimensional coordinates. The characteristics of each passenger are sorted by time to form a characteristic sequence. The characteristic sequence includes multiple fields, such as timestamp, hazard characteristic value, and spatial characteristic value. Different weights are assigned to hazard and spatial characteristics, and the data is fused according to these weights to generate the characteristic sequence data.

[0125] Preferably, the extraction of security inspection hazard features specifically includes:

[0126] Visual features are extracted from security inspection image data in the security inspection cleaning data to obtain security inspection visual feature data;

[0127] Specifically, visual information reflecting the characteristics of luggage items, including shape, texture, and density, is extracted from security inspection image data. The security inspection images are converted to grayscale to reduce color interference; histogram equalization enhances image contrast; and edge detection highlights item outlines. Region segmentation techniques are used to separate different items in the image, such as segmenting different items in luggage; each segmented region is numbered. The following visual features are extracted from the segmented item regions: parameters such as boundary length, area, and roundness. Statistical methods are used to extract item surface texture characteristics (such as smoothness and roughness). The density distribution of items (such as the density distribution of liquids or metals) is detected by changes in the grayscale values ​​of the scanned image. The extracted features are numerically converted into vector representations, generating a set of visual feature data for each item region.

[0128] A time-stamped map is constructed based on the security inspection image data in the security inspection and cleaning data to obtain the time-stamped map data;

[0129] Specifically, a time-stamped graph is constructed for security inspection image data to reflect the changes of items over time during the security inspection process. The acquired security inspection images are grouped according to timestamps, generating an image sequence for each time period, such as by second or frame. The time-grouped image data is modeled as nodes, with each node representing an image within a time period. Edge connections are established between nodes according to chronological order to form the time-stamped graph. Attribute labels corresponding to the image are added to each node, such as the number of items contained or the largest item area. Edge weights are assigned to the connected nodes, based on the degree of change in item characteristics between adjacent time periods; for example, higher edge weights indicate more drastic changes in item characteristics, while lower edge weights indicate relatively stable item characteristics. The output time-stamped graph data includes all time nodes, node attributes, and edge weights between nodes.

[0130] Time feature fusion is performed on time-stamped map data and security visual feature data to obtain security visual feature time map data;

[0131] Specifically, by combining security inspection visual feature data and time-stamped graph data, temporal and spatial visual features are fused to construct a visual feature time graph. The security inspection visual feature data is aligned with the time nodes in the time-stamped graph. For example, the visual feature vectors of items extracted within a certain time period are matched with the corresponding time node attributes. Time series modeling is performed on the edge weights in the time-stamped graph to capture the trend of feature changes between time nodes. For example, if the features of an item remain unchanged across multiple consecutive time nodes, the time trend is stable. If the features change significantly across consecutive time nodes, the temporal gradient is recorded. A fused feature vector is generated for each time node, merging visual features and time trend information into a comprehensive set of features. The generated security inspection visual feature time graph data includes time nodes, visual features, and time change trends.

[0132] Hazard features are extracted from the time-map data of visual features in security checks to obtain hazard feature data.

[0133] Specifically, features reflecting danger are extracted from the visual feature time map, such as changes in unusual items or abnormal time trends. Danger indicators are set according to security inspection rules, such as certain shape or density characteristics exceeding safe limits (e.g., sharp objects, liquids). Items undergo significant changes in the time series (e.g., a sudden change in shape, likely indicating concealment). Each node and edge in the time map is traversed, and a danger indicator score is calculated. For example, for a single node, the presence of dangerous items in the visual features is detected. For edges between nodes, changes in time trends are checked to see if they exceed thresholds. High-risk items and abnormal behaviors are labeled, generating danger feature records including item number, danger type, and danger level. The danger feature information from all nodes and edges is summarized to form security inspection danger feature data. The output security inspection danger feature data includes potentially dangerous items and their time behavior characteristics.

[0134] Preferably, the construction of the time-stamped graph specifically involves:

[0135] Based on the security inspection image data in the security inspection and cleaning data, a time tag map of a single item is constructed to obtain the first time tag map data;

[0136] Specifically, for a single item in security inspection image data (such as an item in luggage), a time-series label map is constructed to reflect the item's state changes at different time points. Individual item regions are segmented from each frame of the security inspection image, and each item is uniquely identified by a number (such as an item ID). For example, region segmentation technology is used to identify each item in luggage, extracting its corresponding region. Image data at each time point is treated as a node. For example, the timestamp of the image frame is used as a node label, indicating the item's state at that time. Visual feature attributes (such as shape, texture, and density) of the item are added to each time point, and the extracted visual features are stored as the node's attribute values. Edge weights are calculated based on the changes in visual features between adjacent time points. For example, if the shape or density features of an item change significantly between two time points, a higher weight is assigned; if the change is small, a lower weight is assigned. Time nodes are connected with edges to generate a single-item time-label map. The output first time-label map data is a time-series map for a single item, containing time nodes, node attributes, and edge weights.

[0137] Based on the security inspection image data in the security inspection and cleaning data, a multi-item time-label map is constructed to obtain the second time-label map data;

[0138] Specifically, for multiple items in security inspection image data, a holistic time-series label map is constructed to reflect the relationships between items over time. The relationships between items in each frame of the image are analyzed. For example, the correlation between items is determined by their location, shape similarity, or region overlap. Each frame of image data corresponds to a time node, and each node contains overall information about all items at that time point. The following multi-item attributes are extracted for each time node: the number of items in that frame; the relative positional relationships of all items; and the similarity between items (such as density and shape). Edge weights are calculated based on changes in item distribution between adjacent time nodes. For example, if the total number of items differs significantly between two time nodes, or if the correlation between items changes significantly, a higher edge weight is assigned. If the change is small, a lower edge weight is assigned. Each time node is connected to its adjacent nodes through edges to form a multi-item time-label map. The output second time-label map data is a holistic time-series map of multiple items, containing nodes (each frame of the image) and edge weights.

[0139] The first time-labeled map data and the second time-labeled map data are fused into a layered map to obtain time-labeled map data;

[0140] Specifically, the first time-labeled map (single item) and the second time-labeled map (multiple items) are merged to construct a hierarchical time-labeled map containing features of both single and multiple items. The time nodes of the single-item and multi-item maps are aligned to ensure a one-to-one correspondence in the time dimension. For example, if an item exists in the first time-labeled map at time point t, then time point t must have a corresponding node in the second time-labeled map. The nodes of the single-item map are considered the first layer, the base layer; the nodes of the multi-item map are considered the second layer, the overall layer. In the first layer, single-item nodes at the same time point are merged, and their average attribute values ​​are calculated to form the merged single-item attribute. In the second layer, the original nodes and attributes of the multi-item map are retained. Connections are established between the corresponding time nodes of the first layer (single-item map) and the second layer (multi-item map), and the edge weights between layers are calculated. For example, if the feature changes of a single item in the first layer significantly affect the multi-item distribution in the second layer, a higher edge weight is assigned. If the impact is small, a lower edge weight is assigned. The nodes of the first and second layers and their connections are uniformly encoded to generate the hierarchical time-labeled map. The generated time-stamped map data includes time features of single and multiple items, reflecting the individual changes and overall relationships of items.

[0141] The construction of the single-item time-stamped graph is specifically as follows:

[0142] Single-item target data is obtained by identifying and extracting single-item target data from the security inspection and cleaning data;

[0143] Specifically, individual items are identified and their features extracted from security inspection image data to generate single-item target data. Target detection is performed on the security inspection images to identify and locate individual items within the images. Boundary boxes for each item are extracted using bounding box detection, and each item is labeled with a unique identifier (e.g., ObjectID). Image segmentation techniques are used to separate the item regions from the image, eliminating background and other interference. Attribute analysis is performed on the extracted item regions, including shape, texture, and density. The center point coordinates and bounding box dimensions of the item in the image are also recorded. Data records are generated for each item, including ObjectID, visual feature vector, and positional features, forming a single-item target data table. The output single-item target data includes the item number and its attribute information.

[0144] Extract time-series labels from single-item target data to obtain single-item time-series label data;

[0145] Specifically, time dimension labels are added to individual item target data to form time-series data for each item. A timestamp is added to each security inspection image and synchronized to the individual item target data. For example, a time field is added to each item's record, indicating the item's time in the current frame. Records of the same item at different time points are compared to extract time-series attributes, such as calculating the time difference of the item's center point coordinates to obtain the rate of position change. The time change value of the item's visual feature vector is calculated. Based on the changes in the item's time dimension, time-series labels are generated. For example, if the item's position changes significantly, it is labeled "movement." If the visual features change significantly, it is labeled "morphological change." Single-item time-series label data containing timestamps, time-series attributes, and change labels is generated. The output time-series label data includes the item's time information and its characteristics changing over time.

[0146] Nodes are constructed based on single-item time-series tag data and single-item target data to obtain single-item time-series node data.

[0147] Specifically, the time-series label data of individual items is converted into nodes in a graph structure. Each time point in time represents a node for an individual item. The unique identifier of a node is formed by a combination of the item number (ObjectID) and a timestamp. The following attributes are added to each node: visual features of the item are obtained from the individual item target data; temporal features of the item are obtained from the individual item time-series label data, including the rate of change of position and the magnitude of change of shape. The node records of each item at different time points are organized into a structured data table, forming the single-item time-series node data. The output single-item time-series node data includes node identifiers and their attributes, for use in graph relationship analysis.

[0148] Neighbor node relationship analysis is performed on the time-series node data of a single item to obtain neighbor node relationship data of the single item;

[0149] Specifically, the relationships between adjacent nodes of a single item are analyzed over time, generating connection weights between nodes. For nodes of the same item, they are sorted by timestamp to determine the preceding and following neighbors of each node. For example, the neighbors of node t are t-1 and t+1. The relationship attributes between adjacent nodes are calculated, including: calculating the Euclidean distance of the visual feature vectors of adjacent nodes as a feature change; calculating the difference in the coordinates of the center points of adjacent nodes as a positional change; and calculating a weighted comprehensive change value based on the visual and positional feature changes. Weights are assigned to the edges between adjacent nodes based on the magnitude of the comprehensive change: a larger edge weight indicates a more drastic change in the node, while a smaller edge weight indicates a more stable change. A neighboring node relationship data table containing node pairs (start and end points) and edge weights is generated. The output single-item neighboring node relationship data includes node relationships and their weights.

[0150] Graphs are constructed based on the neighbor node relationship data and time sequence node data of single items to obtain the first time label graph data.

[0151] Specifically, a complete time-labeled graph is constructed based on single-item time-series node data and neighboring node relationship data. An empty graph data structure is created, storing nodes and edges separately. Each node from the single-item time-series node data is added to the graph, preserving its attributes. Node pairs and their weights are added as edges to the graph based on the single-item neighboring node relationship data. The constructed graph is then structurally optimized, for example, by removing isolated nodes or edges with excessively low weights, retaining only those parts with analytical value. The first time-labeled graph data constructed is a time-series single-item time graph, containing nodes (time points) and their connections (time proximity relationships).

[0152] The construction of the multi-item time-stamped graph is specifically as follows:

[0153] Multi-item node data is obtained by constructing multi-item nodes based on different single-item time-series node data corresponding to the same single-item time-series tag data.

[0154] Specifically, based on single-item time-series node data, information nodes containing multiple items are constructed, forming multi-item node data. All item nodes at the same time point (same time-series label) are extracted from the single-item time-series node data and integrated into a single multi-item node. For example, the multi-item node at time point t contains all items identified at time t. The attributes of the multi-item nodes are aggregated, including: arranging the visual feature vectors of all items by number to form a high-dimensional visual feature vector set; recording the relative positions and spatial distribution of all items; calculating the distances or similarities between items to form a preliminary association matrix; and assigning a unique identifier to each multi-item node, typically composed of a time label and an item set. The output multi-item node data includes the node identifier, the included item set, and its aggregated attributes.

[0155] Based on the multi-item node data, interaction relationship processing, functional relationship processing, and structural relationship processing are performed to obtain multi-item interaction relationship data, multi-item functional relationship data, and multi-item structural coupling relationship data.

[0156] Specifically, the interaction behavior between items in a multi-item node is analyzed to generate interaction relationships. Interaction analysis is performed on items in a multi-item node based on their temporal attributes and spatial location. For example, items that are close to each other have interaction relationships. Items that undergo significant visual changes simultaneously may be performing the same operation or causing interference. Interaction weights are calculated for each pair of items. For example, the closer the items are, the higher the interaction weight. The more synchronized the visual changes, the higher the interaction weight. Multi-item interaction relationship data is output, including item pairs, interaction types, and interaction weights.

[0157] This program analyzes the functional attributes of items and their relationships to generate functional relationships. It infers item functions based on visual features (such as shape and material). For example, liquids and containers have combined functions, or a long rod and a gunstock-like object have a combined relationship. It detects associations between items with similar or complementary functional attributes. For example, items with similar functions may belong to the same category. Items with complementary functions may be used for specific purposes (such as liquids and fuel containers). The program outputs multi-item functional relationship data, including item pairs, functional categories, and functional association weights.

[0158] Analyze the coupling characteristics of items in spatial structure to generate structural relationships. Analyze the positional relationships of items in multiple item nodes. For example, closely arranged items have structural coupling. Items with stable spacing changes belong to the same physical structure. Calculate structural coupling weights based on indicators such as positional overlap and relative positional stability. For example, the more spatially overlapping items, the higher the coupling weight. The more stable the relative positions, the higher the coupling weight. Output multi-item structural coupling relationship data, including item pairs and coupling strength.

[0159] Graphs are constructed on the multi-item node data based on the multi-item interaction relationship data, multi-item functional relationship data, and multi-item structural coupling relationship data, respectively, to obtain multi-item interaction relationship graph data, multi-item functional relationship graph data, and multi-item structural coupling relationship graph data;

[0160] Specifically, three types of graphs are constructed based on multi-item relationship data. An empty graph structure is created for each type of relationship (interaction relationship graph, functional relationship graph, and structural coupling relationship graph). Nodes from the multi-item node data are added to the graphs, preserving their attributes. Edges are added to each type of graph based on the interaction relationship data, functional relationship data, and structural coupling relationship data, preserving edge weights. The three types of graphs are then optimized, for example, by removing edges with lower weights and merging nodes with similar heights. The three relationship graphs are output, respectively describing the interaction, functional, and structural relationships between multiple items.

[0161] Static multi-level graph fusion is performed based on multi-item interaction relationship graph data, multi-item functional relationship graph data, and multi-item structural coupling relationship graph data to obtain second time-labeled graph data.

[0162] Specifically, the multi-item interaction graph, functional relationship graph, and structural coupling graph are merged into a single multi-level graph. The three graphs are treated as different levels to form the initial multi-level graph structure. Nodes in the three graphs are aligned, for example, by merging the same node based on its identifier (time and item set). Edge weights for identical node pairs in the three graphs are merged. For example, the weights of interaction, function, and structural relationships can be weighted and summed according to their respective weight proportions. If an edge is missing in a relationship graph, a default weight value is assigned. The merged multi-level graph is output, where the merged node and edge attributes include the interactions, functions, and structural relationships of multiple items. The generated second time-labeled graph data is a time-based graph that merges multiple relationships and levels, used in subsequent layered graph fusion steps.

[0163] Preferably, the layered graph fusion specifically involves:

[0164] Based on the first time-label map data and the second time-label map data, a time change layer is constructed to obtain the time change layer data;

[0165] Specifically, the temporal evolution characteristics of nodes in the first and second time-labeled graphs are analyzed to construct a temporal variation layer. Node sets at the same time points are extracted from the first and second time-labeled graphs to ensure consistency in the temporal dimension. Changes in node features between consecutive time points are analyzed to extract time-series features. For example, for single-item graph nodes, the temporal rate of change of visual or positional features is calculated. For multi-item graph nodes, the temporal gradient of overall distribution or behavioral features is calculated. Edges between each pair of consecutive time points are weighted according to the degree of feature change (e.g., the greater the change, the higher the weight). Time nodes and their temporal variation edges are combined to form the temporal variation layer. The output temporal variation layer data includes time nodes and their time-series features.

[0166] Spatial relationship layer data is constructed based on the first time-label map data and the second time-label map data.

[0167] Specifically, the spatial distribution relationships of nodes in the first and second time-labeled graphs are analyzed to construct a spatial relationship layer. A set of nodes in the same spatial region is extracted from both time-labeled graphs, and their location coordinates are recorded. The spatial distance between nodes is calculated, generating a spatial adjacency matrix. For example, for single-item graph nodes, the relative spatial positions of items are calculated. For multi-item graph nodes, the similarity of the group's spatial distribution is calculated. Edge weights are generated based on the spatial adjacency matrix: higher edge weights indicate closer spatial relationships (e.g., nodes that are closer together). Spatial nodes and their spatial relationship edges are combined to form the spatial relationship layer. The output spatial relationship layer data includes spatial nodes and their spatial relationship weights.

[0168] The interaction layer is constructed based on the first time tag map data and the second time tag map data to obtain the interaction layer data;

[0169] Specifically, the interaction features between different nodes in the first and second time-labeled graphs are analyzed to construct an interaction layer. Interaction features of each node are extracted, such as: behavioral interactions of single-item nodes (e.g., visual feature synchronization); and group behaviors of multiple-item nodes (e.g., group movement synchronization). Edge weights are assigned based on the interaction strength between nodes. For example, if the interaction strength between nodes is high (e.g., high temporal and spatial overlap), the weight is higher; if the interaction is weak, the weight is lower. Nodes and their interaction relationship edges are combined to form the interaction layer. The output interaction layer data includes the interacting nodes and their interaction relationship weights.

[0170] Graph attention networks were used to extract feature data from the time-varying layer data, spatial relationship layer data, and interaction layer data, respectively.

[0171] Specifically, a graph attention network (GAT) is used to extract features from the data at each layer, generating feature data for the temporal variation layer, spatial relationship layer, and interaction layer. The node features of these layers are then input into the GAT. Attention weights between each node and its neighbors are calculated, dynamically allocating weights based on edge weights and node feature importance. The feature information of neighboring nodes is aggregated through the attention mechanism, updating the feature vector of each node. This generates feature data for each layer, including temporal variation features, spatial relationship features, and interaction features. The output layer feature data includes the node features of each layer and their attention weights.

[0172] Based on the time-varying layer feature data, spatial relationship layer feature data, and interaction layer feature data, inter-layer connections are made between the time-varying layer data, spatial relationship layer data, and interaction layer data to obtain layered graph connection data.

[0173] Specifically, inter-layer connections are established between the time-varying layer, spatial relationship layer, and interaction layer to form a hierarchical graph connection data. Data from the three layers is aligned based on the same node identifier. Cross-layer edges are established for the same node across different layers. For example, a node in the time-varying layer is connected to a corresponding node in the spatial relationship layer, and vice versa. The weights of the cross-layer edges are calculated, incorporating the feature similarity across layers. For example, if a node has high feature similarity across multiple layers, the edge weight is higher. All nodes and edges between layers are recorded as hierarchical graph connection data. The output hierarchical graph connection data includes cross-layer edges and their weights.

[0174] Graph fusion is performed based on the data connected by the hierarchical graph to obtain time-stamped graph data.

[0175] Specifically, the layered graph connectivity data is fused with the feature data of each layer to generate time-stamped graph data. Data from the time variation layer, spatial relationship layer, and interaction layer, along with the layered graph connectivity data, are integrated into a single global graph. Global features of each node are calculated by aggregating node and edge features. For example, a weighted average or weighted summation is performed on the temporal, spatial, and interaction features of the same node. The fused graph structure is optimized, for example, by removing low-weight edges and simplifying redundant nodes. The final time-stamped graph data is generated, recording global node features, edge weights, and hierarchical information. The output time-stamped graph data is a complete graph after layered feature fusion to support threat analysis and decision support.

[0176] Preferably, the spatial feature extraction specifically includes:

[0177] Personnel detection data is obtained by performing personnel detection based on multi-source security screening data from the airport.

[0178] Specifically, all human targets are identified from multi-source airport security data (such as surveillance videos and image sequences). Image frames are extracted from the surveillance video stream at fixed time intervals to ensure real-time performance and computational efficiency. Object detection technology is used to identify people in the image frames, generating a bounding box and unique identifier (ID) for each person. Background modeling and motion detection techniques are used to eliminate background interference (such as luggage and equipment) to ensure accurate detection results. The detection results from each image frame are organized into structured data, including the bounding box, confidence score, and ID for each person. The output human detection data contains all detected people and their location information in each video frame.

[0179] Perform character tracking on the character detection data to obtain character tracking data;

[0180] Specifically, detected individuals are tracked across frames, generating a time-series trajectory for each individual. Individuals are matched in adjacent frames, identifying the same individual through Intersection over Union (IoU) calculation or feature vector matching (such as color or shape). A unique trajectory ID is assigned to each individual, and its positional changes across all time frames are recorded. For occluded or temporarily missing targets, trajectory prediction methods are used to complete the trajectory, ensuring continuity. Trajectory data for each individual is generated, including a timestamp, position sequence, and trajectory ID. The output individual tracking data contains each individual's trajectory ID and its positional information within the time series.

[0181] Facial detection is performed based on person tracking data to obtain facial detection data;

[0182] Specifically, facial regions are detected from tracked individuals. Bounding boxes from the individual tracking data are used to define the facial detection range in each frame, reducing computational complexity. Facial feature points (such as eyes, nose, and mouth) are detected within local regions, generating facial region bounding boxes. Tilted or off-center facial regions are corrected to ensure complete detection. Facial region information for each individual at each time point is output, including bounding box coordinates and detection confidence. The output facial detection data contains the facial regions and related attributes of each individual in each frame.

[0183] Facial expression data is obtained by performing facial label analysis based on facial detection data.

[0184] Specifically, facial expression analysis is performed on detected faces to generate facial expression data. Key facial features (such as the curvature of the corners of the mouth and the position of the eyebrows) are extracted from the facial detection data. Based on the feature extraction results, facial expressions are classified into standard categories (such as happy, nervous, angry, etc.). A confidence score is calculated for each expression, and results with low confidence scores are discarded. The expression category and confidence score for each person in each frame are recorded. The output facial expression data includes the expression classification results and time series for each person.

[0185] By combining facial tracking data and facial tag data to analyze the spatial relationships of facial expression changes, we can obtain facial expression trajectory correlation data.

[0186] Specifically, facial expression changes are correlated with spatial trajectories to generate facial expression trajectory correlation data. The person tracking data and facial expression data are aligned according to timestamps to ensure consistency between expression information and the trajectory. The trends in facial expression data over time are analyzed. For example, the frequency of change from "happy" to "nervous" is calculated. Facial expression change labels are added to the trajectory, such as marking "nervous areas" within the trajectory. Correlation data containing facial expression changes and trajectory locations is generated, indicating the time and spatial location of the facial expression changes. The output facial expression trajectory correlation data contains the temporal and spatial correlation information between facial expression changes and the trajectory.

[0187] Clustering calculations were performed based on person tracking data to obtain clustering feature data for person security check scenarios;

[0188] Specifically, the spatial behavior of individuals is clustered to analyze the distribution and behavioral patterns of people in security check scenarios. Behavioral features are extracted from person tracking data, including dwell time, movement distance, and movement path complexity. The extracted features are normalized to ensure that different features have the same dimensions. Based on these behavioral features, individuals are clustered to generate behavioral categories. For example, categories include "fast passersby," "lingering individuals," and "those going against the flow." A feature description and a list of individuals for each cluster are generated. The output clustered feature data contains the behavioral categories and feature distribution of individuals in the security check scenario.

[0189] Based on the clustering feature data of security check scenarios, spatial behavior features of people are extracted to obtain preliminary security check spatial feature data;

[0190] Specifically, the overall spatial behavior characteristics are extracted from clustering features. The clustering results are analyzed to identify population distribution, density changes, and hotspot areas. Combined with the time dimension, spatial behavior changes over different time periods are extracted. For example, the distribution of people lingering during peak hours. Spatial behavior feature data is generated, including behavior types, hotspot areas, and temporal variations. The output preliminary security check spatial feature data contains group behavior characteristics and their spatiotemporal changes.

[0191] The spatial feature data of the preliminary security check is processed to determine the spatial relationship between the security check equipment and the personnel security check data.

[0192] Specifically, the spatial relationship between people and security screening equipment is analyzed to generate security screening spatial feature data. The fixed locations of security screening equipment and their areas of influence (such as baggage scanners, security gates, and camera shooting areas) are marked in the monitoring scene. The interaction between people and equipment is analyzed by combining their trajectories. For example, the time and frequency of each person passing through the equipment are calculated. Spatial interaction features between people and equipment are extracted, such as "passage frequency," "dwell time," and "location of abnormal behavior." The relationship features between people and equipment are summarized to generate security screening spatial feature data. The output security screening spatial feature data includes the spatial relationship between person behavior and equipment interaction, which is used for further analysis of threat levels and scene optimization.

[0193] Preferably, step S3 specifically includes:

[0194] Step S31: Detect behavioral mutations based on security inspection characteristic sequence data to obtain behavioral mutation point data;

[0195] Specifically, abrupt changes in behavior are detected from security check characteristic sequence data, marking potential locations of abnormal behavior. Key behavioral features (such as passenger dwell time, changes in baggage scanning density, and abrupt changes in behavioral trajectory) are extracted from the security check characteristic sequence data. Time series analysis is performed on the behavioral features to calculate the rate of change of features within each time period. For example, differential or sliding window techniques are used to capture abrupt changes. Abrupt change thresholds are set for behavioral changes; for example, an abnormally increased rate of location change indicates a sudden stop or rapid movement. An abnormal change in baggage density indicates that items have been removed or added. Detected abrupt change points are marked, generating behavioral abrupt change point data, recording the abrupt change time, feature type, and abrupt change intensity. The output behavioral abrupt change point data includes the time point, abrupt change type, and abrupt change intensity.

[0196] Step S32: Correlate mutation behaviors based on the behavioral mutation point data to obtain security inspection mutation feature correlation data;

[0197] Specifically, the correlation between abrupt behavioral changes is analyzed to form security check abrupt change feature correlation data. Based on behavioral abrupt change point data, the temporal and spatial correlations between multiple behavioral changes are analyzed. For example, a passenger's gait change may be related to a change in baggage scanning results. Facial expressions of tension may occur simultaneously with baggage anomaly detection. Correlation weights are calculated for each pair of abrupt behavioral changes, including temporal overlap, spatial distance, and feature similarity. Higher temporal overlap results in higher correlation weights. Closer spatial distances result in higher correlation weights. Based on the correlation weights, abrupt behaviors are classified as strong, medium, or weak correlations. For example, highly correlated abrupt behaviors represent a comprehensive threat event. The correlation pairs of abrupt behaviors and their weights are recorded to form abrupt change feature correlation data. The output security check abrupt change feature correlation data includes abrupt behavior pairs, correlation weights, and correlation types.

[0198] Step S33: Perform cross-modal feature matching based on the security inspection characteristic sequence data to obtain cross-modal feature matching data;

[0199] Specifically, the process involves matching different modalities of security check feature sequences to uncover potential correlations. Feature standardization is performed on the data (e.g., image data, behavioral trajectory data, facial expression data), for example, converting image features into numerical vectors. Behavioral trajectories are discretized into temporal location information. Matching rules are defined between different modal features, such as spatial overlap between image features and behavioral trajectories, and temporal synchronization between facial expression changes and behavioral abrupt changes. The matching degree of modal features is calculated based on the matching rules, for example, using cosine similarity to calculate the matching degree of visual features. Temporal deviation is used to measure the synchronicity between facial expressions and behaviors. The matching results for each pair of modal features are recorded, including the matching degree and matching type. The output cross-modal feature matching data includes modal feature pairs, matching degrees, and matching descriptions.

[0200] Step S34: Generate a multimodal data association graph based on the cross-modal feature matching data to obtain the security inspection cross-modal feature association data.

[0201] Specifically, a multimodal data association graph is generated based on cross-modal feature matching data to reflect the relationships between different modalities. The features of each modality are represented as graph nodes. For example: density feature nodes for luggage scanning results; facial expression change feature nodes; behavioral trajectory change feature nodes. Feature attributes are added to each node, such as feature type, time label, and intensity value. Edges are generated for each pair of matched nodes based on the cross-modal feature matching data, and weights are assigned: the higher the matching degree, the greater the edge weight; the higher the temporal and spatial overlap, the greater the edge weight. Weakly correlated edges are removed, and high-weight edges and important nodes are retained. A cross-modal feature association graph is generated, recording nodes, edges, and their attributes. The output security inspection cross-modal feature association data is a multimodal association graph, which can be used for further threat assessment or decision support.

[0202] Preferably, step S4 specifically includes:

[0203] Step S41: Identify mutation behavior threats based on the security inspection mutation feature association data to obtain mutation behavior threat data;

[0204] Specifically, based on the correlation data of security check mutation characteristics, mutated behaviors that may represent security threats are identified. The categories and attributes of mutated behaviors are extracted from the mutation characteristic correlation data. For example, behavioral mutations may include "abnormal lingering," "changes in carried items," and "going against the flow." Based on a known threat behavior rule base, the mutated behaviors are matched to see if they conform to known threat patterns. For example, an unusually long lingering time and proximity to a restricted area indicates a potential threat. The unusual presence of a high density of items in luggage indicates the carrying of dangerous goods. A preliminary threat score is given to the mutated behaviors, based on the mutation intensity, the number of associated behaviors, and historical data of specific threat scenarios. Identified mutated behaviors are marked as potential threats, and the threat type, location, time, and score are recorded. The output mutated behavior threat data includes the threat type and threat score for each mutated behavior.

[0205] Step S42: Quantify the mutation threat data of the mutation behavior to obtain the first security inspection hazard data;

[0206] Specifically, the threat data of mutated behaviors is quantified to generate overall security check hazard data. Hazard levels are assigned based on threat scores, for example: Low risk (score < 3): No significant threat; Medium risk (score 3-6): Requires further manual review; High risk (score > 6): Represents a direct threat. Multiple threatening behaviors of the same object are comprehensively quantified. For example, if both passenger A's behavioral mutation and baggage mutation are marked as high risk, the overall score is increased. All hazard scores for the same scenario (such as a security check area) are aggregated to generate a region-level hazard score. The quantification results are recorded in a structured form, indicating the hazard object, hazard level, and corresponding behavioral description. The output of the first security check hazard data includes the threat object, hazard level, and scenario risk score.

[0207] Step S43: Perform cross-modal threat labeling based on the cross-modal feature association data of security inspection to obtain cross-modal threat labeling data;

[0208] Specifically, based on cross-modal feature association data from security checks, potential threat features are labeled. The consistency between different modalities in the cross-modal feature association graph is analyzed. For example, baggage images show an unusually high density of items, but the passenger's declaration does not mention carrying similar items. Behavioral trajectories show reverse walking, but identity information indicates the passenger is a special individual. Anomalies in the cross-modal data are extracted and labeled; for example, image feature anomalies are labeled as "high-density items," and behavioral anomalies are labeled as "reverse walking" or "abnormal lingering." High-weighted associations between modalities are combined with threat features to further verify the accuracy of the labeling. Labeled threat feature data is generated, including feature descriptions, modal origins, and association strength. The output cross-modal threat labeling data includes threat feature descriptions and modal association information.

[0209] Step S44: Perform cross-modal threat analysis on the cross-modal threat annotation data to obtain the second security inspection hazard data.

[0210] Specifically, cross-modal threat annotation data is comprehensively analyzed to generate overall cross-modal hazard data. Threat features from cross-modal annotations are fused by time and space. For example, if a passenger's tense facial expression (modality 1) and high-density items in a luggage image (modality 2) appear at the same time, the overall threat score increases. The threat level is upgraded based on the overall strength of the cross-modal features. For example, if multiple modal features show anomalies, the threat level is upgraded from "medium" to "high." High-risk areas are marked based on the spatial distribution of threats. For example, a security checkpoint is marked as a high-risk area. The final cross-modal hazard data is generated, including the threat object, modal features, hazard level, and spatial distribution.

[0211] The output of the second security inspection hazard data includes a comprehensive score and distribution information of cross-modal threats, which is used for decision support.

[0212] Preferably, this application also provides an intelligent airport security data analysis system for performing the intelligent airport security data analysis method described above. The intelligent airport security data analysis system includes:

[0213] Airport multi-source security inspection data acquisition module, used to acquire airport multi-source security inspection data;

[0214] The preliminary security check data analysis module is used to perform preliminary security check data analysis based on multi-source airport security check data to obtain security check characteristic sequence data.

[0215] The security inspection feature association module is used to perform behavioral mutation feature association based on security inspection feature sequence data to obtain security inspection mutation feature association data, and to perform cross-modal feature association based on security inspection feature sequence data to obtain security inspection cross-modal feature association data.

[0216] The security inspection feature hazard assessment module is used to assess the threat level of mutation behavior based on the correlation data of security inspection mutation features to obtain the first security inspection hazard data, and to assess the cross-modal threat level based on the correlation data of security inspection cross-modal features to obtain the second security inspection hazard data;

[0217] The security inspection decision support module is used to assist in security inspection decision-making based on the first security inspection hazard data and the second security inspection hazard data.

[0218] Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended application documents rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of the equivalents of the application documents be incorporated into the invention.

[0219] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for intelligent analysis of airport security inspection data, characterized in that, Includes the following steps: Step S1: Obtain multi-source security inspection data from the airport; Step S2: Perform multimodal data cleaning based on multi-source security inspection data from the airport to obtain cleaned security inspection data; Security inspection hazard features and spatial features are extracted from the security inspection cleaning data to obtain security inspection hazard feature data and security inspection spatial feature data, respectively; the security inspection hazard feature data and security inspection spatial feature data are then fused and encoded to obtain security inspection characteristic sequence data; Step S3: Perform behavioral mutation feature association based on security inspection characteristic sequence data to obtain security inspection mutation feature association data, and perform cross-modal feature association based on security inspection characteristic sequence data to obtain security inspection cross-modal feature association data; Step S4: Based on the security inspection mutation feature correlation data, assess the threat level of mutation behavior to obtain the first security inspection hazard data, and based on the security inspection cross-modal feature correlation data, assess the cross-modal threat level to obtain the second security inspection hazard data; Step S5: Conduct security inspection decision support operations based on the first security inspection hazard data and the second security inspection hazard data; The specific steps for extracting hazardous features during security checks are as follows: Visual features are extracted from security inspection image data in the security inspection cleaning data to obtain security inspection visual feature data; A time-stamped map is constructed based on the security inspection image data in the security inspection and cleaning data to obtain the time-stamped map data; Time feature fusion is performed on time-stamped map data and security visual feature data to obtain security visual feature time map data; Hazard features are extracted from the time-map data of visual features in security checks to obtain security check hazard feature data; The construction of the time-stamped graph is specifically as follows: Based on the security inspection image data in the security inspection and cleaning data, a time tag map of a single item is constructed to obtain the first time tag map data; Based on the security inspection image data in the security inspection and cleaning data, a multi-item time-label map is constructed to obtain the second time-label map data; The first time-labeled map data and the second time-labeled map data are fused into a layered map to obtain time-labeled map data; The construction of the single-item time-stamped graph is specifically as follows: Single-item target data is obtained by identifying and extracting single-item target data from the security inspection and cleaning data; Extract time-series labels from single-item target data to obtain single-item time-series label data; Nodes are constructed based on single-item time-series tag data and single-item target data to obtain single-item time-series node data. Neighbor node relationship analysis is performed on the time-series node data of a single item to obtain neighbor node relationship data of the single item; Graphs are constructed based on the neighbor node relationship data and time sequence node data of single items to obtain the first time label graph data. The construction of the multi-item time-stamped graph is specifically as follows: Multi-item node data is obtained by constructing multi-item nodes based on different single-item time-series node data corresponding to the same single-item time-series tag data. Based on the multi-item node data, interaction relationship processing, functional relationship processing, and structural relationship processing are performed to obtain multi-item interaction relationship data, multi-item functional relationship data, and multi-item structural coupling relationship data. Graphs are constructed on the multi-item node data based on the multi-item interaction relationship data, multi-item functional relationship data, and multi-item structural coupling relationship data, respectively, to obtain multi-item interaction relationship graph data, multi-item functional relationship graph data, and multi-item structural coupling relationship graph data; Static multi-level graph fusion is performed based on multi-item interaction relationship graph data, multi-item functional relationship graph data, and multi-item structural coupling relationship graph data to obtain second time-labeled graph data.

2. The method according to claim 1, characterized in that, Step S1 is as follows: By accessing the API interface of security inspection equipment, real-time airport security inspection data can be collected. Preprocessed security data is obtained by performing multimodal data edge processing on real-time airport security inspection data. Data labels are assigned based on the pre-processed security inspection data to obtain labeled security inspection data; Tag-based security check data is merged into the cloud to generate multi-source security check data for the airport.

3. The method according to claim 1, characterized in that, The specific steps for merging layered graphs are as follows: Based on the first time-label map data and the second time-label map data, a time change layer is constructed to obtain the time change layer data; Spatial relationship layer data is constructed based on the first time-label map data and the second time-label map data. The interaction layer is constructed based on the first time tag map data and the second time tag map data to obtain the interaction layer data; Graph attention networks were used to extract feature data from the time-varying layer data, spatial relationship layer data, and interaction layer data, respectively. Based on the time-varying layer feature data, spatial relationship layer feature data, and interaction layer feature data, inter-layer connections are made between the time-varying layer data, spatial relationship layer data, and interaction layer data to obtain layered graph connection data. Graph fusion is performed based on the data connected by the hierarchical graph to obtain time-stamped graph data.

4. The method according to claim 1, characterized in that, The spatial feature extraction specifically involves: Personnel detection data is obtained by performing personnel detection based on multi-source security screening data from the airport. Perform character tracking on the character detection data to obtain character tracking data; Clustering calculations were performed based on person tracking data to obtain clustering feature data for person security check scenarios; Based on the clustering feature data of security check scenarios, spatial behavior features of people are extracted to obtain preliminary security check spatial feature data; The spatial feature data of the preliminary security check is processed to determine the spatial relationship between the security check equipment and the personnel security check data.

5. The method according to claim 1, characterized in that, Step S3 is as follows: Behavioral mutation detection is performed based on security inspection characteristic sequence data to obtain behavioral mutation point data; Based on the behavioral mutation point data, mutation behavior is correlated to obtain security inspection mutation feature correlation data; Cross-modal feature matching data is obtained by performing cross-modal feature matching based on security inspection characteristic sequence data; Multimodal data association graphs are generated based on cross-modal feature matching data to obtain cross-modal feature association data for security checks.

6. The method according to claim 1, characterized in that, Step S4 is as follows: Based on the correlation data of security inspection mutation characteristics, mutation behavior threat identification is performed to obtain mutation behavior threat data; The mutation threat data is quantified to obtain the first security check hazard data; Cross-modal threat labeling is performed based on cross-modal feature association data of security inspection to obtain cross-modal threat labeling data; Cross-modal threat analysis was performed on the cross-modal threat annotation data to obtain the second security inspection hazard data.

7. An intelligent analysis system for airport security inspection data, characterized in that, For executing the intelligent analysis method for airport security data as described in claim 1, the intelligent analysis system for airport security data includes: Airport multi-source security inspection data acquisition module, used to acquire airport multi-source security inspection data; The preliminary security check data analysis module is used to perform preliminary security check data analysis based on multi-source airport security check data to obtain security check characteristic sequence data. The security inspection feature association module is used to perform behavioral mutation feature association based on security inspection feature sequence data to obtain security inspection mutation feature association data, and to perform cross-modal feature association based on security inspection feature sequence data to obtain security inspection cross-modal feature association data. The security inspection feature hazard assessment module is used to assess the threat level of mutation behavior based on the correlation data of security inspection mutation features to obtain the first security inspection hazard data, and to assess the cross-modal threat level based on the correlation data of security inspection cross-modal features to obtain the second security inspection hazard data; The security inspection decision support module is used to assist in security inspection decision-making based on the first security inspection hazard data and the second security inspection hazard data.

Images