Intelligent analysis method and system for airport security check data

Abstract

The invention relates to the technical field of data processing, in particular to an intelligent analysis method and system for airport security check data. The method comprises the following steps: acquiring airport multi-source security check data; performing security check data preliminary analysis according to the airport multi-source security check data to obtain security check characteristic sequence data; performing behavior mutation feature association according to the security check characteristic sequence data to obtain security check mutation feature association data, and performing cross-modal feature association according to the security check characteristic sequence data to obtain security check cross-modal feature association data; performing mutation behavior threat level evaluation according to the security check mutation feature associated data to obtain first security check danger data, and performing cross-modal threat level evaluation according to the security check cross-modal feature associated data to obtain second security check danger data; and performing security check decision-making auxiliary operation according to the first security check danger data and the second security check danger data. According to the invention, through quantitative threat and intelligent decision support, the intelligent level and efficiency of airport security check are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent analysis method and system for airport security inspection data. Background Art

[0002] With the increasing demand for airport security and the increasing complexity of security threats, traditional manual security inspection methods can no longer meet the security requirements of modern air transportation. Modern airport security inspection systems not only need to inspect luggage, personnel and their belongings, but also need to provide real-time warnings and threat assessments of potential safety hazards during the security inspection process. Existing security inspection systems usually rely on single-modal data input, such as X-ray scan images, behavior monitoring camera videos or manual inspection records. The processing efficiency and accuracy of these single data sources have certain limitations.

[0003] In addition, the analysis of security inspection data and the threat identification process often rely on manual experience, which makes it difficult to deal with complex and diverse threat situations. With the rapid development of technologies such as deep learning, image recognition, and behavioral analysis, traditional security inspection methods are gradually unable to meet the needs of real-time processing of multi-source data and intelligent decision support. In particular, threat identification during the security inspection process involves the fusion analysis of data from different modalities, which places higher demands on existing technologies.

[0004] In the existing technology, although some research has focused on image recognition and behavior analysis, most methods still rely on the processing of a single data source and lack the ability to comprehensively analyze multimodal data. At the same time, there is still no effective intelligent analysis framework for how to accurately identify and evaluate the threat level of different behaviors and how to combine multiple data sources to optimize security inspection decisions. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an airport security data intelligent analysis method and system to solve at least one of the above technical problems.

[0006] The present application provides a method for intelligent analysis of airport security inspection data, comprising the following steps: Step S1: Acquire airport multi-source security inspection data; Step S2: Perform a preliminary analysis of the security inspection data based on the airport multi-source security inspection data to obtain security inspection characteristic sequence data; Step S3: performing behavioral mutation feature association according to the security inspection feature sequence data to obtain security inspection mutation feature association data, and performing cross-modal feature association according to the security inspection feature sequence data to obtain security inspection cross-modal feature association data; Step S4: performing a mutation behavior threat level assessment based on the security inspection mutation feature association data to obtain first security inspection danger data, and performing a cross-modal threat level assessment based on the security inspection cross-modal feature association data to obtain second security inspection danger data; Step S5: Perform security inspection decision-making assistance operations according to the first security inspection risk data and the second security inspection risk data.

[0007] In the present invention, potential dangerous behaviors, such as sudden rapid movement, abnormal pause, etc., can be discovered at an early stage by analyzing the association of behavioral mutation characteristics of security inspection data, and the accuracy of threat identification is improved by detecting the precursors of potential threats. By integrating data of different modalities (such as video surveillance, sensor data, facial recognition, etc.), character behavior and emotional changes are captured, the accuracy of cross-modal feature association analysis is improved, the blind spots of a single data source are avoided, and the ability to identify complex threat situations is enhanced. Accurate decision support is provided by evaluating the real-time threat level of security inspection data. When a potential threat is identified, security inspectors can get real-time action suggestions, take additional inspection measures or alarms in time, improve response speed, and reduce risks caused by human errors. The system can also optimize the decision-making process according to the evaluation results, realize adaptive and intelligent adjustment, and ensure the balance between safety and efficiency. By integrating data from different sources (video, sensor, facial recognition, behavior analysis, etc.), the method can comprehensively analyze the dynamic behavior of inspected objects and personnel, reveal more potential danger signals, and enhance the comprehensiveness and reliability of security inspection.

[0008] Preferably, step S1 specifically includes: Step S11: Collecting real-time airport security data by accessing the API interface of security inspection equipment (such as baggage scanners, behavior monitoring cameras, and identity verification systems); Step S12: performing multimodal data edge processing according to the real-time airport security inspection data to obtain pre-processed security inspection data; Step S13: assigning data labels according to the pre-processed security inspection data to obtain labeled security inspection data; Step S14: Merge the labeled security inspection data into the cloud to generate airport multi-source security inspection data.

[0009] In the present invention, data from different security inspection equipment (such as baggage scanners, cameras, identity verification systems, etc.) are processed at the edge, and the security inspection process can be comprehensively analyzed from multiple angles. For example, the baggage scanner provides item image data, the behavior monitoring camera provides video data, and the identity verification system provides biometric data. Combining these different modal data helps to more comprehensively and accurately evaluate potential threats in security inspection scenarios. Through real-time data acquisition and multimodal data preprocessing, the accuracy and real-time performance of data processing can be effectively improved. Multimodal data fusion and labeled data provide richer information for threat identification and decision analysis. Cloud storage and integration make data management more efficient, while supporting real-time analysis and decision-making of large-scale data.

[0010] Preferably, step S2 specifically includes: Step S21: performing multimodal data cleaning based on the airport multi-source security inspection data to obtain security inspection cleansing data; Step S22: extracting security inspection danger features and space features from the security inspection cleansing data to obtain security inspection danger feature data and security inspection space feature data respectively; Step S23: performing feature fusion encoding on the security inspection hazard feature data and the security inspection space feature data to obtain security inspection characteristic sequence data.

[0011] In the present invention, by cleaning multi-source security inspection data (such as images, sensor data, video surveillance, etc.), noise, redundant information and incomplete data are removed, so that the remaining data is more accurate, clean and efficient. By extracting dangerous features from the cleaned security inspection data, potential security inspection threats can be effectively identified. For example, the abnormal object form is extracted from the luggage scanning data, and the abnormal behavior features are extracted from the behavior monitoring video. The extracted dangerous features provide higher-precision information support for subsequent threat assessment and decision-making. Spatial feature extraction focuses on the analysis of the spatial layout, movement trajectory and position relationship of people and objects in the security inspection area. For example, the abnormal movement trajectory of security personnel, the accumulation of items, the spatial relationship between people and items, etc. are identified, which helps to further judge potential security risks. Especially in complex security inspection environments, the identification of spatial features can effectively support behavioral pattern recognition and abnormal activity detection. By fusing and encoding security inspection dangerous features and spatial features, feature information from different sources and types can be integrated into a unified feature sequence, which not only optimizes the expression form of data, but also effectively captures multi-dimensional information interaction, such as the relationship between the degree of danger of an object and its position in space. Feature fusion coding combines hazard features and spatial features to extract more comprehensive and rich security inspection features, which facilitates pattern recognition, behavior analysis and threat assessment.

[0012] Preferably, the security inspection risk feature extraction is specifically as follows: Performing visual feature extraction on the security inspection image data in the security inspection cleaning data to obtain security inspection visual feature data; A time label graph is constructed according to the security inspection image data in the security inspection cleaning data to obtain time label graph data; Perform time feature fusion according to the time label graph data and the security inspection visual feature data to obtain the security inspection visual feature time graph data; Dangerous features are extracted from the security inspection visual feature time graph data to obtain security inspection dangerous feature data.

[0013] In the present invention, by extracting visual features from security inspection image data, it is possible to identify potential threat item features from the image, such as item shape, material, size, density, etc., which may be ignored or misjudged in traditional security inspection methods. Visual feature extraction can automatically extract fine-grained item features through deep learning or computer vision algorithms, effectively improving the accuracy of recognition. Especially in the case of complex images and diverse items, visual feature extraction provides a stronger basis for item hazard assessment. By tracking the changes of items in the time dimension, the construction of the time label graph can capture the dynamic information of items during the security inspection process, such as the movement trajectory of the items, changes in the scanned image, abnormal pauses and other behaviors. Through the fusion of time features, the system can capture the behavior patterns of items across time periods, thereby identifying time series features related to dangerous items. For example, abnormal changes in items during scanning (such as sudden stops, unnatural morphological changes) can be used as a warning signal. Extracting features related to danger from the data after the fusion of visual features and time features can determine whether they are dangerous goods based on the spatiotemporal behavior patterns of different items. Further improve the system's sensitivity and accuracy to potential threats, avoid false alarms and missed alarms, and improve the reliability of the security inspection system.

[0014] Preferably, the time label graph is constructed as follows: Constructing a single item time label graph according to the security inspection image data in the security inspection cleaning data to obtain first time label graph data; Construct a multi-item time label graph according to the security inspection image data in the security inspection cleaning data to obtain second time label graph data; Performing layered graph fusion on the first time label graph data and the second time label graph data to obtain time label graph data; The single item time label graph is constructed as follows: Perform single-item target recognition and extraction based on the security inspection image data in the security inspection cleaning data to obtain single-item target data; Extract time series labels according to single item target data to obtain single item time series label data; Nodes are constructed according to the single item time series label data and the single item target data to obtain the single item time series node data; Perform neighboring node relationship analysis on single item time series node data to obtain single item neighboring node relationship data; A graph is constructed based on the single item neighboring node relationship data and the single item time series node data to obtain first time label graph data; The construction of multi-item time label graph is as follows: Constructing multiple item nodes according to different single item time series node data corresponding to the same single item time series label data to obtain multiple item node data; Performing interaction relationship processing, functional relationship processing and structural relationship processing according to the multi-item node data to obtain multi-item interaction relationship data, multi-item functional relationship data and multi-item structural coupling relationship data; According to the multi-item interaction relationship data, the multi-item function relationship data and the multi-item structure coupling relationship data, a graph of multi-item node data is constructed to obtain multi-item interaction relationship graph data, multi-item function relationship graph data and multi-item structure coupling relationship graph data respectively; Static multi-level graph fusion is performed based on the multi-item interaction relationship graph data, the multi-item function relationship graph data and the multi-item structure coupling relationship graph data to obtain the second time label graph data.

[0015] In the present invention, by target recognition and time series label extraction of a single object, the dynamic changes of the object during the security inspection process can be clearly captured. For example, whether an object has abnormal behavior (such as sudden stop, abnormal shape change, etc.) during the security inspection process can be accurately recorded and analyzed through time series data. When multiple objects appear in the security inspection image at the same time, the time series label graph of multiple objects is constructed to simultaneously analyze the relationship between different objects and their changes in the time series, which helps to identify the potential danger of certain objects, such as abnormal interactive behavior between multiple objects or combination patterns of suspected hidden dangerous objects. By constructing and analyzing the node relationship of multiple objects, the complex interaction, functional relationship and structural coupling relationship between multiple objects can be identified and modeled. By comprehensively analyzing the dynamic changes of multiple objects in different time periods and the interactions between them, early warning of abnormal objects can be achieved. Especially in a high-density security inspection environment, the combination or proximity of multiple objects may bring higher risks, and traditional methods may find it difficult to detect such complex interactive relationships. Through graph construction, this complexity can be better identified and handled. For the relationship between multiple objects, simple object recognition may not be enough to reveal its potential danger. By constructing multi-item interaction diagrams, functional diagrams, and structural coupling diagrams, we can deeply explore the interaction patterns and structural features between items during the security inspection process, thereby improving the ability to identify potentially dangerous items in complex situations. For example, by analyzing the interaction relationship between multiple items, we can detect that some items are used as tools to hide other items, thereby discovering dangerous signals that traditional security inspection methods cannot identify.

[0016] By fusion of the hierarchical graphs of the first time label graph and the second time label graph, and further fusion of static multi-level graphs, time series data at different levels can be combined to form a more refined time label graph data. It not only improves the utilization efficiency of time series data, but also optimizes the dynamic relationship modeling between multiple items, so that data at different levels complement and strengthen each other, thereby providing a more comprehensive threat assessment. When security inspection equipment has difficulty identifying some potentially dangerous items, the fused data can provide more reliable information for decision-making. By combining static spatiotemporal information (such as the appearance features of items in the image) and dynamic information (such as the movement trajectory and interactive behavior of items), more accurate behavior prediction and threat assessment can be performed at different time points, further improving the prediction ability of threatening items. For security inspection scenarios that require long-term monitoring and high-density items, multi-level graph fusion provides stronger support for threat assessment.

[0017] Preferably, the layered graph fusion is specifically as follows: Constructing a time-varying layer according to the first time label graph data and the second time label graph data to obtain time-varying layer data; Constructing a spatial relationship layer according to the first time label graph data and the second time label graph data to obtain spatial relationship layer data; Constructing an interaction layer according to the first time label graph data and the second time label graph data to obtain interaction layer data; Perform graph attention network extraction on the time-varying layer data, the spatial relationship layer data, and the interaction layer data to obtain the time-varying layer feature data, the spatial relationship layer feature data, and the interaction layer feature data respectively; According to the time-varying layer feature data, the spatial relationship layer feature data and the interactive layer feature data, the time-varying layer data, the spatial relationship layer data and the interactive layer data are connected between layers to obtain layered graph connection data; The graph fusion is performed based on the hierarchical graph connection data to obtain the time label graph data.

[0018] In the present invention, by independently modeling the time change layer, spatial relationship layer and interaction layer, different types of information can be effectively distinguished, so as to finely process time series data, spatial relationships and interactive behaviors between objects. The data processing method of each layer can be optimized independently, making the information extraction of each layer more accurate. The time change layer focuses on the dynamic changes of objects on the time axis, which is suitable for capturing the changing trends of objects during the security inspection process and helping to identify abnormal fluctuations in the state of objects. The spatial relationship layer focuses on the spatial position relationship between objects, and through spatial layout analysis, it can effectively discover patterns in which dangerous goods are hidden or have hidden dangers. The interactive layer deals with the interaction between multiple objects, which can reveal the mutual influence and potential threat behaviors between objects, especially in the security inspection scenario of multiple people or high-density objects.

[0019] In the feature extraction of each layer, the use of graph attention network can effectively capture the relationship between different layers and perform weighted processing, thereby improving the ability to identify key features. Through the attention mechanism, the importance of each node and neighboring nodes can be adaptively adjusted to highlight threat-related features. For example, some potentially threatening items exhibit mutation behavior in time series, and the graph attention network will give priority to these mutation patterns, thereby helping to improve the recognition accuracy of dangerous items.

[0020] By connecting feature data at different levels and fusing graphs, effective integration of multi-level information can be achieved. After the data of each layer is fused, not only the information characteristics of each layer are retained, but also the correlation between layers can be captured, providing more comprehensive and accurate time label graph data. By connecting the time change, spatial relationship and interaction layers, the traditional single-level information structure can be broken and the deep fusion of multi-dimensional features can be achieved. Fusion of graphs at different levels not only retains the information advantages of each level, but also eliminates the limitations of single-level data analysis, ensuring the comprehensiveness and accuracy of judgments. For example, the combination of the interaction layer and the spatial layer can reveal the potential threat patterns that may be generated by objects during time changes, while the combination of the time change layer and the interaction layer can help discover potential dynamic threat behaviors.

[0021] Preferably, the spatial feature extraction is specifically: Perform person detection based on multi-source airport security inspection data to obtain person detection data; Performing person tracking on the person detection data to obtain person tracking data; Performing facial detection according to the person tracking data to obtain facial detection data; Perform facial label analysis based on facial detection data to obtain facial expression data; According to the character tracking data and the facial tag data, the spatial relationship of expression changes is combined to obtain the expression trajectory correlation data; Perform clustering calculation based on the person tracking data to obtain clustering feature data of the person security inspection scene; Extract the spatial behavior features of people based on the clustering feature data of the person security inspection scene to obtain preliminary security inspection space feature data; The preliminary security inspection space feature data is processed with respect to the spatial relationship of the person security inspection equipment to obtain the security inspection space feature data.

[0022] The present invention can realize dynamic monitoring and analysis of the behavior of people in the security inspection area through accurate person detection and tracking, which is particularly important for high-traffic security inspection scenes. It can grasp the position, movement trajectory and behavior changes of people in real time, and timely identify potential abnormal behaviors and threats. Through facial detection and expression analysis technology, the system can extract the facial expression information of people, which is of special significance in security inspection scenes. Changes in facial expressions, such as nervousness, anxiety and other emotions, can be used as potential indicators of abnormal behavior, which helps security personnel identify people with abnormal psychological states. Through the association of expression trajectories and the extraction of spatial behavior features of people, not only can the emotional changes of people be evaluated, but also their behavior paths can be tracked to reveal their action patterns in space. Combining facial expressions and spatial behavior features can help the system determine whether the behavior of people is potentially threatening. Through clustering calculation, the people in the security inspection scene are classified and grouped, and normal and abnormal behavior patterns can be effectively distinguished. For example, people are divided into "normal behavior" and "abnormal behavior" groups according to their behavior types, which further improves the judgment efficiency of security personnel. By processing the spatial relationship between the person security inspection equipment, the relative position and interaction between the person and the equipment during the security inspection process can be revealed. For example, if a person stays in front of the scanning equipment for too long or is too close to the equipment, it indicates the risk of hidden objects.

[0023] Preferably, step S3 is specifically: Step S31: Perform behavior mutation detection according to the security inspection characteristic sequence data to obtain behavior mutation point data; Step S32: performing mutation behavior association according to the behavior mutation point data to obtain security inspection mutation feature association data; Step S33: performing cross-modal feature matching according to the security inspection characteristic sequence data to obtain cross-modal feature matching data; Step S34: Generate a multimodal data association graph based on the cross-modal feature matching data to obtain security inspection cross-modal feature association data.

[0024] The analysis of a single modality in the present invention is often limited by the one-sidedness of information, but through cross-modal feature matching, the system can integrate information from multiple data sources to improve the accuracy of recognition. For example, image recognition technology cannot fully identify the intentions of a person, but combined with behavioral pattern data, audio data, etc., it can conduct a comprehensive analysis of the person's behavior. By combining cross-modal feature matching data to generate security inspection cross-modal feature association data, an intuitive multi-dimensional data association graph can be provided, which can help security personnel identify the relationship between various data sources and reveal the inherent connection between different modal data. According to the changes in multimodal data, the system can flexibly adjust the threat assessment criteria and dynamically identify and predict security risks. For example, if there is a sudden change in the behavior of multiple people in a certain area, the system can automatically adjust the focus of analysis and focus on monitoring the area.

[0025] Preferably, step S4 is specifically: Step S41: performing mutation behavior threat identification according to security inspection mutation feature association data to obtain mutation behavior threat data; Step S42: quantifying the mutation threat of the mutation behavior threat data to obtain first security inspection risk data; Step S43: performing cross-modal threat annotation according to the security inspection cross-modal feature association data to obtain cross-modal threat annotation data; Step S44: performing cross-modal threat analysis on the cross-modal threat annotation data to obtain second security inspection risk data.

[0026] In the present invention, by analyzing the security inspection mutation feature association data, abnormal behaviors or potential threats that occur during the security inspection process can be accurately identified. These behavioral changes are usually related to dangerous events, such as sudden abnormal behaviors, sudden violent behaviors or unusual behaviors of individuals. The quantified threat data allows security inspectors to no longer rely on intuition or experience, but to judge the risk level through specific numerical values ​​and model results, thereby improving the scientificity and accuracy of decision-making. By labeling and analyzing the security inspection cross-modal feature association data for threats, the system can make full use of data from different sensors (such as video surveillance, behavior trajectory, voice recognition, etc.) to conduct a comprehensive assessment of threats. Real-time threat assessment can help security inspectors determine the severity of potential threats in the shortest time and take corresponding actions. For example, if a person's behavior is abnormal and his facial expression appears anxious, the system can combine behavior, facial data and other features for real-time threat analysis to help security inspectors respond more quickly.

[0027] Preferably, the present application also provides an airport security data intelligent analysis system, which is used to execute the airport security data intelligent analysis method as described above, and the airport security data intelligent analysis system includes: Airport multi-source security inspection data collection module, used to obtain airport multi-source security inspection data; The security inspection data preliminary analysis module is used to perform preliminary analysis of the security inspection data based on the airport's multi-source security inspection data to obtain security inspection characteristic sequence data; The security inspection feature association module is used to perform behavioral mutation feature association based on the security inspection feature sequence data to obtain security inspection mutation feature association data, and to perform cross-modal feature association based on the security inspection feature sequence data to obtain security inspection cross-modal feature association data; The security inspection feature risk assessment module is used to perform a mutation behavior threat level assessment based on the security inspection mutation feature association data to obtain first security inspection risk data, and to perform a cross-modal threat level assessment based on the security inspection cross-modal feature association data to obtain second security inspection risk data; The security inspection decision-making assistance module is used to perform security inspection decision-making assistance operations based on the first security inspection risk data and the second security inspection risk data.

[0028] The beneficial effect of the present invention is that the system can comprehensively analyze data from different devices and combine the advantages of various data sources to comprehensively capture potential threats in security inspections. For example, luggage scanners can provide image data, behavior monitoring cameras can provide motion trajectory data, and identity verification systems can provide personal identity information. Through the combination of these multi-source data, threats can be identified more accurately and false positives and false negatives can be reduced. By using mutation feature association, the system can quickly identify abnormal behaviors or sudden dangerous events. For example, if a passenger's behavior suddenly changes dramatically, the system can mark and process it in time to avoid the spread of potential threats. Threats in the security inspection process may appear in various forms, such as behavior, images, sounds, facial expressions, etc. Through the refined association of cross-modal features, the system can more comprehensively and meticulously evaluate multi-dimensional information, thereby improving the accuracy and comprehensiveness of threat identification. By quantifying the threat, security inspectors can make decisions with the help of scientific data, no longer relying on personal judgment, greatly improving the accuracy and efficiency of decision-making. The system can propose optimal countermeasures based on real-time data, such as personnel allocation, equipment mobilization, etc., thereby improving security inspection efficiency. By analyzing cross-modal data in real time, the system can immediately issue an alarm and provide threat assessment when a potential threat occurs, providing security personnel with timely decision-making basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings: Figure 1 A flowchart showing the steps of an intelligent analysis method for airport security data according to an embodiment is shown; Figure 2 A flowchart showing a method for collecting airport multi-source security inspection data according to an embodiment is shown; Figure 3 A flowchart showing a method for preliminary analysis of security inspection data according to an embodiment is shown; Figure 4 A flowchart showing a method for associating security inspection features according to an embodiment is shown; Figure 5 A flowchart of the steps of a security feature risk assessment method according to an embodiment is shown. DETAILED DESCRIPTION

[0030] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In addition, the drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0032] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0033] See also Figures 1 to 5 , the present application provides an airport security data intelligent analysis method, comprising the following steps: Step S1: Acquire airport multi-source security inspection data; Specifically, the process of obtaining airport multi-source security data includes collecting data from multiple security subsystems, such as image data output by security scanners (such as X-ray machines, CT scanners), passenger information data, item identification information, video data collected by surveillance cameras, and voice records of manual inspections.

[0034] Step S2: Perform a preliminary analysis of the security inspection data based on the airport multi-source security inspection data to obtain security inspection characteristic sequence data; Specifically, after completing data collection, it is necessary to perform preliminary processing on multi-source data and extract security inspection characteristic sequence data. Classify and archive data according to the source (passengers, luggage, security inspection equipment). Extract features such as object shape and density from image data; extract identity information (such as age, gender) and action mode (such as time and pace of passing security inspection) from passenger information data. Since different security inspection equipment generates data in different formats, it is necessary to unify the dimensions through normalization. For example, image features are converted into numerical matrices, and speech features are converted into text feature codes.

[0035] Step S3: performing behavioral mutation feature association according to the security inspection feature sequence data to obtain security inspection mutation feature association data, and performing cross-modal feature association according to the security inspection feature sequence data to obtain security inspection cross-modal feature association data; Specifically, monitor the behavior trajectory: analyze the path changes of passengers or luggage in the security inspection process, such as abnormal increase in dwell time, change in the number of items carried, etc. Analyze the feature changes in each time period and identify significant mutations (such as the shape of an item changes from regular to irregular). Match the behavior mutation features with predefined threat patterns (such as a known dangerous goods feature library) and output the association results.

[0036] Integrate multiple modal data (such as images, identity information, and voice) for correlation matching. For example, compare the shape of an object scanned by an X-ray with the information about the object carried in the passenger's voice record. Annotate the correlation strength and abnormal points to form cross-modal feature correlation data.

[0037] Step S4: performing a mutation behavior threat level assessment based on the security inspection mutation feature association data to obtain first security inspection danger data, and performing a cross-modal threat level assessment based on the security inspection cross-modal feature association data to obtain second security inspection danger data; Specifically, the data associated with behavioral mutation characteristics are analyzed to quantify the threat level of mutation behavior. For example, if a passenger's stay time is significantly longer than other passengers and the items they carry have high-risk characteristics, they are assessed as high threat. Using the classification rules, the threat level results (such as low threat, medium threat, high threat) are output.

[0038] Compare cross-modal feature association data with historical dangerous cases, for example, the irregular shape of objects in the image data and the inconsistency in the passenger identity match, and determine the threat level. Weighted processing is performed on the evaluation results to output threat level data.

[0039] Step S5: Perform security inspection decision-making assistance operations according to the first security inspection risk data and the second security inspection risk data.

[0040] Specifically, based on the threat level data, the security inspector is assisted in the next step. The first and second security inspection danger data are combined, and the final threat level list is generated according to the comprehensive scoring rules. Security inspection process optimization suggestions are automatically generated. For example, high-threat passengers are given priority for secondary manual inspections, and luggage carrying abnormal items is unpacked for inspection. The system automatically issues high-threat alerts to security inspectors, including the source of the threat, recommended actions, and supporting evidence (such as image screenshots and behavior records).

[0041] Preferably, step S1 specifically includes: Step S11: Collecting real-time airport security data by accessing the API interface of security inspection equipment (such as baggage scanners, behavior monitoring cameras, and identity verification systems); Specifically, the main security equipment in the airport (such as baggage scanners, behavior monitoring cameras, and identity verification systems) are connected to the data collection server through the local area network. Each device must be configured with a unique device ID to distinguish the source of data. Configure the API interfaces of different devices to ensure that data can be obtained in real time. The API of the baggage scanner can output baggage X-ray image data and item attribute information; the API of the behavior monitoring camera provides real-time video streams; and the API of the identity verification system outputs passenger identity information (such as ID number and boarding pass information). Write a collection program to obtain real-time data by periodically requesting the API interface, such as collecting image data and video frames every second, and synchronizing passenger information in real time after each identity verification is completed. All collected data is marked with timestamps to ensure the synchronization of subsequent analysis.

[0042] Step S12: performing multimodal data edge processing according to the real-time airport security inspection data to obtain pre-processed security inspection data; Specifically, edge computing modules are deployed on the acquisition server, and the distributed computing framework is used to localize the multimodal data to reduce transmission pressure. The image noise reduction algorithm is used to clean up the artifacts and noise in the scanned image and adjust the image contrast; key frames are extracted from the video stream, and intra-frame downsampling technology is used to reduce redundancy. The passenger information output by the identity authentication system is normalized (such as removing spaces and repeated fields). The timestamp is used to align the data of different modes, for example, to ensure that the image of a passenger's luggage is consistent with the time of his or her identity information.

[0043] Step S13: assigning data labels according to the pre-processed security inspection data to obtain labeled security inspection data; Specifically, labeling rules are defined according to different data types. For example, image data automatically generates item category labels based on item type (electronic equipment, liquid, metal); video data labels passenger behavior labels (fast pass, stop, reverse) based on the monitored area; identity verification data automatically labels passenger identity labels (age group, gender, nationality). Based on image preprocessing, an item recognition algorithm is used to partition the image content, output the item category information of each area, and generate corresponding labels. A behavior recognition algorithm is used for video key frames to detect whether passengers have abnormal behavior and mark the behavior labels. Based on the passenger information field, the defined labeling rules are matched to generate corresponding labels.

[0044] Step S14: Merge the labeled security inspection data into the cloud to generate airport multi-source security inspection data.

[0045] Specifically, the labeled security inspection data of the edge node is synchronized to the cloud server using an encrypted transmission protocol (such as HTTPS or TLS). A data integration module is deployed in the cloud server to merge the multimodal data of the same passenger into a unified data entry through the passenger's unique identifier (such as passport number or boarding pass number). All merged data is indexed and stored in a distributed database. It is stored by category, by region, by time, or by passenger.

[0046] Preferably, step S2 specifically includes: Step S21: performing multimodal data cleaning based on the airport multi-source security inspection data to obtain security inspection cleansing data; Specifically, remove redundant, inconsistent, and abnormal data from airport multi-source security data to ensure the accuracy of analysis. Check whether there are null or missing values ​​in the data. For example, if the age field is missing in the passenger identity data, use the average age of passengers on the same flight to fill it; if some frames of image data are missing, interpolate to generate missing frames. Set a reasonable range for each data type and remove abnormal values. For example, if there are negative values ​​or unreasonable large values ​​(exceeding the normal passing time range) in the time record of passengers passing through security checks, they need to be marked as abnormal and removed. Normalize numerical data (such as passing time and item weight); standardize text data (such as passenger identity fields), such as unifying date formats and name capitalization. Merge duplicate passenger information records; use filtering technology to reduce noise for frames with too much noise in image data.

[0047] Step S22: extracting security inspection danger features and space features from the security inspection cleansing data to obtain security inspection danger feature data and security inspection space feature data respectively; Specifically, identify potential risk factors in the data, such as high-risk items, abnormal behavior, and identity risks. Analyze the scanned images of cleaned luggage, segment the object areas in the image, and extract dangerous items (such as liquids and sharp objects) through features such as shape and density. Extract risk features based on identity information, such as comparing flight destinations with a list of high-risk areas; detect whether there are risky passengers marked by the security inspection system (such as a wanted list). Analyze the records of behavioral monitoring cameras and extract abnormal behavior features, such as repeated luggage checks and abnormally long stays. All dangerous feature extraction results are marked as structured data, including dangerous item categories, dangerous behavior types, etc., and stored as security inspection dangerous feature data.

[0048] Capture the spatial behavior characteristics of passengers and luggage in the security inspection area. Extract the spatial trajectory of passengers based on surveillance camera data, and record the changes in their coordinate positions over time as a time series. Analyze the distribution of passengers' stay time in different security inspection areas, and mark abnormal aggregation behavior or retrograde behavior. Combined with X-ray image data, associate the spatial position of items in the luggage with the relative position of passengers when passing through security inspection. The extraction results form spatial feature data, including passenger position trajectory, stay time distribution, etc., which are stored as security inspection spatial feature data.

[0049] Step S23: performing feature fusion encoding on the security inspection hazard feature data and the security inspection space feature data to obtain security inspection characteristic sequence data.

[0050] Specifically, the security inspection hazard feature data and spatial feature data are uniformly fused and encoded to form time-series data. Based on the timestamps of the hazard feature data and spatial feature data, ensure that different modal features correspond one-to-one in the time dimension. For example, align the time point of the passenger's abnormal behavior with the dangerous goods detection result of the corresponding luggage. Convert the hazard features and spatial features into a numerical coding format. For example, 0 represents no danger, 1 represents low risk, 2 represents medium risk, and 3 represents high risk; use three-dimensional coordinates to encode the spatial position of the passenger. Sort the features of each passenger by time to form a feature sequence. The feature sequence includes multiple fields, such as timestamp, hazard feature value, and spatial feature value. Assign different weights to the hazard features and spatial features, fuse them according to the weights, and generate feature sequence data.

[0051] Preferably, the security inspection risk feature extraction is specifically as follows: Performing visual feature extraction on the security inspection image data in the security inspection cleaning data to obtain security inspection visual feature data; Specifically, visual information that can reflect the characteristics of luggage items is extracted from security inspection image data, including shape, texture, density, etc. The security inspection image is grayed to reduce color interference; image contrast is enhanced through histogram equalization; image edges are detected to highlight the outline of items. Different items in the image are separated by region using regional segmentation technology, such as the segmentation of different items in luggage; each segmented region is numbered. The following visual features are extracted from the segmented item regions: parameters such as boundary length, area, and roundness are extracted. Statistical methods are used to extract the surface texture characteristics of the item (such as smoothness and roughness). The density distribution of the item (such as the density distribution of liquids or metals) is detected by changing the gray value of the scanned image. The extracted feature values ​​are converted into vector representations, and a set of visual feature data is generated for each item region.

[0052] A time label graph is constructed according to the security inspection image data in the security inspection cleaning data to obtain time label graph data; Specifically, a time label graph is constructed for the security inspection image data through the time dimension to reflect the changes of items over time during the security inspection process. The collected security inspection images are grouped according to the timestamps, and an image sequence is generated for each time period, such as segmentation by seconds or frames. The time-grouped image data is modeled as nodes, each node represents an image of a time period; edge connections between nodes are established according to the time sequence to form a time label graph. Attribute labels corresponding to the image are added to each node, such as the number of items, the maximum item area, etc. Edge weights are assigned to the connected nodes, and the weights are based on the degree of change of item features in adjacent time periods. For example, the higher the edge weight, the more drastic the change in item features. The lower the edge weight, the more stable the item features are. The output time label graph data includes all time nodes, node attributes, and edge weights between nodes.

[0053] Perform time feature fusion according to the time label graph data and the security inspection visual feature data to obtain the security inspection visual feature time graph data; Specifically, the security inspection visual feature data and the time label graph data are combined, and the time features and spatial visual features are fused to construct a visual feature time graph. The security inspection visual feature data are aligned with the time nodes in the time label graph. For example, the visual feature vectors of the items extracted within a certain time period are matched with the corresponding time node attributes. Time series modeling is performed on the edge weights in the time label graph to capture the trend of feature changes between time nodes. For example: if the features of an item remain unchanged in multiple consecutive time nodes, the time trend is stable. If the features change significantly in consecutive time nodes, the time change gradient is recorded. A fused feature vector is generated for each time node, and the visual features and time trend information are merged into a set of comprehensive features. The generated security inspection visual feature time graph data contains time nodes, visual features, and time change trends.

[0054] Dangerous features are extracted from the security inspection visual feature time graph data to obtain security inspection dangerous feature data.

[0055] Specifically, features that reflect danger are extracted from the visual feature time graph, such as abnormal changes in items or abnormal time trends. Danger indicators are set according to security inspection rules, for example: certain shape or density feature values ​​exceed the safety range (such as sharp objects, liquids). Items change significantly in the time series (such as sudden changes in the shape of items, which is likely to be the behavior of hiding items). Traverse each node and edge in the time graph and calculate the danger indicator score. For example: for a single node, detect whether there are dangerous items in the visual features. For the edges between nodes, detect whether the time trend changes exceed the threshold. Label the detected high-risk items and abnormal behaviors, and generate danger feature records, including item number, danger type, danger level, etc. Summarize the danger feature information in all nodes and edges to form security inspection danger feature data. The output security inspection danger feature data includes potentially dangerous items and their time behavior characteristics.

[0056] Preferably, the time label graph is constructed as follows: Constructing a single item time label graph according to the security inspection image data in the security inspection cleaning data to obtain first time label graph data; Specifically, for a single item in the security inspection image data (such as an item in luggage), a time-series label graph is constructed to reflect the state changes of the item at different time nodes. A single item region is segmented from each frame of the security inspection image, and the item is uniquely identified by a number (such as an item ID). For example, each item in the luggage is identified using a region segmentation technique, and its corresponding region is extracted. The image data at each time point is taken as a node. For example, the timestamp of the image frame is used as a node label to indicate the state of the item at that time point. The visual feature attributes (such as shape, texture, density) of the item are added to each time node, and the extracted visual features are stored as the attribute values ​​of the node. The edge weight is calculated based on the visual feature changes of adjacent time nodes. For example: if the shape or density feature of the item changes greatly between two time points, a higher weight is assigned. If the change is small, a lower weight is assigned. The time nodes are connected with the edges to generate a single item time label graph. The output first time label graph data is a time series graph for a single item, including time nodes, node attributes, and edge weights.

[0057] Construct a multi-item time label graph according to the security inspection image data in the security inspection cleaning data to obtain second time label graph data; Specifically, for multiple items in the security inspection image data, an overall time-series label graph is constructed to reflect the relationship between items in the time dimension. The relationship between items in each frame of the image is analyzed. For example, the correlation between items is judged by the position, shape similarity, or area overlap of the items. Each frame of image data corresponds to a time node, and each node contains the overall information of all items at that time point. The following multi-item attributes are extracted for each time node: the number of items in the frame image. The relative position relationship of all items. The similarity between items (such as density, shape). The edge weight is calculated based on the change in the distribution of items at adjacent time nodes. For example: if the total number of items at two time nodes is greatly different, or the correlation between items changes significantly, a higher edge weight is assigned. If the change is small, a lower edge weight is assigned. Each time node is connected to its adjacent nodes through edges to form a multi-item time label graph. The output second time label graph data is a time series graph for the entire multi-item, including nodes (each frame of the image) and edge weights.

[0058] Performing layered graph fusion on the first time label graph data and the second time label graph data to obtain time label graph data; Specifically, the first time label graph (single item) and the second time label graph (multiple items) are fused to construct a hierarchical time label graph containing single item and multi-item features. The time nodes of the single item graph and the multi-item graph are aligned to ensure that they correspond one-to-one in the time dimension. For example, if an item exists in the first time label graph at time point t, there must be a corresponding node in the second time label graph at time point t. The nodes of the single item graph are regarded as the first layer, as the base layer; the nodes of the multi-item graph are regarded as the second layer, as the overall layer. In the first layer, the single item nodes at the same time point are merged, and their average attribute values ​​are calculated to form the fused single item attributes; in the second layer, the nodes and attributes of the original multi-item graph are retained. Connections are established for the corresponding time nodes of the first layer (single item graph) and the second layer (multi-item graph), and the inter-layer edge weights are calculated. For example, if the feature changes of the single item in the first layer significantly affect the multi-item distribution in the second layer, a higher edge weight is assigned. If the impact is small, a lower edge weight is assigned. The nodes of the first and second layers and their connection relationships are uniformly encoded to generate a hierarchical time label graph. The generated time label graph data contains single-item and multi-item time characteristics, reflecting the individual changes and overall relationships of items.

[0059] The single item time label graph is constructed as follows: Perform single-item target recognition and extraction based on the security inspection image data in the security inspection cleaning data to obtain single-item target data; Specifically, identify individual items from the security inspection image data and extract their features to generate single item target data. Perform target detection on the security inspection image to identify and locate individual items in the image. Extract the bounding box of each item through border detection and mark each item with a unique number (such as ObjectID). Use image segmentation technology to separate the area of ​​the object from the image and exclude background and other interference. Perform attribute analysis on the extracted object area, including: shape, texture, density, etc. The center point coordinates and bounding box size of the object in the image. Generate data records for each item, including ObjectID, visual feature vector, position features, etc., to form a single item target data table. The output single item target data contains the item number and its attribute information.

[0060] Extract time series labels according to single item target data to obtain single item time series label data; Specifically, add a time dimension label to the single-item target data to form single-item time-series data. Add a timestamp to each security inspection image, and synchronize the timestamp to the single-item target data. For example, add a time field to the record of each item to indicate the time of the item in the current frame. Compare the records of the same item at different time points to extract time series attributes, such as calculating the time difference of the coordinates of the center point of the item to obtain the position change rate. Calculate the time change value of the item's visual feature vector. Generate a time series label based on the change of the item in the time dimension. For example: if the position of the item changes significantly, it is marked as "moving". If the visual features change greatly, it is marked as "morphological change". Generate single-item time series label data containing timestamps, time series attributes and change labels. The output time series label data includes the time information of the item and its characteristics that change over time.

[0061] Nodes are constructed according to the single item time series label data and the single item target data to obtain the single item time series node data; Specifically, the single-item time series label data is converted into nodes of the graph structure. The single-item data at each time point is regarded as a node. The unique identifier of the node is formed by the combination of the item number (ObjectID) and the timestamp. The following attributes are added to each node: The visual features of the item are obtained from the single-item target data. The item time series features are obtained from the single-item time series label data, including the position change rate, the morphological change amplitude, etc. The node records of each item at different time points are organized into a structured data table to form the single-item time series node data. The output single-item time series node data contains the node identifier and its attributes for use in graph relationship analysis.

[0062] Perform neighboring node relationship analysis on single item time series node data to obtain single item neighboring node relationship data; Specifically, the relationship between adjacent nodes of a single item in the time dimension is analyzed to generate connection weights between nodes. For nodes of the same item, the nodes are sorted by timestamp to determine the front and back neighbors of each node. For example, the neighbors of node t are t-1 and t+1. The relationship attributes between adjacent nodes are calculated, including: calculating the Euclidean distance of the visual feature vectors of adjacent nodes as the feature change. Calculating the difference in the coordinates of the center points of adjacent nodes as the position change. Calculating the comprehensive change value based on the weighted changes in visual and positional features. According to the size of the comprehensive change, weights are assigned to the edges between adjacent nodes: the larger the edge weight, the more drastic the node change. The smaller the edge weight, the more stable the node change. Generate a neighboring node relationship data table containing node pairs (starting point and end point) and edge weights. The output single item neighboring node relationship data includes node relationships and their weights.

[0063] A graph is constructed based on the single item neighboring node relationship data and the single item time series node data to obtain first time label graph data; Specifically, a complete time label graph is constructed based on the single item time series node data and the neighboring node relationship data. Create an empty graph data structure to store nodes and edges separately. Add each node in the single item time series node data to the graph while retaining its attributes. According to the single item neighboring node relationship data, add node pairs and their weights as edges in the graph. Structural optimization is performed on the constructed graph, such as removing isolated nodes or edges with too small weights, and retaining parts with analytical value. The first time label graph data constructed is a time-series single item time graph, which contains nodes (time points) and their connection relationships (time neighbor relationships).

[0064] The construction of multi-item time label graph is as follows: Constructing multiple item nodes according to different single item time series node data corresponding to the same single item time series label data to obtain multiple item node data; Specifically, based on the single-item time series node data, an information node containing multiple items is constructed to form multi-item node data. All item nodes at the same time point (same time series label) are extracted from the single-item time series node data and integrated into a multi-item node. For example, the multi-item node at time point t contains all items identified at time t. The attributes of the multi-item node are aggregated, including: arranging the visual feature vectors of all items by number to form a high-dimensional visual feature vector set. The relative position and spatial distribution of all items are recorded. The distance or similarity between items is counted to form a preliminary association matrix. A unique identifier is assigned to each multi-item node, which is usually composed of a time label and an item set. The output multi-item node data includes the node identifier, the included item set and its aggregated attributes.

[0065] Performing interaction relationship processing, functional relationship processing and structural relationship processing according to the multi-item node data to obtain multi-item interaction relationship data, multi-item functional relationship data and multi-item structural coupling relationship data; Specifically, the interaction behaviors between items in a multi-item node are analyzed to generate interaction relationships. For items in a multi-item node, interaction analysis is performed based on the time series attributes and spatial positions of the items. For example, items that are close to each other have interaction relationships. Items that undergo significant visual changes at the same time have common operations or interference. Interaction weights are calculated for each pair of items. For example, the closer the distance, the higher the interaction weight. The more synchronized the visual changes, the higher the interaction weight. Output multi-item interaction relationship data, including item pairs, interaction types, interaction weights, etc.

[0066] Analyze the functional attributes of items and their associations to generate functional relationships. Infer the functions of items based on their visual features (e.g., shape, material). For example, liquids and containers have combined functions, or long poles and butt-like objects have combined relationships. Detect associations between items with similar or complementary functional attributes. For example, items with similar functions may belong to the same category. Items with complementary functions may be used for specific purposes (e.g., liquid and fuel containers). Output multi-item functional relationship data, including item pairs, functional categories, functional association weights, etc.

[0067] Analyze the coupling characteristics of items in spatial structure and generate structural relationships. Analyze the position relationship of items in multi-item nodes. For example: closely arranged items have structural coupling. Items with stable spacing changes belong to the same physical structure. Calculate the structural coupling weight based on indicators such as position overlap and relative position stability. For example: the more spatial position overlap, the higher the coupling weight. The more stable the relative position, the higher the coupling weight. Output multi-item structural coupling relationship data, including item pairs, coupling strength, etc.

[0068] According to the multi-item interaction relationship data, the multi-item function relationship data and the multi-item structure coupling relationship data, a graph of multi-item node data is constructed to obtain multi-item interaction relationship graph data, multi-item function relationship graph data and multi-item structure coupling relationship graph data respectively; Specifically, three types of graphs are constructed based on multi-item relationship data. An empty graph structure (interaction relationship graph, functional relationship graph, structural coupling relationship graph) is created for each relationship. Nodes in the multi-item node data are added to the graph, retaining their attributes. According to the interaction relationship data, functional relationship data, and structural coupling relationship data, edges are added to each graph respectively, retaining the edge weights. The three graphs are optimized separately, such as removing edges with lower weights and merging highly similar nodes. Three types of relationship graph data are output, respectively describing the interaction, function, and structural relationships between multiple items.

[0069] Static multi-level graph fusion is performed based on the multi-item interaction relationship graph data, the multi-item function relationship graph data and the multi-item structure coupling relationship graph data to obtain the second time label graph data.

[0070] Specifically, the multi-item interaction relationship graph, functional relationship graph and structural coupling relationship graph are fused into a multi-level graph. The three graphs are respectively used as different levels to form an initial multi-level graph structure. The nodes in the three-level graph are aligned, for example, the same node is merged based on the node identifier (time and item set). The edge weights of the same node pairs in the three-level graph are fused. For example: the weights of the interaction, function and structural relationships can be weighted and summed according to the weight ratio. If an edge is missing in a relationship graph, a default weight value is assigned. The fused multi-level graph is output, and the attributes of the nodes and edges are fused to include the interaction, function and structural relationships of multiple items. The generated second time label graph data is a time graph that integrates multiple relationships and multiple levels for use in the subsequent layered graph fusion step.

[0071] Preferably, the layered graph fusion is specifically as follows: Constructing a time-varying layer according to the first time label graph data and the second time label graph data to obtain time-varying layer data; Specifically, the time evolution characteristics of the nodes in the first time label graph and the second time label graph are analyzed to construct a time change layer. The node set at the same time point is extracted from the first and second time label graphs to ensure the consistency of the time dimension. The changes in node features between consecutive time points are analyzed to extract time series features. For example: for a single-item graph node, the time change rate of the visual feature or position feature is calculated. For multi-item graph nodes, the time change gradient of the overall distribution or behavioral feature is calculated. Weights are assigned to the edges between each pair of consecutive time points, and the weights are based on the degree of feature change (e.g., the greater the change, the higher the weight). The time nodes and their time change edges are combined into a time change layer. The output time change layer data includes the time nodes and their time series features.

[0072] Constructing a spatial relationship layer according to the first time label graph data and the second time label graph data to obtain spatial relationship layer data; Specifically, the spatial distribution relationship of the nodes in the first and second time label graphs is analyzed to construct a spatial relationship layer. The node set in the same spatial area is extracted from the two time label graphs, and their position coordinates are recorded. The spatial distance between nodes is calculated to generate a spatial adjacency matrix. For example: for a single-item graph node, the spatial relative position between items is calculated. For multi-item graph nodes, the similarity of the spatial distribution of the group is calculated. Edge weights are generated according to the spatial adjacency matrix: the higher the edge weight, the closer the spatial relationship (for example, nodes with closer distances). The spatial nodes and their spatial relationship edges are combined into a spatial relationship layer. The output spatial relationship layer data includes spatial nodes and their spatial relationship weights.

[0073] Constructing an interaction layer according to the first time label graph data and the second time label graph data to obtain interaction layer data; Specifically, the interaction features between different nodes in the first and second time label graphs are analyzed to construct an interaction layer. The interaction features of each node are extracted, for example: the behavioral interaction of single-item nodes (such as visual feature synchronization). The group behavior of multi-item nodes (such as group movement synchronization). The edge weights are assigned based on the interaction strength between nodes. For example: if the interaction behavior strength between nodes is high (such as high overlap in time and space), the weight is high. If the interaction is weak, the weight is low. The nodes and their interaction relationship edges are combined into an interaction layer. The output interaction layer data includes the interaction nodes and their interaction relationship weights.

[0074] Perform graph attention network extraction on the time-varying layer data, the spatial relationship layer data, and the interaction layer data to obtain the time-varying layer feature data, the spatial relationship layer feature data, and the interaction layer feature data respectively; Specifically, the graph attention network is used to extract features from each layer of data to generate feature data for the time-varying layer, spatial relationship layer, and interaction layer. The node features of the time-varying layer, spatial relationship layer, and interaction layer are input into the graph attention network (GAT). The attention weight between each node and its neighbors is calculated, and the weight is dynamically assigned based on the edge weight and the importance of the node feature. The feature information of neighboring nodes is aggregated through the attention mechanism, and the feature vector of each node is updated. The feature data of each layer is generated, including time-varying features, spatial relationship features, and interaction features. The output layer feature data includes the node features of each layer and its attention weight.

[0075] According to the time-varying layer feature data, the spatial relationship layer feature data and the interactive layer feature data, the time-varying layer data, the spatial relationship layer data and the interactive layer data are connected between layers to obtain layered graph connection data; Specifically, the time-varying layer, the spatial relationship layer, and the interaction layer are connected to each other to form layered graph connection data. The data of the three layers are aligned according to the same node identifier. Cross-layer edges are established between different layers for the same node. For example: a node in the time-varying layer is connected to the corresponding node in the spatial relationship layer. A node in the spatial relationship layer is connected to the corresponding node in the interaction layer. The weights of the cross-layer edges are calculated to integrate the feature similarities of each layer. For example: if the feature similarity of the node in multiple layers is high, the edge weight is higher. The nodes and edges between all layers are recorded as layered graph connection data. The output layered graph connection data includes cross-layer edges and their weights.

[0076] The graph fusion is performed based on the hierarchical graph connection data to obtain the time label graph data.

[0077] Specifically, the layered graph connection data is fused with the feature data of each layer to generate the time label graph data. The data of the time change layer, spatial relationship layer and interaction layer as well as the layered graph connection data are integrated into a global graph. The global features of each node are calculated by aggregating the features of nodes and edges. For example: weighted average or weighted sum of the time features, spatial features and interaction features of the same node. The fused graph structure is optimized, such as removing low-weight edges and simplifying redundant nodes. The final time label graph data is generated, recording the global node features, edge weights and hierarchical information. The output time label graph data is a complete graph after the fusion of layered features to support threat analysis and auxiliary decision-making.

[0078] Preferably, the spatial feature extraction is specifically: Perform person detection based on multi-source airport security inspection data to obtain person detection data; Specifically, all human targets are identified from the airport's multi-source security data (such as surveillance videos and image sequences). Image frames are extracted from the surveillance video stream at fixed time intervals to ensure real-time and computational efficiency. Object detection technology is used to identify people in image frames and generate a bounding box and unique ID for each person. Background modeling and motion detection technology are used to eliminate non-human background interference (such as luggage and equipment) to ensure accurate detection results. The detection results in each frame of the image are organized into structured data, including the bounding box, confidence, ID, etc. of each person. The output person detection data contains all people detected in each frame of the video and their location information.

[0079] Performing person tracking on the person detection data to obtain person tracking data; Specifically, the detected characters are tracked across frames to generate a time series trajectory for each character. Characters are matched in adjacent frames, and the same character is identified by calculating the degree of overlap (IoU) or matching feature vectors (such as color, shape). A unique trajectory ID is assigned to each character, and its position changes in all time frames are recorded. For occluded or temporarily missing targets, the trajectory is completed by trajectory prediction methods to ensure trajectory continuity. Generate trajectory data for each character, including timestamp, position sequence, and trajectory ID. The output character tracking data contains the trajectory ID of each character and its position information in the time series.

[0080] Performing facial detection according to the person tracking data to obtain facial detection data; Specifically, the facial region is detected from the tracked person. The bounding box of the person tracking data is used to limit the facial detection range in each frame to reduce the computational complexity. Facial feature points (such as eyes, nose, and mouth) are detected in the local area to generate a facial region bounding box. The tilted or non-orthogonal facial region is corrected to ensure that the detected facial region is complete. The facial region information of each person at each time point is output, including the bounding box coordinates and the detection confidence. The output facial detection data contains the facial region and related attributes of each person in each frame.

[0081] Perform facial label analysis based on facial detection data to obtain facial expression data; Specifically, facial expression analysis is performed on the detected face to generate facial expression data. Key facial features (such as the curvature of the mouth corners and the position of the eyebrows) are extracted from the facial detection data. Based on the feature extraction results, facial expressions are classified into standard categories (such as happiness, tension, anger, etc.). Confidence is calculated for each expression, and low-confidence results are eliminated. The expression category and confidence of each character in each frame of the image are recorded. The output facial expression data contains the expression classification results and time series of each character.

[0082] According to the character tracking data and the facial tag data, the spatial relationship of expression changes is combined to obtain the expression trajectory correlation data; Specifically, the expression changes of the character are associated with the spatial trajectory to generate expression trajectory associated data. The character tracking data and facial expression data are aligned according to the timestamp to ensure that the expression information is consistent with the trajectory. The changing trend of the expression data in the time dimension is analyzed. For example, the frequency of change from "happy" to "nervous" is calculated. Expression change labels are added to the trajectory, such as marking the "nervous area" in the trajectory. Associated data containing expression changes and trajectory positions are generated, indicating the time and spatial position of the expression changes. The output expression trajectory associated data contains the temporal and spatial association information between expression changes and trajectories.

[0083] Perform clustering calculation based on the person tracking data to obtain clustering feature data of the person security inspection scene; Specifically, the spatial behaviors of characters are clustered to analyze the distribution and behavior patterns of people in security inspection scenarios. Behavioral features are extracted from character tracking data, including dwell time, moving distance, and complexity of moving paths. The extracted features are normalized to ensure that different features have the same dimension. Characters are clustered based on behavioral features to generate behavioral categories. For example, they are classified into "fast passers-by", "stranded persons", and "reversers". A feature description of each cluster category and a list of its personnel are generated. The output cluster feature data contains the behavior categories and feature distribution of characters in security inspection scenarios.

[0084] Extract the spatial behavior features of people based on the clustering feature data of the person security inspection scene to obtain preliminary security inspection space feature data; Specifically, the overall spatial behavior characteristics are extracted from the clustering characteristics. The distribution of people, density changes, hot spots, etc. in the clustering results are analyzed. Combined with the time dimension, the spatial behavior changes in different time periods are extracted. For example, the distribution of stranded people during peak hours. Generate spatial behavior feature data, including behavior type, hot spots, time changes, etc. The output preliminary security inspection spatial feature data contains group behavior characteristics and their spatiotemporal change information.

[0085] The preliminary security inspection space feature data is processed with respect to the spatial relationship of the person security inspection equipment to obtain the security inspection space feature data.

[0086] Specifically, the spatial relationship between people and security inspection equipment is analyzed to generate security inspection space feature data. The fixed position of the security inspection equipment and its influence range (such as baggage scanners, security inspection gates, and camera shooting areas) are marked in the monitoring scene. Combined with the trajectory of the person, the interaction relationship between the person and the equipment is analyzed. For example, the time and frequency of each person passing through the equipment is calculated. The spatial interaction features between people and equipment are extracted, such as "passing frequency", "detention time", "location where abnormal behavior occurs", etc. The relationship features between people and equipment are summarized to generate security inspection space feature data. The output security inspection space feature data contains the spatial relationship between the interaction between people's behavior and equipment, which is used to further analyze the threat level and scene optimization.

[0087] Preferably, step S3 is specifically: Step S31: Perform behavior mutation detection according to the security inspection characteristic sequence data to obtain behavior mutation point data; Specifically, detect behavioral mutation points from the security inspection feature sequence data and mark potential abnormal behavior locations. Extract key behavioral features (such as passenger stay time, changes in baggage scan item density, behavioral trajectory mutations, etc.) from the security inspection feature sequence data. Perform time series analysis on behavioral features and calculate the rate of change of features in each time period. For example, use differential or sliding window technology to capture mutation points. Set mutation thresholds for behavioral changes, for example: an abnormal increase in the rate of position change indicates a sudden stop or rapid movement. An abnormal change in baggage item density indicates that items have been removed or added. Mark the detected abnormal change points, generate behavioral mutation point data, and record the mutation time, feature type, and mutation strength. The output behavioral mutation point data includes time point, mutation type, and mutation strength.

[0088] Step S32: performing mutation behavior association according to the behavior mutation point data to obtain security inspection mutation feature association data; Specifically, the correlation between mutation behaviors is analyzed to form security inspection mutation feature association data. Based on the behavior mutation point data, the temporal and spatial correlations between multiple behavior mutations are analyzed. For example, the change in a passenger's pace may be related to the mutation of the baggage scanning result. The nervousness in the facial expression and the abnormal baggage detection appear at the same time. The association weight is calculated for each pair of mutation behaviors, including time overlap, spatial distance, feature similarity, etc. The higher the time overlap, the higher the association weight. The closer the spatial distance, the higher the association weight. According to the association weight, it is classified as strong association, medium association or weak association. For example, a highly associated mutation behavior represents a comprehensive threat event. The association pairs of mutation behaviors and their weights are recorded to form mutation feature association data. The output security inspection mutation feature association data includes mutation behavior pairs, association weights and association types.

[0089] Step S33: performing cross-modal feature matching according to the security inspection characteristic sequence data to obtain cross-modal feature matching data; Specifically, the different modal data in the security inspection feature sequence are matched to discover potential related information. The data of different modalities (such as image data, behavior trajectory data, and facial expression data) are subjected to feature standardization processing, for example, image features are converted into numerical vectors. The behavior trajectory is discretized into time point location information. The matching rules between different modal features are defined, for example, the spatial overlap between image features and behavior trajectories. The temporal synchronization between facial expression changes and behavioral mutations. The matching degree of modal features is calculated according to the matching rules, for example, the matching degree of visual features is calculated using cosine similarity. The synchronization of expression and behavior is measured using time deviation. The matching results of each pair of modal features are recorded, including the matching degree and matching type. The output cross-modal feature matching data includes modal feature pairs, matching degrees, and matching descriptions.

[0090] Step S34: Generate a multimodal data association graph based on the cross-modal feature matching data to obtain security inspection cross-modal feature association data.

[0091] Specifically, a multimodal data association graph is generated based on cross-modal feature matching data to reflect the relationship between different modalities. The features of each modality are represented as graph nodes. For example: density feature node of baggage scanning results. Feature node of facial expression change. Feature node of behavior trajectory change. Feature attributes are added to each node, such as feature type, time label, intensity value, etc. According to the cross-modal feature matching data, edges are generated for each pair of matching nodes and weights are assigned: the higher the matching degree, the greater the edge weight. The higher the time and space overlap, the greater the edge weight. Weakly associated edges are removed, and high-weight edges and important nodes are retained. A cross-modal feature association graph is generated to record nodes, edges and their attributes. The output security inspection cross-modal feature association data is a multimodal association graph for further threat assessment or decision support.

[0092] Preferably, step S4 is specifically: Step S41: performing mutation behavior threat identification according to security inspection mutation feature association data to obtain mutation behavior threat data; Specifically, based on the security inspection mutation feature association data, identify mutation behaviors that may represent security threats. Extract the categories and attributes of mutation behaviors from the mutation feature association data. For example, behavior mutations may include "abnormal stay", "changes in carried items", "reverse travel", etc. Based on the known threat behavior rule base, match whether the mutation behavior conforms to the known threat pattern. For example: abnormally long stay time and proximity to restricted areas indicate potential threats. The abnormal appearance of high-density items in luggage indicates the carrying of dangerous items. Perform a preliminary threat score on the mutation behavior, and the score is based on the mutation strength, the number of associated behaviors, and the historical data of specific threat scenarios. Mark the identified mutation behavior as a potential threat, and record the threat type, location, time, and score. The output mutation behavior threat data includes the threat type and threat score of each mutation behavior.

[0093] Step S42: quantifying the mutation threat of the mutation behavior threat data to obtain first security inspection risk data; Specifically, the mutation behavior threat data is quantified to generate overall security inspection risk data. The risk level is set according to the threat score, for example: low risk (score < 3): the behavior does not pose a significant threat. Medium risk (score 3-6): further manual review is required. High risk (score > 6): represents a direct threat. Comprehensively quantify multiple threat behaviors of the same object. For example: if passenger A's behavioral mutation and luggage mutation are both marked as high risk, the comprehensive score will be improved. Aggregate all risk scores for the same scene (such as a security inspection area) to generate a regional-level risk score. Record the quantification results in a structured form, indicating the dangerous object, risk level and corresponding behavior description. The first security inspection risk data output includes the threat object, risk level and scene risk score.

[0094] Step S43: performing cross-modal threat annotation according to the security inspection cross-modal feature association data to obtain cross-modal threat annotation data; Specifically, based on the security inspection cross-modal feature association data, potential threat features are annotated. The consistency between different modes in the cross-modal feature association graph is analyzed. For example: the luggage image shows abnormally high-density items, but the passenger declaration does not mention carrying similar items. The behavior trajectory shows retrograde, but the identity information indicates that the passenger is a special person. The abnormal points in the cross-modal data are extracted and annotated, for example: the image feature anomaly is annotated as "high-density items". The behavioral anomaly is annotated as "retrograde" or "abnormal stay". The high-weight association relationship between modalities is combined with the threat feature to further verify the accuracy of the annotation. Generate annotated threat feature data, including feature description, modal source and association strength. The output cross-modal threat annotation data includes threat feature description and modal association information.

[0095] Step S44: performing cross-modal threat analysis on the cross-modal threat annotation data to obtain second security inspection risk data.

[0096] Specifically, a comprehensive analysis is performed on the cross-modal threat annotation data to generate cross-modal overall danger data. The cross-modal annotated threat features are fused in time and space. For example, if a passenger's facial expression is tense (modality 1) and a high-density item in the luggage image (modality 2) appears at the same time point, the comprehensive threat score increases. The threat level is upgraded according to the comprehensive strength of the cross-modal features. For example, if all multi-modal features show abnormalities, the threat level is upgraded from "medium" to "high". High-risk areas are labeled according to the spatial distribution of threats. For example, a security check channel is labeled as a high-risk area. The final cross-modal danger data is generated, including threat objects, modal features, danger levels, and spatial distribution.

[0097] The output second security inspection hazard data contains the comprehensive score and distribution information of cross-modal threats for decision-making assistance.

[0098] Preferably, the present application also provides an airport security data intelligent analysis system, which is used to execute the airport security data intelligent analysis method as described above, and the airport security data intelligent analysis system includes: Airport multi-source security inspection data collection module, used to obtain airport multi-source security inspection data; The security inspection data preliminary analysis module is used to perform preliminary analysis of the security inspection data based on the airport's multi-source security inspection data to obtain security inspection characteristic sequence data; The security inspection feature association module is used to perform behavioral mutation feature association based on the security inspection feature sequence data to obtain security inspection mutation feature association data, and to perform cross-modal feature association based on the security inspection feature sequence data to obtain security inspection cross-modal feature association data; The security inspection feature risk assessment module is used to perform a mutation behavior threat level assessment based on the security inspection mutation feature association data to obtain first security inspection risk data, and to perform a cross-modal threat level assessment based on the security inspection cross-modal feature association data to obtain second security inspection risk data; The security inspection decision-making assistance module is used to perform security inspection decision-making assistance operations based on the first security inspection risk data and the second security inspection risk data.

[0099] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0100] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for intelligent analysis of airport security data, characterized in that: The following steps are involved: Step S1: Acquire airport multi-source security inspection data; Step S2: Perform a preliminary analysis of the security inspection data based on the airport multi-source security inspection data to obtain security inspection characteristic sequence data; Step S3: performing behavioral mutation feature association according to the security inspection feature sequence data to obtain security inspection mutation feature association data, and performing cross-modal feature association according to the security inspection feature sequence data to obtain security inspection cross-modal feature association data; Step S4: performing a mutation behavior threat level assessment based on the security inspection mutation feature association data to obtain first security inspection danger data, and performing a cross-modal threat level assessment based on the security inspection cross-modal feature association data to obtain second security inspection danger data; Step S5: Perform security inspection decision-making assistance operations according to the first security inspection risk data and the second security inspection risk data.

2. The method according to claim 1, characterized in that Step S1 is specifically as follows: Collect real-time airport security data by accessing the API interface of security equipment (such as baggage scanners, behavior monitoring cameras, and identity verification systems); Perform multimodal data edge processing based on the airport's real-time security inspection data to obtain pre-processed security inspection data; Assign data labels based on the pre-processed security inspection data to obtain labeled security inspection data; Merge labeled security inspection data into the cloud to generate multi-source security inspection data for airports.

3. The method according to claim 1, characterized in that Step S2 is specifically as follows: Perform multimodal data cleaning based on multi-source airport security inspection data to obtain security inspection cleansing data; Perform security inspection hazard feature extraction and space feature extraction on the security inspection cleansing data to obtain security inspection hazard feature data and security inspection space feature data respectively; The security inspection hazard feature data and the security inspection space feature data are feature fused and encoded to obtain security inspection feature sequence data.

4. The method according to claim 3, characterized in that The security inspection risk feature extraction is specifically as follows: Performing visual feature extraction on the security inspection image data in the security inspection cleaning data to obtain security inspection visual feature data; A time label graph is constructed according to the security inspection image data in the security inspection cleaning data to obtain time label graph data; Perform time feature fusion according to the time label graph data and the security inspection visual feature data to obtain the security inspection visual feature time graph data; Dangerous features are extracted from the security inspection visual feature time graph data to obtain security inspection dangerous feature data.

5. The method according to claim 4, characterized in that The time label graph is constructed as follows: Constructing a single item time label graph according to the security inspection image data in the security inspection cleaning data to obtain first time label graph data; Construct a multi-item time label graph according to the security inspection image data in the security inspection cleaning data to obtain second time label graph data; Performing layered graph fusion on the first time label graph data and the second time label graph data to obtain time label graph data; The single item time label graph is constructed as follows: Perform single-item target recognition and extraction based on the security inspection image data in the security inspection cleaning data to obtain single-item target data; Extract time series labels according to single item target data to obtain single item time series label data; Nodes are constructed according to the single item time series label data and the single item target data to obtain the single item time series node data; Perform neighboring node relationship analysis on single item time series node data to obtain single item neighboring node relationship data; A graph is constructed based on the single item neighboring node relationship data and the single item time series node data to obtain first time label graph data; The construction of multi-item time label graph is as follows: Constructing multiple item nodes according to different single item time series node data corresponding to the same single item time series label data to obtain multiple item node data; Performing interaction relationship processing, functional relationship processing and structural relationship processing according to the multi-item node data to obtain multi-item interaction relationship data, multi-item functional relationship data and multi-item structural coupling relationship data; According to the multi-item interaction relationship data, the multi-item function relationship data and the multi-item structure coupling relationship data, a graph of multi-item node data is constructed to obtain multi-item interaction relationship graph data, multi-item function relationship graph data and multi-item structure coupling relationship graph data respectively; Static multi-level graph fusion is performed based on the multi-item interaction relationship graph data, the multi-item function relationship graph data and the multi-item structure coupling relationship graph data to obtain the second time label graph data.

6. The method according to claim 5, characterized in that The specific details of layered graph fusion are: Constructing a time-varying layer according to the first time label graph data and the second time label graph data to obtain time-varying layer data; Constructing a spatial relationship layer according to the first time label graph data and the second time label graph data to obtain spatial relationship layer data; Constructing an interaction layer according to the first time label graph data and the second time label graph data to obtain interaction layer data; Perform graph attention network extraction on the time-varying layer data, the spatial relationship layer data, and the interaction layer data to obtain the time-varying layer feature data, the spatial relationship layer feature data, and the interaction layer feature data respectively; According to the time-varying layer feature data, the spatial relationship layer feature data and the interactive layer feature data, the time-varying layer data, the spatial relationship layer data and the interactive layer data are connected between layers to obtain layered graph connection data; The graph fusion is performed based on the hierarchical graph connection data to obtain the time label graph data.

7. The method according to claim 3, characterized in that The spatial feature extraction is specifically as follows: Perform person detection based on multi-source airport security inspection data to obtain person detection data; Performing person tracking on the person detection data to obtain person tracking data; Performing facial detection according to the person tracking data to obtain facial detection data; Perform facial label analysis based on facial detection data to obtain facial expression data; According to the character tracking data and the facial tag data, the spatial relationship of expression changes is combined to obtain the expression trajectory correlation data; Perform clustering calculation based on the person tracking data to obtain clustering feature data of the person security inspection scene; Extract the spatial behavior features of people based on the clustering feature data of the person security inspection scene to obtain preliminary security inspection space feature data; The preliminary security inspection space feature data is processed with respect to the spatial relationship of the person security inspection equipment to obtain the security inspection space feature data.

8. The method according to claim 1, characterized in that Step S3 is specifically as follows: Conduct behavioral mutation detection based on security inspection characteristic sequence data to obtain behavioral mutation point data; According to the behavior mutation point data, mutation behavior association is performed to obtain security inspection mutation feature association data; Perform cross-modal feature matching according to the security inspection characteristic sequence data to obtain cross-modal feature matching data; A multimodal data association graph is generated based on the cross-modal feature matching data to obtain security inspection cross-modal feature association data.

9. The method according to claim 1, characterized in that: Step S4 is specifically as follows: Perform mutation behavior threat identification based on security inspection mutation feature correlation data to obtain mutation behavior threat data; Quantify the mutation threat of the mutation behavior threat data to obtain the first security inspection danger data; Perform cross-modal threat annotation based on the security inspection cross-modal feature correlation data to obtain cross-modal threat annotation data; Perform cross-modal threat analysis on the cross-modal threat annotation data to obtain the second security inspection risk data.

10. An airport security data intelligent analysis system, characterized in that: Used to execute the airport security data intelligent analysis method as claimed in claim 1, the airport security data intelligent analysis system comprises: Airport multi-source security inspection data collection module, used to obtain airport multi-source security inspection data; The security inspection data preliminary analysis module is used to perform preliminary analysis of the security inspection data based on the airport's multi-source security inspection data to obtain security inspection characteristic sequence data; The security inspection feature association module is used to perform behavioral mutation feature association based on the security inspection feature sequence data to obtain security inspection mutation feature association data, and to perform cross-modal feature association based on the security inspection feature sequence data to obtain security inspection cross-modal feature association data; The security inspection feature risk assessment module is used to perform a mutation behavior threat level assessment based on the security inspection mutation feature association data to obtain first security inspection risk data, and to perform a cross-modal threat level assessment based on the security inspection cross-modal feature association data to obtain second security inspection risk data; The security inspection decision-making assistance module is used to perform security inspection decision-making assistance operations based on the first security inspection risk data and the second security inspection risk data.

Images