Dynamic portrait verification and inspection result generation system based on machine learning

Abstract

The invention relates to the field of artificial intelligence and image processing, in particular to a dynamic portrait verification and check result generation system based on machine learning, which comprises multi-modal data acquisition, recursive feedback and dynamic feature optimization, feature completion and correction, and spatial-temporal feature verification and correlation analysis. Performing time sequence optimization through a recurrent neural network, complementing missing features through a generative adversarial network, and correcting time-space consistency by using Bayesian reasoning; the system is combined with a Markov chain Monte Carlo method to dynamically adjust the weight of the multi-modal data, and more accurate time sequence characteristic data is generated; the space-time relevance of the feature data is analyzed through the graph neural network, a final verification result is generated, the identity verification accuracy and consistency in the dynamic scene are ensured, and the method is particularly suitable for passenger verification and security systems in high-flow dynamic scenes such as civil aviation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and image processing, and in particular to a dynamic portrait verification and inspection result generation system based on machine learning. Background Art

[0002] In the field of civil aviation, passenger identity verification and security checks are the core links to ensure flight safety and efficient passenger passage. With the rapid development of the global aviation industry, airports and aviation hubs need to handle thousands of passengers every day. The traditional manual identity verification method is inefficient and cannot meet the modern airport's demand for efficient and safe passage. Therefore, dynamic portrait verification technology based on machine learning has emerged. Its purpose is to achieve fast and accurate identity recognition of passengers through intelligent means, improve security inspection efficiency, and ensure safety.

[0003] Most existing technologies (Chinese invention patent, publication number: CN118553002B, name: Face recognition system and method based on cloud platform four-layer architecture AFC system) use convolutional neural network (CNN) in deep learning to perform face recognition in static images, or use simple frame difference to deal with dynamic scenes. However, these methods have the following shortcomings when dealing with actual dynamic portrait verification scenes: Static image recognition relies on single-frame information. When passengers move quickly or their postures change, it is impossible to effectively capture and process changes in facial features in continuous frames, resulting in unstable recognition results. Most existing technologies rely only on RGB image data. Under complex lighting conditions or short-term occlusions at airports, this single data source model is easily disturbed and cannot guarantee high-precision verification results. Existing methods fail to effectively integrate information in time series, resulting in verification results being limited to single-frame data, making it difficult to achieve cross-frame information association and completion, especially in rapidly changing dynamic scenes, where the robustness of the system is significantly reduced. Summary of the invention

[0004] In view of the many problems existing in the above-mentioned prior art, the present invention provides a dynamic portrait verification and inspection result generation system based on machine learning. The present invention optimizes the timing of the input multimodal data through recursive feedback and dynamic feature optimization modules, adjusts the dynamic weight of the data in combination with the Markov chain Monte Carlo method, uses a generative adversarial network (GAN) to complete the missing features, and performs spatiotemporal consistency correction through Bayesian reasoning; finally, the spatiotemporal correlation of the data is verified through a graph neural network to ensure the accuracy and spatiotemporal consistency of the passenger identity verification results in dynamic scenarios; the present invention can greatly improve the accuracy of dynamic portrait verification in practical applications, and ensure that the system can still quickly and accurately generate verification results in complex dynamic scenarios.

[0005] A dynamic portrait verification and inspection result generation system based on machine learning, comprising: A multimodal data acquisition module, used to collect RGB image data, infrared thermal image data, depth map data and millimeter wave profile data of passengers, and perform feature extraction and cleaning on the collected data to generate first fusion feature data; A recursive feedback and dynamic feature optimization module, which is used to perform time series optimization on the first fused feature data using a recursive neural network to generate time series optimized feature data, and calculate the dynamic weight of the collected data using a Markov chain Monte Carlo method, and perform weighted fusion on the collected data based on the dynamic weight to generate second optimized feature data; A feature completion and correction module, used to perform feature completion on the second optimized feature data using a generative adversarial network, generate completed feature data by generating missing or occluded facial features, and perform spatiotemporal consistency correction on the completed feature data based on Bayesian reasoning to generate corrected completed feature data; The spatiotemporal feature verification and association analysis module is used to construct a spatiotemporal graph based on the corrected and completed feature data, and analyze the multi-frame feature correlation of the corrected and completed feature data through a graph neural network. It combines recursive feedback to dynamically correct the temporal features and generate corrected associated feature data to ensure accurate verification of spatiotemporal consistency and dynamic behavior.

[0006] Preferably, the recursive neural network in the recursive feedback and dynamic feature optimization module generates timing optimized feature data by comparing the first fused feature data of the previous frames with the first fused feature data of the current frame. The timing optimized feature data is obtained by recursively accumulating calculations on each frame of data to ensure timing continuity and reduce the interference of noise on feature extraction.

[0007] Preferably, the dynamic weights in the recursive feedback and dynamic feature optimization module are calculated by the Markov chain Monte Carlo method, and the calculation formula is:

[0008] in, For the Dynamic weights of modalities; Representing modality In the Weight contribution in the frame; Indicates the frame number; Indicates the total number of modes; The dynamic weights are weighted and optimized for each modal feature by a recursive accumulation method.

[0009] Preferably, the recursive feedback and dynamic feature optimization module uses an attention mechanism to dynamically adjust the weights of each modal feature during the recursive feedback process, and generates second optimized feature data by adaptively adjusting the weights of important features in the time series. The adaptive adjustment is updated based on the feature weight change trend of the previous frames.

[0010] Preferably, the generative adversarial network in the feature completion and correction module uses a convolutional generative network to complete the missing facial features from the second optimized feature data. The convolutional generative network generates features for local areas of the features through a generator, and uses a discriminator to evaluate the authenticity of the generated features. The generation process is continuously generated based on feature extraction and reconstruction of the convolutional operation layer.

[0011] Preferably, the Bayesian reasoning in the feature completion and correction module is used to perform spatiotemporal correction on the completed feature data, and the calculation formula of the correction process is:

[0012] in, is the posterior probability of the completed feature data in the current frame; is the feature likelihood of the previous frames; is the prior probability of completing the feature data; is the probability of observing the data.

[0013] The correction process adjusts the spatiotemporal consistency of the features of the current frame according to the inference results of the previous frames.

[0014] Preferably, the Bayesian reasoning ensures the continuity of the completed feature data of the current frame in the time series and eliminates feature mutations caused by missing data or noise by performing continuous reasoning calculations on the completed feature data of previous frames.

[0015] Preferably, the spatiotemporal feature verification and association analysis module performs aggregation analysis on node features in the spatiotemporal graph through a graph convolutional network, wherein the nodes represent facial feature information of each frame, and the edges represent feature changes between adjacent frames. The graph convolutional network analyzes the spatiotemporal correlation between adjacent frames through an aggregation operation of neighborhood features to generate spatiotemporal correlation feature data.

[0016] Preferably, the spatiotemporal feature verification and association analysis module is combined with a recursive feedback mechanism, and the spatiotemporal association data of adjacent frames are dynamically corrected, and the recursive feedback is used to cumulatively calculate the changes in node features.

[0017] Preferably, the corrected associated feature data generated by the spatiotemporal feature verification and association analysis module is used to verify the dynamic behavior and facial features of the passenger. The corrected associated feature data ensures the consistency of facial features and behaviors in dynamic scenes, and further optimizes the accuracy of dynamic behavior verification through recursive correction.

[0018] Compared with the prior art, the advantages and beneficial effects of the present invention are: The present invention optimizes time series data through recursive neural networks to generate more accurate time series features, solving the problem of feature data distortion and noise interference in dynamic scenes; Based on multimodal data, the present invention dynamically adjusts the weight of each modal data, improves the comprehensive processing capability of different data sources, and ensures the effectiveness of data fusion in complex environments; The present invention generates missing or obscured facial features, so that even when some data is missing, a complete portrait verification result can be obtained, which greatly improves the recognition accuracy in dynamic scenes; The present invention ensures the temporal continuity and consistency of data by performing spatiotemporal correction on the complementary features, thereby further improving the reliability of dynamic behavior verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a structural block diagram of the present invention; Figure 2 This is a structural diagram of the recursive feedback and dynamic feature optimization module of the present invention; Figure 3 This is a structural diagram of the feature completion and correction module of the present invention; Figure 4 This is a structural diagram of the spatiotemporal feature verification and association analysis module of the present invention. DETAILED DESCRIPTION

[0020] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure.

[0021] like Figure 1 As shown, a dynamic portrait verification and inspection result generation system based on machine learning includes: A multimodal data acquisition module, used to collect RGB image data, infrared thermal image data, depth map data and millimeter wave profile data of passengers, and perform feature extraction and cleaning on the collected data to generate first fusion feature data; The multimodal data acquisition module is designed to collect information of passengers in different dimensions through the collaborative work of multiple sensors to support dynamic portrait verification and the generation of inspection results. In actual application scenarios in the civil aviation field, when passengers pass through the security check channel, the system simultaneously collects the passengers' RGB image data, infrared thermal image data, depth map data and millimeter wave profile data through RGB cameras, infrared thermal image sensors, depth sensors and millimeter wave radars. These data can accurately capture the facial details, body contours and dynamic behavior characteristics of passengers, ensuring the recognition accuracy and robustness of the system under different environmental conditions.

[0022] The collection of multimodal data is based on the principles and technologies of different sensors. First, RGB image data captures the color image of the passenger through the RGB camera, providing basic facial visual information. This information is the core data source for facial recognition, which can not only be used for static facial feature extraction, but also dynamically capture facial micro-expressions and posture changes. Secondly, infrared thermal image data obtains the temperature distribution information of the passenger's skin through infrared sensors, which can continue to play a role in low-light environments, especially when detecting camouflage or facial occlusion (such as passengers wearing masks or camouflage materials). This thermal image data can detect abnormal temperature distribution on the face, thereby enhancing the system's ability to identify potential threats. Depth map data is captured by depth sensors, providing three-dimensional structural information of the passenger's face and body through optical ranging or structured light technology, which is particularly suitable for rapid capture and processing in dynamic scenes, so that the system can still accurately identify the passenger's facial structure when the passenger passes through the security inspection area quickly. Millimeter wave contour data is obtained by millimeter wave radar, which constructs the passenger's body contour by emitting millimeter wave signals and analyzing the reflected waveform. This type of data performs well in detecting hidden objects worn under clothing and can effectively assist in security checks, especially when detecting whether passengers are carrying prohibited items or hidden objects, providing stronger perspective than other sensors.

[0023] In the application of the present invention, the above sensors generate RGB image data, infrared thermal map data, depth map data and millimeter wave profile data respectively, which are fused and processed in multimodal manner to generate first fused feature data. The core of feature extraction is to extract distinguishing features from each type of data, such as extracting facial geometry from RGB data, extracting temperature distribution features from infrared data, extracting three-dimensional structural information from depth map data, and extracting human body contours and object detection features from millimeter wave data. The feature extraction of each modal data is processed by a strict algorithm to convert the raw data into an operable feature vector, and then in the feature fusion stage, weighted fusion is performed through a recursive neural network (RNN) or a convolutional neural network (CNN). Specifically, the fusion process is based on the time series and feature weights of each modal data, and by dynamically adjusting the contribution of each modality, the system ensures that the most discernible modality is given priority in different environments. For example, under poor light conditions, the weights of infrared and millimeter wave data will be increased accordingly, thereby ensuring that the system can still accurately perform dynamic portrait verification.

[0024] Through the fusion processing of this multimodal data, the present invention significantly improves the accuracy and anti-interference ability of dynamic portrait verification in security inspection applications in the civil aviation field. In practical applications, when passengers quickly pass through the security inspection channel, the system can capture facial expressions through RGB images, detect whether there is facial disguise through infrared thermal images, accurately reconstruct the three-dimensional structure of the face through depth maps, and detect whether the passenger is carrying prohibited items through millimeter wave radar. These sensors work together to form a stable multimodal data acquisition network, which enables the system to not only maintain a high recognition accuracy under various complex conditions, but also effectively deal with adverse factors such as occlusion, low light and rapid movement.

[0025] like Figure 2 As shown, the recursive feedback and dynamic feature optimization module is used to use a recursive neural network to perform time series optimization on the first fused feature data to generate time series optimized feature data, and calculate the dynamic weight of the collected data through the Markov chain Monte Carlo method, and perform weighted fusion on the collected data based on the dynamic weight to generate second optimized feature data; The core role of the recursive feedback and dynamic feature optimization module in the present invention is to optimize the timing of the first fused feature data through a recursive neural network (RNN), and use the Markov chain Monte Carlo method to calculate the dynamic weight of the data to achieve weighted fusion of multimodal data, thereby generating the second optimized feature data. In the application of civil aviation, especially in scenes with fast passenger movement, changing facial expressions or complex external environments, the role of this module is particularly important, because it can not only dynamically adjust the weights of each modal feature, but also optimize the timing features to ensure the accuracy and robustness of dynamic portrait verification.

[0026] Recurrent neural network (RNN) is the core tool of this module for processing time series data. It can capture the changes of time series features by recursively processing multiple frames of data. Specifically, when passengers pass through the security check channel, the system will continue to collect multimodal data of their faces and bodies (such as RGB images, infrared images, depth maps, and millimeter wave profile data), which form a time series. RNN processes this data frame by frame in a recursive manner, combines the information of the previous frames with the current frame, and generates time-series optimized feature data. This time-series optimization process can effectively reduce the impact of passengers' rapid movements or sudden changes in facial expressions, ensuring that the system can capture stable feature expressions of passengers. Compared with traditional single-frame analysis methods, the advantage of RNN is that it can use historical data to reasonably predict and correct the features of the current frame, making the system more stable when dealing with dynamic changes.

[0027] After generating the time-series optimized feature data, the system needs to further perform weighted fusion on the data of different modalities, and this process uses the Markov chain Monte Carlo method. The core of this method is to obtain the dynamic weight of each modality in the current frame by probabilistically calculating the contribution of each modality in each frame of data. The introduction of the Markov chain Monte Carlo method enables the system to dynamically adjust the weights of different modalities according to factors such as the environment and data quality, thereby ensuring that the contribution of each data in the fusion process is optimal. Taking the civil aviation security inspection scene as an example, when the lighting conditions are poor, the weight of RGB image data may decrease, while the weight of infrared thermal map data and millimeter wave profile data will increase, because the latter can provide more reliable information in low-light environments. Similarly, when the passenger's face is partially obscured, the weight of the depth map and millimeter wave data will increase to ensure that the system can still accurately capture the key features of the passenger.

[0028] Through the calculation of the above dynamic weights, the system performs weighted fusion of the features of each modality to generate the second optimized feature data. The key to this process is that the system not only considers the characteristic performance of each modality in the current frame, but also combines the weight change trend of the previous frames through the feedback mechanism of the recursive neural network, thereby making the weighted fusion process more stable and accurate. The fused second optimized feature data is an important input for the system to perform dynamic portrait verification. It can effectively integrate multimodal information and provide a highly accurate and robust feature expression, so that the subsequent verification steps can still maintain a high recognition rate in complex dynamic scenes.

[0029] This module has shown remarkable results in the application of civil aviation. Through recursive feedback and dynamic feature optimization, the system can maintain efficient and stable recognition effects when dealing with dynamic passenger behaviors and complex scenes. Especially when passengers pass through security checks quickly or their faces are partially blocked, the module can ensure the optimal fusion of multimodal data through recursive optimization of timing features and adaptive adjustment of dynamic weights. This optimization process not only improves the accuracy of recognition, but also enhances the system's anti-interference ability, avoiding recognition errors caused by the absence or failure of single modal data.

[0030] For example, when a passenger passes through security checks, some modal data may not be reliable due to changes in light, facial expressions, or dynamic occlusion. At this time, the system will automatically optimize the timing features through a recursive feedback mechanism so that these fluctuating or missing features will not have too much negative impact on the overall recognition. At the same time, the dynamic weights calculated by the Markov Chain Monte Carlo method can adjust the fusion method of each modal data in real time to ensure that the system can make full use of the most reliable feature sources. For example, when the RGB image is blurred due to rapid movement of passengers, infrared and millimeter wave data will play a greater role, ensuring that the system can still stably identify the dynamic behavior and facial features of passengers.

[0031] Preferably, the recursive neural network in the recursive feedback and dynamic feature optimization module generates timing optimized feature data by comparing the first fused feature data of the previous frames with the first fused feature data of the current frame. The timing optimized feature data is obtained by recursively accumulating calculations on each frame of data to ensure timing continuity and reduce the interference of noise on feature extraction.

[0032] In the present invention, the core task of the recursive feedback and dynamic feature optimization module is to optimize the first fusion feature data in time series through a recursive neural network (RNN) to generate time series optimized feature data and ensure the continuity and stability of the time series features when processing dynamic changes. The application of this module in the civil aviation field, especially in the scenario of dynamic portrait verification and inspection result generation, can effectively cope with the challenges of rapid passenger movement, posture changes and external environmental interference. Its core principle is to recursively compare and optimize the data of the current frame with the previous frames to ensure the consistency of each frame of data in the time series, while reducing the impact of noise on system feature extraction.

[0033] The main function of the recursive neural network (RNN) is to process time series data, combining the historical features of the previous frames with the data of the current frame through a recursive structure to generate more stable and optimized time series optimized feature data. Specifically, the RNN reads the first fused feature data frame by frame through its internal hidden layer state, and recursively transmits the feature information of each frame. When passengers pass through security checks quickly, each frame of data captured by the system may fluctuate due to the influence of light, angle or noise, and the RNN can predict the feature trend of the current frame by accumulating and comparing the data of the previous frames, so as to correct the current frame and ensure the continuity of the time series features. For example, when a passenger's face is suddenly partially blocked, the RNN can use the feature information of the previous frames for compensation and prediction to generate a more stable feature representation. This mechanism enables the system to maintain stable recognition of passenger features when dealing with dynamic changes, without affecting the overall recognition effect due to the abnormality of one or two frames.

[0034] The key to this recursive processing is to perform recursive cumulative calculations on the data of each frame. In the processing of each frame, RNN not only considers the feature data of the current frame, but also weightedly combines the information of the previous frames to generate a continuous time series feature. This cumulative calculation not only enhances the stability of the features of each frame, but also effectively reduces the interference caused by noise. For example, during the security check, the image quality of some frames may deteriorate due to slight changes in posture or changes in ambient light. The system can smooth these short-term fluctuations through this recursive cumulative calculation, ensuring that the overall feature expression of the system is more continuous and stable.

[0035] To better understand its effect in practical applications, an example can be used. When a passenger walks quickly through the security check area, the system continuously collects multiple frames of facial data. At certain moments, a single frame of data may not accurately capture the complete facial features due to a slight turn of the passenger's head or interference from shadows. In this case, the RNN automatically predicts and corrects the features of the current frame by recursively accumulating facial information from the previous frames. This method ensures that the system can still generate stable feature data even under extreme conditions, avoiding recognition failures due to incomplete or distorted single frame data. This is particularly important in the field of civil aviation, because passengers often pass through the security check channel quickly in a short period of time, and the system needs to obtain high-quality feature data and complete verification within a limited time window. Through the timing optimization of recursive neural networks, the system can better cope with these challenges, ensuring that the passenger's feature data is still continuous and reliable even in dynamic scenes.

[0036] The recursive feedback and dynamic feature optimization module shows significant advantages in processing dynamic portrait verification, especially when dealing with dynamic passenger behavior, complex scene changes and external interference. First, through the optimization of time series features by recursive neural networks, the system can effectively integrate the features of continuous frames to ensure that the facial features of passengers can still be accurately identified in rapidly changing scenes. Secondly, recursive cumulative calculations further enhance the system's ability to resist short-term noise and data fluctuations. For example, when a passenger turns his head quickly or his face is partially obscured, the system can still maintain high-quality feature generation and recognition. In addition, the introduction of RNN greatly improves the robustness of the system, enabling it to always maintain efficient recognition capabilities when facing different passengers, complex lighting and dynamic changes.

[0037] For example, when a passenger quickly walks through the security check channel, the system captures the facial features of multiple consecutive frames through a recursive neural network. If a frame causes data distortion due to lighting or angle problems, the system can automatically use the features of the previous frames to compensate and optimize, ensuring that the final generated time-optimized feature data can accurately reflect the passenger's true facial features. This optimization method not only improves the recognition accuracy of the system, but also significantly reduces the risk of missed detection and false detection, providing more efficient and safer protection for the security inspection process in the civil aviation field.

[0038] Preferably, the dynamic weights in the recursive feedback and dynamic feature optimization module are calculated by the Markov chain Monte Carlo method, and the calculation formula is:

[0039] in, For the Dynamic weights of modalities; Representing modality In the Weight contribution in the frame; Indicates the frame number; Indicates the total number of modes; The dynamic weights are weighted and optimized for each modal feature by a recursive accumulation method.

[0040] The dynamic weight calculation in the recursive feedback and dynamic feature optimization module is one of the core steps of the present invention. Its purpose is to enable the system to dynamically adjust the importance of each modality in feature optimization according to the contribution of each modality by weighted fusion of multimodal data. Specifically, during the dynamic portrait verification process, the system will face complex situations such as rapid movement of passengers and changes in ambient lighting. At this time, the modal data collected by different sensors may show large differences at certain moments. In order to ensure the optimal fusion of multimodal data, the present invention uses the Markov chain Monte Carlo method to calculate the dynamic weight of each modality, thereby optimizing the feature fusion process and ensuring that the system can adjust the modal weights in real time to cope with these changes.

[0041] The core role of the Markov Chain Monte Carlo method in the present invention is to adaptively calculate the dynamic weights of multimodal data. The basic principle is to calculate the contribution of each modality in the time series through recursively collected data, and determine the dynamic weight of the modality based on these contributions. The introduction of this method enables the system to avoid keeping a fixed weight for each modality in the entire time series when processing multimodal data, but to dynamically adjust the weight according to the performance of different modalities in each frame.

[0042] The core idea of ​​this formula is to calculate the contribution of each mode in different frames (i.e. ), to determine the importance of each modality in the entire time series. In specific implementation, the system first extracts features from the modal data of each frame to obtain the initial weight contribution of each modality. Then, through recursive accumulation in the time series, the system calculates the total contribution of each modality. , and compared with the contribution of other modes , and finally generate the dynamic weight of each mode This weight value is used in the subsequent weighted fusion process to ensure that the fusion of multimodal data is based on the actual performance of each frame of data and the dynamic changes of the current environment.

[0043] In civil aviation applications, the calculation of dynamic weights provides flexibility and adaptability to the system. For example, when passengers pass through security checks, the RGB camera, infrared thermal image sensor, depth sensor, and millimeter wave radar work simultaneously, and the collected modal data may show instability due to changes in the environment or passenger behavior. Suppose at a certain moment, the light suddenly dims, the data quality of the RGB image decreases, while the quality of the infrared thermal image and millimeter wave data is relatively high. At this time, the dynamic weight calculated by the Markov Chain Monte Carlo method can adaptively increase the weight of the infrared and millimeter wave data and reduce the weight of the RGB data, thereby ensuring that the feature expression after the fusion of multimodal data is still accurate.

[0044] This dynamic weight calculation method can also effectively cope with the fluctuations in modal data that may occur when passengers quickly pass through the security check channel. For example, the rapid movement of passengers may cause poor modal data quality in some frames, such as the depth map may be distorted due to the large distance, while the millimeter wave data can accurately capture the passenger's body contour. In this case, the dynamic weight calculated by the system will give priority to the weight of the millimeter wave data, making the final feature data more reliable and accurate.

[0045] Take the actual scenario of passengers passing through security check as an example. Assume that the system simultaneously obtains data from the RGB camera, infrared thermal image, depth sensor, and millimeter wave radar. At a certain moment, the passenger's face is blurred in the RGB image due to the angle problem, and the infrared thermal image partially loses data due to occlusion, while the millimeter wave data can still accurately capture the passenger's body contour. In this case, the system will first obtain the weighted contribution value of each frame modality through recursive processing. The Markov Chain Monte Carlo method is used to calculate the dynamic weight of each modality. After calculation through the formula, the system will dynamically increase the weight of the millimeter wave data and reduce the weight of the RGB and infrared data to ensure that the final fused feature data has the best performance. This method not only improves the accuracy of the system, but also effectively reduces the risk of failure or quality degradation of single modality data.

[0046] Preferably, the recursive feedback and dynamic feature optimization module uses an attention mechanism to dynamically adjust the weights of each modal feature during the recursive feedback process, and generates second optimized feature data by adaptively adjusting the weights of important features in the time series. The adaptive adjustment is updated based on the feature weight change trend of the previous frames.

[0047] In the present invention, the recursive feedback and dynamic feature optimization module uses an attention mechanism to dynamically adjust the weights of each modal feature during the recursive feedback process. Specifically, the introduction of the attention mechanism enables the system to focus on important features in the time series when processing multimodal data, and to adaptively adjust the weights of these features to generate a second optimized feature data that is more robust and accurate. In the application scenarios of the civil aviation field, this mechanism is highly adaptable to handling dynamic changes in passengers, especially in complex situations such as passengers passing through security checks quickly, uneven lighting or partial occlusion. It can ensure that the system always pays attention to the most critical feature data, thereby improving the accuracy of dynamic portrait verification.

[0048] The attention mechanism is a method commonly used in deep learning. It emphasizes important features and weakens unimportant parts by adaptively assigning different weights to different parts of the input data. In the present invention, the role of the attention mechanism is reflected in the dynamic adjustment of the weights of different modal features in the time series. When multimodal data is processed by a recursive neural network (RNN), the attention mechanism automatically determines which features are more important at the current moment based on the feature performance of each frame, and assigns these features higher weights. Unlike the traditional fixed weight method, the attention mechanism can dynamically adjust the modal feature weights of the current frame based on the feature weight change trend of the previous frames, making the expression of feature data in the time series more accurate and flexible.

[0049] In terms of specific implementation, the attention mechanism first generates preliminary feature weights for each frame through feature extraction. These weights are not static, but are dynamically adjusted over time and with the weight change trend of the previous frames. For example, the system may detect that some modal features of the passenger's face change dramatically in the first few frames. For example, when the lighting conditions change or the facial expression changes, the importance of RGB data may decrease, while the importance of infrared or millimeter wave data may increase. In this case, the attention mechanism decides whether to give higher weights to these modal features in the current frame by analyzing the weight change trend of the previous frames of data. This adaptive adjustment process ensures that the system can always focus on the most valuable features in dynamically changing scenes and avoid the influence of irrelevant information on the final feature data.

[0050] To ensure the efficiency of these dynamic adjustments, the system continuously optimizes the timing of features through a recursive feedback mechanism. The features of each frame are not only based on the data of the current frame, but also refer to the trend of feature weight changes in the previous frames. The attention mechanism tracks and updates these changing trends through recursive feedback to generate the second optimized feature data. For example, when passengers pass through security checks quickly, the features of the previous frames may show large fluctuations. At this time, the system quickly adjusts the weights of each mode through the attention mechanism to ensure that the important features of the current frame receive more attention and reduce the impact of noise data.

[0051] In the dynamic scenario of civil aviation security inspection, passenger behavior and environmental conditions will change frequently, which requires the system to have adaptive feature processing capabilities. By introducing the attention mechanism, the system can adaptively and dynamically adjust the modal features according to the actual situation, thereby improving the accuracy of recognition. For example, when a passenger walks quickly through the security inspection area, the image captured by the RGB camera may be blurred due to changes in light. At this time, the attention mechanism will reduce the weight of the RGB data and increase the weight of the infrared or millimeter wave data to ensure that the system can still accurately capture the facial or contour features of the passenger. Similarly, if the facial features of the previous frames are occluded, the system can use recursive feedback and the attention mechanism to dynamically adjust the contribution of the modality to avoid occlusion affecting the accurate verification of the passenger's identity.

[0052] The advantage of this mechanism is its high flexibility and adaptability. In a complex civil aviation environment, dynamic behaviors of passengers, such as posture changes, fluctuations in lighting conditions, and even partial facial occlusion, can cause single-modal data to fail. However, through the attention mechanism, the system is able to dynamically assign weights to modal features to ensure that important data is always prioritized. This adaptive adjustment not only improves the accuracy of dynamic portrait verification, but also effectively enhances the system's ability to cope with various complex situations.

[0053] Suppose in actual applications, when a passenger passes through security checks, the image captured by the RGB camera becomes partially blurred due to light changes, but the infrared thermal image and millimeter wave data are still relatively clear. At this point, the system's recursive feedback mechanism will analyze the quality of the RGB data through the data of the previous few frames, and dynamically adjust its weight through the attention mechanism. Specifically, the system will automatically reduce the weight of the RGB image and increase the weight of the infrared and millimeter wave data to ensure that the final generated second optimized feature data is still accurate and reliable. The attention mechanism plays an important role in this process. Through the flexible adjustment of different modal features, it enables the system to adaptively respond to data fluctuations in complex scenes and ensure that the dynamic portrait verification of passengers is not interfered with by external factors.

[0054] like Figure 3 As shown, the feature completion and correction module is used to perform feature completion on the second optimized feature data using a generative adversarial network, generate the completed feature data by generating missing or occluded facial features, and perform spatiotemporal consistency correction on the completed feature data based on Bayesian reasoning to generate corrected completed feature data; The feature completion and correction module plays a vital role in the present invention, especially when some features of the passenger's face are missing or blocked, the module uses the generative adversarial network (GAN) and Bayesian reasoning technology to complete and correct the data to generate complete and consistent feature data. This is of special significance for dynamic portrait verification in the civil aviation field, and can ensure that in the actual security inspection process, even if the passenger's face is partially blocked or some features are lost due to posture, the system can still guarantee high-quality facial verification results through the generation and correction mechanism.

[0055] Generative adversarial network (GAN) is the core technology of this module, which is used to complete features in the case of missing or occluded features. GAN consists of two main parts: generator and discriminator. The generator is responsible for generating missing facial features, while the discriminator drives the generator to generate more realistic feature data by judging whether the generated features are real. Specifically in the present invention, when the system detects that there are missing or occluded parts of the face in the second optimized feature data, the generator of GAN will generate the missing part of the features based on the existing feature data. This process relies on the generator's ability to learn facial features. It can infer the shape, color and other details of the missing part from the surrounding known features. Through continuous interaction with the discriminator, the generator will gradually optimize the generation results to ensure that the completed facial features are highly consistent with the real data.

[0056] In civil aviation scenarios, when passengers quickly pass through security checks, some occluded or incomplete feature data often appear. For example, passengers may turn their heads, wear glasses, or be affected by changes in lighting, which causes some facial features to be unable to be fully captured by RGB cameras or depth sensors. At this time, GAN can complete the occluded or missing areas by learning the feature information of other known parts, ensuring that the final generated completed feature data is complete enough to support subsequent verification.

[0057] After completion, the system also needs to correct the spatiotemporal consistency of these supplementary features to ensure that the generated features are not only credible in the current frame, but also consistent throughout the entire time series. To this end, the present invention introduces Bayesian reasoning to achieve spatiotemporal consistency correction. Bayesian reasoning can reasonably infer the supplementary features of the current frame based on the data of the previous frames through the combination of prior probability and observation data, ensuring that the supplementary features are consistent with other features in the time series. The correction mechanism of Bayesian reasoning enables the system to infer and update the features of each frame, ensuring that the generated features are not only visually reasonable, but also consistent with the continuity in the time series. For example, when a passenger is in dynamic motion, because the facial features of a certain frame are partially obscured, after the system is completed by GAN, it will use Bayesian reasoning to infer a reasonable supplementary feature based on the feature changes of the previous frames, so that its performance in the entire time series is more natural and consistent.

[0058] Through the feature completion and correction module, the system can greatly improve the robustness and accuracy of dynamic portrait verification. In the security inspection scenarios in the civil aviation field, passengers often lose some facial features due to various reasons, such as turning their heads, occlusion, or unsatisfactory equipment acquisition angles. This situation may seriously affect the recognition rate of traditional portrait recognition systems. The feature completion and correction module in the present invention can complete the missing facial features through a generative adversarial network, ensuring that even in the case of incomplete faces, the system can still obtain complete facial feature data and perform accurate verification.

[0059] For example, when a passenger's face is partially obscured due to wearing a hat or turning his head, GAN can generate complete facial data based on the unobstructed facial features of the passenger, and Bayesian reasoning will correct the generated features based on prior information to ensure that they meet the continuity and consistency of the entire time series. This method can not only effectively compensate for the deficiencies in a single frame of data, but also ensure that the generated data will not conflict with other frames in the time series through the correction mechanism.

[0060] Suppose that when a passenger passes through security check, part of his face is blocked due to turning his head, and the facial image obtained by the system through the RGB camera has obvious omissions. At this time, the generator of the generative adversarial network will infer the outline and details of the missing part based on the remaining known facial features (such as eyes, nose, etc.) to generate a complete facial feature. At the same time, the discriminator in GAN will discriminate the generated features, driving the generator to continuously optimize the generated results to ensure that the generated facial features are highly matched with the real features. Then, the system uses Bayesian reasoning to correct the spatiotemporal consistency of these completed features, and infers a completion result that is logical in the time series based on the facial features of the previous frames and the dynamic features of the current frame. The final generated corrected and completed feature data can not only maintain visual authenticity, but also ensure its coherence in the time series.

[0061] Preferably, the generative adversarial network in the feature completion and correction module uses a convolutional generative network to complete the missing facial features from the second optimized feature data. The convolutional generative network generates features for local areas of the features through a generator, and uses a discriminator to evaluate the authenticity of the generated features. The generation process is continuously generated based on feature extraction and reconstruction of the convolutional operation layer.

[0062] In the present invention, the generative adversarial network (GAN) in the feature completion and correction module completes the missing facial features from the second optimized feature data through the convolutional generative network, ensuring that the system can fully model and verify the facial features of passengers in dynamic scenes. The working principle of this module is based on the feature extraction and reconstruction of the convolution operation layer. The generator gradually generates the missing facial features through the convolutional network, and the discriminator is used to evaluate the authenticity of the generated features. This process is particularly suitable for dynamic portrait verification in the civil aviation field, especially when the passenger's face is partially occluded or the features are lost, which can ensure that the system can still perform efficient and accurate facial recognition.

[0063] Generative adversarial networks (GANs) consist of two main parts: a generator and a discriminator. In the present invention, the core architecture of the generator is based on a convolutional generative network (CGAN), which uses convolutional layers to extract and generate features. Specifically, when the system detects that certain facial features are occluded or missing, the generator gradually generates the missing facial features based on the existing second optimized feature data. The generator first extracts information from known feature areas and expands and reconstructs this information through convolutional operation layers to simulate the missing facial features. The generated features are reconstructed through layers of convolutional networks, so that the missing parts not only match the existing facial features, but also naturally transition to the entire facial structure. This generation method is not only limited to static feature completion, but can also dynamically track the changes in the passenger's face in a time series, so that the generated features are highly consistent with the passenger's dynamic behavior.

[0064] The features generated by the generator are not static. They are evaluated by the discriminator to ensure that the generated features are realistic enough. The task of the discriminator is to compare the generated facial features with the real facial feature data to determine whether the generated features are realistic enough. In specific implementation, the discriminator analyzes the generated features layer by layer through a convolutional neural network (CNN) to evaluate its details, texture, and continuity with surrounding features. When the generated features are not realistic enough, the discriminator will feedback the results to the generator, and the generator will continuously optimize the generation process based on the feedback from the discriminator until the generated features are highly similar to the real features. This adversarial process ensures that the final generated facial features are not only highly realistic in appearance, but also can be integrated into the overall facial structure to maintain the consistency of facial features.

[0065] The advantage of the convolutional generative network lies in its powerful feature extraction capability. Through multiple layers of convolution operations, the network is able to capture low-level features (such as edges, contours) and high-level features (such as facial details, textures) in facial images, thereby gradually improving the detailed performance of the generated features during the generation process. For example, when the generator detects that the passenger's face is occluded, it can first extract similar features from the non-occluded part, and generate the occluded area, such as the facial contour, eyes or mouth, through the convolution layer. With the feedback from the discriminator, the generator can gradually optimize these generated features so that they are highly consistent with the actual facial structure.

[0066] In applications in the civil aviation field, dynamic behavior and facial occlusion of passengers are common phenomena, especially when going through security checks, passengers may wear glasses, hats, masks and other items that partially occlude their faces, or lose facial features due to head rotation. Traditional facial recognition technology may have recognition errors or incomplete features when dealing with these situations. The generative adversarial network in the present invention can complete facial features through a convolutional generative network in these situations, ensuring that even under unfavorable conditions, the system can still obtain complete facial feature data for accurate verification.

[0067] For example, when a passenger wears a mask, the system can automatically generate the mouth and lower jaw covered by the mask based on the passenger's upper facial features through the convolutional generation network. The generator will first make a preliminary guess based on the known features of the passenger's eyes, nose and other areas, and then gradually generate the lower facial area that matches these features. The generation process is continuously optimized through the feedback of the discriminator to ensure that the generated mouth and lower jaw not only visually match the real face, but also can naturally transition with the passenger's facial contour. In this way, even under complex occlusion conditions, the system can still perform efficient facial recognition to ensure the smooth progress of the security inspection process.

[0068] In addition, the generative adversarial network can not only complete static facial features, but also handle feature loss caused by dynamic changes in the passenger's face. For example, a passenger turns his head quickly when passing through security check, resulting in the loss of facial features in some frames. The system can extract information from the features of adjacent frames through a convolutional generative network and generate the missing parts in the current frame. The generation process is based on the continuity of the time series, so that the generated features can be consistent with the dynamic behavior of the passenger, ensuring the consistency and stability of facial verification.

[0069] Assume that the upper half of a passenger's face is partially blocked by a hat when passing through security check, and the facial image collected by the system through the RGB camera is incomplete. At this time, the generator of the convolutional generative network will first extract key information from the passenger's lower facial features, such as the contours of the mouth, jaw and other parts. Based on these known feature information, the generator will gradually generate the features of the upper half of the face blocked by the hat, including the forehead, eyes and other areas. The generation process uses multiple layers of convolution operations, and the generator can gradually build high-level features (such as eye shape, eyebrow texture) from low-level features (such as edges and contours). When the generated features are verified and optimized by the discriminator, the system finally generates a complete and realistic facial feature data, which can ensure that even in the case of occlusion, the system can still accurately complete the dynamic portrait verification of passengers.

[0070] Preferably, the Bayesian reasoning in the feature completion and correction module is used to perform spatiotemporal correction on the completed feature data, and the calculation formula of the correction process is:

[0071] in, is the posterior probability of the completed feature data in the current frame; is the feature likelihood of the previous frames; is the prior probability of completing the feature data; is the probability of observing the data.

[0072] The correction process adjusts the spatiotemporal consistency of the features of the current frame according to the inference results of the previous frames.

[0073] In the present invention, Bayesian reasoning in the feature completion and correction module is used to perform spatiotemporal consistency correction on the generated completed feature data, ensuring that the features generated in a dynamic environment are not only reasonable and true in the current frame, but also coherent and consistent throughout the time series. This module is crucial in dynamic portrait verification systems, especially in civil aviation applications, as it ensures that the system can still perform effective correction and optimization through the context of the time series when facing rapid movement of passengers, changes in facial expressions, and environmental interference.

[0074] Bayesian reasoning is a technique based on probabilistic reasoning that can combine prior information and currently observed data to infer the posterior probability of an event. In the present invention, the role of Bayesian reasoning is to analyze the feature data of the previous frames to infer whether the complement features in the current frame are consistent with the previous features, and adjust the complement features accordingly. This reasoning mechanism can effectively handle dynamic changes in time series, ensuring that even if facial features are blocked or lost in some frames, the system can generate facial features that are consistent with spatiotemporal consistency.

[0075] In practical applications, Bayesian reasoning continuously infers the complement features of each frame, combines the observation information of the previous frames (likelihood probability of feature data) and the complement features of the current frame, infers the rationality of the features of the current frame, and fine-tunes them. For example, suppose that in a certain frame, the passenger's facial features are blurred or missing due to rapid movement. The system completes these missing features through a generative adversarial network, but in order to ensure that the generated features can be consistent with the previous features, Bayesian reasoning will combine the facial feature data of the previous frames to infer the optimal form of the generated features, so that the completed facial features are still consistent in the time series.

[0076] The correction process of Bayesian reasoning not only relies on the generation result of the current frame, but also ensures that the facial features in the entire time series are coherent and conform to the logic of dynamic changes based on the feature change trend of the previous frames of data. This means that when passengers show rapid changes in facial expressions in dynamic scenes or lose some features due to occlusion, the system can perform effective spatiotemporal consistency correction through Bayesian reasoning, reduce the inconsistency between features, and thus improve the overall verification accuracy and robustness of the system.

[0077] The introduction of Bayesian reasoning significantly improves the robustness of the system in handling dynamic portrait verification, especially in the face of dynamically changing scenes, and can ensure that the generated facial feature data remains consistent in the time series. For example, during civil aviation security checks, passengers may experience unstable feature data in certain frames due to rapid movement or changes in facial expressions. In this case, a single-frame generative adversarial network may generate a reasonable facial feature that is inconsistent with the previous and subsequent temporal sequences. Through Bayesian reasoning, the system can infer and correct the complementary features of the current frame based on the feature data of the previous frames, ensuring that the generated features are not only reasonable in the current frame, but also coherent throughout the entire time series.

[0078] For example, when a passenger loses some facial features due to turning his head during security check, the system generates the missing features through a generative adversarial network. However, in order to ensure that the generated features are consistent with the features of the previous frames, Bayesian reasoning will combine the feature information in the previous frames for inference to ensure that the generated features not only conform to the environment of the current frame, but also remain consistent with the previous features. For example, Bayesian reasoning may infer the shape and position of the mouth in the current frame through the facial posture of the previous frames, thereby generating facial features consistent with the time series. This correction based on the information of the previous frames ensures that the system works efficiently in dynamic scenarios, especially when passengers pass through security checks quickly, the ambient light changes, or the face is blocked. Bayesian reasoning can infer the best complementary features through historical data and improve the accuracy of verification.

[0079] Assume that in a certain frame, the passenger's facial features are partially blocked due to head rotation. The system uses a generative adversarial network to generate facial features of the blocked part. However, the generated features alone may not be completely consistent with the facial information of the previous frames. At this time, the role of Bayesian reasoning becomes apparent. The system uses the facial features of the previous frames (i.e., likelihood probability ) and the prior probability of generating features , infer the posterior probability of the feature data in the current frame Based on this inference result, the system will correct the generated features so that they not only look reasonable in the current frame, but also remain consistent with previous facial features.

[0080] Through this reasoning process, the system can automatically correct the generated features so that they show highly consistent facial features throughout the entire time series, ensuring that the final generated facial features are not only accurate but also have good spatiotemporal coherence.

[0081] Preferably, the Bayesian reasoning ensures the continuity of the completed feature data of the current frame in the time series and eliminates feature mutations caused by missing data or noise by performing continuous reasoning calculations on the completed feature data of previous frames.

[0082] The main role of Bayesian reasoning in the present invention is to ensure the continuity of the complementary feature data of the current frame in the time series and eliminate feature mutations caused by missing data or noise by continuously reasoning and calculating the complementary feature data of the previous frames. This is crucial for dynamic portrait verification systems, especially when passengers pass through security check channels quickly. The system needs to process multimodal data, which may cause features in some frames to be lost or interfered by noise due to motion, light changes or partial occlusion. Bayesian reasoning infers the information of previous frames and combines the data of the current frame for correction, thereby ensuring that the facial features in the entire time series are consistent and natural.

[0083] The core principle of Bayesian reasoning is to use prior knowledge and observation data for inference. In the present invention, the prior knowledge comes from the feature data of the previous frames, which represent the dynamic change trend of the passenger's facial features. The observation data is the complementary feature data generated by the generative adversarial network (GAN) of the current frame. The task of Bayesian reasoning is to infer the most likely complementary features of the current frame by analyzing these previous frames of data and the current data, so as to ensure that it is consistent with the previous and subsequent frame data.

[0084] Through this reasoning process, the system can correct the features generated by the current frame so that it not only looks realistic at the local level, but also conforms to the overall change law in the time series and avoids sudden feature mutations. For example, when a passenger moves quickly, the facial features in a certain frame may appear abnormal due to noise or data loss. At this time, Bayesian reasoning infers the reasonable features of the current frame by combining the change trends of the previous frames, thereby correcting the mutation caused by noise or missing data.

[0085] In practical applications in the field of civil aviation, Bayesian reasoning can effectively cope with the dynamic behavior changes of passengers when passing through security checks. When passengers quickly pass through the security check channel, facial features may experience data loss or noise due to uneven lighting, angle changes, or short-term occlusion. In this case, the traditional facial recognition system may cause inaccurate recognition due to abnormal data in a certain frame. The Bayesian reasoning in the present invention infers the reasonable complementary features of the current frame by continuously reasoning about the feature data of the previous frames, ensuring the natural transition of facial features in the entire time series.

[0086] For example, suppose a passenger is missing a frame of data due to facial occlusion when passing through security check. The generative adversarial network completes the features of the frame, but the generated features may be inconsistent with the changing trend of the previous and next frames. Bayesian reasoning determines whether the features of the current frame are reasonable by reasoning about the features of the previous frames, and fine-tunes them based on the observed trends of the previous frames, so that the generated features not only match the current frame, but also remain consistent with other frames in the time series. This process effectively eliminates feature mutations caused by occlusion, noise or other data instability factors, and improves the system's dynamic portrait verification effect.

[0087] Bayesian reasoning can also handle sudden changes in facial features caused by dynamic passenger behavior. For example, when a passenger suddenly turns his head or moves quickly, the facial features of a certain frame may be quite different from those of the previous frame. Traditional static feature completion methods may not be able to correctly handle such dynamic changes and are prone to feature mutations. However, through Bayesian reasoning, the system can correct the features of the current frame through continuous reasoning of the previous frames of data to make it conform to the change logic in the time series, thereby avoiding unreasonable feature jumps. For example, when a passenger's facial expression changes due to rapid movement, Bayesian reasoning can infer the expression state that the current frame should have through the expression change trend of the previous frames, thereby completing the missing facial features of the current frame.

[0088] Suppose that in a certain frame, the passenger's face is partially obscured due to head rotation, and the system uses a generative adversarial network to complete the lost facial features. However, since the completed features of a single frame may be inconsistent with the previous frames, Bayesian reasoning uses the data of the previous frames to infer and correct to ensure the continuity of the completed features in the time series. Specifically, Bayesian reasoning analyzes the changing trends of facial features in the previous frames, such as facial posture, expression, etc., and infers the reasonable features that should be presented in the current frame. Through this reasoning process, the system can eliminate feature mutations caused by data loss or noise, ensuring that the final generated facial features are coherent and natural in the time series.

[0089] like Figure 4 As shown, the spatiotemporal feature verification and association analysis module is used to construct a spatiotemporal graph based on the corrected and completed feature data, and analyze the multi-frame feature correlation of the corrected and completed feature data through a graph neural network, dynamically correct the timing features in combination with recursive feedback, and generate corrected associated feature data to ensure accurate verification of spatiotemporal consistency and dynamic behavior.

[0090] The spatiotemporal feature verification and association analysis module is one of the important modules used in the present invention to ensure spatiotemporal consistency and dynamic behavior verification. Its core function is to construct a spatiotemporal graph based on the corrected and completed feature data, analyze the multi-frame correlation of feature data using a graph neural network (GNN), and dynamically correct the features in the time series in combination with a recursive feedback mechanism, and finally generate corrected associated feature data to ensure the temporal and spatial consistency and accuracy of the dynamic portrait verification of passengers. This module is particularly suitable for dynamic portrait verification application scenarios in the civil aviation field, and can effectively handle complex situations such as rapid movement of passengers, changes in facial expressions, and partial occlusion, to ensure the accuracy of the verification results.

[0091] The construction of the spatiotemporal graph is the basis of this module. The spatiotemporal graph represents each frame feature data in the time series as a node in the graph, and the edges between the nodes represent the feature connections between adjacent frames or related frames. Through this graph structure, the system can capture the complex relationship between the features of each frame in the time series, especially the correlation between multimodal data and the change of temporal features. This spatiotemporal graph can not only reflect the spatial distribution of single-frame features, but also capture the continuous changes of multi-frame features in the time dimension.

[0092] In order to analyze the correlation between nodes in the spatiotemporal graph, the present invention introduces a graph neural network (GNN). The role of GNN is to learn the global correlation information of each frame feature by iteratively updating the nodes and edges of the spatiotemporal graph, and adjust the feature representation of each node in the process. Specifically, the graph neural network will propagate information to the nodes of adjacent frames, that is, the data of each frame feature will be passed to the features of the adjacent frames through the edges, thereby updating the feature representation of each frame. Through multiple iterations, GNN can capture the global feature correlation in the entire time series, so as to better understand the relationship between the features of each frame in the spatiotemporal dimension.

[0093] For example, in the application scenario of civil aviation security inspection, when passengers pass through security inspection, the system will capture multiple frames of facial feature data. In some frames, the passenger's face may be partially lost or distorted due to occlusion or light changes. By constructing a spatiotemporal graph, the system can connect the associations between these frames. By analyzing these associations, GNN can supplement and correct the missing or distorted features in a frame to make it consistent with the features of the previous and next frames. Through this information propagation mechanism, the system can ensure that the spatiotemporal consistency and association of facial feature data are maintained in dynamic scenes.

[0094] In addition, the recursive feedback mechanism is also a key part of this module. The recursive feedback mechanism dynamically adjusts the features of each frame by recursively processing the time series features. This mechanism combined with GNN ensures that the system can not only capture the association between multiple frames in the spatial dimension, but also optimize the continuity of features in the time dimension. After the GNN analysis is completed, the recursive feedback mechanism will further dynamically correct the time series features according to the feature change trend of each frame to ensure that the generated corrected correlation feature data is coherent and consistent in the time series.

[0095] Through this module, the system can effectively solve the problem of inconsistent features caused by passenger movement, expression changes or data loss during dynamic portrait verification. In civil aviation security inspection scenarios, passengers usually pass through security inspection equipment quickly, which means that the system needs to continuously capture multiple frames of facial feature data within a limited time. Due to the rapid movement of passengers or changes in angles, the data in some frames may be incomplete or inaccurate. At this time, the features of a single frame are difficult to provide reliable verification basis. Through the spatiotemporal feature verification and correlation analysis module, the system can correlate and analyze multi-frame feature data to ensure that even if data is missing or distorted in a certain frame, it can be supplemented and corrected through the correlation between the previous and next frames.

[0096] For example, when a passenger passes through security check, some facial features are lost due to the rotation of the head. GNN will analyze the feature relationship between the previous and next frames and infer the facial features that should be shown in the frame, thereby generating consistent correction-related feature data during the correction process. At the same time, the recursive feedback mechanism will dynamically adjust these completed features to ensure that they remain consistent with other features in the time series. The final generated correction-related feature data can not only accurately reflect the dynamic facial features of the passenger, but also ensure that the system maintains a high degree of verification accuracy when dealing with rapidly changing dynamic behaviors.

[0097] Assuming that in a certain frame, the passenger's facial features are blurred due to changes in light, the system constructs a spatiotemporal graph through the spatiotemporal feature verification and association analysis module, and performs association analysis on the previous and next frames through the graph neural network. GNN learns the facial feature information in the previous frames and propagates this information to the current frame, thereby completing or correcting the blurred features in the current frame. Then, the recursive feedback mechanism further optimizes the feature representation of the current frame by recursively processing the feature change trends of the current frame and the previous frames, ensuring that the frame maintains consistency with the feature data of other frames. Finally, the corrected associated feature data generated by the system can reflect the true facial features of the passenger, ensuring the accuracy and consistency of facial verification.

[0098] Preferably, the spatiotemporal feature verification and association analysis module performs aggregation analysis on node features in the spatiotemporal graph through a graph convolutional network, wherein the nodes represent facial feature information of each frame, and the edges represent feature changes between adjacent frames. The graph convolutional network analyzes the spatiotemporal correlation between adjacent frames through an aggregation operation of neighborhood features to generate spatiotemporal correlation feature data.

[0099] Graph convolutional network (GCN) is a neural network specifically designed to process graph-structured data. The nodes in the graph represent the facial feature information of each frame in the time series, and the edges represent the feature change relationship between adjacent frames. Through the graph convolutional network, the system can aggregate the features between adjacent frames, analyze the correlation between these frames, and ultimately generate more complete and temporally consistent feature data.

[0100] In the present invention, the working mechanism of the graph convolutional network mainly includes the following key steps: Construction of the spatiotemporal graph: The spatiotemporal graph uses the facial features of each frame as nodes and the feature changes between adjacent frames (such as changes in facial expressions or posture changes caused by movement) as edges. Through this graph structure, the system can model all frames in the time series in a holistic manner, rather than processing each frame of data in isolation.

[0101] Aggregation of neighborhood features: The graph convolutional network analyzes the features of each node and its neighborhood (i.e., adjacent frames) through aggregation operations. Specifically, GCN combines the feature data of each frame with the feature data of adjacent frames and aggregates these features together through convolution operations to obtain a richer feature representation. This aggregation process not only considers the data of a single frame, but also considers the associated information between adjacent frames to ensure that the generated feature data is not only valid in the current frame, but also consistent in the time series.

[0102] Generation of spatiotemporal correlation feature data: Through the multi-layer convolution operation of GCN, the system can continuously update and optimize the feature representation of each frame. Finally, the system generates feature data containing spatiotemporal correlation, which is called spatiotemporal correlation feature data. These data not only reflect the facial features of each frame, but also ensure the continuity and consistency in the time series through the correlation analysis between adjacent frames.

[0103] For example, in actual civil aviation applications, when passengers quickly pass through security checks, the system will capture multi-frame feature data of the passengers' faces through multimodal sensors. In some frames, facial features may be partially missing or blurred due to the rapid movement or occlusion of passengers. At this time, GCN can aggregate the feature data of adjacent frames and complete and optimize the missing or blurred parts through other frame features in the context. The resulting spatiotemporal correlation feature data ensures that the system can provide accurate and coherent facial feature verification results even when processing dynamic scenes.

[0104] By introducing graph convolutional networks, the system can capture more complex spatiotemporal correlations in dynamic portrait verification, significantly improving the robustness and accuracy of verification. In dynamic scenes, passengers' behaviors, postures, and facial expressions change frequently, and the facial features of a single frame may not be able to fully reflect these changes. Traditional single-frame feature processing methods tend to ignore the correlation between multiple frames, resulting in missing or inaccurate information during the verification process. The graph convolutional network can capture more comprehensive spatiotemporal information by aggregating multi-frame features, thereby improving the accuracy of dynamic portrait verification.

[0105] For example, in the scenario of passenger security inspection, GCN can model facial features of multiple frames. Even if a frame of data is distorted due to light changes or partial occlusion, the system can still supplement and correct it through the associated information in adjacent frames. This approach ensures that the system can verify facial features with higher accuracy when dealing with rapidly changing dynamic behaviors. For example, when the features in a frame are inconsistent with the previous and next frames due to changes in the passenger's facial expression, GCN can dynamically correct the features of the frame through the correlation analysis of the previous and next frames, and finally generate accurate and consistent facial verification results.

[0106] Assume that during the security check, a passenger passes through the security check equipment quickly, causing the facial features in a certain frame to be blurred or distorted. The system uses a graph convolutional network to aggregate and analyze the facial features of multiple frames. GCN updates the feature representation of the current frame by combining the features of the frame with the features of its adjacent frames. In this process, the system gradually captures the key information in the previous and next frames through convolution operations, and propagates this information to the feature representation of the current frame. In the end, the system generates a facial feature data containing the spatiotemporal correlation of multiple frames, ensuring that even if a frame of data is distorted, the system can still generate a coherent facial verification result.

[0107] Preferably, the spatiotemporal feature verification and association analysis module is combined with a recursive feedback mechanism, and the spatiotemporal association data of adjacent frames are dynamically corrected, and the recursive feedback is used to cumulatively calculate the changes in node features.

[0108] The spatiotemporal feature verification and association analysis module combines a recursive feedback mechanism to dynamically correct the spatiotemporal association data of adjacent frames and accumulate the changes in node features to ensure the consistency and coherence of facial features in the time series. The recursive feedback mechanism accumulates the feature change trends of adjacent frames so that the feature representation of each frame not only depends on the data of a single frame, but also combines the features of the previous frames in the time series, thereby dynamically adjusting and optimizing facial features. This mechanism is crucial for dynamic portrait verification in the civil aviation field, especially when passengers pass through security checks quickly, their facial expressions change, or there is a short-term occlusion, to ensure that the system generates accurate and coherent verification results. The working principle of the recursive feedback mechanism relies on the recursive accumulation calculation of node features, which is specifically reflected in the association analysis of adjacent frames and the feedback processing of multiple frames. The facial features of each frame are used as nodes in the graph, and the edges between nodes represent the correlation between adjacent frames. The system uses a graph convolutional network (GCN) to perform a preliminary aggregation analysis of the features of these nodes and their neighborhoods to capture the feature changes between each frame. On this basis, the recursive feedback mechanism further accumulates the features of multiple frames, combines the feature change trends of the previous frames into the features of the current frame, and thus forms spatiotemporal correlation feature data.

[0109] For example, when a passenger passes through a security inspection device, the facial features in a certain frame may be inconsistent with the previous and next frames, or there may be slight distortion or noise due to the passenger's rapid movement or posture changes. In this case, the recursive feedback mechanism will infer the reasonable features of the current frame by analyzing the feature change trends of the previous frames, and make corrections and adjustments based on the cumulative feedback information of the previous frames. This dynamic correction process can avoid sudden changes or anomalies in single-frame data and ensure that the facial features in the time series remain coherent and consistent. The recursive feedback mechanism combined with the cumulative calculation of spatiotemporal feature verification not only enhances the system's adaptability to dynamic changes, but also significantly improves the accuracy and robustness of facial verification, especially in complex scenarios where multiple frames of data need to be combined for analysis. For example, in the case of a passenger's face being partially occluded, through recursive feedback, the system can infer the reasonable features of the occluded part based on the features of the previous frames, thereby completing and correcting the facial features of the current frame, and finally generating a consistent verification result. In this way, the system can achieve efficient and accurate dynamic portrait verification in dynamic scenes, ensuring the safety and efficiency of the security inspection process.

[0110] Preferably, the corrected associated feature data generated by the spatiotemporal feature verification and association analysis module is used to verify the dynamic behavior and facial features of the passenger. The corrected associated feature data ensures the consistency of facial features and behaviors in dynamic scenes, and further optimizes the accuracy of dynamic behavior verification through recursive correction.

[0111] The corrected correlation feature data generated by the spatiotemporal feature verification and correlation analysis module is used to verify the dynamic behavior and facial features of passengers, ensure the consistency of facial features and behaviors in dynamic scenes, and further optimize the accuracy of dynamic behavior verification through recursive correction. The core of this module is to generate corrected correlation feature data by comprehensively analyzing and processing the spatiotemporal correlation of multi-frame data of passengers' facial features and dynamic behaviors, thereby ensuring that the system can continuously and accurately verify the identity and behavior of passengers in complex scenes such as rapid movement of passengers, changes in facial posture or short-term occlusion. Its principle is based on taking facial features in time series as graph nodes, aggregating and analyzing these nodes through graph convolutional networks (GCNs), and combining recursive feedback mechanisms to realize cumulative calculation and correction of feature changes, thereby generating facial and behavioral feature data containing global spatiotemporal correlation.

[0112] When the system processes the dynamic behavior and facial features of passengers, each frame of the facial image is regarded as a node in the spatiotemporal graph, and the edges between nodes represent the feature associations and changes between adjacent frames. The graph convolutional network can capture the spatiotemporal correlation between adjacent frames by performing aggregation calculations on these adjacent nodes. This aggregation analysis not only helps to correct feature changes caused by motion, light changes, or facial occlusion, but also combines multi-frame features to generate a more robust facial feature representation. In this process, the recursive feedback mechanism recursively corrects the feature data of each frame, and further optimizes the feature representation of the current frame based on the feature change trends of the previous frames. This correction process can better reflect the dynamic behavior and facial features of passengers by accumulating feature change information from multiple frames, ensuring feature consistency and accuracy of dynamic performance in continuous time series.

[0113] In civil aviation security inspection scenarios, this module is particularly important for processing dynamic behavior verification of passengers. When passengers pass through the security inspection channel quickly, the system will capture facial features and dynamic behavior data of multiple consecutive frames, but due to the rapid movement or short-term occlusion of passengers, the data of some frames may be lost, blurred or distorted. In this case, the system cannot rely solely on the features of a single frame for verification, but instead associates the features of multiple frames through the spatiotemporal feature verification and association analysis module to generate correction data containing spatiotemporal correlation. Through the graph convolutional network and recursive feedback mechanism, the system can analyze and correct the feature change trend between the previous and next frames, ensuring that even if the facial features in a certain frame are incomplete, the system can still complete and correct the features of the current frame through the information of the previous and next frames, thereby generating high-precision corrected correlation feature data.

[0114] For example, when a passenger turns his head and the facial features of a certain frame are lost or distorted, the system associates the feature information of the previous and next frames with the current frame through spatiotemporal graph analysis, uses graph convolutional networks to aggregate and analyze these features, and then combines the recursive feedback mechanism to infer the reasonable facial features of the current frame, ensuring that the features of this frame are consistent with the previous and next frames. In this way, the system can generate accurate facial feature data in dynamic scenes, and can still perform efficient and accurate dynamic portrait verification even when passengers pass through security checks quickly or are disturbed by the environment.

[0115] Correcting the associated feature data not only ensures the consistency of facial features, but also accurately verifies the dynamic behavior of passengers. In some complex scenarios, the dynamic behavior of passengers (such as rapid movement, changes in expression, etc.) may cause single-frame features to be unable to fully reflect their true identity characteristics. Correcting the associated feature data can better identify and verify these behavioral changes through the aggregation and recursive correction of multi-frame features, avoiding verification errors caused by abnormal single-frame data. For example, when a passenger turns his head quickly during the security check, a single-frame data may not provide sufficient verification basis, but through spatiotemporal correlation analysis and recursive correction, the system can infer the passenger's true facial features and dynamic behavior based on the feature change trends of the previous and next frames, ensuring the continuity and accuracy of the verification.

[0116] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A dynamic portrait verification and inspection result generation system based on machine learning, characterized in that: include: A multimodal data acquisition module, used to collect RGB image data, infrared thermal image data, depth map data and millimeter wave profile data of passengers, and perform feature extraction and cleaning on the collected data to generate first fusion feature data; A recursive feedback and dynamic feature optimization module, which is used to perform time series optimization on the first fused feature data using a recursive neural network to generate time series optimized feature data, and calculate the dynamic weight of the collected data using a Markov chain Monte Carlo method, and perform weighted fusion on the collected data based on the dynamic weight to generate second optimized feature data; A feature completion and correction module, used to perform feature completion on the second optimized feature data using a generative adversarial network, generate completed feature data by generating missing or occluded facial features, and perform spatiotemporal consistency correction on the completed feature data based on Bayesian reasoning to generate corrected completed feature data; The spatiotemporal feature verification and association analysis module is used to construct a spatiotemporal graph based on the corrected and completed feature data, and analyze the multi-frame feature correlation of the corrected and completed feature data through a graph neural network. It combines recursive feedback to dynamically correct the temporal features and generate corrected associated feature data to ensure accurate verification of spatiotemporal consistency and dynamic behavior.

2. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1 is characterized in that: The recursive neural network in the recursive feedback and dynamic feature optimization module generates timing optimized feature data by comparing the first fused feature data of the previous frames with the first fused feature data of the current frame. The timing optimized feature data is obtained by recursively accumulating each frame of data to ensure timing continuity and reduce the interference of noise on feature extraction.

3. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1 is characterized in that: The dynamic weights in the recursive feedback and dynamic feature optimization module are calculated using the Markov chain Monte Carlo method, and the calculation formula is: in, For the Dynamic weights of modalities; Representing modality In the Weight contribution in the frame; Indicates the frame number; Indicates the total number of modes; The dynamic weights are weighted and optimized for each modal feature by a recursive accumulation method.

4. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1 is characterized in that: The recursive feedback and dynamic feature optimization module uses an attention mechanism to dynamically adjust the weights of each modal feature during the recursive feedback process, and generates second optimized feature data by adaptively adjusting the weights of important features in the time series. The adaptive adjustment is updated based on the feature weight change trend of the previous frames.

5. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1 is characterized in that: The generative adversarial network in the feature completion and correction module uses a convolutional generative network to complete the missing facial features from the second optimized feature data. The convolutional generative network generates features for local areas of features through a generator and uses a discriminator to evaluate the authenticity of the generated features. The generation process is continuously generated based on feature extraction and reconstruction of the convolutional operation layer.

6. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1 is characterized in that: The Bayesian reasoning in the feature completion and correction module is used to perform spatiotemporal correction on the completed feature data. The calculation formula of the correction process is: in, is the posterior probability of the completed feature data in the current frame; is the feature likelihood of the previous frames; is the prior probability of completing the feature data; is the probability of observing the data. The correction process adjusts the spatiotemporal consistency of the features of the current frame according to the inference results of the previous frames.

7. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 6 is characterized in that: The Bayesian reasoning ensures the continuity of the completed feature data of the current frame in the time series by performing continuous reasoning calculations on the completed feature data of the previous frames, and eliminates feature mutations caused by data missing or noise.

8. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1 is characterized in that: The spatiotemporal feature verification and association analysis module aggregates and analyzes the node features in the spatiotemporal graph through a graph convolutional network, wherein the nodes represent the facial feature information of each frame, and the edges represent the feature changes between adjacent frames. The graph convolutional network analyzes the spatiotemporal correlation between adjacent frames through the aggregation operation of neighborhood features to generate spatiotemporal correlation feature data.

9. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1, characterized in that: The spatiotemporal feature verification and association analysis module is combined with a recursive feedback mechanism to dynamically correct the spatiotemporal association data of adjacent frames, and recursive feedback is used to cumulatively calculate changes in node features.

10. The dynamic portrait verification and inspection result generation system based on machine learning according to claim 1, characterized in that: The corrected associated feature data generated by the spatiotemporal feature verification and association analysis module is used to verify the dynamic behavior and facial features of the passenger. The corrected associated feature data ensures the consistency of facial features and behaviors in dynamic scenes, and further optimizes the accuracy of dynamic behavior verification through recursive correction.

Images