High-speed rail gate anti-misclamping control method and system based on multi-modal perception

Abstract

The application relates to the technical field of image recognition, and discloses a high-speed rail gate anti-misclamping control method and system based on multi-modal perception, wherein the method comprises the following steps: an asymmetric double-flow feature extraction mechanism; registering infrared features to visible light coordinates through a deep deformation field; generating fusion features through decoupling tensor fusion; calculating a minimum safety gap through physical model guided gap measurement; and generating motor signals through risk potential field model prediction control. Compared with the prior art, the direct superposition of visible light and infrared images causes fusion artifacts, especially under the condition of heterogenous image resolution and geometric mismatch, and the technical problem that pixel-level alignment and fusion of visible light details and infrared temperature information cannot be realized. Due to the asymmetric feature extraction, deep deformation field registration and decoupling tensor fusion, pixel-level accurate alignment of visible light texture and infrared temperature information is realized, and the body positioning precision and the reliability of the anti-clamping decision are improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for preventing accidental clamping of high-speed rail turnstiles based on multimodal perception. Background Technology

[0002] Currently, anti-pinch safety control at high-speed rail turnstiles primarily employs single-modal sensing solutions such as infrared light curtains, monocular visible light cameras, or ultrasonic sensors. These solutions detect passenger limbs and determine safe clearances by collecting a single type of signal. Infrared light curtains rely on beam obstruction to detect object presence, but they lack sensitivity to small limbs, transparent materials, or low-reflectivity materials. Visible light cameras, on the other hand, are limited by drastic changes in lighting, shadow interference, and target occlusion, making it difficult to reliably extract precise limb contours. To improve perception robustness in complex traffic scenarios, some solutions have attempted to simultaneously deploy visible light cameras and infrared thermal imaging sensors. These sensors acquire RGB texture images and temperature distribution images of the scene, respectively. By pixel-level overlay or channel stitching of the two images, a preliminary multimodal fusion representation is formed to assist in subsequent limb localization and anti-pinch decisions.

[0003] For example, when fusing visible light and infrared images within a turnstile, the fundamental differences in imaging mechanisms, spectral responses, spatial resolutions, and field of view between the two sensors mean that directly performing pixel overlay or simple channel stitching on the original images often introduces severe artifacts, edge misalignment, and contour blurring into the fusion result. When passengers quickly pass through with luggage or their limbs are at the edge of the gate blades, the limb boundaries presented in the visible light image due to rich textures are geometrically misaligned with the contours presented in the infrared image due to temperature differences. This cross-modal spatial mismatch leads to systematic deviations in the positional representation of key limb points in the fusion features, resulting in significant errors in the calculated limb-to-gate blade distance. At the decision-making level, this manifests as an overestimation or underestimation of the safety clearance, which can easily cause accidental clamping or incorrect passage during gate closing, seriously affecting the safety and efficiency of high-speed rail platform passage.

[0004] Therefore, there is an urgent need for a method that can still achieve pixel-level accurate registration and fusion of visible light details and infrared temperature information at the feature level under the condition of heterogeneous image resolution and geometric mismatch, so as to improve the accuracy of limb three-dimensional position detection under complex lighting, occlusion and rapid movement conditions, thereby enhancing the reliability of gate anti-pinch control decisions. Summary of the Invention

[0005] To address the aforementioned technical shortcomings, the purpose of this invention is to propose a high-speed rail turnstile anti-mispinch control method based on multimodal perception. This method aims to solve the technical problem in the prior art where directly superimposing visible light and infrared images leads to fusion artifacts, especially under conditions of resolution and geometric mismatch between heterogeneous images, making it impossible to achieve pixel-level alignment and fusion of visible light details and infrared temperature information.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a high-speed rail turnstile anti-mispinch control method based on multimodal perception.

[0007] The high-speed rail turnstile anti-mispinch control method based on multimodal sensing includes:

[0008] Step S10: Obtain a pair of visible light images and infrared thermal images collected with a unified time stamp within the gate channel. Based on the pair of visible light images and infrared thermal images, perform the task of extracting features of passenger limbs and personal objects using an asymmetric dual-stream feature extraction mechanism, and output a visible light multi-scale feature map and an infrared multi-scale feature map.

[0009] Step S20: Based on the visible light multi-scale feature map and the infrared multi-scale feature map, a depth perception and deformation field estimation network mechanism is used to perform the spatial registration task, and output the infrared feature map registered to the visible light coordinate system.

[0010] Step S30: Based on the infrared feature map registered to the visible light coordinate system and the corresponding scale visible light feature map, a cross-modal tensor fusion mechanism with structure and texture decoupling is used to perform the feature fusion task and output the fused feature map;

[0011] Step S40: Based on the fused feature map, a physical model-guided temporal three-dimensional attitude and gate dynamic gap measurement mechanism is used to perform the task of calculating the safety gap between the limb and the gate, and the minimum safety gap is output;

[0012] Step S50: Based on the minimum safety gap, the risk potential field and model predictive control mechanism are used to execute the motor control command generation task and output the gate motor control signal.

[0013] Preferably, in step S10, the steps of acquiring a pair of visible light images and infrared thermal images collected with a unified spatiotemporal stamp within the gate channel, and performing feature extraction tasks for passenger limbs and personal objects using an asymmetric dual-stream feature extraction mechanism based on the visible light images and infrared thermal images, and outputting visible light multi-scale feature maps and infrared multi-scale feature maps, specifically include:

[0014] Step S101: Using the gate wing opening and closing status signal within the same gate passage cycle as the trigger reference, collect visible light images and infrared thermal images with the same timestamp, and input them into the visual Transformer network and infrared convolutional network respectively for initial feature extraction.

[0015] Step S102: In the shallow output of the visual Transformer network, retain the texture features of passenger clothing edges, luggage outlines and door wing boundaries, and in the temporal consistency module of the infrared convolutional network, retain the thermal response features of human skin, limb ends and under occlusion conditions, and suppress feature jumps caused by gate light strip reflection, glass reflection and environmental heat source disturbance.

[0016] Step S103: Generate visible light multi-scale feature maps and infrared multi-scale feature map ,in, The resolution level is indicated; the visible light multi-scale feature map includes a visible light mid-level feature map and a visible light structural feature map, and the infrared multi-scale feature map includes an infrared mid-level feature map and an infrared temperature feature map.

[0017] Preferably, step S20, which involves performing spatial registration based on the visible light multi-scale feature map and the infrared multi-scale feature map using a depth sensing and deformation field estimation network mechanism, and outputting an infrared feature map registered to the visible light coordinate system, specifically includes:

[0018] Step S201: Obtain the visible light mid-layer feature map and infrared mid-spheric feature map Input to a depth-aware deformation field estimation network, which then predicts the initial warping displacement using a multi-scale spatial convolutional encoder. ;

[0019] Step S202: The initial warpage displacement is refined using a depth modulation formula to obtain a refined displacement field. The depth modulation formula is as follows:

[0020] ;

[0021] in, This represents the initial warping displacement. This represents the initial displacement of the infrared feature point in the lateral direction of the visible light coordinate system. This represents the initial displacement of the infrared feature point in the longitudinal direction of the visible light coordinate system. To refine the displacement field, This refers to the precise lateral displacement. For longitudinal fine-tuning of displacement; This is a depth prior map generated based on the depth calibration results of the turnstile channel; Learnable deep modulated convolutional kernels; Modulation intensity factor; This is pixel-by-pixel multiplication; It is the hyperbolic tangent activation function;

[0022] Step S203: Based on the refined displacement field and bicubic interpolation resampling, the infrared temperature feature map is mapped to the visible light coordinate system to obtain an infrared feature map registered to the visible light coordinate system.

[0023] Preferably, in step S202, the modulation term in the depth modulation formula The modulation term is used to constrain the deformation prediction amplitude based on different depth regions within the gate channel. Specifically, in the near-field region near the gate wing edge, the modulation term is close to 1 to maintain limb contour registration accuracy; in the region far from the gate wing edge, the modulation term is processed by a learnable depth modulation convolution kernel. The deformation prediction amplitude is adaptively adjusted to compensate for the nonlinear shift caused by the inconsistency in the viewing angles of the infrared and visible light lenses.

[0024] Preferably, in step S30, the step of performing feature fusion based on the infrared feature map registered to the visible light coordinate system and the corresponding scale visible light feature map using a cross-modal tensor fusion mechanism that decouples structure and texture, and outputting the fused feature map, specifically includes:

[0025] Step S301: Decompose the visible light structural feature map, the visible light mid-layer feature map, the infrared feature map registered to the visible light coordinate system, and the infrared temperature feature map through the structure extraction branch and the texture extraction branch respectively to obtain the visible light structural features. Visible light texture features Infrared structural features and infrared temperature characteristics ;

[0026] Step S302: Using a four-dimensional weighted tensor approximated by tensor ring decomposition with a low rank, perform cross-modal fusion on the visible light structural features, visible light texture features, infrared structural features, and infrared temperature features. The fusion formula is as follows:

[0027] ;

[0028] in, To fuse feature maps; For the first Visible light structural features of each channel; For the first Visible light texture features of each channel; For the first Infrared structural features of each channel; For the first Infrared temperature characteristics of each channel; The fusion weights are the corresponding channel combinations in the four-dimensional weight tensor; , , , All are feature channel indexes; This indicates that the summation is performed on all channel combinations. Indicates the convolution operation;

[0029] Step S303: Through the cross-modal fusion, the passenger's limb contour, skin thermal response, baggage obstruction boundary, and door wing edge structure are uniformly mapped to the same feature space to generate the fused feature map.

[0030] Preferably, in step S40, the step of performing the task of calculating the safety gap between the limb and the gate based on the fused feature map using a physical model-guided temporal three-dimensional attitude and gate dynamic gap measurement mechanism, and outputting the minimum safety gap, specifically includes:

[0031] Step S401: The depth refinement head predicts the depth residual map of the passenger's limb region based on the fused feature map, and fuses the depth residual map with the gate channel depth prior map to obtain the final depth map;

[0032] Step S402: The gate motion state perception module maps the real-time angle and angular velocity of the gate motor into a gate motion state feature vector, and performs attention interaction between the gate motion state feature vector and the fused feature map;

[0033] Step S403: Predict the pixel coordinates of limb key points from the fused feature map, index the corresponding depth values ​​from the final depth map, and calculate the real-time 3D distance from the limb key points to the edge of the gate blades by combining the camera intrinsic parameters and the 3D line segment equation of the gate blades. Take the minimum value among all limb key point distances as the minimum safety gap. The formula for calculating the real-time 3D distance is:

[0034] ;

[0035] ;

[0036] in, for Time of the first Real-time 3D distance from each limb key point to the 3D line segment of the gate blade; for Minimum safe gap at any given time; and The first The horizontal and vertical pixel coordinates of key points of a limb in the visible light coordinate system; and These are the horizontal and vertical coordinates of the camera's principal point, respectively. For the first The depth values ​​of key limb points in the final depth map; and These refer to the camera's horizontal focal length and vertical focal length, respectively. for The three-dimensional line segment of the gate blade is obtained by real-time angle mapping of the gate motor. Candidate points on the three-dimensional line segment of the gate blade; Let L2 be the norm of the vector; Indicate candidate points Located in the three-dimensional line segment of the gate fan blade superior; This means traversing all limb key points and taking the minimum distance.

[0037] Preferably, step S50, which involves using a risk potential field and model predictive control mechanism based on the minimum safety gap to generate motor control commands and output gate motor control signals, specifically includes:

[0038] Step S501: Construct a dynamic risk potential field based on the coordinates of key human body points and the 3D model of the gate blades, and control the flow at the specified time. The risk is quantified by determining the safe distance between points on the gate blades and key points on the human body. The formula for the risk potential field strength is:

[0039] ;

[0040] in, To control the timing At the gate fan blade point The strength of the risk field at the location; To control the timing Candidate points on the gate blades; To control the timing No. Three-dimensional coordinates of key points of a limb; The angular velocity of the gate motor; This is the gain coefficient; To avoid constants with a denominator of zero; This indicates the summation of all key points of the limbs;

[0041] Step S502: In each control cycle, solve the model predictive control problem in the finite time domain based on the minimum safety gap and the risk potential field strength. The optimization objective is to minimize the objective function that includes the risk potential field norm, the motor acceleration smoothing term, and the deviation term between the gate angular velocity and the target velocity, so as to obtain the motor acceleration control sequence.

[0042] Step S503: Extract the first term of the motor acceleration control sequence as the motor control command for the current control cycle, and convert the motor control command into a gate motor control signal and output it to the gate motor driver.

[0043] This invention also provides a high-speed rail turnstile anti-mispinch control system based on multimodal sensing, including:

[0044] The feature extraction module is used to acquire visible light images and infrared thermal images collected with a unified time stamp within the gate channel. Based on the visible light images and infrared thermal images, an asymmetric dual-stream feature extraction mechanism is used to perform the feature extraction task of passenger limbs and personal objects, and output visible light multi-scale feature maps and infrared multi-scale feature maps.

[0045] The spatial registration module is used to perform spatial registration tasks based on the visible light multi-scale feature map and the infrared multi-scale feature map using a depth perception and deformation field estimation network mechanism, and outputs an infrared feature map registered to the visible light coordinate system.

[0046] The feature fusion module is used to perform a feature fusion task based on the infrared feature map registered to the visible light coordinate system and the visible light feature map of the corresponding scale, using a cross-modal tensor fusion mechanism that decouples structure and texture, and outputs a fused feature map.

[0047] The gap calculation module is used to perform the task of calculating the safety gap between the limb and the gate based on the fused feature map and the time-series three-dimensional posture and gate dynamic gap measurement mechanism guided by the physical model, and output the minimum safety gap.

[0048] The control generation module is used to generate motor control commands based on the minimum safety gap using a risk potential field and model prediction control mechanism, and output gate motor control signals.

[0049] The present invention also provides a high-speed rail turnstile anti-mispinch control device based on multimodal perception. The high-speed rail turnstile anti-mispinch control device based on multimodal perception includes: a memory, a processor, and a high-speed rail turnstile anti-mispinch control program based on multimodal perception stored in the memory and executable on the processor. When the high-speed rail turnstile anti-mispinch control program based on multimodal perception is executed by the processor, it implements the above-mentioned method.

[0050] The present invention also provides a computer program product, the computer program product including a high-speed rail turnstile anti-mispinch control program based on multimodal perception, the high-speed rail turnstile anti-mispinch control program based on multimodal perception implementing the above method when executed by a processor.

[0051] The beneficial effects of this invention are as follows:

[0052] 1. By combining an asymmetric dual-stream feature extraction mechanism with depth-aware deformation field estimation, infrared feature maps are accurately registered to the visible light coordinate system, solving the fusion artifact problem caused by spatial misalignment of heterogeneous images. The asymmetric network specializes in different modalities, and combined with the deformation field refinement process modulated by depth priors, it can overcome the nonlinear distortions in the near and far fields, providing spatially consistent intermediate features for subsequent fusion. This ensures that the expression of limb edges corresponds accurately in different modalities, which is unattainable by direct image-level alignment.

[0053] 2. A cross-modal tensor fusion mechanism decoupling structure and texture is employed. Visible light structural contours and texture details are decomposed and then fused with infrared thermal features and temperature distribution in a high-order interactive manner. This preserves the unique information of each mode while avoiding boundary blurring caused by the direct mixing of redundant or conflicting information between modes. The fused feature map can still clearly present limb boundaries under conditions such as sudden changes in illumination and local occlusion, thereby improving the stability of 3D localization of key points and minimum safety gap calculation. This makes the risk potential field and the input for model predictive control decisions more reliable, indirectly improving the accuracy of the anti-pinch response. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the first embodiment of the high-speed rail turnstile anti-mispinch control method based on multimodal sensing of the present invention.

[0055] Figure 2 This is a schematic diagram of the three-dimensional position of the limb key points and the gate line segment of the first embodiment of the high-speed rail gate anti-mispinch control method based on multimodal perception of the present invention.

[0056] Figure 3 This is a schematic diagram of the acceleration smoothing effect of the first embodiment of the high-speed rail turnstile anti-mispinch control method based on multimodal perception of the present invention.

[0057] Figure 4 This is a schematic diagram of the output motor control signal of the first embodiment of the high-speed rail gate anti-mis-clamping control method based on multimodal sensing of the present invention. Detailed Implementation

[0058] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0059] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Example 1: As Figure 1The diagram shown is a flowchart of the first embodiment of the high-speed rail turnstile anti-mispinch control method based on multimodal perception of the present invention, which proposes the first embodiment of the high-speed rail turnstile anti-mispinch control method based on multimodal perception of the present invention.

[0061] In the first embodiment, the high-speed rail turnstile anti-pinch control method based on multimodal sensing includes:

[0062] Step S10: Obtain a pair of visible light images and infrared thermal images collected with a unified time stamp within the gate channel. Based on the pair of visible light images and infrared thermal images, perform the task of extracting features of passenger limbs and personal objects using an asymmetric dual-stream feature extraction mechanism, and output a visible light multi-scale feature map and an infrared multi-scale feature map.

[0063] In this step, the asymmetric dual-stream feature extraction mechanism addresses the differences in imaging characteristics between visible light images and infrared thermal images by employing heterogeneous backbone networks for feature extraction. The visible light image stream uses a high-resolution representation network as its backbone, maintaining fine-grained spatial information at each resolution level and fully preserving visible light details such as clothing textures and limb contours. The infrared thermal image stream uses a compact visual Transformer network as its backbone, capturing the contextual relationships of heat source distribution throughout the image through a self-attention mechanism, while its compact design controls computational burden. A ConvGRU temporal consistency module is embedded between the dual-stream backbones. This module takes historical frame fusion features and current frame initial features as input, updates its state through gated recurrent units, and outputs current frame features carrying temporal information, thereby suppressing feature jumps caused by single-frame occlusion, instantaneous motion blur, or environmental interference. The final generated visible light multi-scale feature map { } and infrared multi-scale feature map { It contains multiple resolution levels, among which the middle-level feature maps and With rich semantics and accurate spatial positioning, it will be directly used as the input for deformation field estimation in step S20, providing a feature basis for subsequent spatial registration.

[0064] After asymmetric dual-stream extraction and ConvGRU temporal fusion, the multi-scale features output by the visible light stream maintain high-resolution spatial details while mitigating feature fragmentation caused by single-frame occlusion through inter-frame state updates. The infrared stream utilizes the global receptive field of the Transformer, enabling stable representation of the passenger's heat source subject even under drastic lighting changes or partial occlusion, unaffected by visible light texture interference. Both sets of features already possess good temporal coherence and modal complementarity before spatial registration. For example, when a limb is partially occluded in a frame, features from historical frames can supplement the missing information of the current frame through the ConvGRU gating mechanism, resulting in a more robust mid-level feature map. The heat source profile and The visible light edges in the model are more complete and continuous within their respective modes. This stable intermediate representation reduces the risk of erroneous displacement predictions due to feature jumps during subsequent deformation field estimation, creating favorable conditions for high-precision heterogeneous feature registration.

[0065] In existing multimodal perception schemes for high-speed rail turnstiles, feature extraction often uses convolutional networks with the same structure to process both visible light and infrared images simultaneously. This ignores the rich texture and high resolution of visible light images, while infrared images have low signal-to-noise ratios and blurred edges. This results in infrared stream features being mixed with a large amount of noise, or the visible light stream wasting computational power on ineffective details. At the same time, the independent processing of each frame discards temporal information. In scenarios where passengers pass through in staggered passages or limbs are briefly obscured by luggage or others, the feature representation of the same target between adjacent frames may be inconsistent. This step selects suitable architectures for the asymmetric backbone, allowing the visible light stream to focus on spatial details and the infrared stream to focus on global modeling of heat sources. Then, lightweight temporal fusion is achieved through ConvGRU embedding. This obtains spatiotemporally consistent multi-scale features without significantly increasing computational overhead, making full use of dual-modal features in both spatial and temporal dimensions and compensating for the lack of features in a single frame.

[0066] For example, when a passenger pushes a suitcase through a turnstile, the suitcase temporarily obscures part of the legs in the visible light image. However, due to the temperature difference between the suitcase and the legs, the infrared image still shows the heat source outline of the obscured legs. The high-resolution representation network extracts the sharp edges of the suitcase and the texture details of the unobscured upper body from the visible light image, while the compact vision transformer captures the continuous thermal radiation area of ​​the legs from the infrared image. Since the suitcase causes a loss of leg information in the single-frame visible light features, the ConvGRU module fuses the leg feature state from the previous frame when the legs were unobscured, and updates it together with the current frame's infrared thermal features, resulting in a higher-resolution infrared mid-layer feature map output for this frame. The outline of the heat source in the middle leg remains intact and continuous, while the visible light mid-layer feature map... The upper body details remain clear. This set of spatiotemporally stable mid-level features is then passed to step S20, and the deformation field estimation network will use this to more accurately infer the mapping relationship between infrared features and the visible light coordinate system.

[0067] Step S20: Based on the visible light multi-scale feature map and the infrared multi-scale feature map, a depth perception and deformation field estimation network mechanism is used to perform the spatial registration task, and output the infrared feature map registered to the visible light coordinate system.

[0068] In this step, the deep-sensing deformation field estimation network mechanism uses visible light mid-layer feature maps. and infrared mid-spheric feature map As input, an initial dense displacement vector field is first predicted by a multi-scale dilated convolutional encoder. , This vector field describes the coarse horizontal and vertical offset of each pixel on the infrared feature map relative to the visible light coordinate system. Subsequently, a spatial adaptive scaling of the initial displacement field is achieved using a depth modulation formula. Finally, based on the refined displacement field and the adaptive kernel width inferred from the local Jacobian determinant value of the displacement field, the... Non-uniform cubic B-spline interpolation resampling is performed to obtain an infrared feature map spatially registered to the visible light coordinate system.

[0069] Through refined deformation field processing using depth prior modulation, the network can adaptively adjust the deformation prediction amplitude according to the inherent geometric distortion characteristics of different distance regions in the scene. In the near-field region with a small depth prior value, the tanh term approaches zero, and the modulation factor approaches 1, essentially maintaining the fine deformation learned by the dilated convolutional encoder. In the far-field region with a large depth prior value, the nonlinear distortion caused by the optical axis deviation between the infrared and visible light sensors becomes more pronounced. The learnable convolutional kernel can generate modulation values ​​greater than or less than 1, thereby selectively amplifying or reducing the displacement amplitude to compensate for far-field distortion. After non-uniform cubic B-spline interpolation, the heat source contour in the infrared feature map is precisely distorted to coincide with the corresponding limb edge in the visible light feature map, ensuring strict spatial alignment of the two modal features. This spatial consistency ensures that during subsequent structure and texture decoupling and fusion, the same limb point is in the same spatial coordinates in both features, thus avoiding contour aliasing or artifacts caused by spatial misalignment.

[0070] Traditional visible light and infrared image registration often employs affine transformations or optical flow estimation based on image brightness. These methods assume the scene satisfies rigid body or linear deformation, making it difficult to handle complex nonlinear distortions caused by sensor installation deviations and optical distortions within the turnstile channel. When a limb moves from the near field to the far field, the deformation pattern changes, and a single global transformation matrix often results in ghosting at the limb edges. Direct fusion can lead to systematic errors in keypoint localization. This step performs dense deformation field estimation at the feature level rather than the image level, avoiding additional blurring caused by interpolation of the original infrared image. Simultaneously, a channel depth prior is introduced for modulation, ensuring near-field fidelity and far-field adaptive compensation of the deformation field. Combined with Jacobian determinant control of the interpolation kernel width, the kernel is automatically tightened in areas of large deformation gradient and widened in smooth regions, maintaining edge sharpness. This approach better reflects the actual characteristics of geometric mismatch at different distances within the high-speed rail turnstile channel, enabling the registered infrared features to be accurately embedded into the visible light space, improving the reliability of subsequent fusion.

[0071] For example, suppose a child stands close to the inner panel of a turnstile, their body very close to the camera, while their arm, extended into the passage, is farther from the camera. The depth prior image (Dprior) has a small value near the child and a large value in the distant arm region. The initial deformation field has roughly pulled the infrared thermal image to the visible light coordinate system, but there is still a slight offset at the far end of the arm due to far-field distortion. After depth modulation, at the corresponding pixel position at the far end of the arm, due to the large Dprior value, the tanh term may output a positive value after learning, and the modulation factor is greater than 1, appropriately amplifying the displacement field amplitude to compensate for the large distortion at that location. Meanwhile, at the near end, the modulation factor is close to 1, maintaining the original fine matching. After non-uniform B-spline interpolation, the heat source contour of the child's arm in the infrared feature map is accurately aligned to the texture edge of the same arm in the visible light feature map. The spatial representation of the arm boundary in both modalities is consistent, providing accurate spatial alignment input for subsequent decoupled tensor fusion.

[0072] Step S30: Based on the infrared feature map registered to the visible light coordinate system and the corresponding scale visible light feature map, a cross-modal tensor fusion mechanism with structure and texture decoupling is used to perform the feature fusion task and output the fused feature map;

[0073] Structure and texture decoupling allows the network to explicitly separate geometric shape information and apparent material information in an image, enabling cross-modal interaction based on different attributes. After decoupling, cross-modal tensor fusion only requires combining corresponding or complementary information dimensions, such as visible light structure. With infrared structure Interactive joint calibration of limb boundary positions, visible light texture With infrared temperature Interaction enhances the expressive power of occluded or camouflaged limbs while avoiding interference from mismatched features such as visible light textures and infrared structures. The four-dimensional tensor outer product provides high-order interaction possibilities across all cross-modal and cross-attribute channel pairs, while the low-rank tensor ring approximation significantly compresses the number of parameters while maintaining this powerful interaction capability, making computational overhead manageable. The fused feature map outperforms simple linear fusion in terms of the sharpness of limb contour edges and modal consistency of internal regions, providing a high-quality two-dimensional feature foundation for subsequent three-dimensional position calculations.

[0074] Traditional multimodal fusion methods, such as channel stitching or weighted summation, rely on the assumption that different modal feature spaces are aligned and statistically compatible. However, the response distributions of visible light and infrared feature channels differ significantly, and direct mixing can easily lead to the obscuring of texture details or the blurring of temperature boundaries. In high-speed rail turnstiles, factors such as summer ground convection can cause non-limb hotspots in infrared images. Ordinary fusion methods mix these hotspot interference signals with the actual limb temperature information with equal weight, resulting in blurred limb edges or false positive contours. The decoupling and tensor fusion mechanism in this step forces the network to focus on meaningful interactions between different attribute dimensions during the learning process. Simultaneously, it suppresses irrelevant or harmful cross-channel combinations through low-rank weighted tensors, selectively filtering artifacts and noise during the fusion stage. This ensures that the fused features can still clearly represent limb boundaries under conditions such as sudden changes in illumination and local occlusion, improving the stability of subsequent keypoint localization.

[0075] For example, in high-temperature summer conditions, the ground of a turnstile may exhibit uneven temperature distribution and false hot spots in infrared images due to the interaction of air conditioning cool air and external hot air. If ordinary channel stitching is used, these ground hot spots will be indiscriminately mixed with visible light textures in subsequent convolutions, forming blurred blocks resembling limb boundaries in the fused feature map. However, in this step, the structure extraction branch extracts the visible light structure... With infrared temperature Breaking it down, during tensor fusion, the interaction weights between the ground hotspot channel and the visible light limb structure channel learned during training are very low, therefore the structure... It will not strongly respond to false ground temperature channels; at the same time, it truly represents the temperature of the passenger's legs. Then with visible light structure Significant activation is generated in the tensor outer product. In the final fused feature map, ground interference is effectively suppressed, while the outline of the passenger's legs is clearly visible, which provides a more reliable input for step S40 to accurately detect limb key points.

[0076] Step S40: Based on the fused feature map, a physical model-guided temporal three-dimensional attitude and gate dynamic gap measurement mechanism is used to perform the task of calculating the safety gap between the limb and the gate, and the minimum safety gap is output;

[0077] The physical model-guided temporal 3D attitude and gate dynamic gap measurement mechanism in this step uses fused feature maps as the main input to comprehensively calculate the minimum safe gap between the limbs and the gate blades. This step includes three sub-processes: First, the depth refinement head is based on the predicted fine depth residual map. The depth map is obtained by adding it to the channel depth prior map, which gives the three-dimensional Z coordinate of each pixel in the camera coordinate system; secondly, the gate motion state sensing module obtains the real-time angle of the motor. and angular velocity The system generates a gate motion state feature vector through fully connected mapping and feature embedding, and interacts with image features using an attention mechanism, enabling the network to explicitly perceive the current and future three-dimensional spatial positions of the gate blades; third, it predicts the pixel coordinates of limb key points from the depth prior image. The corresponding depth value Zk is indexed from the final depth map, and combined with the camera intrinsic parameters and the three-dimensional line segment equation of the gate blades output by the gate motion model. Calculate the distance from each key point to the target point using the three-dimensional distance formula. The shortest Euclidean distance is used, and the minimum value among all keypoint distances is taken as the minimum safety gap for the current frame. This value is directly passed to step S50 for control decision.

[0078] By introducing a depth residual refinement mechanism, the final depth map recovers the detailed depth variations of the limb surface based on the depth prior, correcting the limb edge depth deviation caused by the smoothness or insufficient accuracy of the prior map, making the 3D backprojection of key points more accurate. The gate motion state is injected into the feature map through attention interaction, enabling the network to simultaneously consider the real-time motion trend of the gate blades when predicting key points, avoiding lag in gap calculation caused by ignoring blade motion. 3D distance calculation directly uses camera intrinsics and the gate physical model. The 2D coordinates and depth values ​​of the key points predicted from the feature map are jointly backprojected to points in the camera coordinate system, and then the Euclidean distance is calculated with the 3D line segments of the gate blades. The entire measurement chain maintains a clear geometric relationship from 2D features to 3D gap values, requiring no additional spatial transformation or post-processing, reducing accumulated errors and ensuring... It can reflect the true relative position of the limbs and the fan blades in a timely manner.

[0079] Existing high-speed rail turnstiles typically rely on infrared light curtains or individual depth sensors for gap determination, which presents challenges in time synchronization and spatial calibration errors. For example, when using independent depth cameras and visible light cameras, inconsistencies in their frame rates and exposure times can lead to mismatches between depth and color images during rapid limb movements. This is especially problematic during the rapid closing of the gate blades, where even minor synchronization differences can cause the measured gap to deviate significantly from the true value. Furthermore, traditional methods often assume the gate blades are fixed planes, ignoring the real-time rotational position changes of the blades driven by the motor. This results in calculated gaps relative to past timestamps rather than the current blade position. This new approach integrates depth residual prediction, gate motion state, and key point detection into a unified network architecture. This ensures all information is based on the same timestamp, and the gate motion model is expressed using real-time angle and angular velocity parameters. The calculated 3D distance corresponds to the precise spatial position of the blades at the current moment, eliminating reliance on externally filtered and delayed data. Consequently, the measurement of the minimum safe gap in high-speed dynamic scenarios is more timely and physically accurate.

[0080] For example, if a passenger suddenly reaches out to block the gate when it is closed to approximately 45 degrees, at which point the motor's angular velocity... The depth is still relatively large. The depth refinement head predicted a negative depth residual for the extended arm region, correcting the overestimated depth value in the depth prior map due to the slender arm, thus obtaining an accurate arm Zk. The gate motion state perception module is based on the current... and Output fan blade 3D line segments The line segment is rotating clockwise. (This is related to calculating the key points of the arm.) At this distance, the fingertip, being the deepest point, has the smallest three-dimensional Euclidean distance to the edge of the fan blade, at 12 centimeters. The value accurately reflects the current urgency, no longer obscured by depth overestimation or blade position lag. Subsequent control algorithms can then make emergency braking decisions based on this, avoiding distance misjudgments that may occur when relying solely on two-dimensional images or independent depth sensors.

[0081] Specifically, such as Figure 2 As shown in the figure, the 3D scatter plot displays the 3D coordinates (blue dots) of the limb keypoints detected from the fused feature map after backprojection, and the 3D line segment L3D(t) of the gate blade generated based on the real-time motor angle (red line segment). Traditional methods often assume the gate is a fixed plane and ignore its rotational position. This application updates the line segment equation in real time through the gate motion model and considers the actual position of the blade when calculating the gap, making the 3D distance formula meaningful. The green dashed line in the figure represents the shortest distance from the keypoint to the line segment, and the minimum value is taken as the final value. This method ensures that the distance calculation is consistent with physical reality.

[0082] Step S50: Based on the minimum safety gap, the risk potential field and model predictive control mechanism are used to execute the motor control command generation task and output the gate motor control signal.

[0083] The risk potential field and model predictive control mechanism in this step use the minimum safety gap output in step S40. and coordinates of key points on the human body Using this as the core input, the gate motor control signal is dynamically generated. In each control cycle, a finite-time model predictive control (MPC) problem is solved. The objective function includes three terms: the integral term of the risk potential norm over the blades, the motor acceleration smoothing term, and the deviation term between the gate speed and the target speed. The motor acceleration control sequence is obtained through online optimization. The first term of the sequence is taken as the current gate motor control signal, driving the motor to perform acceleration, deceleration, or reverse actions.

[0084] The constructed dynamic risk potential field transforms the dispersed limb key points and the three-dimensional relative positions of the fan blades into a continuous potential energy surface in space. Once the edge of the fan blade approaches any key point, the potential field value increases sharply. This provides the MPC optimizer with dense spatial penalty information about the safe state, avoiding the decision-making bias caused by single-point distance thresholds. Through rolling optimization, the MPC controller not only considers the risk at the current moment but also predicts the potential field evolution and actuator limitations over several future control cycles based on the system dynamics model, thus making forward-looking control decisions. The acceleration smoothing term suppresses abrupt changes in motor commands, reducing mechanical shock and passenger discomfort. The speed deviation term ensures that the gate can smoothly return to the preset normal operating speed after the risk is eliminated or eliminated. Overall, this mechanism integrates risk quantification and motion prediction, enabling the anti-pinch action to both quickly brake in dangerous situations and smoothly transition in safe situations, improving control smoothness.

[0085] Traditional turnstile anti-pinch control often uses fixed threshold logic, such as immediately triggering braking when the infrared light curtain is blocked or the ultrasonic ranging is less than a preset value. This hard threshold method cannot distinguish between small floating objects and real limbs; low-risk items such as packaging bags and backpack straps can also cause sudden stops, resulting in frequent false braking. Furthermore, the braking process is rigid and does not dynamically adjust the braking intensity according to the closing speed and distance. In addition, distance fluctuations near the threshold can cause repeated start-stop cycles, shortening the motor's lifespan. This step, through risk potential field modeling, integrates distance, key point distribution, and fan blade movement speed into a smooth risk index without hard boundaries, avoiding threshold oscillations. Based on this, MPC solves a constrained optimization problem, calculating a motor acceleration trajectory that is both safe and as gentle as possible. This achieves a pre-prediction and controllable deceleration strategy, rather than suddenly locking up after exceeding the safe distance, thus improving passage efficiency and passenger experience while ensuring that the safety clearance is not violated.

[0086] For example, when a passenger has successfully passed through the gate, but their backpack strap is still moving with their body within the passageway, the key points detected by the fusion of visible light and infrared features include not only the upper body of the passenger but also several isolated points on the backpack strap. When constructing the risk potential field, the key points on the backpack strap are relatively far from the fan blades, and due to the motor angular velocity... The deceleration had already begun in the previous moment, and the value was relatively small. Therefore, the potential field strength contributed by these key points was limited, and the overall potential field value did not reach a dangerous level. During the MPC solution, based on the system model's prediction that if the current deceleration rate was maintained, the potential field value would not increase significantly in the next few cycles, and the acceleration smoothing term ensured that the control sequence did not produce abrupt changes. The optimization result was to maintain the current gentle deceleration trajectory. Finally, the motor control signal executed gentle braking, and the gate closed smoothly after the backpack strap completely exited the channel, without any accidental triggering of reverse opening. This fine control makes the anti-pinch system more tolerant of slender, low-risk carried items.

[0087] Specifically, such as Figure 3 As shown in the figure, this diagram compares the abrupt acceleration changes of traditional emergency stops with the smooth acceleration sequence optimized by MPC. Traditional control methods involve abrupt acceleration changes, which have a significant impact on the motor and mechanical structure; MPC, through smoothing term constraints, allows the acceleration to change gradually, achieving flexible braking, extending equipment lifespan, and reducing passenger shock. Figure 4 As shown in the figure, the acceleration control sequence obtained by MPC solution is illustrated, with only the first term executed as the current motor control signal. The output signal is continuous and has a reasonable amplitude, indicating that the control command is smooth and executable. This method achieves closed-loop control by combining the risk potential field, prediction model, and real-time feedback, making the gate motor control command both safe and efficient, no longer relying on simple hard threshold logic.

[0088] Example 2: Furthermore, the high-speed rail turnstile anti-pinch control system based on multimodal perception provided by the present invention, employing the high-speed rail turnstile anti-pinch control method based on multimodal perception in the above embodiments, can solve the technical problem of high-speed rail turnstile anti-pinch control based on multimodal perception. The beneficial effects of the high-speed rail turnstile anti-pinch control system based on multimodal perception provided by the present invention are the same as those of the high-speed rail turnstile anti-pinch control method based on multimodal perception provided in the above embodiments, and other technical features in the high-speed rail turnstile anti-pinch control system based on multimodal perception are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0089] Example 3: This invention provides a high-speed rail turnstile anti-pinch control device based on multimodal perception. The device includes at least one processor and a memory communicatively connected to the processor. The memory stores instructions executable by the processor, which are then executed to enable the processor to perform the high-speed rail turnstile anti-pinch control method based on multimodal perception described in Example 1. The high-speed rail turnstile anti-pinch control device based on multimodal perception in this invention can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. This high-speed rail turnstile anti-pinch control device based on multimodal perception is merely an example and should not limit the functionality or scope of the invention. The high-speed rail turnstile anti-pinch control device based on multimodal perception can include a processing unit (such as a central processing unit, graphics processing unit, etc.), which can execute various appropriate actions and processes according to the program stored in read-only memory or the program loaded from the storage device into random access memory. The random access memory also stores various programs and data required for the operation of the high-speed rail turnstile anti-pinch control device based on multimodal perception. The processing unit, read-only memory, and random access memory are interconnected via a bus. I / O interfaces are also connected to the bus. Typically, the following systems can be connected to the I / O interface: input devices including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tape, hard disks, etc.; and communication devices. The communication device allows the high-speed rail turnstile anti-pinch control device based on multimodal perception to communicate wirelessly or wiredly with other devices to exchange data. While multimodal sensing-based anti-pinch control devices for high-speed rail turnstiles with various systems have been described, it should be understood that implementation of all the described systems is not required. More or fewer systems may be implemented alternatively.

[0090] Example 4: This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multimodal perception-based high-speed rail turnstile anti-pinch control method described above. The computer program product provided by this invention can solve the technical problem of multimodal perception-based high-speed rail turnstile anti-pinch control. Compared with the prior art, the beneficial effects of the computer program product provided by this invention are the same as those of the multimodal perception-based high-speed rail turnstile anti-pinch control method provided in the above embodiments, and will not be repeated here.

[0091] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a read-only memory. When the computer program is executed by a processing device, it performs the functions defined in the methods of the embodiments disclosed in this invention.

[0092] It should be understood that the various parts disclosed in this invention can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0093] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A high-speed rail turnstile anti-mispinch control method based on multimodal sensing, characterized in that, The methods include: Step S10: Obtain a pair of visible light images and infrared thermal images collected with a unified time stamp within the gate channel. Based on the pair of visible light images and infrared thermal images, perform the task of extracting features of passenger limbs and personal objects using an asymmetric dual-stream feature extraction mechanism, and output a visible light multi-scale feature map and an infrared multi-scale feature map. Step S20: Based on the visible light multi-scale feature map and the infrared multi-scale feature map, a depth perception and deformation field estimation network mechanism is used to perform the spatial registration task, and output the infrared feature map registered to the visible light coordinate system. Step S30: Based on the infrared feature map registered to the visible light coordinate system and the corresponding scale visible light feature map, a cross-modal tensor fusion mechanism with structure and texture decoupling is used to perform the feature fusion task and output the fused feature map; Step S40: Based on the fused feature map, a physical model-guided temporal three-dimensional attitude and gate dynamic gap measurement mechanism is used to perform the task of calculating the safety gap between the limb and the gate, and the minimum safety gap is output; Step S50: Based on the minimum safety gap, the risk potential field and model predictive control mechanism are used to execute the motor control command generation task and output the gate motor control signal.

2. The high-speed rail turnstile anti-mispinch control method based on multimodal sensing as described in claim 1, characterized in that, In step S10, the steps of acquiring a pair of visible light images and infrared thermal images with a unified spatiotemporal stamp within the gate channel, and performing feature extraction of passengers' limbs and personal objects using an asymmetric dual-stream feature extraction mechanism based on the visible light images and infrared thermal images, and outputting visible light multi-scale feature maps and infrared multi-scale feature maps, specifically include: Step S101: Using the gate wing opening and closing status signal within the same gate passage cycle as the trigger reference, collect visible light images and infrared thermal images with the same timestamp, and input them into the visual Transformer network and infrared convolutional network respectively for initial feature extraction. Step S102: In the shallow output of the visual Transformer network, retain the texture features of passenger clothing edges, luggage outlines and door wing boundaries, and in the temporal consistency module of the infrared convolutional network, retain the thermal response features of human skin, limb ends and under occlusion conditions, and suppress feature jumps caused by gate light strip reflection, glass reflection and environmental heat source disturbance. Step S103: Generate visible light multi-scale feature maps and infrared multi-scale feature map ,in, The resolution level is indicated; the visible light multi-scale feature map includes a visible light mid-level feature map and a visible light structural feature map, and the infrared multi-scale feature map includes an infrared mid-level feature map and an infrared temperature feature map.

3. The high-speed rail turnstile anti-mispinch control method based on multimodal sensing as described in claim 1, characterized in that, Step S20, which involves performing spatial registration based on the visible light multi-scale feature map and the infrared multi-scale feature map using a depth sensing and deformation field estimation network mechanism, and outputting an infrared feature map registered to the visible light coordinate system, specifically includes: Step S201: Obtain the visible light mid-layer feature map and infrared mid-spheric feature map Input to a depth-aware deformation field estimation network, which predicts the initial warping displacement using a multi-scale spatial convolutional encoder. ; Step S202: The initial warpage displacement is refined using a depth modulation formula to obtain a refined displacement field. The depth modulation formula is as follows: ； in, This represents the initial warping displacement. This represents the initial displacement of the infrared feature point in the lateral direction of the visible light coordinate system. This represents the initial displacement of the infrared feature point in the longitudinal direction of the visible light coordinate system. To refine the displacement field, This refers to the precise lateral displacement. For longitudinal fine-tuning of displacement; This is a depth prior map generated based on the depth calibration results of the turnstile channel; Learnable deep modulated convolutional kernels; Modulation intensity factor; This is pixel-by-pixel multiplication; It is the hyperbolic tangent activation function; Step S203: Based on the refined displacement field and bicubic interpolation resampling, the infrared temperature feature map is mapped to the visible light coordinate system to obtain an infrared feature map registered to the visible light coordinate system.

4. The high-speed rail turnstile anti-mispinch control method based on multimodal sensing as described in claim 3, characterized in that, In step S202, the modulation term in the depth modulation formula The modulation term is used to constrain the deformation prediction amplitude based on different depth regions within the gate channel. Specifically, in the near-field region near the gate wing edge, the modulation term is close to 1 to maintain limb contour registration accuracy; in the region far from the gate wing edge, the modulation term is processed by a learnable depth modulation convolution kernel. The deformation prediction amplitude is adaptively adjusted to compensate for the nonlinear shift caused by the inconsistency in the viewing angles of the infrared and visible light lenses.

5. The high-speed rail turnstile anti-mispinch control method based on multimodal sensing as described in claim 1, characterized in that, Step S30, which involves performing a feature fusion task based on the infrared feature map registered to the visible light coordinate system and the corresponding scale visible light feature map using a cross-modal tensor fusion mechanism that decouples structure and texture, and outputting the fused feature map, specifically includes: Step S301: Decompose the visible light structural feature map, the visible light mid-layer feature map, the infrared feature map registered to the visible light coordinate system, and the infrared temperature feature map through the structure extraction branch and the texture extraction branch respectively to obtain the visible light structural features. Visible light texture features Infrared structural features and infrared temperature characteristics ; Step S302: Using a four-dimensional weighted tensor approximated by tensor ring decomposition with a low rank, perform cross-modal fusion on the visible light structural features, visible light texture features, infrared structural features, and infrared temperature features. The fusion formula is as follows: ； in, To fuse feature maps; For the first Visible light structural features of each channel; For the first Visible light texture features of each channel; For the first Infrared structural features of each channel; For the first Infrared temperature characteristics of each channel; The fusion weights are the corresponding channel combinations in the four-dimensional weight tensor; , , , All are feature channel indexes; This indicates that the summation is performed on all channel combinations. Indicates the convolution operation; Step S303: Through the cross-modal fusion, the passenger's limb contour, skin thermal response, baggage obstruction boundary, and door wing edge structure are uniformly mapped to the same feature space to generate the fused feature map.

6. The high-speed rail turnstile anti-mispinch control method based on multimodal sensing as described in claim 1, characterized in that, In step S40, the step of performing the task of calculating the safety gap between the limb and the gate based on the fused feature map using a physical model-guided temporal three-dimensional attitude and gate dynamic gap measurement mechanism, and outputting the minimum safety gap, specifically includes: Step S401: The depth refinement head predicts the depth residual map of the passenger's limb region based on the fused feature map, and fuses the depth residual map with the gate channel depth prior map to obtain the final depth map; Step S402: The gate motion state perception module maps the real-time angle and angular velocity of the gate motor into a gate motion state feature vector, and performs attention interaction between the gate motion state feature vector and the fused feature map; Step S403: Predict the pixel coordinates of limb key points from the fused feature map, index the corresponding depth values ​​from the final depth map, and calculate the real-time 3D distance from the limb key points to the edge of the gate blades by combining the camera intrinsic parameters and the 3D line segment equation of the gate blades. Take the minimum value among all limb key point distances as the minimum safety gap. The formula for calculating the real-time 3D distance is: ； ； in, for Time of the first Real-time 3D distance from each limb key point to the 3D line segment of the gate blade; for Minimum safe gap at any given time; and The first The horizontal and vertical pixel coordinates of key points of a limb in the visible light coordinate system; and These are the horizontal and vertical coordinates of the camera's principal point, respectively. For the first The depth values ​​of key limb points in the final depth map; and These refer to the camera's horizontal focal length and vertical focal length, respectively. for The three-dimensional line segment of the gate blade is obtained by real-time angle mapping of the gate motor. Candidate points on the three-dimensional line segment of the gate blade; Let L2 be the norm of the vector; Indicate candidate points Located in the three-dimensional line segment of the gate fan blade superior; This means traversing all limb key points and taking the minimum distance.

7. The high-speed rail turnstile anti-mispinch control method based on multimodal sensing as described in claim 1, characterized in that, Step S50, which involves generating motor control commands based on the minimum safety gap using a risk potential field and model predictive control mechanism, and outputting the gate motor control signal, specifically includes: Step S501: Construct a dynamic risk potential field based on the coordinates of key human body points and the 3D model of the gate blades, and control the flow at the specified time. The risk is quantified by determining the safe distance between points on the gate blades and key points on the human body. The formula for the risk potential field strength is: ； in, To control the timing At the gate fan blade point The strength of the risk field at the location; To control the timing Candidate points on the gate blades; To control the timing No. Three-dimensional coordinates of key points of a limb; The angular velocity of the gate motor; This is the gain coefficient; To avoid constants with a denominator of zero; This indicates the summation of all key points of the limbs; Step S502: In each control cycle, solve the model predictive control problem in the finite time domain based on the minimum safety gap and the risk potential field strength. The optimization objective is to minimize the objective function that includes the risk potential field norm, the motor acceleration smoothing term, and the deviation term between the gate angular velocity and the target velocity, so as to obtain the motor acceleration control sequence. Step S503: Extract the first term of the motor acceleration control sequence as the motor control command for the current control cycle, and convert the motor control command into a gate motor control signal and output it to the gate motor driver.

8. A high-speed rail turnstile anti-mispinch control system based on multimodal sensing, applied to the high-speed rail turnstile anti-mispinch control method based on multimodal sensing as described in any one of claims 1 to 7, characterized in that, The high-speed rail turnstile anti-mispinch control system based on multimodal perception includes: The feature extraction module is used to acquire visible light images and infrared thermal images collected with a unified time stamp within the gate channel. Based on the visible light images and infrared thermal images, an asymmetric dual-stream feature extraction mechanism is used to perform the feature extraction task of passenger limbs and personal objects, and output visible light multi-scale feature maps and infrared multi-scale feature maps. The spatial registration module is used to perform spatial registration tasks based on the visible light multi-scale feature map and the infrared multi-scale feature map using a depth perception and deformation field estimation network mechanism, and outputs an infrared feature map registered to the visible light coordinate system. The feature fusion module is used to perform a feature fusion task based on the infrared feature map registered to the visible light coordinate system and the visible light feature map of the corresponding scale, using a cross-modal tensor fusion mechanism that decouples structure and texture, and outputs a fused feature map. The gap calculation module is used to perform the task of calculating the safety gap between the limb and the gate based on the fused feature map and the time-series three-dimensional posture and gate dynamic gap measurement mechanism guided by the physical model, and output the minimum safety gap. The control generation module is used to generate motor control commands based on the minimum safety gap using a risk potential field and model prediction control mechanism, and output gate motor control signals.

9. A high-speed rail turnstile anti-mispinch control device based on multimodal sensing, characterized in that, The high-speed rail turnstile anti-mispinch control device based on multimodal perception includes: a memory, a processor, and a high-speed rail turnstile anti-mispinch control program based on multimodal perception stored in the memory and executable on the processor. When the high-speed rail turnstile anti-mispinch control program based on multimodal perception is executed by the processor, it implements the high-speed rail turnstile anti-mispinch control method based on multimodal perception as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a high-speed rail turnstile anti-mispinch control program based on multimodal perception. When the high-speed rail turnstile anti-mispinch control program based on multimodal perception is executed by the processor, it implements the high-speed rail turnstile anti-mispinch control method based on multimodal perception as described in any one of claims 1 to 7.

Images