Low light gesture control method for in-vehicle environment and system therefor

Abstract

The application discloses a low-light gesture control method and system for an in-vehicle environment, and relates to the technical field of gesture control.The control method comprises the following steps: S1, a data synchronous acquisition and registration step; S2, a stable depth data generation step; S3, a light self-adaptive adjustment step; S4, a multi-dimensional reliability parameter acquisition step; S5, a comprehensive reliability generation step; and S6, a control output or inhibition step.The application synchronously acquires depth data and black light structure image data under the same visual angle in the vehicle through a TOF image module and a black light imaging module, and utilizes hand contour information in the black light structure image to participate in the fusion processing of the TOF depth data, so that the influence of depth jump, local cavities and contour fracture in the hand edge region on gesture recognition input is reduced.

Description

Technical Field

[0001] This invention relates to the field of gesture control technology, and in particular to a low-light gesture control method and system for use in vehicle environments. Background Technology

[0002] Gesture control technology refers to an interactive method in which the device captures the movement trajectory of the hand by moving it above the camera or sensor and converts it into operation commands.

[0003] With the development of smart cockpits, non-contact gesture recognition has become an important way for drivers and passengers to operate in-vehicle equipment (such as multimedia, air conditioning, and seats). In low-light conditions such as night driving, tunnels, or backlighting, drivers or passengers often need to operate in-vehicle multimedia, air conditioning, or driver assistance systems in a non-contact manner.

[0004] However, the lighting conditions inside the vehicle are complex, especially when driving at night, in tunnels, or in backlit environments, the system's stability suffers from the following issues: 1. Poor image quality: The accuracy of existing imaging systems drops significantly in low light conditions; 2. Depth jitter: A single TOF depth system is prone to boundary breakage, depth jitter, and discontinuity of hand contours under complex lighting conditions, leading to misjudgment of gestures.

[0005] 3. Safety risks: Due to unstable sensing quality, false triggering can easily occur, affecting driving safety; Therefore, we propose a low-light gesture control method and system for in-vehicle environments. Summary of the Invention

[0006] The purpose of this invention is to provide a low-light gesture control method and system for in-vehicle environments to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a low-light gesture control method for use in a vehicle interior environment, comprising the following steps: S1. Data Synchronization Acquisition and Registration Steps: Using a common reference package and hardware-triggered synchronization, the TOF image module and black light imaging module synchronously acquire the original TOF depth data, the corresponding measurement confidence map, and the black light structure image data from the same viewpoint inside the vehicle; the TOF depth data is subjected to distortion correction and coordinate mapping by pre-calibrated intrinsic and extrinsic parameter matrices, and the TOF depth data is upsampled to the same resolution as the black light structure image data through interpolation processing, so that the two data streams are aligned to a unified pixel coordinate system; S2. Stable Depth Data Generation Steps: Perform multi-scale spatial gradient pyramid decomposition on the registered black light structure image data, extract gradient features at each scale layer and construct corresponding scale gradient guided weights, and fuse them to obtain comprehensive gradient guided weights; use the TOF measurement confidence map as an auxiliary weight and fuse it with the comprehensive gradient guided weights to obtain a composite weight map; based on the composite weight map, perform adaptive spatiotemporal fusion processing on the current frame TOF depth data and multiple frames of historical depth data aligned by motion vector compensation to obtain optimized depth data; S3, Illumination Adaptive Adjustment Step: Real-time acquisition of the ambient light level inside the vehicle, and dynamic adjustment of the gradient-sensitive adjustment parameters corresponding to the multi-scale gradient pyramid in step S2 based on the light level, so that the spatiotemporal fusion strategy adapts to the light conditions. S4. Multidimensional credibility parameter acquisition steps: Transmit multidimensional driving state parameters such as vehicle speed, acceleration, and steering wheel angle to the gesture control system via the vehicle bus; calculate the state factors corresponding to each parameter using a piecewise Sigmoid function, and obtain the driving state weights through weighted product operation; input the optimized depth data into the lightweight gesture recognition model deployed at the edge for inference, and output the visual stability score corresponding to the gesture recognition; calculate the spatial constraint score based on the hand spatial position and sensor distance using a hysteresis soft boundary function; S5. Comprehensive Credibility Generation Steps: Fusion of driving state weights, visual stability scores and spatial constraint scores, smoothing of the fusion result by a sliding time window and filtering out outliers, and generation of comprehensive credibility of gesture control. S6. Control output or suppress step: Compare the overall confidence level with the pre-calibrated confidence level threshold, and output the vehicle control command or suppress the current gesture operation based on the comparison result.

[0008] Preferably, the specific steps of multi-scale spatial gradient pyramid decomposition and composite weight construction in step S2 are as follows: Step S2.1: Perform L-level Gaussian pyramid decomposition on the registered black light structure image. Calculate the horizontal and vertical gradients at each scale layer l (l=0,1,...,L-1) to obtain the gradient magnitude field at that scale. Where L≥3, different scale layers capture edge information of different spatial frequencies. The coarse-scale layer responds to the overall contour edge of the hand, while the fine-scale layer responds to the fine structural edges such as fingertips and joints. Step S2.2: Construct gradient-guided weights for the gradient magnitude field of each scale layer, and synthesize the weights of each layer into a comprehensive gradient-guided weight Wg(x,y) through a cross-scale weighted fusion strategy. During cross-scale fusion, the fusion coefficients of each scale layer are dynamically allocated according to the current ambient light level. Step S2.3: Normalize the measurement confidence map Ct(x,y) output by the TOF sensor to the [0,1] interval, and perform a pixel-by-pixel multiplication operation with the comprehensive gradient guidance weight Wg(x,y) to obtain the composite weight map Wc(x,y)=Wg(x,y)·Ct(x,y), so that the historical frame constraints are automatically enhanced in the region of low TOF measurement confidence.

[0009] Preferably, the specific steps of adaptive spatiotemporal fusion in step S2 are as follows: Step S2.4: Calculate the motion vector field of each pixel based on the difference of stable depth data from adjacent frames, and perform motion vector compensation alignment on the historical depth data of the previous N frames to eliminate inter-frame spatial misalignment caused by hand movement; wherein the number of historical reference frames N is adaptively adjusted according to the speed of hand movement: when the global mean of the motion vector field is less than the preset static threshold, N takes a larger value to enhance the temporal domain smoothing effect; when the global mean exceeds the preset dynamic threshold, N takes a smaller value to preserve the motion response speed; Step S2.5: Use the composite weight map Wc(x,y) to perform weighted fusion of the current frame TOF depth data and the motion-compensated N frames of historical depth data: in flat, high-confidence regions with high Wc(x,y) values, the current frame is dominant; in edge or low-confidence regions with low Wc(x,y) values, the weighted average of multiple frames of historical data is dominant, and a stable depth map with continuous contours is output.

[0010] Preferably, the specific steps of the adaptive adjustment of ambient light in step S3 are as follows: Step S3.1: The ambient light sensor integrated in the sensor module collects the ambient illuminance value Lux(t) in the vehicle in real time and maps the illuminance value to the light level E(t); the light level is divided into at least three intervals: normal light interval, low light interval and extremely dark interval. Step S3.2: Dynamically adjust the gradient sensitivity adjustment parameter αl of each scale layer in the multi-scale gradient pyramid according to the illumination level E(t): In the normal illumination range, the αl of each layer is taken to make the fusion weight moderately sensitive to gradient changes; in the low light range, the αl value of the fine scale layer is increased to suppress the pseudo gradient response amplified by noise, while keeping the αl of the coarse scale layer unchanged to maintain the contour guiding ability; in the extremely dark range, the αl value of each layer is further increased and the number of historical frame reference frames N is increased to enhance the temporal domain constraint.

[0011] Preferably, the calculation steps for the driving state weights in step S4 are as follows: Step S4.1: Obtain the vehicle state vector in real time through the vehicle CAN bus, including real-time vehicle speed v(t), real-time acceleration a(t), and steering wheel angle θ(t); Step S4.2: Calculate a state factor for each state parameter through a piecewise Sigmoid function: The vehicle speed state factor fv(t)=1 / (1+exp(kv·(|v(t)|-v0)), where v0 is the vehicle speed value corresponding to the center point of the Sigmoid function, and kv is the slope adjustment parameter; this function outputs a value close to 1 in the low-speed section, transitions smoothly near v0, and outputs a value close to 0 in the high-speed section, achieving the piecewise control effect of full permission at low speed, strong suppression at high speed, and smooth transition at medium speed; The acceleration state factor fa(t) and the steering angle state factor fθ(t) adopt the same Sigmoid function form, and are calculated independently using their respective corresponding center point parameters and slope parameters; Step S4.3: Perform a weighted product operation on each state factor to obtain the driving state weight Wv=fv(t)^wv·fa(t)^wa·fθ(t)^wθ, where wv, wa, and wθ are the exponential weights of each factor, which are determined by regression through the in-vehicle driving scene calibration dataset.

[0012] Preferably, the calculation steps of the hysteresis soft boundary space constraint score in step S4 are as follows: Step S4.4: Obtain the spatial distance d(t) between the hand feature point and the in-vehicle sensor, expand the inner and outer boundaries of the interaction interval (dmin, dmax) by a transition width Δ respectively, to form an inner transition zone [dmin-Δ, dmin] and an outer transition zone [dmax, dmax+Δ]; Step S4.5: Within the core interaction interval (dmin, dmax), the space constraint score Sd=1; within the inner transition zone and the outer transition zone, the space constraint score smoothly attenuates from 1 to 0 through the Sigmoid soft boundary function; outside the transition zones, Sd=0; Step S4.6: Apply a hysteresis mechanism to the space constraint score: set an entry threshold Th_enter and an exit threshold Th_exit, where Th_enter>Th_exit. When the hand enters from outside the interaction interval, activation is triggered only when Sd≥Th_enter; after activation, when the hand moves outward toward the boundary, deactivation is not triggered until Sd<Th_exit, so as to avoid repeated chattering near the boundary.

[0013] Preferably, the specific steps of comprehensive credibility calculation and robust processing in step S5 are as follows: Step S5.1: Perform a product operation on the driving state weight Wv, the visual stability score Ws and the space constraint score Sd to obtain the instantaneous credibility Cinst=Ws·Wv·Sd; Step S5.2: Perform median filtering on the instantaneous credibility sequence of the most recent M frames to filter out peak outliers caused by CAN bus packet loss or single-frame misjudgment of the neural network; Step S5.3: Perform weighted average smoothing on the median-filtered sequence within a sliding time window K to obtain the final comprehensive confidence Ccar. The time window K is adaptively adjusted according to the current hand movement speed: K takes a larger value for static gestures to enhance decision stability, and K takes a smaller value for dynamic gestures to ensure response speed.

[0014] Preferably, the specific implementation of data registration in step S1 is as follows: Step S1.1: During the system manufacturing stage, the intrinsic parameters of the TOF image module and the black light imaging module are calibrated using a calibration board to obtain their respective focal length, principal point and distortion coefficients. The extrinsic rotation matrix and translation vector between the two modules are obtained through stereo calibration. Step S1.2 During runtime, based on the pre-calibrated intrinsic and extrinsic parameter matrices, distortion correction and coordinate transformation are performed on the pixel coordinates of each pixel in the TOF depth data, and the data is projected onto the pixel coordinate system of the black light structure image. Step S1.3: The sparse TOF depth data after projection is upsampled on the resolution grid of the black light structure image through interpolation processing to obtain a TOF depth map and its measurement confidence map that are spatially aligned with the black light structure image and have the same resolution. Step S1.4: During vehicle operation, periodically extract hand feature points from the black light structure image and determine the registration offset between the hand feature points and the TOF depth data projection points; when the registration offset continuously exceeds the preset offset threshold, fine-tune the external parameter rotation matrix and translation vector between the TOF image module and the black light imaging module to compensate for the optical axis drift caused by vehicle vibration.

[0015] Preferably, the lightweight gesture recognition network deployed at the edge in step S4 is designed and deployed as follows: The lightweight gesture recognition network adopts an encoder-classifier architecture based on depthwise separable convolutions as building blocks. The encoder part receives dual-channel data consisting of a single-channel input from a stable depth map and a single-channel input from a black light image. After extracting spatial features through multiple layers of depthwise separable convolutions, it outputs a feature vector through global average pooling. The classifier part outputs the gesture category probability distribution through a fully connected layer. The network is trained using a dataset that includes multiple lighting conditions and hand poses inside the vehicle during the training phase, and soft label guidance is obtained from a large teacher network through knowledge distillation. During the deployment phase, it is deployed on the NPU of the vehicle edge computing unit after optimization by INT8 quantization and operator fusion, and the inference latency does not exceed a single frame acquisition cycle.

[0016] Secondly, the present invention provides a low-light gesture control system for in-vehicle environments, implementing the method described above, the system comprising: The sensor collaborative acquisition unit includes a TOF image module, a black light imaging module, and an ambient light sensor packaged on the same optical reference bracket. The three share a hardware synchronous trigger signal line. The TOF image module outputs raw depth data and measurement confidence map, the black light imaging module outputs low-light high-contrast structural image, and the ambient light sensor outputs ambient illuminance value. The vehicle-mounted edge computing unit includes a heterogeneous computing processor, which has a CPU core and an NPU core. The CPU core runs a heterogeneous resolution registration module, a multi-scale gradient pyramid decomposition and adaptive spatiotemporal fusion algorithm, a driving state weight calculation module, and a comprehensive credibility calculation and robust processing module. The NPU core runs a lightweight gesture recognition network quantized by INT8. The vehicle bus interface module connects to the vehicle control system via the CAN bus, acquires real-time data on vehicle speed, acceleration, and steering wheel angle, and outputs gesture control commands to the vehicle controller via the bus. The sensor collaborative acquisition unit and the vehicle-mounted edge computing unit transmit image data through the MIPICSI interface, and the CPU core and NPU core inside the vehicle-mounted edge computing unit exchange data through shared memory.

[0017] The technical effects and advantages of this invention are as follows: This invention simultaneously acquires depth data and black light structural image data from the same viewpoint inside the vehicle using a Time-of-Flight (TOF) imaging module and a black light imaging module. After registration, gradient edge information from the black light structural image is used to participate in TOF depth data fusion. Since the black light structural image can still provide hand contour boundaries in low-light environments, the system no longer relies solely on the raw depth results of a single frame of TOF in the hand edge region. Instead, it combines historical depth data for constraint, thereby reducing the impact of depth jumps, local holes, and contour breaks in the hand edge region on gesture recognition input.

[0018] This invention divides the illumination range into normal illumination, low light, and extremely dark ranges based on the illuminance values ​​collected by the ambient light intensity detection module, and adjusts the fusion method of multi-scale gradient weights according to different illumination ranges. When the illumination is strong, the fusion process retains more fine-scale edge information; when the illumination is weak, the fusion process enhances the effect of coarse-scale contour information and historical frame constraints, thereby reducing the impact of low-light noise, false edges, and single-frame depth jitter on stable depth data.

[0019] Before outputting vehicle control commands, this invention uses visual stability score, driving state weight, and spatial constraint results together to calculate comprehensive credibility, and compares the comprehensive credibility with a preset credibility threshold. When the vehicle is at high speed, accelerating rapidly, turning sharply, or the hand is outside the preset interaction range, even if the visual recognition result shows a similar hand gesture, the output of control commands will be suppressed due to the reduced driving state weight or unmet spatial constraints. This reduces false triggering caused by non-interactive actions, vehicle motion disturbances, and single-frame recognition fluctuations. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention provides, for example Figures 1-2 The method for low-light gesture control in a vehicle interior environment, as shown, includes the following specific steps: Step 1: Simultaneously acquire TOF raw depth data and black light structure image data from the same viewpoint inside the vehicle. The black light structure image data is high-contrast image data of the hand under low light or dim light conditions.

[0023] The specific details of acquiring TOF raw depth data and black light structure image data in step 1 are as follows: A hardware-synchronized TOF image module and a black light imaging module are used to acquire data from the same viewpoint. The TOF image module and the black light imaging module are packaged on the same optical reference plane, and shutter synchronization is achieved through hardware trigger pins to avoid fusion errors caused by sampling time difference and spatial offset.

[0024] Specifically, the TOF imaging module and the black light imaging module are packaged on the same optical reference plane. The specific implementation method is as follows: an integrated optical bracket is used to fix the photosensitive chip and lens group of the two modules on the same reference mounting surface to ensure that the optical axes of the two modules are parallel and the viewing angles are completely coincident, thus eliminating viewing angle offset from a physical level; the two modules share the same optical window or use a beam splitter structure to ensure that the light incident path is consistent and to avoid field of view deviation and hand imaging position misalignment caused by differences in installation position.

[0025] Further, the specific implementation of data registration in step 1 is as follows: During the system manufacturing stage, the intrinsic parameters of the TOF image module and the black light imaging module are calibrated using a calibration board to obtain their respective focal length, principal point, and distortion coefficients. The extrinsic rotation matrix and translation vector between the two modules are obtained through stereo calibration. During runtime, based on the pre-calibrated intrinsic and extrinsic parameter matrices, distortion correction and coordinate transformation are performed on each pixel coordinate of the TOF depth data, projecting it onto the pixel coordinate system of the black light structure image. The projected sparse TOF depth data is upsampled on the resolution grid of the black light structure image through interpolation processing to obtain a TOF depth map that is spatially aligned with the black light structure image and has the same resolution. Its measurement confidence map; during vehicle operation, the hand edge feature points or joint points in the black light structure image are periodically extracted, and the corresponding point positions after the current TOF depth data is projected onto the black light structure image coordinate system are calculated. When the registration offset continuously exceeds the preset offset threshold, the rotation matrix and translation vector of the external parameters between the TOF image module and the black light imaging module are finely adjusted based on the pre-calibrated external parameters, and the projection mapping of the TOF depth data to the black light structure image coordinate system is re-executed using the corrected external parameters to compensate for the optical axis drift caused by vehicle vibration; the intrinsic parameter matrix adopts the factory calibration result and is not adjusted in real time during vehicle operation.

[0026] Step 1 can eliminate sampling time difference, avoid dynamic scene fusion error, eliminate viewpoint offset, improve depth fusion accuracy, adapt to complex vehicle environment, improve system robustness, ensure perception accuracy in low light environment, and provide a foundation for real-time fusion at the edge.

[0027] Step 2: Based on the spatial features extracted from the black light structure image data, construct weight rules and perform spatiotemporal fusion processing on the original TOF depth data to obtain stable depth data with continuous contours.

[0028] By employing a technical approach that combines black light spatial gradient feature guidance, gradient-sensitive weighting functions, and spatiotemporal differentiated weighted fusion, we have achieved precise stabilization of TOF depth data. This approach not only solves the problems of low-light failure, depth jitter, and contour fragmentation in existing technologies, but also adapts to the computing power and real-time requirements of automotive edge computing, providing key perception layer technology support for precise gesture control in low-light environments inside vehicles.

[0029] The specific steps for constructing weight rules and performing spatiotemporal fusion processing in step 2 are as follows: Step 2.1: Extract the spatial gradient features of the black light structure image data, and construct a gradient-sensitive weight function based on the spatial gradient features. The value of the weight function decreases as the magnitude of the spatial gradient increases. Step 2.2: Use a weighting function to perform weighted spatiotemporal fusion of the current TOF raw depth data and the previous TOF historical depth data, weaken the weight of the TOF raw depth data in the hand edge region, and strengthen the constraint effect of the temporal depth data to eliminate edge burrs and contour fragmentation problems.

[0030] Furthermore, in multi-scale spatial gradient pyramid processing, black light structured images can be decomposed into coarse-scale, meso-scale, and fine-scale layers. The coarse-scale layer is mainly used to extract the outer contour of the palm and the overall boundary of the hand; the meso-scale layer is mainly used to extract the finger connection area and the palm-finger transition boundary; and the fine-scale layer is mainly used to extract the local edges near the fingertips, finger gaps, and joints. The system calculates the gradient magnitude field of each scale layer separately and then sums the gradient guidance weights corresponding to each scale layer according to preset fusion coefficients to obtain the comprehensive gradient guidance weights. When the ambient light is strong, the fusion coefficients of the fine-scale and meso-scale layers are relatively increased to preserve gesture details; when the ambient light is weak, the fusion coefficient of the coarse-scale layer is relatively increased to reduce the impact of low-light noise and false edges on the fusion result.

[0031] In one embodiment, the vehicle-mounted edge computing unit also receives the in-vehicle ambient illuminance value collected by the ambient light intensity detection module and determines the lighting level based on the ambient illuminance value. The lighting level is divided into a normal lighting range, a low-light range, and an extremely dark range; wherein, the normal lighting range is the range with an ambient illuminance value greater than or equal to 50 Lux, the low-light range is the range with an ambient illuminance value greater than or equal to 5 Lux and less than 50 Lux, and the extremely dark range is the range with an ambient illuminance value less than 5 Lux. The above-mentioned illuminance ranges can be calibrated and adjusted according to the cabin structure, sensor installation position, and in-vehicle light source arrangement of different vehicle models.

[0032] In the normal illumination range, the gradient sensitivity adjustment parameter α is set to the first parameter value; in the low light range, the gradient sensitivity adjustment parameter α is set to the second parameter value, which is greater than the first parameter value; in the extremely dark range, the gradient sensitivity adjustment parameter α is set to the third parameter value, which is greater than the second parameter value. In addition, the participation ratio of historical depth data in spatiotemporal fusion is increased to reduce the impact of low light noise and false edges on stable depth data.

[0033] The formula for calculating the weight function is as follows: W(x,y)=exp(-α·∣) I(x,y)|); Where α is the gradient-sensitive adjustment parameter, | I(x, y)| represents the spatial gradient magnitude of the black light structure image at pixel coordinates (x, y). I(x, y) = ( I / x, I / y) represents the horizontal and vertical spatial gradients of the black light image.

[0034] The calculation formula for spatiotemporal weighted fusion is as follows: D′(x,y,t)=W(x,y)·D(x,y,t)+(1-W(x,y))·D(x,y,t-1); Where D′(x, y, t) is the stable depth map at the current time, D(x, y, t) is the original TOF depth map at the current time, D(x, y, t-1) is the historical TOF depth data at the previous time, and W(x, y) is the weight function value at pixel coordinates (x, y).

[0035] Among them, at the edge of the hand: gradient When I is large, W(x, y) approaches 0 in the flat region: gradient When I is very small, W(x,y) is close to 1.

[0036] Specifically, in step 2.1, the synchronously acquired black light structure image I(x, y) is preprocessed with a 3×3 Gaussian filter to eliminate image noise under low light conditions and preserve the effective spatial features of the hand; the horizontal gradient of the black light image at pixel coordinates (x, y) is calculated. I / x and vertical gradient I / y is used to calculate the spatial gradient magnitude, which characterizes the edge intensity of a pixel; based on the gradient magnitude, an exponentially decaying weight function W(x, y) = exp(-α·∣) is constructed. I(x,y)|), In this embodiment, the gradient sensitivity adjustment parameter α is set to 0.05 to adapt to the intensity of hand edge features inside the vehicle; In the hand edge region, W(x,y) approaches 0 due to the large gradient magnitude, and In the flat hand region, W(x,y) approaches 1 due to the small gradient magnitude, thus realizing the differentiated weight allocation between edge and flat regions. Step 2.2: The vehicle-mounted edge computing unit caches the previous TOF historical depth data D(x, y, t-1) to ensure that it has the same resolution as the current TOF original depth map D(x, y, t) and meets the pixel-level fusion requirements. The weight function value W(x, y) is substituted pixel by pixel into the fusion formula D′(x, y, t) = W(x, y)·D(x, y, t) + (1-W(x, y))·D(x, y, t-1) to complete the weighted fusion: the weight of the original depth data is weakened in the hand edge region and the constraints of the historical time series data are strengthened, while the original depth data is retained in the flat region. The fused data is normalized and a stable depth map D′(x, y, t) with continuous contours is output as the input data for subsequent neural network gesture recognition.

[0037] Step 2 uses a fusion method guided by black light gradient features to correct anomalies in the original depth edges using historical time-series data, eliminating burrs and fragmentation at the source and improving the continuity of the depth map contour. Relying on the high contrast characteristics of black light images under low light, gradient features are stably extracted and weight rules are constructed to ensure that depth fusion remains effective even under extreme low light conditions. Differentiated weights are used to achieve accurate edge region repair and fidelity preservation in flat areas, taking into account both the continuity and real-time performance of depth data. Furthermore, the stable depth map output in Step 2 provides high-precision input for gesture recognition, significantly reducing misjudgments caused by depth issues and ensuring driving safety from the perception layer.

[0038] Step 3: Obtain vehicle driving state parameters and calculate driving state weights. At the same time, perform neural network gesture recognition on the stable depth data to obtain visual stability scores, and combine the hand spatial position judgment to obtain spatial constraint results.

[0039] Step 3 acquires vehicle driving status parameters and calculates driving status weights via the vehicle bus. It then combines this with a neural network to perform gesture recognition on the stable depth data to obtain a visual stability score. Simultaneously, it determines spatial constraints based on hand spatial position. From the three dimensions of vehicle driving status, gesture recognition confidence, and operational space validity, this provides multi-dimensional core judgment criteria for subsequent comprehensive credibility calculations. This step deeply integrates vehicle dynamics with gesture perception results, improving the accuracy of gesture recognition and laying the foundation for avoiding false triggers caused by vehicle body swaying and invalid spatial movements during driving, thus meeting the safety interaction requirements of in-vehicle driving.

[0040] In step 3, the vehicle driving state parameters include the vehicle's real-time speed and real-time acceleration. The driving state weight is obtained by performing an exponential decay coupling operation on the vehicle speed and acceleration. The driving state weight decreases synchronously as the vehicle speed and the absolute value of the acceleration increase. The specific steps for calculating the driving state weight in step 3 are as follows: Step 3.1: Obtain the vehicle state vector V(t) = [v(t), a(t)] through the vehicle bus. The vehicle state vector includes the real-time vehicle speed v(t) and the real-time acceleration a(t). Step 3.2: Calculate the driving state weight Wv based on the vehicle state vector using exponential decay operation. The driving state weight Wv decreases synchronously as the vehicle speed and the absolute value of acceleration increase.

[0041] The formula for calculating the driving state weight Wv is as follows: Wv=e -α∣v（t）∣-β∣a（t）∣ ; Where α is the vehicle speed weight adjustment coefficient, β is the acceleration weight adjustment coefficient, |v(t)| is the absolute value of the real-time vehicle speed, and |a(t)| is the absolute value of the real-time acceleration.

[0042] Specifically, 3.1, the vehicle edge computing unit connects to the vehicle control system via the vehicle CAN bus, collects vehicle dynamic data in real time at a frequency of 50Hz, extracts the vehicle state vector, where v(t) is the real-time vehicle speed and a(t) is the real-time vehicle acceleration, and performs filtering and noise reduction processing on the collected data to eliminate interference in signal transmission; a driving state weight calculation model is constructed, and preset adjustment coefficients are substituted: the speed weight adjustment coefficient α is 0.06, and the acceleration weight adjustment coefficient β is 0.08, adapting the influence weight of vehicle speed and acceleration in the vehicle driving scenario; the absolute value of real-time vehicle speed |v(t)| and the absolute value of real-time acceleration |a(t)| are substituted into the driving state weight formula Wv=e -α∣v（t）∣-β∣a（t）∣ The driving state weight Wv is calculated frame by frame, and the value of Wv is in the range of [0, 1], realizing the dual coupling constraint of vehicle speed and acceleration on the weight; 3.2 Input the stable depth map D′(x, y, t) with continuous contour output from step 2 into the lightweight neural network gesture recognition model. This model is trained and optimized using an in-vehicle gesture dataset and can quickly recognize typical in-vehicle control gestures and output a visual stability score Ws. The value of Ws ranges from [0, 1]. The higher the score, the stronger the confidence and stability of gesture recognition. Calculate the actual spatial distance d(t) between the hand feature points and the in-vehicle sensor based on the stable depth map D′(x, y, t), and compare it with the preset in-vehicle gesture interaction interval (dmin, dmax) to obtain the spatial constraint result: when d(t) ∈ (dmin, dmax), the spatial constraint result is 1, otherwise it is 0, ensuring that the gesture operation is within the effective spatial range.

[0043] Step 3 calculates driving state weights and reduces them synchronously as vehicle speed and absolute acceleration increase, achieving dual suppression of vehicle speed and driving stability. This filters invalid gestures from the vehicle state level, significantly reducing the probability of false triggers during dynamic driving. By combining visual stability scores with spatial constraint results, the confidence of gesture recognition is ensured through neural networks, while irrelevant actions outside the interaction range are filtered out through spatial constraints. This dual screening makes the determination of valid gestures more accurate. Real-time linkage between vehicle state and gesture perception data is achieved through the vehicle bus, making the effectiveness of gesture control strongly correlated with the vehicle driving state. This makes gesture control more in line with the safety requirements of dynamic driving in vehicles, achieving collaborative adaptation between the perception layer and the application layer.

[0044] Step 4: Calculate the overall credibility of gesture control based on driving state weights, visual stability scores, and spatial constraint results.

[0045] Step 4 obtains the comprehensive credibility of gesture control by coupling the driving state weight, visual stability score and spatial constraint results. It integrates the core judgment dimensions of vehicle dynamic driving state, gesture recognition confidence and operation space effectiveness, forming a multi-dimensional and comprehensive gesture effectiveness evaluation system. It comprehensively verifies gesture operation and provides accurate and scientific judgment basis for the output or suppression of subsequent gesture control commands. From the evaluation level, it avoids the false triggering problem that is easily caused by single-dimensional judgment, further improves the safety and reliability of vehicle low-light gesture control, and adapts to the complex interaction needs of vehicle dynamic driving.

[0046] Step 5: Compare the overall confidence level with the preset threshold, and execute the vehicle control command output or gesture operation suppression logic based on the comparison result.

[0047] In one embodiment, a preset confidence threshold can be pre-calibrated and stored in a threshold mapping table based on the vehicle driving mode and ambient light level. For example, when the vehicle is in parking mode, the preset confidence threshold can be set to a first threshold; when the vehicle is in low-speed driving mode, the preset confidence threshold can be set to a second threshold; and when the vehicle is in normal driving mode, the preset confidence threshold can be set to a third threshold, wherein the third threshold is greater than the second threshold, and the second threshold is greater than the first threshold. When the ambient light level is in the low-light or extremely dark range, the threshold can be increased based on the corresponding driving mode threshold to improve the prudence of control command output in low-light environments.

[0048] In a specific application scenario, a vehicle enters a tunnel from a daytime road, and the ambient illuminance inside the vehicle decreases from the normal lighting range to the low-light range. After the ambient light intensity detection module detects the decrease in illuminance, the onboard edge computing unit increases the fusion ratio of the coarse-scale layer gradient guidance weights and enhances the constraint effect of historical frame depth data on the hand edge region, ensuring the hand contour remains continuous under low-light conditions. When the vehicle accelerates or the steering wheel angle increases after entering the tunnel, the onboard bus interface module acquires vehicle speed, acceleration, and steering wheel angle data, with the driving state weight decreasing dynamically with the vehicle. If the system detects occupant hand movements at this time, but the overall confidence level is below a preset confidence threshold, no control commands are output to the onboard controller. When the vehicle's state returns to stability and the hand is within a preset interaction range, and the overall confidence level reaches the preset confidence threshold, the system outputs the corresponding onboard control commands.

[0049] Step 5 compares the comprehensive credibility obtained from multi-dimensional verification with a preset threshold and executes differentiated decision logic for command output or operation suppression. This is the final decision-making stage for in-vehicle low-light gesture control. Based on the triple evaluation results of preceding vehicle status, visual recognition, and spatial constraints, this step forms a rigid judgment rule for high-credibility triggering and low-credibility shielding. This ensures both the accurate execution of effective gesture operations and maximizes operational safety during dynamic vehicle operation, adapting to the full-scenario needs of gesture control in low-light in-vehicle environments.

[0050] The specific steps for calculating the overall credibility in step 5 are as follows: Step 5.1: Multiply the driving state weight Wv, the visual stability score, and the spatial constraint result to obtain the comprehensive credibility. Step 5.2: The visual stability score is output by the neural network gesture recognition algorithm, with a value range of [0, 1], representing the confidence level of gesture recognition; Step 5.3: The spatial constraint result is a binary indicator function. The value is 1 when the distance between the hand and the sensor is within the preset interaction range inside the vehicle, and 0 otherwise.

[0051] The formula for calculating the overall credibility Ccar is as follows: Ccar=Ws·Wv·I[d(t)∈(dmin,dmax)]; Where Ws is the visual stability score, output by the neural network gesture recognition algorithm, with a value range of [0, 1], representing the confidence level of gesture recognition, Wv is the driving state weight, and I[d(t)∈(dmin, dmax)] is the spatial constraint indicator function, which takes the value of 1 when the distance d(t) between the hand and the sensor is within the preset interaction interval (dmin, dmax), and otherwise takes the value of 0, to ensure that the gesture is within the fixed interaction interval.

[0052] Specifically, in step 5.1, the system synchronously acquires three parameters output from the previous steps in the same time sequence: driving state weight Wv, visual stability score Ws output by the neural network gesture recognition algorithm, and spatial constraint indicator function I[d(t)∈(dmin,dmax)] obtained based on hand distance judgment, to ensure that the data is aligned in time and reliable in calculation; Step 5.2: Multiply the three parameters according to the formula to obtain the final comprehensive confidence level: Ccar=Ws·Wv·I[d(t)∈(dmin,dmax)], where: Ws takes values ​​in the range of [0,1], and the larger the value, the higher the confidence level of gesture recognition; Wv is the weight of vehicle driving state, which decreases as vehicle speed and acceleration increase; the spatial constraint indicator function I[d(t)∈(dmin,dmax)] is a binary result, which takes 1 when the hand lands within the preset interaction distance interval (dmin,dmax), and 0 otherwise, and is used to exclude invalid actions in non-operation areas; Step 5.3: Compare the calculated overall credibility Ccar with the system's preset credibility threshold: If Ccar is greater than or equal to the preset threshold, the current gesture is determined to be a valid and safe operation command, and the system outputs the corresponding control signal to the vehicle controller to realize gesture control of multimedia, air conditioning, lighting and other devices; if Ccar is less than the preset threshold, it is determined to be an invalid gesture, interference action or operation in an unsafe scenario, and the system directly executes the gesture operation suppression logic and does not output any vehicle control command to avoid accidental triggering.

[0053] Step 5 integrates three core parameters—vehicle driving status, visual recognition confidence, and operational space validity—to calculate comprehensive credibility, achieving multi-dimensional and comprehensive verification. This makes gesture validity judgment more accurate and reduces judgment bias at its source. Through a binary space constraint indicator function, only gestures within the interaction range are included in the valid judgment range, accurately filtering invalid actions outside the range and significantly reducing the probability of false triggers caused by irrelevant interference. Driving status weights are incorporated into the comprehensive credibility calculation; when vehicle dynamics are poor, the weights are reduced, directly lowering the comprehensive credibility. This achieves adaptive suppression of gesture control in dynamic driving scenarios, ensuring driving safety from the vehicle status perspective. By calculating comprehensive credibility through a clear formula and setting rigid preset thresholds, clear judgment rules for high-credibility triggering and low-credibility blocking are formed, ensuring driving safety while also considering the convenience of gesture operation.

[0054] All data extraction, fusion, processing, and identification steps are completed locally in the vehicle edge computing unit. Raw sensor data is not uploaded to the cloud, ensuring real-time data processing and user privacy. The gesture control commands control at least one of the following in-vehicle devices: in-vehicle multimedia, air conditioning, head-up display, side window sunshades, and rear seat entertainment system.

[0055] A low-light gesture control system for in-vehicle environments, which implements the aforementioned low-light gesture control method for in-vehicle environments, includes: The sensor collaborative acquisition unit includes a TOF image module, a black light imaging module, and an ambient light sensor packaged on the same optical reference bracket. The three share a hardware synchronous trigger signal line. The TOF image module outputs raw depth data and measurement confidence map, the black light imaging module outputs low-light high-contrast structural images, and the ambient light sensor outputs ambient illuminance values. The vehicle-mounted edge computing unit includes a heterogeneous computing processor, which has a CPU core and an NPU core. The CPU core runs a heterogeneous resolution registration module, a multi-scale gradient pyramid decomposition and adaptive spatiotemporal fusion algorithm, a driving state weight calculation module, and a comprehensive credibility calculation and robust processing module. The NPU core runs a lightweight gesture recognition network quantized by INT8. The vehicle bus interface module connects to the vehicle control system via the CAN bus, acquires real-time data on vehicle speed, acceleration, and steering wheel angle, and outputs gesture control commands to the vehicle controller via the bus. The sensor collaborative acquisition unit and the vehicle-mounted edge computing unit transmit image data through the MIPICSI interface, and the CPU core and NPU core inside the vehicle-mounted edge computing unit exchange data through shared memory.

[0056] In a specific application scenario, the same vehicle cabin, the same sensor installation location, and the same hand gestures were used as test conditions. The control method was a gesture recognition method using only TOF depth maps, and the experimental method was the low-light gesture control method of this invention. During the test, the ambient lighting inside the vehicle was set to three conditions: normal lighting, low light, and extreme darkness. Normal lighting corresponds to an ambient illuminance value greater than or equal to 50 Lux, low light corresponds to an ambient illuminance value greater than or equal to 5 Lux and less than 50 Lux, and extreme darkness corresponds to an ambient illuminance value less than 5 Lux. The vehicle status was set to parking, low-speed driving, and dynamic interference driving. Dynamic interference driving included vehicle acceleration, changes in steering wheel angle, and interference from the passenger's hand or clothing movements. Preset in-vehicle control gestures were continuously executed under each condition, and the effective recognition rate, the proportion of hand edge depth holes, the number of false triggers in non-interactive areas, and the number of false triggers in dynamic driving scenarios were recorded.

[0057] In extremely dark environments, when using only TOF depth maps for gesture recognition, depth holes and fragmented contours easily appear in the hand edge region, leading to a decrease in the effective recognition rate. With the method of this invention, the high-contrast hand contour in the black light structured image data participates in constructing gradient-guided weights, and adaptive spatiotemporal fusion is performed by combining the TOF measurement confidence map and multi-frame historical depth data. This continuously repairs depth holes in the hand edge region, and stable depth data maintains a relatively complete hand contour. Furthermore, in dynamic scenarios such as vehicle acceleration or steering wheel angle changes, the driving state weight decreases with the vehicle's dynamic changes, and the overall confidence level decreases accordingly, thereby suppressing low-confidence gesture operations. When hand movements are located in non-interactive areas, the spatial constraint score decreases, causing the system not to output vehicle control commands, thus reducing false triggers caused by non-interactive area movements and dynamic driving interference.

[0058] As demonstrated by the test scenarios above, the method of this invention can constrain the gesture control process from three aspects: low-light perception stability, dynamic driving safety, and suppression of false triggering in non-interactive areas. Compared with methods that rely solely on TOF depth maps for gesture recognition, this invention does not simply increase image brightness or directly output recognition results. Instead, it guides TOF depth data through spatiotemporal fusion using black light structured image data and uses vehicle driving status, visual stability score, and spatial constraint results together for comprehensive credibility judgment, thereby improving the stability and safety of gesture control in low-light and dynamic in-vehicle environments.

[0059] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A low-light gesture control method for in-vehicle environments, characterized in that, Includes the following steps: S1. Data Synchronization Acquisition and Registration Steps: Using a common reference package and hardware-triggered synchronization, the TOF image module and black light imaging module synchronously acquire TOF depth data, corresponding measurement confidence maps, and black light structure image data from the same viewpoint inside the vehicle; distortion correction and coordinate mapping are completed for the TOF depth data using a pre-calibrated intrinsic and extrinsic parameter matrix, and the TOF depth data is upsampled to the same resolution as the black light structure image data through interpolation processing, so that the two data streams are aligned to a unified pixel coordinate system; S2. Stable Depth Data Generation Steps: Perform multi-scale spatial gradient pyramid decomposition on the registered black light structure image data, extract gradient features at each scale layer and construct corresponding scale gradient guided weights, and fuse them to obtain comprehensive gradient guided weights; use the TOF measurement confidence map as an auxiliary weight and fuse it with the comprehensive gradient guided weights to obtain a composite weight map; based on the composite weight map, perform adaptive spatiotemporal fusion processing on the current frame TOF depth data and multiple frames of historical depth data aligned by motion vector compensation to obtain optimized depth data; S3, Illumination Adaptive Adjustment Step: Real-time acquisition of the ambient light level inside the vehicle, and dynamic adjustment of the gradient-sensitive adjustment parameters corresponding to the multi-scale spatial gradient pyramid in step S2 based on the light level, so that the spatiotemporal fusion strategy adapts to changes in illumination conditions. S4. Steps for obtaining multi-dimensional credibility parameters: Transmit multi-dimensional driving state parameters such as vehicle speed, acceleration and steering wheel angle to the gesture control system via the vehicle bus; calculate the state factors corresponding to each parameter using a piecewise Sigmoid function, and obtain the driving state weights through weighted product operation. The lightweight gesture recognition model deployed at the edge of the optimized depth data input is used for inference, and the visual stability score corresponding to the gesture recognition is output. Based on the spatial position of the hand and the distance to the sensor, the spatial constraint score is calculated using a hysteretic soft boundary function; S5. Comprehensive Credibility Generation Steps: The driving state weight, visual stability score and spatial constraint score are fused together. Outliers are filtered out from the fusion result and smoothed in the sliding time window to generate the comprehensive credibility of gesture control. S6. Control output or suppress step: Compare the overall confidence level with the pre-calibrated confidence level threshold, and output the vehicle control command or suppress the current gesture operation based on the comparison result.

2. The low-light gesture control method for in-vehicle environment according to claim 1, characterized in that, The specific steps of multi-scale spatial gradient pyramid decomposition and composite weight graph construction in step S2 are as follows: Step S2.1: Perform L-level Gaussian pyramid decomposition on the registered black light structure image. Calculate the horizontal and vertical gradients at each scale layer l, where l = 0, 1, ..., L-1, to obtain the gradient magnitude field at the corresponding scale. Where L ≥ 3, different scale layers capture edge information at different spatial frequencies. The coarse-scale layer responds to the overall contour edge of the hand, while the fine-scale layer responds to the fine structural edges of the fingertips and joints. Step S2.2: Construct gradient-guided weights for the gradient magnitude field of each scale layer, and synthesize the weights of each layer into a comprehensive gradient-guided weight Wg(x,y) through a cross-scale weighted fusion strategy. During cross-scale weighted fusion, the fusion coefficients of each scale layer are dynamically allocated according to the current ambient light level. Step S2.3: Normalize the measurement confidence map Ct(x,y) output by the TOF sensor to the [0,1] interval, and perform a pixel-by-pixel multiplication operation with the comprehensive gradient guidance weight Wg(x,y) to obtain the composite weight map Wc(x,y)=Wg(x,y)·Ct(x,y), so that the historical frame constraints are automatically enhanced in the region of low TOF measurement confidence.

3. The low-light gesture control method for in-vehicle environment according to claim 2, characterized in that, The specific steps of adaptive spatiotemporal fusion in step S2 are as follows: Step S2.4: Calculate the motion vector field of each pixel based on the difference of stable depth data of adjacent frames, and perform motion vector compensation and alignment on the historical depth data of the previous N frames to eliminate inter-frame spatial misalignment caused by hand movement. The historical reference frame number N is adaptively adjusted according to the hand movement speed: when the global mean of the motion vector field is less than the preset static threshold, N takes a larger value to enhance the time domain smoothing effect; when the global mean exceeds the preset dynamic threshold, N takes a smaller value to preserve the motion response speed. Step S2.5: Use the composite weight map Wc(x,y) to perform weighted fusion of the current frame TOF depth data and the motion-compensated N frames of historical depth data: in flat, high-confidence regions with high Wc(x,y) values, the current frame is dominant; in edge or low-confidence regions with low Wc(x,y) values, the weighted average of multiple frames of historical data is dominant, and a stable depth map with continuous contours is output.

4. The low-light gesture control method for in-vehicle environment according to claim 1, characterized in that, The specific steps of adaptive illumination adjustment in step S3 are as follows: Step S3.1: The ambient light sensor integrated in the sensor module collects the ambient illuminance value Lux(t) in the vehicle in real time and maps the illuminance value to the light level E(t); the light level is divided into at least three intervals: normal light interval, low light interval and extremely dark interval. Step S3.2: Dynamically adjust the gradient sensitivity adjustment parameter αl of each scale layer in the multi-scale spatial gradient pyramid according to the illumination level E(t): In the normal illumination range, each layer takes a smaller value of αl so that the fusion weight is moderately sensitive to gradient changes; In the low-light region, the αl value of the fine-scale layer is increased to suppress the pseudo-gradient response amplified by noise, while the αl value of the coarse-scale layer remains unchanged to maintain the contour guiding capability; in the extremely dark region, the αl value of each layer is further increased and the number of historical frame reference frames N is increased to enhance the temporal domain constraint.

5. The low-light gesture control method for in-vehicle environment according to claim 1, characterized in that, The calculation steps for the driving state weight in step S4 are as follows: Step S4.1: Obtain the vehicle state vector in real time through the vehicle CAN bus, including real-time vehicle speed v(t), real-time acceleration a(t), and steering wheel angle θ(t); Step S4.2: Calculate the state factor for each state parameter using the piecewise Sigmoid function: The vehicle speed state factor fv(t)=1 / (1+exp(kv·(|v(t)|-v0)), where v0 is the vehicle speed value corresponding to the center point of the Sigmoid function, and kv is the slope adjustment parameter; the piecewise Sigmoid function corresponding to the vehicle speed state factor fv(t) outputs a value close to 1 in the low-speed section, transitions smoothly near v0, and outputs a value close to 0 in the high-speed section, achieving the piecewise control effect of full permission at low speed, strong suppression at high speed, and smooth transition at medium speed; The acceleration state factor fa(t) and the steering angle state factor fθ(t) adopt the same Sigmoid function form, and are calculated independently using their respective corresponding center point parameters and slope parameters; Step S4.3: Obtain the driving state weight Wv=fv(t) by performing a weighted product operation on each state factor. wv ·fa(t) wa ·fθ(t) wθ , where wv, wa, and wθ are the exponential weights of each factor, determined by regression analysis using an in-vehicle driving scenario calibration dataset.

6. The low-light gesture control method for in-vehicle environment according to claim 1, characterized in that, The calculation steps of the hysteresis soft boundary function in step S4 are as follows: Step S4.4: Acquire the spatial distance d(t) between the hand feature point and the vehicle-mounted sensor, expand each of the inner and outer boundaries of the interaction interval (dmin, dmax) by a transition width Δ, to form an inner transition zone [dmin-Δ, dmin] and an outer transition zone [dmax, dmax+Δ]; Step S4.5: Within the core interaction interval (dmin, dmax), the spatial constraint score Sd=1; within the inner transition zone and the outer transition zone, the spatial constraint score smoothly decays from 1 to 0 through the Sigmoid soft boundary function; outside the transition zones, Sd=0; Step S4.6: Apply a hysteresis mechanism to the spatial constraint score: set an entry threshold Th_enter and an exit threshold Th_exit, where Th_enter>Th_exit. When the hand enters from outside the interaction interval, activation is triggered only when Sd≥Th_enter; after activation, when the hand moves outward toward the boundary, deactivation is triggered only when Sd<Th_exit, so as to avoid repeated chattering near the boundary.

7. The low-light gesture control method for in-vehicle environment according to claim 1, characterized in that, The specific steps for calculating the comprehensive credibility in step S5 are as follows: Step S5.1: Perform a multiplication operation on the driving state weight Wv, the visual stability score Ws and the spatial constraint score Sd, to obtain the instantaneous credibility Cinst=Ws·Wv·Sd; Step S5.2: Perform median filtering on the instantaneous credibility sequence of the latest M frames, to filter out peak abnormal values caused by CAN bus packet loss or single-frame misjudgment of the neural network; Step S5.3: Perform weighted average smoothing on the median-filtered sequence within the sliding time window K, to obtain the final comprehensive credibility Ccar, wherein the time window K is adaptively adjusted according to the current hand movement speed: a larger value is taken for K in the case of static gestures to enhance decision stability, and a smaller value is taken for K in the case of dynamic gestures to ensure response speed.

8. The low-light gesture control method for in-vehicle environment according to claim 1, characterized in that, The specific implementation manner of data registration in step S1 is as follows: Step S1.1: At the system factory stage, use a calibration board to perform internal parameter calibration on the TOF image module and the black-light imaging module respectively, to obtain their respective focal lengths, principal points and distortion coefficients, and obtain the external parameter rotation matrix and translation vector between the two modules through stereo calibration; Step S1.2: During operation, based on the pre-calibrated internal parameter matrix and external parameter matrix, perform distortion correction and coordinate transformation on each pixel coordinate of the TOF depth data, and project the pixel coordinate to the pixel coordinate system of the black-light structured image; Step S1.3: The sparse TOF depth data after projection is upsampled on the resolution grid of the black light structure image through interpolation processing to obtain a TOF depth map and its measurement confidence map that are spatially aligned with the black light structure image and have the same resolution. Step S1.4: During vehicle operation, periodically extract hand feature points from the black light structure image and determine the registration offset between the hand feature points and the TOF depth data projection points; when the registration offset continuously exceeds the preset offset threshold, fine-tune the external parameter rotation matrix and translation vector between the TOF image module and the black light imaging module to compensate for the optical axis drift caused by vehicle vibration.

9. The low-light gesture control method for in-vehicle environment according to claim 3, characterized in that, The design and deployment method of the lightweight gesture recognition model deployed at the edge in step S4 is as follows: The lightweight gesture recognition model adopts an encoder-classifier architecture based on depthwise separable convolution as the building block. The encoder part receives dual-channel data consisting of a single-channel input from a stable depth map and a single-channel input from a black light image. After extracting spatial features through multiple layers of depthwise separable convolution, it outputs a feature vector through global average pooling. The classifier part outputs the gesture category probability distribution through a fully connected layer. The lightweight gesture recognition model is trained using a dataset containing multiple lighting conditions and hand gestures inside the vehicle during the training phase, and soft label guidance is obtained from a large teacher network through knowledge distillation. During the deployment phase, it is deployed on the NPU of the vehicle edge computing unit after optimization through INT8 quantization and operator fusion, and the inference latency does not exceed a single frame acquisition cycle.

10. A low-light gesture control system for in-vehicle environments, comprising the method described in any one of claims 1 to 9, characterized in that, The system includes: The sensor collaborative acquisition unit includes a TOF image module, a black light imaging module, and an ambient light sensor packaged on the same optical reference bracket. The three share a hardware synchronous trigger signal line. The TOF image module outputs depth data and measurement confidence map, the black light imaging module outputs low-light high-contrast structural image, and the ambient light sensor outputs ambient illuminance value. The vehicle-mounted edge computing unit includes a heterogeneous computing processor, which has a CPU core and an NPU core. The CPU core runs a heterogeneous resolution registration module, a multi-scale spatial gradient pyramid decomposition and adaptive spatiotemporal fusion algorithm, a driving state weight calculation module, and a comprehensive credibility processing module. The NPU core runs a lightweight gesture recognition model quantized by INT8. The vehicle bus interface module connects to the vehicle control system via the CAN bus, acquires real-time data on vehicle speed, acceleration, and steering wheel angle, and outputs gesture control commands to the vehicle controller via the bus. The sensor collaborative acquisition unit and the vehicle-mounted edge computing unit transmit image data through the MIPI CSI interface, and the CPU core and NPU core inside the vehicle-mounted edge computing unit exchange data through shared memory.

Images