A multimodal fusion hand image enhancement method and system

Abstract

The application discloses a kind of multi-modal fusion hand image enhancement method and system, including the two-way gradient channel between the differentiable physical simulation module of constructing parameterized three-dimensional hand model and two-dimensional image processing module;Iterative optimization is executed in channel, each iteration includes: initialize three-dimensional hand model posture parameter, generate two-dimensional projection mask by differentiable rendering, update segmentation network parameter, region analysis network parameter and three-dimensional hand model posture parameter by minimizing joint loss function;Sampling obtains intermediate segmentation mask sequence and posture parameter sequence, generates region evolution feature vector;Extract static morphological feature vector and structural feature vector, generate multi-dimensional region morphological control vector by fusing three;The vector is converted into image enhancement control instruction, selects and combines image processing operator to carry out pixel-level processing, and outputs enhanced hand region image;Difference, high-fidelity pixel-level enhancement to target hand region is realized.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, specifically relating to a method and system for enhancing hand images through multimodal fusion. Background Technology

[0002] In applications such as remote control, medical imaging, and immersive interactive interfaces, hand images captured by optical devices are crucial data sources for pose measurement, motion analysis, and status display. The quality of the hand region in these images directly determines the effectiveness of subsequent processing. However, due to the non-rigid nature of the hand, the flexibility of its joints, and the fact that it is often in motion during capture, the raw hand images generally suffer from a variety of image quality degradation problems, including blurred edge contours, noise interference in skin texture areas, insufficient contrast of key structures such as finger joints, and motion blur caused by rapid hand movements.

[0003] To improve hand image quality, various techniques have been employed, but all have drawbacks. Currently, most hand image enhancement methods use general-purpose approaches. These methods employ pre-defined image processing flows and fixed parameters, such as binarization with a uniform threshold and blurring or sharpening filters with fixed kernels, to process all hand images. Their fundamental flaw lies in their lack of specificity for the complex and varied shapes and degradation patterns of the hand. For instance, strong denoising might be effective for a clenched, static hand image, but applying the same parameters to an open, rapidly moving hand image could severely damage fine contour information such as the fingertips, or fail to effectively correct motion blur. This approach, which ignores the specificity of image content, struggles to achieve the optimal balance between improving visual quality and maintaining structural accuracy.

[0004] In addition, there are learning-based end-to-end hand image transformation methods. These methods train a model to directly learn the mapping from low-quality hand images to clear hand images. Although they perform well on some datasets, they have inherent limitations: First, the processing is uncontrollable, and users cannot guide the enhancement direction according to specific task requirements. For example, in gesture measurement, it is necessary to prioritize the geometric accuracy of skeletal joints, and in skin diagnosis assistance, it is necessary to prioritize the authenticity of tone and texture. Second, their effectiveness is severely limited by the training data. When the lighting, pose, or motion pattern of the input hand image exceeds the range of the training set, the output image may exhibit unpredictable artifacts such as structural distortion and texture distortion, and the model has difficulty in quickly adjusting to adapt to new scenarios.

[0005] Furthermore, some existing enhancement schemes that attempt to achieve adaptiveness often rely solely on simple global statistical features of the hand region, such as the overall average skin tone and average brightness of the area. These features cannot deeply analyze the multi-dimensional and structured quality of the hand image. They cannot quantitatively assess the different noise levels in the flat areas of the palm and the finger edges, cannot distinguish between overall soft blur caused by slight defocus and directional blur caused by specific pointing movements, and cannot accurately characterize the degree of local contrast loss in minute structures such as interphalangeal joint wrinkles. Due to the lack of a quantitative analysis parameter system that can accurately and comprehensively describe the multimodal morphological details and complex quality defects of the hand region, existing technologies struggle to drive a collaborative processing chain composed of multiple image processing operators to perform refined and differentiated pixel-level restoration of hand images. Summary of the Invention

[0006] The purpose of this invention is to address the problems existing in the prior art by providing a multimodal fusion method for hand image enhancement. This method can fuse complementary static morphological features, regional evolution features, and structural features to construct a comprehensive and quantitative multi-dimensional regional morphological control vector that describes the specific quality status of the hand region. This control vector is then converted into image enhancement control instructions to achieve differentiated, high-fidelity pixel-level enhancement of the target hand region.

[0007] To achieve the above objectives, the present invention discloses a multimodal fusion method for hand image enhancement, comprising the following steps:

[0008] Step S1: Construct a bidirectional gradient channel between the differentiable physical simulation and the two-dimensional image processing module of the parameterized three-dimensional hand model; perform multiple iterative optimizations within the bidirectional gradient channel to obtain the optimized two-dimensional hand region segmentation mask and the pose parameters of the three-dimensional hand model;

[0009] Step S2: Sample and obtain the intermediate segmentation mask sequence and the corresponding intermediate pose parameter sequence of the 3D hand model during the iteration process. Generate a fixed-length region evolution feature vector based on the intermediate segmentation mask sequence and the intermediate pose parameter sequence. The region evolution feature vector represents the shape evolution law of the hand region during the optimization process.

[0010] Step S3: Extract static morphological feature vectors from the optimized 2D hand region segmentation mask; extract structural feature vectors from the optimized 3D hand model pose parameters, wherein the structural feature vectors include at least the length ratio of each finger bone and the width-to-height ratio of the palm in the 3D model; adaptively weight and fuse the static morphological feature vectors, structural feature vectors, and region evolution feature vectors to generate multi-dimensional region morphological control vectors.

[0011] Step S4: Convert the multidimensional region morphology control vector obtained in step S3 into image enhancement control instructions, adaptively select and combine image processing operators, apply image processing operators to perform pixel-level processing on the hand region in the original image, and output the enhanced hand region image.

[0012] Furthermore, before iterative optimization, the pose parameters of the 3D hand model are initialized. Specifically, the input image is first segmented and the region state is analyzed to obtain the initial segmentation mask and the initial region morphology distribution parameters. The parameters include at least the region area parameter, density parameter, aspect ratio parameter, centroid position parameter, and principal axis direction parameter. Then, based on the initial region morphology distribution parameters and combined with the preset region morphology-pose mapping relationship, the pose parameters of the 3D hand model are initialized.

[0013] The preset region morphology-pose mapping relationship is a pre-generated morphology-pose mapping table used to establish a deterministic correspondence between the parameters observed in the two-dimensional image domain and the three-dimensional model domain. Its construction process can be based on biomechanical rules and knowledge of human hand anatomy and kinematics to associate common and stable hand poses with their morphological features under two-dimensional projection; or it can be based on statistical learning, using a labeled dataset containing "two-dimensional hand image - corresponding three-dimensional real pose", to automatically learn the statistical correlation between two-dimensional morphological parameters and three-dimensional joint angles through data analysis, and store this relationship in the mapping table after discretization or functionalization.

[0014] Furthermore, each iteration of the optimization process includes: generating a two-dimensional projection mask through differentiable rendering based on the pose parameters of the 3D hand model after the previous update, and updating the segmentation network parameters, region analysis network parameters, and 3D hand model pose parameters by minimizing the joint loss function.

[0015] Furthermore, the bidirectional gradient channel includes:

[0016] The first forward channel is used to map the two-dimensional hand region segmentation mask and region morphological distribution parameters output by the image processing module into input constraints for the differentiable physical simulation module. Specifically, it includes a differentiable mapping network, which is used to convert the hand segmentation probability map into virtual torque vectors applied to the joints of the three-dimensional model. The virtual torque vectors are input as dynamic driving signals to the differentiable physical simulation module, driving the three-dimensional hand model to generate motion state changes according to physical laws, and generating a three-dimensional hand motion state sequence with continuous time steps.

[0017] The second reverse channel is used to convert the state of the 3D hand model in the differentiable physics simulation module into an image domain supervision signal through differentiable rendering, so as to calculate the loss and backpropagate it to the image processing module. Specifically, it includes a differentiable neural renderer, which generates a synthesized 2D optical flow map based on the motion sequence of the 3D hand model. The synthesized 2D optical flow map is compared with the real optical flow map extracted from real continuous video frames to calculate the optical flow reconstruction loss. The optical flow reconstruction loss and the mask reconstruction loss corresponding to the synthesized hand segmentation mask generated by the differentiable neural renderer together constitute the image domain supervision signal.

[0018] The gradient of the image domain supervision signal propagates backward along the following path: updating rendering parameters through the differentiable neural renderer; updating dynamic parameters through the differentiable physics simulation module; updating torque mapping parameters through the differentiable mapping network; and finally propagating to the image processing module to update segmentation network parameters and region analysis network parameters.

[0019] By iteratively performing forward and backward propagation, the system outputs three-dimensional hand motion that tends to be consistent with two-dimensional image observations.

[0020] Furthermore, the joint loss function L is composed of a weighted sum of the segmentation loss term, the geometric alignment loss term, and the joint angle constraint loss:

[0021]

[0022] in, , , These are weighting coefficients, and their values ​​are positive real numbers. For the segmentation loss term, standard cross-entropy loss is used to supervise the consistency between the predicted hand segmentation map and the ground truth annotation; For the geometric alignment loss term, Dice loss is used to optimize the contour matching degree between the predicted hand region and the target region; The joint angle limitation loss term is designed as a smooth quadratic penalty term, which penalizes any joint angle when it exceeds its physiological range of motion, thereby ensuring that the generated hand posture conforms to biomechanical constraints.

[0023] The set of actual pixel mask values; The set of pixel values ​​for the hand region prediction mask output by the 2D image processing module; The set of pixel values ​​for a two-dimensional projection mask generated by differentiable rendering of a parametric 3D hand model; This represents the current angle state of all joints in the hand.

[0024] The joint loss function, through constraints in different dimensions, enables the method to maintain pixel-domain consistency, spatial domain accuracy, and dynamic rationality. The weight settings reflect the guidance of system optimization at different stages or for different needs. The optimal combination of weighting coefficients is determined by grid search on the validation set. Based on the segmentation mask accuracy, 3D geometric alignment error, and pose physiological rationality scores on the validation set, the weighting coefficients are dynamically fine-tuned until the overall performance of the model is optimal.

[0025] Furthermore, the segmentation loss term adopts the cross-entropy loss function, specifically:

[0026] ;

[0027] in, Let i be the actual mask value of the i-th pixel. This represents the value of the predicted mask at the i-th pixel position. Represented by natural constant Logarithmic operations with base 0;

[0028] The geometric alignment loss term is expressed as:

[0029]

[0030] in, This represents the value of the predicted mask at the i-th pixel position. The summation symbol is a preset value used to avoid the smallest positive number with a denominator of zero. This indicates that the corresponding values ​​of all pixels in the mask are summed.

[0031] The joint angle limitation loss term is expressed as:

[0032] ;

[0033] in, This represents the current angle state of all joints in the hand. This represents the total number of joints in the hand. For the first The current angle value of each joint. From 1 to Traverse all joints, For the first The maximum permissible absolute angle preset for each joint. Indicates the first The absolute value of each joint angle.

[0034] Furthermore, the process of generating a fixed-length region evolution feature vector is as follows:

[0035] Step S21: For each segmentation mask in the intermediate segmentation mask sequence, calculate its shape feature description vector, which includes at least the area, compactness, and Hu moment invariant features of the mask;

[0036] Step S22: Establish the correspondence between the attitude parameter vector of each iteration step in the intermediate attitude parameter sequence and a segmentation mask in the intermediate segmentation mask sequence; for each iteration step, concatenate its attitude parameter vector with its corresponding shape feature description vector to form the joint feature vector of that iteration step.

[0037] Step S23: Input the sequence of joint feature vectors arranged in iteration order into a bidirectional long short-term memory network for processing;

[0038] Step S24: Concatenate the forward hidden state and the backward hidden state of the last time step of the bidirectional long short-term memory network. The concatenation result is a fixed-length region evolution feature vector.

[0039] Furthermore, a two-stage gating fusion mechanism is adopted in the process of generating multi-dimensional region morphological control vectors, specifically as follows:

[0040] Phase 1: Using static morphological feature vectors and regional evolution feature vectors as inputs, the output is a fusion weight with values ​​ranging from [0,1]. They are then merged into an intermediate feature vector, calculated using the following formula:

[0041] ;

[0042] in, Represents a static morphological feature vector; Represents the feature vector of regional evolution. Represents the intermediate feature vector; The fusion weights are the static morphological feature vector and the regional evolution feature vector.

[0043] The second stage: Taking the intermediate feature vector and structural feature vector as input, the output is a fusion weight β with a value range of [0,1], which is then fused into an optimized multi-dimensional region morphological control vector. The calculation formula is as follows:

[0044] ;

[0045] in, Represents structural feature vectors. This represents the multidimensional region morphological control vector obtained from the final fusion. The fusion weights are the intermediate feature vector and the structural feature vector.

[0046] The aforementioned two-stage gating fusion mechanism achieves hierarchical adaptive fusion of multimodal features of hand images, including static morphology, regional evolution, and three-dimensional structure. This effectively avoids information interference, redundancy, and dilution of effective features caused by direct fusion of features from different dimensions. It can dynamically adjust the contribution weights of each feature based on the actual morphology and quality degradation mode of the hand image, compensating for the limitations of single-feature representation. This constructs a feature system that comprehensively and quantitatively characterizes the overall state of the hand's appearance, dynamic evolution, and inherent structure. Simultaneously, it suppresses interference from invalid features, improving the robustness and effectiveness of the fused features. This makes the feature fusion process interpretable and controllable, adapting to the hand image enhancement needs of different application scenarios. Ultimately, it generates precise multi-dimensional regional morphology control vectors, providing reliable feature basis for subsequent adaptive and differentiated pixel-level enhancement of hand images.

[0047] Furthermore, in step S4, based on the multi-dimensional region morphological control vector, corresponding image enhancement control commands are generated through predefined parameter-command mapping rules. The predefined parameter-command mapping rules are specifically embodied in a queryable rule knowledge base, which stores the deterministic mapping relationship from multi-dimensional vectors to specific processing commands.

[0048] Furthermore, the image processing operator includes at least a denoising filter, a sharpening kernel, and a local contrast enhancement function;

[0049] A denoising filter is used to suppress noise introduced during image acquisition; a sharpening kernel is used to enhance the edge contour information of the image; and a local contrast enhancement function is used to improve the grayscale contrast of local areas of the image.

[0050] A first threshold is preset to evaluate the degree of motion blur, and a second threshold is preset to evaluate the sharpness of the outline.

[0051] When the motion blur level is higher than the first threshold, a serial processing flow is adopted: first, the denoising filter is executed, and then the local contrast enhancement is performed on the denoising result; when the contour sharpness is lower than the second threshold, a parallel processing architecture is adopted: the sharpening kernel and the local contrast enhancement function are executed in parallel, and the processing results of the two are merged according to the preset weights.

[0052] When the motion blur level is higher than the first threshold and the contour sharpness is lower than the second threshold, a hybrid processing architecture is adopted: first, the sharpening kernel and local contrast enhancement are fused in parallel, and then the denoising filtering process is performed on the fused result.

[0053] Based on the same inventive concept, the present invention also discloses a multimodal fusion hand image enhancement system, which performs the above-described method, specifically including:

[0054] A multimodal quality analyzer for hand images is used to construct a bidirectional gradient channel between a differentiable physical simulation module and a 2D image processing module for a parameterized 3D hand model. It initializes the pose parameters of the 3D hand model and performs iterative optimization within the channel. Each iteration includes: generating a 2D projection mask through differentiable rendering; updating the segmentation network parameters, region analysis network parameters, and 3D hand model pose parameters by minimizing the joint loss function; sampling to obtain intermediate segmentation mask sequences and pose parameter sequences to generate region evolution feature vectors; and extracting static morphological feature vectors and structural feature vectors, fusing them to generate a multidimensional region morphological control vector.

[0055] A parameter instruction converter is used to receive multi-dimensional region morphology control vectors and convert the vectors into an image enhancement control instruction set by querying a preset mapping rule table; the control instruction set is used to specify the selection of image processing operators, parameter configuration, and execution logic.

[0056] An adaptive processing engine, communicatively connected to a parameter instruction converter, is used to receive the image enhancement control instruction set, select and instantiate corresponding image processing operators from a pre-stored operator library according to the instruction set, and combine multiple operators into a processing chain according to the execution logic; the operator library includes at least a denoising filter, a sharpening kernel, and a local contrast enhancement function;

[0057] The pixel processing unit, connected to the adaptive processing engine, is used to load the processing chain, perform pixel-level processing on the input raw hand region image, and output an enhanced hand region image.

[0058] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0059] 1) This invention utilizes a joint optimization framework that combines segmentation loss, geometric alignment loss, and physical constraint loss, enabling the system to improve the processing effect of two-dimensional hand images while ensuring that the results remain consistent in the pixel domain, spatial domain, and dynamic domain; effectively solving the technical problem that it is difficult to coordinate the optimization of regional quality improvement and physical law compliance in traditional methods.

[0060] 2) This invention adopts a two-level gating fusion mechanism. First, static morphological features and regional evolution features are adaptively screened and initially fused. Then, the intermediate features and structural features are fused in a second time. The resulting multi-dimensional regional morphological control vector can accurately reflect the comprehensive state features of the hand region, providing a reliable basis for subsequent processing.

[0061] 3) This invention makes the complex image enhancement process transparent, interpretable, and controllable by converting multi-dimensional region morphology control vectors into specific image enhancement instructions. Users can adjust the instruction generation strategy according to different application requirements, which significantly improves the practicality of the system and the convenience of engineering deployment while ensuring the enhancement effect. Attached Figure Description

[0062] Figure 1 This is a flowchart of the hand image enhancement method using multimodal fusion in Example 1. Detailed Implementation

[0063] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0064] Example 1: A multimodal fusion method for hand image enhancement

[0065] like Figure 1 As shown, this invention discloses a multimodal fusion method for hand image enhancement, comprising the following steps:

[0066] Step S1: Construct a bidirectional gradient channel between the parameterized 3D hand model's differentiable physical simulation and the 2D image processing module; In this embodiment, the 2D image processing module includes a segmentation network module and a region analysis network module; The segmentation network module adopts a convolutional neural network based on the U-Net architecture. This module uses an encoder-decoder architecture based on U-Net, specifically designed to accurately extract the hand region contour from the input image; The encoder consists of 4 downsampling stages, each containing a convolutional layer, a batch normalization layer, and a ReLU activation function, progressively extracting and compressing image features.

[0067] The decoder gradually restores the spatial resolution through upsampling operations and then concatenates and fuses it with the feature maps of the corresponding layers in the encoder, thereby effectively reconstructing the boundary details of the hand region. This network, trained end-to-end, can stably output a 2D segmentation mask for the hand region with the same resolution as the input image.

[0068] The region analysis network module employs a lightweight MobileNet architecture centered on depthwise separable convolutions, responsible for efficient feature analysis and state quantization of the segmentation mask. The network replaces standard convolutions with depthwise separable convolutions, significantly reducing the number of parameters and computational complexity while maintaining feature extraction capabilities. The network's final fully connected output layer generates an initial region morphology distribution parameter. This parameter is a set of data describing the contour shape, such as resembling "fully open," "half-clenched," or "clenched fist," and its value represents the relative probability or weight of the current hand region belonging to the corresponding state.

[0069] In this embodiment, the MANO model published by Max Planck Intelligent Systems Research is used as the three-dimensional hand model. This model is differentiable and supports projecting the three-dimensional hand onto a two-dimensional plane and calculating the loss gradient through differentiable rendering.

[0070] In this embodiment, the bidirectional gradient channel includes:

[0071] First forward path: Mapping of 2D region information to 3D model dynamic constraints

[0072] The first forward channel is implemented by a differentiable mapping network. This network is constructed using a feedforward neural network architecture, with its input layer dimension matching the output dimension of the image processing module and its output layer dimension matching the dynamic degrees of freedom of the 3D hand model. The overall mapping relationship can be expressed as:

[0073]

[0074] in, For mapping functions; Represents a two-dimensional hand region segmentation mask; Represents a vector of regional morphological distribution parameters; Denotes the virtual torque vector, where This represents the number of joint degrees of freedom in a 3D hand model.

[0075] The specific implementation steps include: first, spatially encoding the two-dimensional hand region segmentation mask output by the two-dimensional image processing module, converting it into a fixed-dimensional feature vector. Then, the regional morphological distribution parameter vector With feature vectors Feature concatenation is performed to form a fused feature representation. : Next, the fusion features will be... The input is fed into a differentiable mapping network, which performs a nonlinear transformation through three fully connected layers, ultimately outputting a virtual moment vector. Finally, the virtual torque vector As an external driving mechanical constraint input to the differentiable physics simulation module, it is used to drive the three-dimensional hand model to generate posture evolution that conforms to the two-dimensional observation constraints.

[0076] Second reverse channel: Feedback of 3D model state to 2D image domain monitoring signal

[0077] The second inverse channel is implemented by a differentiable neural renderer built on differentiable rasterization technology, capable of converting 3D geometric information into 2D image signals. The differentiable neural renderer is implemented using the PyTorch3D or NVIDIA Kaolin framework. This channel is implemented through a differentiable rendering function.

[0078] Let the state of the 3D hand model be at adjacent time steps t and t+1. and They are respectively:

[0079] ;

[0080] in, Represents the set of three-dimensional positions of the model vertices at time step t; This represents the joint pose parameters at time step t;

[0081] Through differentiable rendering function The following two-dimensional synthesized signal is generated:

[0082] Projection mask signal:

[0083]

[0084] in, This represents the two-dimensional synthetic hand segmentation mask obtained by projecting the state of the three-dimensional model. This represents a dedicated differentiable rendering subfunction for generating the projection mask signal;

[0085] Synthetic optical flow signal:

[0086]

[0087] in, This represents a two-dimensional composite optical flow map generated from changes in adjacent three-dimensional states. This represents a dedicated differentiable rendering subfunction for generating the synthetic optical flow signal;

[0088] The set of synthesized signals is represented as :

[0089] As a supervisory signal in the image domain, it calculates the error with the corresponding result output by the two-dimensional image processing module, and propagates the error information back along the differentiable path to update the segmentation network parameters, region analysis network parameters, and related parameters of the three-dimensional model.

[0090] By alternately executing the first forward channel and the second reverse channel, a bidirectional consistency constraint is achieved between the two-dimensional image domain observation information and the three-dimensional physical model state, so that the system can optimize the two-dimensional segmentation results while ensuring the physical rationality and structural consistency of the three-dimensional hand posture.

[0091] Before iterative optimization, the pose parameters of the 3D hand model are initialized. Specifically, the segmentation network module and the region analysis network module are used to perform initial image segmentation and region analysis on the input image to obtain the initial segmentation mask and region morphology distribution parameters. The parameters include at least the region area parameter, density parameter, aspect ratio parameter, centroid position parameter and principal axis direction parameter.

[0092] Based on the initial region morphological distribution parameters and combined with the preset region morphology-pose mapping relationship, the region morphology-pose mapping relationship is a pre-generated morphology-pose mapping table, which is used to establish a deterministic correspondence between the parameters observed from the two-dimensional image domain and the three-dimensional model domain. Its construction process is based on biomechanical rules and, according to the knowledge of human hand anatomy and kinematics, associates common and stable hand postures with their morphological features shown under two-dimensional projection, such as "contour compactness 0.8 → corresponding to a 30-degree bend in the thumb joint and a 20-degree bend in the index finger of the 3D model", and initializes the posture parameters of the three-dimensional hand model.

[0093] The optimization process is performed within the bidirectional gradient channel constructed above, specifically as follows:

[0094] S11. Based on the pose parameters of the 3D hand model updated last time, the current 3D hand model is rendered using a differentiable renderer to generate a 2D projection mask; that is, the standard outline of the 3D hand model projected onto the 2D plane, which strictly follows the skeletal structure and joint rules of the 3D model.

[0095] S12. Minimize the joint loss function to update the segmentation network parameters, region analysis network parameters, and 3D hand model pose parameters. This joint loss function L is composed of a weighted sum of the segmentation loss term, the geometric alignment loss term, and the joint angle constraint loss, specifically expressed as follows:

[0096]

[0097] in, , , ; For the segmentation loss term, standard cross-entropy loss is used to supervise the consistency between the predicted hand segmentation map and the ground truth annotation; For the geometric alignment loss term, Dice loss is used to optimize the contour matching degree between the predicted hand region and the target region; The joint angle limitation loss term is designed as a smooth quadratic penalty term, which penalizes any joint angle when it exceeds its physiological range of motion, thereby ensuring that the generated hand posture conforms to biomechanical constraints.

[0098] in, The set of actual pixel mask values. This is the set of pixel values ​​for the hand region prediction mask output by the 2D image processing module. This is the set of pixel values ​​for a 2D projection mask generated from a parametric 3D hand model using differentiable rendering. This represents the current angle state of all joints in the hand.

[0099] The segmentation loss term uses the cross-entropy loss function, specifically:

[0100]

[0101] in, Let i be the actual mask value of the i-th pixel. This represents the value of the predicted mask at the i-th pixel position. Represented by natural constant Logarithmic operations with base 0;

[0102] The geometric alignment loss term is expressed as:

[0103] ;

[0104] in, This represents the value of the projection mask at the i-th pixel position. The default positive number is used to avoid the denominator being zero, and the summation sign is used. This indicates that the corresponding values ​​of all pixels in the mask are summed.

[0105] The joint angle limitation loss term is expressed as follows:

[0106] ;

[0107] in, This represents the current angle state of all joints in the hand. This represents the total number of joints in the hand. For the first The current angle value of each joint. From 1 to Traverse all joints, For the first The maximum permissible absolute angle preset for each joint. Indicates the first The absolute value of each joint angle.

[0108] S13, Computation-based joint loss function Backpropagation is performed using the PyTorch automatic differentiation framework.

[0109] First-order moment decay rate using Adam optimizer Second-order moment decay rate Simultaneously update the following three sets of trainable parameters, setting the learning rate to 1×10. -4 ;

[0110] Segmentation network parameters: Update the parameters of all convolutional and batch normalization layers in the U-Net segmentation network. The encoder of this network adopts a 4-layer VGG-like structure, with each layer containing two 3×3 convolutions (stride 1) followed by a 2×2 max pooling (stride 2). The decoder uses transposed convolutions for upsampling and concatenates them with the corresponding features from the encoder. The initial weights are loaded from a model pre-trained on the EgoHands dataset.

[0111] Region morphology analysis network parameters: Update the parameters of all deep separable convolutional layers and linear bottleneck layers of the MobileNet v2 network; the network input is a 256×256 resolution binary mask, extracts features through the inverse residual structure, and finally outputs a 4-dimensional morphological distribution vector through the fully connected layer; the initial weights are based on the ImageNet pre-trained model and fine-tuned using a self-built morphological annotation dataset.

[0112] 3D hand model pose parameters: Update the joint angles of the MANO hand model This corresponds to the rotation angles of 15 joints. The update considers the physiological constraints of the joints: the thumb abduction angle is limited to... The flexion angle of each finger is limited to .

[0113] S14. Input the original RGB input image (640×480 resolution) into the updated U-Net network, perform forward inference, and the network outputs a probability map with the same resolution as the input. Obtain the updated segmentation mask by binarizing with a 0.5 threshold. :

[0114]

[0115] in, The height of the input image. The width of the input image represents its spatial resolution. This mask reduces jagged artifacts in the finger edge region by approximately 40% compared to the initial mask.

[0116] S15. Iterative Update of Morphology and 3D Pose

[0117] Morphological parameter update: Scale to 256×256 and input into the updated MobileNet v2 network, outputting a normalized morphological distribution vector. :

[0118] ,

[0119] in, The first element in the morphological distribution vector represents the first element. There are several components, where K is the dimension of the shape distribution vector. In this embodiment, K=4. If we get [0.15, 0.72, 0.13, 0.00], it means that the current hand shape is judged as "open" 15%, "half-clenched" 72%, "clenched fist" 13%, and "bent" 0%.

[0120] 3D pose update: Employs a radial basis function-based interpolator, based on... The corresponding attitude parameter basis vector is retrieved from the pre-constructed morphology-attitude mapping table. The specific calculation is as follows:

[0121] ,in, Updated 3D hand pose parameter vector, weights It is the first The weights of each baseline pose are determined based on the similarity between the current morphological distribution and the baseline morphology, and satisfy normalization constraints. For the pre-built morphology-pose mapping table, the first The reference attitude parameter vector corresponding to each reference shape;

[0122] S16. Repeat steps S11-S15, recording the changes in the two-dimensional hand region segmentation mask in each iteration: n is the number of iterations, and the maximum number of iterations is set to 10; This indicates the degree of change in the two-dimensional hand region segmentation mask between two adjacent iterations; M (n) This represents the two-dimensional hand region segmentation mask output after the nth iteration; M (n-1) This represents the two-dimensional hand region segmentation mask output after the (n-1)th iteration; IoU ( M (n) , M (n-1) The cross-joint ratio (CLU) represents the ratio between the two-dimensional hand region segmentation mask output in the nth iteration and the two-dimensional hand region segmentation mask output in the (n-1)th iteration.

[0123] Termination condition: When the number of iterations reaches 10 or two consecutive iterations... The loop terminates when the value is less than 0.005.

[0124] Output results: The segmentation mask obtained in the last iteration is used as the final segmentation mask output; the corresponding 3D hand model pose parameters and morphological distribution vector are also output.

[0125] Experimental data: On 1000 test images, the proposed method converged after an average of 4.7 iterations, with the final mean Intersection over Union (mIoU) reaching 92.3% ± 2.1%, an improvement of 11.5 percentage points compared to the initial mask; the mean per joint position error (MPJPE) corresponding to the 3D pose was 7.8 mm ± 1.3 mm, meeting the accuracy requirements for subsequent applications.

[0126] Step S2: During the iteration process of step S1, the intermediate segmentation mask sequence and its corresponding 3D hand model pose parameter sequence are sampled and recorded at preset iteration step intervals. The storage format adopts PyTorch Tensor format, the data type is float32, the image spatial resolution is [H, W], and the pose parameter tensor dimension is [J, 3], where J is the total number of hand joints. A fixed-length region evolution feature vector is generated based on the intermediate segmentation mask sequence and the intermediate pose parameter sequence. The feature vector represents the shape evolution law of the hand region during the optimization process, specifically:

[0127] Step S21: Calculate the shape feature description vector for each segmentation mask in the intermediate segmentation mask sequence. The area is obtained by multiplying the number of pixels with a value of 1 in the mask by the area of ​​a single pixel. The area of ​​a single pixel is determined according to the resolution of the input image. For example, when the resolution of the input image is 640×480, the area of ​​a single pixel is (1 / 640)×(1 / 480) square inches. The compactness is the ratio of the square of the perimeter of the mask to the area. The perimeter is obtained by calculating the sum of the Euclidean distances of the edge pixels of the mask. The Hu moment invariant features are obtained by the standard seven geometric moment invariant calculation method. Its rotation, scaling and translation invariance ensures that the core features of the hand region state can be stably captured.

[0128] Step S22: Establish the correspondence between the attitude parameter vector of each iteration step in the intermediate attitude parameter sequence and a segmentation mask in the intermediate segmentation mask sequence; concatenate the attitude parameter vector of each iteration step with the corresponding shape feature description vector according to the dimensions. The attitude parameter vector is 45-dimensional with 15 joints, each corresponding to 3 angle parameters. The shape feature vector is 9-dimensional with area, compactness, and 7 Hu moments. After concatenation, a 54-dimensional joint feature vector is formed, which fully integrates the attitude and shape information of a single iteration.

[0129] Step S23: The joint feature vector sequence arranged in iteration order is input into a bidirectional long short-term memory network for processing. This network is used to describe the regional morphological change trend during segmentation and pose optimization. The network is set to 2 layers with a hidden layer dimension of 128. A dropout layer is added after each layer with a dropout rate of 0.2 to avoid overfitting. The parameter configuration comprehensively considers feature complexity and computational efficiency, and can fully explore the evolutionary patterns in the time sequence. The forward hidden state and backward hidden state of the last time step of the bidirectional long short-term memory network are concatenated to obtain a 256-dimensional fixed-length feature vector. This vector condenses the dynamic evolution information of the hand region state.

[0130] Step S24: Concatenate the forward hidden state and the backward hidden state of the last time step of the bidirectional long short-term memory network. The concatenation result is a fixed-length region evolution feature vector.

[0131] Step S3: Extract static appearance feature vectors from the optimized 2D hand region segmentation mask. A ResNet18 convolutional neural network is selected, with weights pre-trained on the ImageNet dataset. The first 8 convolutional layers are frozen, the last 4 layers are fine-tuned, and the average pooling output is taken as the feature vector after removing the fully connected layers. The dimension is 512. This network effectively solves the gradient vanishing problem in deep networks through residual connections and can fully extract the deep appearance features of the hand region mask.

[0132] Structural feature vectors are extracted from the posture parameters of the optimized 3D hand model, including the length ratio of each phalanx and the width-to-height ratio of the palm. The length ratio of the phalanx is obtained by calculating the Euclidean distance from the corresponding 3D coordinates of the bones in the model. For example, the length ratio of the proximal phalanx of the index finger to the middle phalanx. The width-to-height ratio of the palm is the ratio of the 3D distance between the maximum width and the maximum height of the palm. These features have strong stability, are not affected by changes in hand gestures, and can reflect the inherent structural information of the hand.

[0133] A 256-dimensional fixed-length feature vector is generated, which carries the temporal information of gesture evolution. A two-stage gating fusion method is used to generate the final classification feature vector. The gating network consists of two fully connected layers, with the first layer having a dimension of 1024 and the second layer having a dimension of 512. The Sigmoid activation function is selected to ensure that the output weights are in the range [0,1].

[0134] The two-stage gating fusion mechanism generates a multi-dimensional region morphological control vector, specifically:

[0135] Phase 1: Using static morphological feature vectors and regional evolution feature vectors as inputs, the output is a fusion weight with values ​​ranging from [0,1]. They are then merged into an intermediate feature vector, calculated using the following formula:

[0136] ;

[0137] in, Represents a static morphological feature vector; Represents the feature vector of regional evolution. Represents the intermediate feature vector; The fusion weights are the static morphological feature vector and the regional evolution feature vector.

[0138] The second stage: Taking the intermediate feature vector and structural feature vector as input, the output is a fusion weight with a value range of [0,1]. They are then integrated into an optimized multi-dimensional region morphological control vector, calculated using the following formula:

[0139] ;

[0140] in, Represents structural feature vectors. This represents the multidimensional region morphological control vector obtained after final fusion. The fusion weights are the intermediate feature vector and the structural feature vector.

[0141] The components of the multi-dimensional region morphological control vector obtained in steps S4 and S3 correspond to different morphological features, such as the motion blur degree component. Noise level component Outline sharpness component Sharpening requirements Contrast requirement components Each component is normalized to the [0,1] interval; image enhancement control instructions are generated based on the control vector; a predefined parameter-instruction mapping rule base is established, which is a structured knowledge base that stores deterministic mapping relationships from multidimensional vector space to specific processing instructions. The rule base adopts a structure of lookup table and decision tree concatenation: first, the control vector is quantized into discrete indexes with a step size of 0.05, and the corresponding instruction group is retrieved through the lookup table; then, the final instruction is selected from the instruction group by a 3-layer decision tree based on the vector detail features.

[0142] Based on the generated instructions, the system dynamically calls the corresponding image processing operators from the operator library. The core operators in this embodiment include, but are not limited to, the following three types, and their specific parameters are dynamically modulated by the aforementioned multidimensional region morphology control vector:

[0143] Denoising filter: Adaptive bilateral filtering is used; parameter settings: dynamically modulated by the noise level component, spatial kernel radius. grayscale kernel radius ; Control the Gaussian weights in the spatial domain, Gaussian weights are used to calculate the grayscale difference between pixels. The pixel neighborhood window size is... ,in This is the rounding function.

[0144] Sharpening kernel: Use Laplacian sharpening (weak contours) or Unsharp Masking (medium / strong contours).

[0145] Parameter settings: Sharpening intensity is determined by the sharpening requirement component. modulation;

[0146] When the outline is clear Select 3x3 Laplacian sharpening, sharpening factor Output ;

[0147] in, The input image is the original hand region; Output the Laplacian-sharpened hand image;

[0148] This indicates that the second-order derivative of the 3×3 Laplacian operator is performed on the input hand image. Its core function is to extract the edge contour information of the hand image, such as the edge pixels of the knuckles and fingertips.

[0149] When the outline is clear Select Unsharp Masking, Gaussian Blur Kernel Sharpening weights Output .

[0150] in Output is the hand image after unsharpening the mask. The core of Unsharp Masking is to use edge masks, preserving only the edges and details of the hand image while filtering out pixels in flat areas, ensuring that sharpening only applies to the hand's contour / texture areas.

[0151] Local contrast enhancement function: Employs contrast-limited adaptive histogram equalization (CLAHE).

[0152] Parameter settings: The cropping and limiting parameters are determined by the contrast requirement component. Dynamic modulation, cropping and limiting threshold ,

[0153] in, The base clipping limit is set to 2.0, which is the default limit when the contrast requirement is 0, ensuring that the contrast enhancement is at a mild and safe level when there is no additional enhancement requirement.

[0154] The limiting increment coefficient is set to 5.0 to amplify the influence of the contrast demand component, making the limiting value more sensitive to changes in demand.

[0155] The contrast requirement component is one of the components in the multidimensional region shape control vector. It is dynamically calculated from the local contrast, edge intensity and other features of the image. The value range is usually between [0,1]. The larger the value, the stronger the contrast enhancement requirement of the current image.

[0156] In this embodiment, the image block size is 8×8, with 256 gray levels, and the pixel interpolation method is bilinear interpolation. The combination relationship between operators is achieved through the following logical judgment: In this embodiment, a first threshold is set. The second threshold is used to assess the degree of motion blur. Used to evaluate contour sharpness; sequential processing: when the motion blur component in the control vector... When this occurs, it indicates significant motion blur in the image. A serial pipeline architecture is then employed: first, a denoising filter is applied to suppress noise amplification; then, a sharpening kernel is applied to restore edges; and finally, local contrast enhancement is performed. The mathematical expression is: In the formula, This indicates noise reduction processing; Indicates sharpening; This indicates that the final contrast adjustment is performed on the sharpened image;

[0157] Parallel processing: When the contour sharpness component in the control vector This indicates that the image edges are blurred and the contrast is insufficient. To prevent excessive edge smoothing caused by serial processing, a parallel fusion architecture is adopted: the sharpening kernel and the local contrast enhancement function are processed in parallel, and the results are fused through adaptive weights.

[0158] The fusion formula is: ,

[0159] in, It is the final output image after parallel fusion, which is a weighted fusion product of the two processing results: sharpening and contrast enhancement. The sharpened image is generated by the Laplacian sharpening or Unsharp Masking sharpening operator to enhance the edge contours of the hand image, such as fingertips, knuckles, and skin texture. The image, after contrast enhancement processing, is generated by contrast enhancement operators such as CLAHE. It is used to improve the local contrast of hand images and enhance the visibility of details in low-light or low-contrast scenes. It is the fusion weighting coefficient, dynamically calculated from the contour sharpness component in the multi-dimensional region morphology control vector, with a value range of [0,1]. .

[0160] Mixed processing: If both of the above conditions are met simultaneously, that is... and Then, parallel fusion is performed first, and then the fusion result is denoised, that is: ,in ;

[0161] Based on the operator type, dynamic parameters, and combination process determined in the above steps, the hand region (ROI) in the original image is processed pixel by pixel. After processing, the enhanced hand region image is output to the next module or saved as the final result.

[0162] Example 2: Multimodal Fusion Hand Image Enhancement System

[0163] This invention also discloses a multimodal fusion hand image enhancement system. The system is deployed on a computing platform containing GPU acceleration units, such as an NVIDIA Jetson AGX Orin edge computing module or a server equipped with an NVIDIA RTX4090 GPU. The software architecture is implemented based on the PyTorch framework, specifically including:

[0164] A hand image multimodal quality analyzer is used to construct a bidirectional gradient channel between a differentiable physical simulation module of a parameterized 3D hand model and a 2D image processing module. Multiple iterative optimizations are performed within this channel to obtain optimized 2D hand region segmentation masks and 3D hand model pose parameters. During the iteration process, intermediate segmentation mask sequences and their corresponding 3D hand model pose parameter sequences are sampled at fixed intervals. A fixed-length region evolution feature vector is generated based on the intermediate segmentation mask sequences and pose parameter sequences. This vector is used to extract static morphological features from the optimized 2D hand region segmentation mask and structural features from the optimized 3D hand model pose parameters. The static morphological features, region evolution features, and structural features are adaptively weighted and fused to generate a multidimensional region morphological control vector.

[0165] The parameter instruction converter module, using an ARM Cortex-A78 CPU core as the execution unit, receives a multi-dimensional region morphological control vector and converts it into an image enhancement control instruction set by querying a preset mapping rule table. This control instruction set specifies the selection of image processing operators, parameter configuration, and execution logic. The rule table storage area, located in the L3 cache, stores 2000 "morphological vector-enhancement instruction" mapping rules defined by expert experience, with an access latency of <50ns. When the rule table is not matched, a lightweight random forest model, quantized to INT8 format, is invoked, and inference is completed on the NPU with a latency of <2ms. This module reads the morphological control vector generated by the analyzer via the PCIe bus, and after querying or inference, outputs a structured image enhancement control instruction set in JSON format.

[0166] An adaptive processing engine, communicating with a parameter instruction converter, receives the image enhancement control instruction set, selects and instantiates corresponding image processing operators from a pre-stored operator library according to the instruction set, and combines multiple operators into a processing chain according to the execution logic. The operator library includes at least a denoising filter, a sharpening kernel, and a local contrast enhancement function. Each operator is pre-compiled into a CUDA kernel and stored in the GPU memory. During runtime, the instruction parser dynamically arranges multiple CUDA kernels into a computation graph within 1ms, supporting sequential or parallel execution, and instantiation is completed through the CUDA Graph mechanism.

[0167] The pixel processing unit is a dedicated image processing pipeline connected to the adaptive processing engine. It is used to load the processing chain, perform pixel-level processing on the input raw hand region image, and output an enhanced hand region image.

[0168] This system achieves end-to-end real-time processing capabilities, from multimodal analysis to pixel-level enhancement, through dedicated hardware acceleration and software pipeline design.

[0169] While the present invention has been described above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for enhancing hand images through multimodal fusion, characterized in that, Includes the following steps: Step S1: Construct a bidirectional gradient channel between the differentiable physical simulation module and the two-dimensional image processing module of the parameterized three-dimensional hand model; perform multiple iterative optimizations within the bidirectional gradient channel to obtain the optimized two-dimensional hand region segmentation mask and the pose parameters of the three-dimensional hand model; Step S2: Sample and obtain the intermediate segmentation mask sequence and the corresponding intermediate pose parameter sequence of the 3D hand model during the iteration process, and generate a fixed-length region evolution feature vector based on the intermediate segmentation mask sequence and the intermediate pose parameter sequence. Step S3: Extract static morphological feature vectors from the optimized 2D hand region segmentation mask; extract structural feature vectors from the optimized 3D hand model pose parameters; adaptively weight and fuse the static morphological feature vectors, structural feature vectors, and region evolution feature vectors to generate multi-dimensional region morphological control vectors; wherein, the generation of multi-dimensional region morphological control vectors adopts a two-stage gating fusion mechanism, specifically: Phase 1: Using static morphological feature vectors and regional evolution feature vectors as inputs, the output is a fusion weight with values ​​ranging from [0,1]. They are then merged into an intermediate feature vector, calculated using the following formula: ； in, This represents the intermediate feature vector. Represents a static morphological feature vector. Represents the feature vector of regional evolution. The fusion weights are the static morphological feature vector and the regional evolution feature vector. The second stage: Taking the intermediate feature vector and structural feature vector as input, the output is a fusion weight β with a value range of [0,1], which is then fused into an optimized multi-dimensional region morphological control vector. The calculation formula is as follows: ； in, This represents the multidimensional region morphological control vector obtained after final fusion. Represents structural feature vectors. The fusion weights are the intermediate feature vectors and the structural feature vectors. Step S4: Convert the multidimensional region morphology control vector obtained in step S3 into image enhancement control instructions, adaptively select and combine image processing operators, apply the image processing operators to perform pixel-level processing on the hand region in the original image, and output the enhanced hand region image.

2. The hand image enhancement method based on multimodal fusion according to claim 1, characterized in that, Before iterative optimization, the initialization of the pose parameters of the 3D hand model is also included. Specifically, the input image is first segmented and the region state is analyzed to obtain the initial segmentation mask and the initial region morphology distribution parameters. The parameters include at least the region area parameter, density parameter, aspect ratio parameter, centroid position parameter and principal axis direction parameter. Then, based on the initial region morphology distribution parameters and combined with the preset region morphology-pose mapping relationship, the pose parameters of the 3D hand model are initialized.

3. The multimodal fusion hand image enhancement method according to claim 1 or 2, characterized in that, Each iteration of the optimization process in step S1 includes: generating a two-dimensional projection mask based on the pose parameters of the three-dimensional hand model after the last update through differentiable rendering, and updating the segmentation network parameters, region analysis network parameters, and three-dimensional hand model pose parameters by minimizing the joint loss function.

4. The hand image enhancement method based on multimodal fusion according to claim 3, characterized in that, The bidirectional gradient channel in step S1 includes: The first forward channel is used to map the segmentation mask and region morphological distribution parameters output by the two-dimensional image processing module into the input constraints of the differentiable physical simulation module. The second reverse channel is used to convert the state of the three-dimensional hand model of the differentiable physics simulation module into an image domain supervision signal through differentiable rendering, so as to calculate the loss and backpropagate it to the two-dimensional image processing module.

5. The hand image enhancement method based on multimodal fusion according to claim 3, characterized in that, The joint loss function L is composed of the segmentation loss term. Geometric alignment loss term and joint angle limit loss item The weighted summation consists of: ; in, , , These are weighting coefficients, and their values ​​are positive real numbers. The set of actual pixel mask values; The set of pixel values ​​for the hand region prediction mask output by the 2D image processing module; The set of pixel values ​​for a two-dimensional projection mask generated by differentiable rendering of a parametric 3D hand model; This represents the current angle state of all joints in the hand. The geometric alignment loss term is expressed as: ; in, This represents the value of the predicted mask at the i-th pixel position, where i is the pixel index. This represents the value of the projection mask at the i-th pixel position. For the preset positive numbers, the summation symbol is... This indicates that the corresponding values ​​of all pixels in the mask are summed. The joint angle limitation loss term is expressed as: ; in, This represents the total number of joints in the hand. For the first The current angle value of each joint. From 1 to Traverse all joints, For the first The maximum permissible absolute angle preset for each joint. Indicates the first The absolute value of each joint angle.

6. The hand image enhancement method based on multimodal fusion according to claim 1, characterized in that, The process of generating a fixed-length region evolution feature vector in step S2 is as follows: Step S21: For each segmentation mask in the intermediate segmentation mask sequence, calculate its shape feature description vector, which includes at least the area, compactness, and Hu moment invariant features of the mask; Step S22: Establish the correspondence between the attitude parameter vector of each iteration step in the intermediate attitude parameter sequence and a segmentation mask in the intermediate segmentation mask sequence; for each iteration step, concatenate its attitude parameter vector with its corresponding shape feature description vector to form the joint feature vector of that iteration step. Step S23: Input the sequence of joint feature vectors arranged in iteration order into a bidirectional long short-term memory network for processing; Step S24: Concatenate the forward hidden state and the backward hidden state of the last time step of the bidirectional long short-term memory network. The concatenation result is a fixed-length region evolution feature vector.

7. The hand image enhancement method based on multimodal fusion according to claim 1, characterized in that, In step S4, based on the multi-dimensional region morphology control vector, corresponding image enhancement control commands are generated through predefined parameter-command mapping rules.

8. The hand image enhancement method based on multimodal fusion according to claim 7, characterized in that, The image processing operator includes at least a denoising filter, a sharpening kernel, and a local contrast enhancement function. A preset first threshold is used to evaluate the degree of motion blur, and a second threshold is used to evaluate the contour sharpness. The order and parallel relationship of the combined image processing operators are as follows: when the degree of motion blur is higher than the first threshold, the order of denoising first and then enhancement is adopted; when the contour sharpness is lower than the second threshold, the parallel processing flow of enhancement and sharpening is adopted; if both of the above conditions are met at the same time, the sharpening and local contrast enhancement are first fused in parallel, and then the denoising processing is performed on the result of the parallel fusion processing.

9. A multimodal fusion hand image enhancement system, characterized in that, The system performs the method described in any one of claims 1-8, specifically including: A hand image multimodal quality analyzer is used to execute the methods described in steps S1-S3 to generate multidimensional region morphological control vectors; A parameter-instruction converter is used to receive a multi-dimensional region morphology control vector and convert the vector into an image enhancement control instruction set through a predefined parameter-instruction mapping rule; the control instruction set is used to specify the selection of image processing operators, parameter configuration, and execution logic. An adaptive processing engine, connected to the parameter instruction converter, is used to receive an image enhancement control instruction set, select and instantiate corresponding image processing operators from a pre-stored operator library according to the instruction set, and combine multiple operators into a processing chain according to the execution logic; the operator library includes at least a denoising filter, a sharpening kernel, and a local contrast enhancement function; The pixel processing unit, connected to the adaptive processing engine, is used to load the processing chain, perform pixel-level processing on the input raw hand region image, and output an enhanced hand region image.

Images