A three-dimensional shape measurement method based on HF-TransNet

Abstract

The present application relates to a kind of three-dimensional shape measurement method based on HF-TransNet.In edge projection profile measurement method, the efficient high-precision height measurement of single-frame stripe pattern is crucial, deep learning method can directly carry out height map regression, but it is susceptible to the influence of complex background.The present application proposes a hierarchical fusion Transformer network, called HF-TransNet, specifically, to enhance the interaction between global and local features in single-frame stripe image, a partial Transformer unit is proposed, a double-branch parallel architecture is used to fuse the global context features of Transformer and the local texture features of CNN, a dynamic displacement feature fusion module is designed, which gradually captures fine stripe details and macro geometric information through displacement convolution, meanwhile, HF Block adopts a multi-branch parallel attention scheme, effectively aggregating shallow texture features and deep semantic information, experimental results show that the model proposed in the present application achieves a root mean square error of 0.0009 mm, which is 50% lower than UNet, and the structural similarity index is 99.91%.

Description

Technical Field

[0001] This invention belongs to the field of structured light 3D measurement, specifically involving a 3D shape measurement method based on HF-TransNet. Background Technology

[0002] Three-dimensional measurement technology plays a crucial role in key fields such as interferometric synthetic aperture radar, magnetic resonance imaging, and cultural relic preservation. With the continuous improvement of application requirements, rapidly and accurately acquiring the three-dimensional shape information of the measured object has become a core technological challenge that urgently needs to be overcome in this field.

[0003] Among numerous 3D measurement methods, structured light technology has become one of the mainstream technologies for 3D shape measurement due to its advantages such as high precision, non-contact measurement, and low cost. This technology projects a specifically coded optical pattern onto the object being measured and uses the deformation of the pattern on the object's surface to calculate 3D geometric information. As an important branch of structured light technology, fringe projection profilometry (FPP) technology projects a periodic sinusoidal fringe pattern and reconstructs the 3D shape of an object based on phase information, and has been widely used in precision manufacturing, quality inspection, and other fields.

[0004] Traditional fringe projection contour measurement techniques are mainly divided into two categories: multi-frame phase-shifting methods and single-frame phase extraction methods. Multi-frame phase-shifting methods project multiple fringe patterns with different phases and calculate the absolute phase using a phase-shifting algorithm. This method offers high accuracy but requires projecting 3-4 or even more images, resulting in a long measurement time and making it difficult to meet the real-time measurement needs of dynamic scenes. Single-frame phase extraction methods project a single fringe pattern and directly learn phase information from it using a deep learning network. While this method has advantages such as single-frame acquisition and high measurement efficiency, it still suffers from unsatisfactory measurement accuracy, a large number of model parameters, low computational efficiency, and difficulty in fully capturing global contextual information and long-distance spatial dependencies. These limitations provide a clear direction for the development of new fringe projection contour measurement techniques.

[0005] In recent years, deep learning technology has made groundbreaking progress in the field of computer vision, bringing new solutions to structured light 3D measurement. The deep learning-based single-frame fringe phase extraction method can directly learn phase information from the fringe pattern, avoiding the complex mathematical transformation process of traditional methods. Compared with traditional methods, deep learning methods exhibit stronger robustness in handling noise, texture interference, and discontinuous phase.

[0006] Despite the considerable potential of deep learning methods in structured light-based 3D measurements, several key challenges remain:

[0007] First, compared to multi-frame phase-shifting techniques, the measurement accuracy of current deep learning methods remains unsatisfactory, especially in high-precision industrial metrology applications requiring sub-millimeter level accuracy. This accuracy gap significantly limits the deployment of deep learning methods in quality control and precision manufacturing scenarios.

[0008] Secondly, existing methods suffer from low computational efficiency, manifested in excessive model parameters and redundant architecture, which not only increases memory requirements but also prolongs inference time.

[0009] Furthermore, the main reliance on traditional convolutional neural network architectures has inherent limitations in capturing global contextual information and modeling long-range spatial dependencies, which are crucial for understanding complex stripe patterns and maintaining phase consistency in extended spatial domains.

[0010] Patent No. 202511307297.0 discloses an image measurement system and method based on deep learning and multi-feature fusion. The method includes: acquiring image data of a target object using an image acquisition device; extracting features from the image data using a deep learning model to automatically obtain multiple feature information; fusing the extracted features and strengthening key feature representations using weighted fusion or a fusion strategy based on a self-attention mechanism; and then using a measurement calculation module to measure the size, position, and shape of the target object based on the fused features. The problems with this method are: it is mainly geared towards general image measurement tasks. Although it can output measurement results of the target object through deep learning and multi-feature fusion, it does not establish a dedicated feature extraction and fusion mechanism for height reconstruction tasks in structured light stripe images, making it difficult to fully capture subtle texture changes, long-distance spatial dependencies, and macroscopic geometric information in a single frame of stripe patterns. Furthermore, its weighted fusion or self-attention fusion strategy is relatively broad, and in scenarios with complex lighting, complex geometry, and discontinuous surfaces, it may still suffer from insufficient feature representation, high computational overhead, and limited real-time performance.

[0011] Based on the above analysis, how to achieve high-precision three-dimensional measurement has become a key issue that urgently needs to be addressed. Summary of the Invention

[0012] This invention integrates convolutional neural networks and the Transformer mechanism through the HF-TransNet network, providing a three-dimensional shape measurement method based on HF-TransNet.

[0013] To achieve the above objectives, the present invention provides the following technical solution:

[0014] A three-dimensional shape measurement method based on HF-TransNet, characterized by comprising the following steps:

[0015] Step 1: Use a public dataset for model training. Collect several sets of structured light stripe patterns to form a dataset. Randomly divide the dataset into a training set, a validation set, and a test set. There is no overlap between the sample sets.

[0016] Step 2: Construct a 3D measurement model using the HF-TransNet network;

[0017] Step 3: In the HF-TransNet network, the bottleneck part adopts the Partial Transformer Unit module, and the HF Block module is introduced in the feature extraction part. Through the CBAM attention mechanism and the dual-branch parallel architecture, a hierarchical and progressive feature extraction architecture is constructed; finally, the optimal model is obtained through multiple rounds of training iterations.

[0018] Step 4: Input the test set images into the trained detection model with optimal weights for performance testing, thereby obtaining the measurement results for each image.

[0019] Furthermore, step 2 specifically includes:

[0020] First, the divided training set images are input into the HF-TransNet network for training;

[0021] Then, the model performance is optimized by adjusting parameters such as the learning rate and the number of iterations;

[0022] Finally, the optimal model weights are obtained through training.

[0023] Furthermore, in step 3:

[0024] The HF Block module employs a five-branch parallel architecture (Branch 0-Branch 4) to simultaneously capture multi-level feature representations, from shallow details to deep semantics.

[0025] Branch 0 uses the DSFF module combined with batch normalization and CBAM attention mechanism;

[0026] Branch 1 through 3 3. Convolution, batch normalization and CBAM attention mechanism are used to extract features, and both focus on shallow feature learning;

[0027] Branch 2-Branch 4 achieve deep semantic feature learning through a progressively deeper convolutional block design, containing two, three, and four convolutional blocks respectively, with each layer employing 3... 3. Combination of convolution, batch normalization, and LeakyReLU activation function.

[0028] Furthermore, in step 3

[0029] In the feature processing stage, the N / 4 channel segmentation method is adopted to evenly distribute the input features to each branch. By adding elements one by one, the advantages of residual learning are maintained while realizing the organic integration of multi-scale and multi-level features.

[0030] Furthermore, the CBAM attention mechanism refines and enhances the original features through element-wise multiplication operations, specifically as follows:

[0031] Let the input features be First, the CBAM module learns the relationships between different feature channels, specifically expressed as follows:

[0032] Global average pooling and global max pooling:

[0033]

[0034]

[0035] Multilayer Perceptron (MLP) Mapping:

[0036]

[0037]

[0038] Merge and generate channel attention maps:

[0039]

[0040] Final output: ;

[0041] Secondly, the CBAM module learns the spatial relationships between different features, expressed as:

[0042] Average pooling and channel max pooling:

[0043]

[0044]

[0045] in, It is a channel index, with a range of values. . This represents the feature map of the c-th channel, with size H. W;

[0046] splicing and blending:

[0047]

[0048] Generate spatial attention map:

[0049]

[0050] Furthermore, the dual-branch parallel architecture employs a CNN branch to extract local features, uses lightweight convolutional operations to accurately capture spatial neighborhood correlations, and uses an MHSA-CGLU branch (Transformer branch) to establish long-distance feature dependencies using a multi-head self-attention mechanism. Specifically:

[0051] At the input end, the features are divided into two parts, which are input into the CNN branch and the MHSA-CGLU branch respectively, as expressed in the following expression:

[0052]

[0053]

[0054]

[0055] in, (transformer channel ratio) represents the proportion of Transformer channels. Setting this ratio... The value is 0.25, meaning 25% of the channels are allocated to the MHSA-CGLU Block and 75% of the channels are allocated to the HF Block branch;

[0056] The HF Block branch uses a concatenated convolution structure, expressed by the formula:

[0057]

[0058] The MHSA-CGLU branch uses the MHSA and CGLU modules for feature processing.

[0059]

[0060] After each branch is processed, the output features of the two branches are concatenated and then fused through a convolutional layer.

[0061] .

[0062] Furthermore, the multi-head self-attention mechanism captures the dependencies between different positions simultaneously through multiple attention heads. Each attention head independently calculates its attention score and performs a weighted summation of the input features to generate richer features. The specific method is as follows:

[0063] First, the input features N is the sequence length, D is the dimension of the input features, and the query (Q), key (K), and value (V) matrices are generated through three linear transformations (fully connected layers), expressed as:

[0064]

[0065]

[0066]

[0067] in, , and It is a learnable weight matrix. and These are the key and value dimensions for each attention head;

[0068] Will , and Matrix partitioning A person's attention

[0069]

[0070]

[0071]

[0072] in, It's about the number of heads to focus on. It is the key dimension of each head;

[0073] For each attention head Calculate the attention scores and perform a weighted sum:

[0074]

[0075] in, It is a scaling factor used to prevent the attention score from becoming too large during gradient descent;

[0076] The outputs of all attention heads are concatenated together, and the final output feature is generated through a linear transformation.

[0077] Furthermore, the DSFF module adopts a multi-scale parallel processing mechanism, consisting of three parts: Conv, ShiftConv, and InformationFusion. ShiftConv divides the input channels into five groups in a 1:1:1:1:1 ratio and performs left shift, right shift, up shift, down shift, and identity transformation respectively. It captures the spatial dependencies in the four main directions through spatial position shifting operations, thus obtaining spatial feature modeling.

[0078] Furthermore, the multi-scale parallel processing mechanism is as follows: different scale branches of 1×1, 3×3, and 5×5 are used to process the input features simultaneously. The scale=1 branch maintains the traditional channel transformation capability, while the scale>1 branch uses ShiftConv to model spatial features. Then, the multi-scale features are synergistically fused through the feature fusion layer, which enhances the expression of complementary features while reducing the interference of redundant information, thereby constructing an optimized feature model.

[0079] Furthermore, in step 2, before training the network, the size of the training set images is uniformly adjusted.

[0080] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0081] 1. To address the problem of insufficient global-local feature coupling in single-striped patterns, this invention designs a ParallelTransformer Unit (PTU). The PTU uses a dual-branch parallel architecture to complementarily fuse global self-attention features based on Transformer with local texture features based on CNN, achieving efficient fusion and collaborative optimization of local fine-grained features and global semantic information, thereby improving the feature representation capability and high reconstruction accuracy under complex striped patterns.

[0082] 2. In order to enhance detail reconstruction, the present invention designs a hierarchical fusion module (HF Block) that adopts a multi-branch parallel attention scheme to effectively aggregate shallow texture features and deep semantic information. By introducing an attention mechanism and a dual-branch parallel structure, the HF Block constructs a hierarchical feature extraction architecture, realizes multi-scale fusion of shallow texture details and deep semantic information, and improves the expressive power of complex stripe features and the accuracy of detail reconstruction.

[0083] 3. This invention improves image detail capture by using the Dynamic Shift Feature Fusion (DSFF) module, which gradually captures fine stripe details and macroscopic geometric information through displacement convolution.

[0084] 4. This invention divides the acquired structured light stripe images into training, validation, and test sets according to a preset ratio. Then, it constructs an HF-TransNet 3D measurement model. HF-TransNet innovatively integrates convolutional neural networks and Transformer mechanisms, providing a new solution for 3D measurement tasks. The divided images are input into the network for training, and the model performance is optimized by adjusting parameters such as the learning rate and number of iterations. A Partial Transformer Unit (PTU) module is introduced into the bottleneck part of the HF-TransNet network to leverage the complementary advantages of CNN and Transformer. Simultaneously, an HF Block module is introduced in the feature extraction part to construct a hierarchical feature extraction architecture, enhancing the model's understanding of global contextual information, local texture features, and multi-scale feature information. Finally, the optimal model weights are obtained through multiple rounds of training iterations. The test set images are input into the model with the optimal weights for testing, obtaining the 3D measurement results for each image. This method achieves a root mean square error (RMSE) of 0.0009 mm, which is 50% lower than UNet; the structural similarity index (SSIM) reaches 99.91%, and the inference time per frame is only 0.0191 s. Attached Figure Description

[0085] Figure 1 This is a schematic diagram illustrating the specific steps of the 3D measurement method based on HF-TransNet of the present invention.

[0086] Figure 2 This is a schematic diagram of the HF Block structure in this example.

[0087] Figure 3 This is a schematic diagram of the CBAM structure in this example.

[0088] Figure 4 This is a schematic diagram of the DSFF structure in this example.

[0089] Figure 5 This is a schematic diagram of the Partial Transformer Unit in this example.

[0090] Figure 6 This is a schematic diagram of the MHSA structure in this example.

[0091] Figure 7 This is a schematic diagram of the structure of CGLU in this example.

[0092] Figure 8 This is a schematic diagram of the results of HF-TransNet measuring a 3D image (single target) in this example.

[0093] Figure 9 This is a schematic diagram of the results of HF-TransNet measuring three-dimensional images (multi-target) in this example. Detailed Implementation

[0094] To more clearly illustrate the purpose, technical solution, and advantages of this invention, the technical solution will be described in detail below with reference to the accompanying drawings of specific embodiments of this invention. It should be noted that the described embodiments are only some examples of this invention, and not all examples. All other embodiments derived by those skilled in the art based on these examples without creative effort are included within the protection scope of this invention.

[0095] This invention employs a U-Net-based symmetric encoder-decoder architecture, progressively encoding input features from 64 to 1024 channels and incorporating multi-scale skip connections to preserve key feature information. The encoder utilizes a combination of a hierarchical fusion module (HFBlock) and a downsampling module to extract hierarchical spatial features, and integrates partial Transformer Units at network bottlenecks to capture long-range dependencies and global contextual information. Specifically, the HF Block enhances the extraction capability of stripe pattern features by integrating the CBAM attention mechanism and the DSFF module. The decoder path reconstructs spatial resolution using transposed convolution and upsampling modules, while utilizing 1 1. Convolutional layers are used to align channel features across skip connections. This hybrid design effectively balances the advantages of CNN's local feature extraction with Transformer's global attention mechanism, achieving accurate and highly detailed reconstruction while maintaining computational efficiency.

[0096] This embodiment proposes a 3D image measurement method based on HF-TransNet, such as... Figure 1 As shown, the specific implementation steps are as follows:

[0097] Step 1: Train the model and evaluate its performance using a public dataset.

[0098] This dataset contains 1523 sets of structured light stripe patterns, with each image having a resolution of 640. The dataset is randomly divided into training, validation, and test sets in a ratio of 8:1:1, with no overlap between the training, validation, and test samples, to ensure the effectiveness of model training and the reliability of the results.

[0099] The ground truth data for the dataset was obtained using the multi-frequency phase-shift fringe projection method. This method employs a four-step phase-shift algorithm for phase unpacking at four different frequencies, ensuring high-precision measurement of depth information. The data acquisition system utilizes the RVBUSTRVC-X mini 3D measurement camera, which consists of a triangulation configuration comprised of a binocular camera and a structured light projector.

[0100] The experimental sample consisted of dozens of small plaster sculptures, covering objects with varying geometric complexities and surface textures; the system's working distance was set at approximately 600 mm to ensure adequate measurement accuracy and field of view. During data acquisition, the object being measured could be a single target or a combination of multiple separate objects.

[0101] To enhance the diversity of the dataset, objects are randomly rotated and placed at different positions within the field of view during the acquisition process to improve the model's generalization ability.

[0102] Step 2: Construct a 3D measurement model using the HF-TransNet network.

[0103] Before training the network, the size of the training set images is adjusted uniformly.

[0104] The divided training set images are input into the HF-TransNet network for training. During the training process, the model performance is optimized by adjusting parameters such as the learning rate and the number of iterations. Finally, the optimal model weights are obtained through training, enabling accurate measurement of 3D objects.

[0105] Step 3: In the HF-TransNet network, the bottleneck part adopts the Partial Transformer Unit module, and the HF Block module is introduced in the feature extraction part to build a hierarchical and progressive feature extraction architecture; finally, the optimal model is obtained through multiple rounds of training iterations.

[0106] The HF-TransNet network architecture is a typical encoder-decoder architecture, such as... Figure 2 As shown, the network consists of an input layer, an encoder, a bottleneck layer, a decoder, and an output layer.

[0107] HF-TransNet employs a skip connection mechanism to fuse shallow, detailed features from the encoder with deep, semantic features from the decoder, effectively compensating for spatial information loss during downsampling. This structure balances local edge details with global semantic information. It achieves an innovative fusion of convolutional neural networks and the Transformer mechanism, providing a new solution for 3D measurement tasks. In the HF-TransNet network, the encoder consists of HF Blocks, max pooling, convolution, normalization, and activation functions.

[0108] To address the challenges of insufficient multi-scale fusion capabilities and lack of normalization processing in traditional UNet feature extraction, this invention designs the HF Block (hierarchical fusion module). By introducing the CBAM attention mechanism and a dual-branch parallel architecture, a hierarchical and progressive feature extraction architecture is constructed, such as... Figure 2 As shown.

[0109] The HF Block module adopts a five-branch parallel architecture (Branch 0-Branch 4) to capture multi-level feature representations from shallow details to deep semantics.

[0110] Branch 0 uses the DSFF module combined with batch normalization and CBAM attention mechanism;

[0111] Branch 1 through 3 3. Convolution, batch normalization, and CBAM attention mechanism are used to extract features, and both focus on shallow feature learning;

[0112] Branch 2-Branch 4 achieve deep semantic feature learning through a progressively deeper convolutional block design, containing two, three, and four convolutional blocks respectively, with each layer employing 3... 3. Combination of convolution, batch normalization, and LeakyReLU activation function.

[0113] In the feature processing stage, a channel segmentation strategy (N / 4) is adopted to evenly distribute the input features to each branch, reducing the computational burden; finally, through the element-wise addition fusion method, the advantages of residual learning are maintained while realizing the organic integration of multi-scale and multi-level features.

[0114] In the HF Block, the CBAM attention mechanism is introduced. This architecture systematically integrates the Channel Attention Module (CAM) and the Spatial Attention Module (SAM), with the following structure: Figure 3 As shown.

[0115] This attention mechanism enables adaptive recalibration of feature maps in both channel and spatial dimensions, and refines and enhances the original features through element-wise multiplication operations.

[0116] This multi-dimensional feature enhancement strategy significantly improves the model's ability to represent complex texture features, thereby achieving higher feature extraction accuracy and model robustness in fine visual tasks such as stripe pattern analysis. The specific method is as follows:

[0117] Let the input features be First, the CBAM module learns the relationships between different feature channels, and the specific expression is shown below:

[0118] Global average pooling and global max pooling:

[0119]

[0120]

[0121] Multilayer Perceptron (MLP) Mapping:

[0122]

[0123]

[0124] Merge and generate channel attention maps:

[0125]

[0126] Final output:

[0127] Secondly, the CBAM module learns the spatial relationships between different features, and its specific expression is shown below:

[0128] Average pooling and channel max pooling:

[0129]

[0130]

[0131] in, It is a channel index, with a range of values. . This represents the feature map of the c-th channel, with size H. W.

[0132] splicing and blending:

[0133]

[0134] Generate spatial attention map:

[0135] .

[0136] In Branch0 of the HF Block, the DSFF module employs a multi-scale parallel processing mechanism, with the structure as follows: Figure 4 As shown, this module mainly consists of three parts: Conv, ShiftConv, and Information Fusion.

[0137] ShiftConv, the core innovation of this module, divides the input channels into five groups in a 1:1:1:1:1 ratio, performing left shift, right shift, up shift, down shift, and identity transformation respectively. The key advantage of this design is that it effectively captures spatial dependencies in four main directions without adding any trainable parameters, achieving lightweight and efficient spatial feature modeling simply through spatial position shifting operations. Compared to traditional convolution operations, ShiftConv can obtain a wider spatial receptive field with lower computational cost, making it particularly suitable for processing image content with directional features.

[0138] The DSFF module processes input features simultaneously using branches at different scales (1×1, 3×3, and 5×5). The branch with scale=1 retains traditional channel transformation capabilities, while the branch with scale>1 utilizes ShiftConv for spatial feature extraction. This multi-scale parallel processing mechanism enables the module to capture both local details and global contextual information, and the introduction of ShiftConv further enhances the spatial modeling capabilities of each scale branch. Finally, a feature fusion layer integrates the multi-scale features, eliminating information redundancy and improving the quality of feature representation.

[0139] This design is suitable for deployment in the shallow layers of multi-branch networks, providing richer multi-scale low-level feature representations for complex reconstruction tasks, and significantly improving the feature representation capabilities of the network while maintaining computational efficiency.

[0140] In the field of structured light 3D measurement, Convolutional Neural Networks (CNNs) and Transformer architectures have shown significant advantages in local feature extraction and global dependency modeling, respectively. However, existing models often employ a single architecture design, resulting in the inability of the models to effectively integrate local fine-grained features with global semantic information.

[0141] Specifically, while traditional CNN architectures can efficiently extract local spatial features and maintain computational efficiency, their limited receptive field makes it difficult to capture long-distance spatial dependencies. In contrast, the Transformer architecture has a significant advantage in capturing global features, but it increases the computational burden when processing high-resolution feature maps and is relatively inadequate in perceiving local structural details.

[0142] The Partial Transformer Unit employs a dual-branch parallel processing mechanism. The structure of the Partial Transformer Unit is as follows: Figure 5 As shown, efficient fusion and collaborative optimization of local and global features are achieved. This is used to guide and adaptively adjust contextual information during multi-scale feature fusion. Specifically, the CNN branch is dedicated to local feature extraction, employing lightweight convolutional operations to accurately capture spatial neighborhood correlations, providing the model with rich local texture information and edge structure details. Meanwhile, the MHSA-CGLU branch (Transformer branch) utilizes a multi-head self-attention mechanism to establish long-distance feature dependencies, achieving effective modeling of global semantic information.

[0143] At the input end, the features are divided into two parts, which are fed into the CNN branch and the MHSA-CGLU branch respectively, as shown in the following expression:

[0144]

[0145]

[0146]

[0147] in, (transformer channel ratio) represents the proportion of Transformer channels. In the experiment, it is set... The ratio is 0.25, meaning 25% of the channels are allocated to the MHSA-CGLU Block and 75% of the channels are allocated to the HF Block branch.

[0148] The HF Block branch uses a concatenated convolutional structure to reduce computational complexity and enhance feature representation. Its formula can be expressed as:

[0149]

[0150] The MHSA-CGLU branch uses the MHSA and CGLU modules for feature processing.

[0151]

[0152] After each branch is processed, the output features of the two branches are concatenated and then fused through a convolutional layer.

[0153]

[0154] The introduction of the PTB module aims to integrate the advantages of CNN and Transformer architectures, solve the limitations of local and global feature fusion, and balance computational efficiency and model performance.

[0155] Multi-head self-attention (MHSA) is a core component of the Transformer architecture, designed to capture the dependency structure between different locations simultaneously using multiple attention heads. Figure 6 As shown, each attention head independently calculates an attention score and performs a weighted summation of the input features to generate a richer feature representation. The specific method is as follows:

[0156] First, the input features are transformed using three linear transformations (fully connected layers) to generate query (Q), key (K), and value (V) matrices, as shown in the following expression. Here, N is the sequence length, and D is the dimension of the input features.

[0157]

[0158]

[0159]

[0160] in, , and It is a learnable weight matrix. and These are the key and value dimensions of each attention head, respectively.

[0161] Will , and Matrix partitioning One point of attention:

[0162]

[0163]

[0164]

[0165] in, It's about the number of heads to focus on. It is the key dimension of each head.

[0166] For each attention head Calculate the attention scores and perform a weighted sum:

[0167]

[0168] in, It is a scaling factor used to prevent the attention score from becoming too large during gradient descent.

[0169] The outputs of all attention heads are concatenated together, and a linear transformation is applied to generate the final output feature: in, This represents the final learnable weight matrix.

[0170] CGLU (Convolutional Gated Linear Unit) is a neural network module that combines convolutional operations and gating mechanisms. It mainly consists of linear transformations, depthwise convolutions, activation functions, and element-wise multiplication operations, with a structure as follows: Figure 7 As shown.

[0171] Let the input features be Where C is the number of channels, and H and W are the height and width of the feature map, respectively. First, two parallel linear transformation branches are used: the output of branch one directly participates in subsequent calculations; the output of branch two is processed by 3... 3. Depthwise convolution and GELU activation function processing. Finally, the results of the two branches are multiplied element-wise to obtain the final output of CGLU, as shown in the following expression:

[0172]

[0173]

[0174] in, and For linear projection weights, and For the bias term, DWConv represents the depthwise convolution operation;

[0175] CGLU utilizes local feature information extracted by depthwise convolution to adjust the gating signal, thereby realizing a gating channel attention mechanism based on nearest neighbor features, which effectively improves model performance.

[0176] Step 4: Input the test set images into the trained detection model with optimal weights for performance evaluation, thereby obtaining the measurement results for each image. The measurement results are as follows: Figure 8 and Figure 9 As shown.

[0177] Table 1 shows the ablation experiment results of HF-TransNet. The experimental setup is as follows: a network containing only PTU (module A), a module with HF Block without CBAM attention (module B), and a module with HF Block containing CBAM attention (module C).

[0178] UNet 7.7 0.9967 0.0018 0.0184 A 8.6 0.9985 0.0015 0.0186 A+B 22.4 0.9987 0.0011 0.0188 A+B+C (HF-TransNet) 25.7 0.9991 0.0009 0.0191

[0179] Table 2 Comparison results of HF-TransNet and different models

[0180] UNet 7.7 0.0018 0.9967 0.0184 TransUNet 32.1 0.0019 0.9967 0.0219 Attention-UNet 31.4 0.0017 0.9979 0.0322 UNet++ 47.9 0.0019 0.9975 0.0557 ResUNet 59.9 0.0024 0.9973 0.0563 VURNet 21.6 0.0064 0.9958 0.0121 hNet 8.6 0.0016 0.9973 0.0195 UHRNet 30.3 0.0030 0.9978 0.0224 HF-TransNet (Ours) 25.7 0.0009 0.9991 0.0191

[0181] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional shape measurement method based on HF-TransNet, characterized in that, Includes the following steps: Step 1: Use a public dataset for model training. Collect several sets of structured light stripe patterns to form a dataset. Randomly divide the dataset into a training set, a validation set, and a test set. There is no overlap between the sample sets. Step 2: Construct a 3D measurement model using the HF-TransNet network; Step 3: In the HF-TransNet network, the bottleneck part adopts the Partial Transformer Unit module, and the HF Block module is introduced in the feature extraction part. Through the CBAM attention mechanism and the dual-branch parallel architecture, a hierarchical and progressive feature extraction architecture is constructed. Finally, the optimal model is obtained through multiple rounds of training iterations; Step 4: Input the test set images into the trained detection model with optimal weights for performance testing, thereby obtaining the measurement results for each image.

2. The three-dimensional shape measurement method based on HF-TransNet according to claim 1, characterized in that, Step 2 specifically includes: First, the divided training set images are input into the HF-TransNet network for training; Then, the model performance is optimized by adjusting parameters such as the learning rate and the number of iterations; Finally, the optimal model weights are obtained through training.

3. The three-dimensional shape measurement method based on HF-TransNet according to claim 1, characterized in that, In step 3: The HF Block module employs a five-branch parallel architecture (Branch 0-Branch 4) to simultaneously capture multi-level feature representations, from shallow details to deep semantics. Branch 0 uses the DSFF module combined with batch normalization and CBAM attention mechanism; Branch 1 through 3 3. Convolution, batch normalization and CBAM attention mechanism are used to extract features, and both focus on shallow feature learning; Branch 2-Branch 4 achieve deep semantic feature learning through a progressively deeper convolutional block design, containing two, three, and four convolutional blocks respectively, with each layer employing 3...

3. Combination of convolution, batch normalization, and LeakyReLU activation function.

4. The three-dimensional shape measurement method based on HF-TransNet according to claim 3, characterized in that, In step 3 In the feature processing stage, the N / 4 channel segmentation method is adopted to evenly distribute the input features to each branch. By adding elements one by one, the advantages of residual learning are maintained while realizing the organic integration of multi-scale and multi-level features.

5. The three-dimensional shape measurement method based on HF-TransNet according to claim 3, characterized in that, The CBAM attention mechanism refines and enhances the original features through element-wise multiplication operations, specifically as follows: Let the input features be First, the CBAM module learns the relationships between different feature channels, specifically expressed as follows: Global average pooling and global max pooling: Multilayer Perceptron (MLP) Mapping: Merge and generate channel attention maps: Final output: ; Secondly, the CBAM module learns the spatial relationships between different features, expressed as: Average pooling and channel max pooling: in, It is a channel index, with a range of values. . This represents the feature map of the c-th channel, with size H. W; splicing and blending: Generate spatial attention map:

6. The three-dimensional shape measurement method based on HF-TransNet according to claim 3, characterized in that, The dual-branch parallel architecture employs a CNN branch to extract local features, uses lightweight convolutional operations to accurately capture spatial neighborhood correlations, and uses an MHSA-CGLU branch (Transformer branch) to establish long-distance feature dependencies using a multi-head self-attention mechanism. Specifically: At the input end, the features are divided into two parts, which are input into the CNN branch and the MHSA-CGLU branch respectively, as expressed in the following expression: in, (transformer channel ratio) represents the proportion of Transformer channels. Setting this ratio... The value is 0.25, meaning 25% of the channels are allocated to the MHSA-CGLU Block and 75% of the channels are allocated to the HF Block branch; The HF Block branch uses a concatenated convolution structure, expressed by the formula: The MHSA-CGLU branch uses the MHSA and CGLU modules for feature processing. After each branch is processed, the output features of the two branches are concatenated and then fused through a convolutional layer. 。 7. A three-dimensional shape measurement method based on HF-TransNet according to claim 6, characterized in that, The multi-head self-attention mechanism captures the dependencies between different positions simultaneously through multiple attention heads. Each attention head independently calculates its attention score and performs a weighted summation of the input features to generate richer features. The specific method is as follows: First, the input features N is the sequence length, D is the dimension of the input features, and the query (Q), key (K), and value (V) matrices are generated through three linear transformations (fully connected layers), expressed as: in, , and It is a learnable weight matrix. and These are the dimensions of the keys and values ​​for each attention head; Will , and Matrix partitioning A person's attention in, It's about the number of heads to focus on. It is the key dimension of each head; For each attention head Calculate the attention scores and perform a weighted sum: in, It is a scaling factor used to prevent the attention score from becoming too large during gradient descent. It concatenates the outputs of all attention heads and generates the final output feature through a linear transformation.

8. The three-dimensional shape measurement method based on HF-TransNet according to claim 3, characterized in that, The DSFF module adopts a multi-scale parallel processing mechanism and consists of three parts: Conv, ShiftConv and Information Fusion. ShiftConv divides the input channels into five groups in a 1:1:1:1:1 ratio and performs left shift, right shift, up shift, down shift and identity transformation respectively. It captures the spatial dependencies in four main directions through spatial position shifting operations to obtain spatial feature modeling.

9. A three-dimensional shape measurement method based on HF-TransNet according to claim 3, characterized in that, The multi-scale parallel processing mechanism is as follows: different scale branches of 1×1, 3×3, and 5×5 are used to process the input features simultaneously. The scale=1 branch maintains the traditional channel transformation capability, while the scale>1 branch uses ShiftConv to model spatial features. Then, the multi-scale features are synergistically fused through the feature fusion layer, which enhances the expression of complementary features while reducing the interference of redundant information, thereby constructing an optimized feature model.

10. A three-dimensional shape measurement method based on HF-TransNet according to claim 1, characterized in that, In step 2, before training the network, the size of the training set images is adjusted uniformly.

Images