A three-dimensional topography reconstruction method combining local and global focusing features

Abstract

The application discloses a three-dimensional profile reconstruction method combining local and global focusing features. The method comprises the following steps: firstly, a multi-focus image sequence is obtained by vertically adjusting the relative distance of the object to be measured to the camera photosensitive imaging surface; secondly, a focusing feature set is obtained by performing convolution operation, deformable convolution operation and pooling operation on the multi-focus image sequence; then, the focusing feature set is input into a local-global feature joint module to obtain a local-global feature set; finally, the local-global feature set is mapped into a depth map. The method can improve the reconstruction efficiency while maintaining high precision, and can effectively meet the application requirements across scenes.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction technology, specifically relating to a three-dimensional topography reconstruction method that combines local and global focusing features. Background Technology

[0002] 3D topography reconstruction, as a typical computer vision technology, is mainly used in processes such as model design and quality inspection in the precision manufacturing field. With the increasing demand for refined 3D modeling of mesoscopic scenes and multi-focus image fusion in the consumer electronics field, existing 3D topography reconstruction methods cannot meet the application needs of these diverse scenarios.

[0003] Currently, 3D reconstruction methods for measuring the surface topography of objects can be broadly categorized into two types: optical reconstruction and graphics reconstruction. Optical reconstruction acquires the surface topography of the object by projecting light rays and receiving reflected signals. For example, 3D laser scanning technology uses a laser rangefinder to detect the object's shape. First, light rays are projected onto the object; the signal is scattered by the object's surface to obtain point cloud data. Then, outliers in the point cloud data are removed, and downsampling is performed for refinement, resulting in the 3D point cloud data of the object. Since the accuracy of optical reconstruction is indirectly dependent on the accuracy of the optical equipment, high accuracy and low cost are mutually exclusive. Graphics reconstruction mainly involves acquiring images of the object's topography and supplementing them with appropriate depth cues for 3D reconstruction. This can be roughly divided into manual feature selection and neural network prediction. Manual feature selection first requires finding features as key information points based on the object's surface topography. Then, based on these key information points, feature registration or deep clustering is performed on the object's topography image. Finally, a 3D topography function is fitted to the object, and the model is iteratively repaired to obtain the 3D model of the object. Because imaging principles and noise interference vary across different scenes, various modeling methods are typically designed for specific application domains, lacking scene adaptability. Neural network prediction, however, integrates and refines data from multiple scenes to form an automated decision-making model, effectively overcoming the high cost of optical reconstruction and the limitations of manual feature modeling due to a lack of prior scene knowledge. However, currently, neural network prediction only performs feature analysis on images using local convolutional kernels, without incorporating global feature information to effectively guide the fitting of the object's shape.

[0004] The current research indicates that existing 3D shape reconstruction methods suffer from three main shortcomings: optical reconstruction's stringent requirements for optical equipment make it impossible to balance cost and accuracy; manual feature modeling in graphics reconstruction relies on prior scene knowledge, resulting in poor model portability in complex and variable scenes; and neural network prediction fails to utilize global feature information, leading to insufficient accuracy when fitting the shape of the object under test. With the increasing demand in related fields in China, achieving scene-adaptive 3D shape reconstruction methods is one of the most pressing issues to consider at this stage.

[0005] As shown above, we believe that in neural network-based 3D topography reconstruction methods, the high practicality of the network structure and the effective utilization of global information are key to solving the cross-scenario application of 3D topography reconstruction. Therefore, this patent, through the design of a focused feature enhancement module and a local-global feature joint module, enables the low-cost and high-efficiency deployment and application of 3D topography reconstruction methods. Summary of the Invention

[0006] To overcome the shortcomings of existing methods, the purpose of this invention is to provide a three-dimensional topography reconstruction method that combines local and global focusing features.

[0007] The technical solution of this invention is: a three-dimensional topography reconstruction method combining local and global focusing features, comprising the following steps:

[0008] Step 1: By vertically adjusting the relative distance between the object under test and the camera's photosensitive imaging surface, focused images of different regions are generated, thus obtaining a multi-focus image sequence of the object under test. n represents the number of multifocus image sequences, and its value ranges from 1 to n to N.

[0009] Step 2: Process the multifocus image sequence obtained in Step 1 The image sequence feature set is obtained by performing a convolution operation according to equation (1). n′ represents the number of feature sets, and its value ranges from 1 ≤ n′ ≤ N′.

[0010]

[0011] Where conv1 and conv2 represent the parameters of the neural network convolutional layer. For convolution operators;

[0012] Step 3: Process the image sequence feature set obtained in Step 2 According to equation (2), I′ n′ The initial focused feature I is obtained by performing deformable convolution on the feature at position P. low ,

[0013]

[0014] Among them W k and P k This represents the pre-specified convolution weights and convolution offsets at the k-th position, where K represents the total number of deformable convolution kernels;

[0015] Step 4: Apply the initial focusing feature I obtained in Step 3 low The focused feature set is obtained by performing a maximum pooling operation on it according to equation (3). Its value range is 1≤k≤K.

[0016]

[0017] Where F k This indicates that the k-th feature map is in the feature matrix I. loc Max pooling output value, x kpq Represents the characteristic matrix I loc The k-th eigenvalue located at (p, q);

[0018] Step 5: Apply the focusing feature set obtained in Step 4 The feature separation module obtains new features {F}. l ,F g}, where F l F represents the input feature of the local feature modeling branch. g This represents the input features of the global feature modeling branch;

[0019] Step 6: Input feature F for the local feature modeling branch in Step 5 l According to equation (4), depthwise convolution is performed to obtain the local features l(F) of the multi-focus image sequence. l ),

[0020]

[0021] Where conv3 represents the depth convolution parameter in the neural network layer;

[0022] Step 7: Input feature F for the global feature modeling branch in Step 5 g Self-attention correction is performed according to equation (5) to obtain the global features g(F) of the multi-focus image sequence. g ),

[0023]

[0024] Where g i (F g ) represents the input feature F at position i. g The association between θ and other location features is constructed. i (F g) represents the input feature F at position i. g In the feature embedding process in the θ space, φ j (F g )and These represent the input features F at position j, respectively. g In φ and The feature embedding process in space, where N represents the number of feature points, T represents the transpose of the feature vector, exp represents the exponential function with the natural constant e as the base, and ∑(·) represents the accumulation of feature values;

[0025] Step 8: Analyze the local features l(F) of the multi-focus image sequence obtained in Step 6. l The global features g(F) of the multifocus image sequence obtained in step 7 and step 7. g According to equation (6), the feature joint operation is performed to obtain the local-global feature set F. cou ,

[0026] F cou =f(l(F) l ),g(F g (6)

[0027] Where f(·,·) represents the joint function of local and global features;

[0028] Step 9: Process the local-global features F obtained in Step 8 cou The depth map D of the object to be measured is obtained iteratively through the feature mapping module of equation (7).

[0029]

[0030] Where conv4, conv5, and conv6 represent the convolution parameters in the neural network layers. This is the deconvolution operator.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] (1) The focusing feature enhancement module and the local-global feature joint module constructed in this invention take into account the characteristics of local focusing information positioning and global focusing information coupling, and can realize the cross-scene and cross-domain application of the three-dimensional shape reconstruction method.

[0033] (2) The three-dimensional topography reconstruction method proposed in this invention improves reconstruction efficiency while maintaining high accuracy, and can meet the requirements of reconstruction accuracy and efficiency in different scenarios. Attached Figure Description

[0034] Figure 1 This is a flowchart of a three-dimensional topography reconstruction method that combines local and global focusing features, as disclosed in this invention.

[0035] Figure 2 This is a schematic diagram of a three-dimensional topography reconstruction method that combines local and global focusing features, as disclosed in this invention. Detailed Implementation

[0036] like Figure 1 , Figure 2 As shown in the figure, the three-dimensional topography reconstruction method combining local and global focusing features described in this embodiment includes the following steps:

[0037] Step 1: By vertically adjusting the relative distance between the object under test and the camera's photosensitive imaging surface, focused images of different regions are generated, thus obtaining a multi-focus image sequence of the object under test. n represents the number of multifocus image sequences, and its value ranges from 1 to n to N.

[0038] Step 2: Process the multifocus image sequence obtained in Step 1 The image sequence feature set is obtained by performing a convolution operation according to equation (1). n′ represents the number of feature sets, and its value ranges from 1 ≤ n′ ≤ N′.

[0039]

[0040] Where conv1 and conv2 represent the parameters of the neural network convolutional layer. For convolution operators;

[0041] Step 3: Process the image sequence feature set obtained in Step 2 According to equation (2), I′ n′ The initial focused feature I is obtained by performing deformable convolution on the feature at position P. low ,

[0042]

[0043] Among them W k and P k This represents the pre-specified convolution weights and convolution offsets at the k-th position, where K represents the total number of deformable convolution kernels;

[0044] Step 4: Apply the initial focusing feature I obtained in Step 3 low The focused feature set is obtained by performing a maximum pooling operation on it according to equation (3). Its value range is 1≤k≤K.

[0045]

[0046] Where F k This indicates that the k-th feature map is in the feature matrix I. loc Max pooling output value, xkpq Represents the characteristic matrix I loc The k-th eigenvalue located at (p, q);

[0047] Step 5: Apply the focusing feature set obtained in Step 4 The feature separation module obtains new features {F}. l ,F g}, where F l F represents the input feature of the local feature modeling branch. g This represents the input features of the global feature modeling branch;

[0048] Step 6: Input feature F for the local feature modeling branch in Step 5 l According to equation (4), depthwise convolution is performed to obtain the local features l(F) of the multi-focus image sequence. l ),

[0049]

[0050] Where conv3 represents the depth convolution parameter in the neural network layer;

[0051] Step 7: Input feature F for the global feature modeling branch in Step 5 g Self-attention correction is performed according to equation (5) to obtain the global features g(F) of the multi-focus image sequence. g ),

[0052]

[0053] Where g i (F g ) represents the input feature F at position i. g The association between θ and other location features is constructed. i (F g ) represents the input feature F at position i. g In the feature embedding process in the θ space, φ j (F g )and These represent the input features F at position j, respectively. g In φ and The feature embedding process in space, where N represents the number of feature points, T represents the transpose of the feature vector, exp represents the exponential function with the natural constant e as the base, and ∑(·) represents the accumulation of feature values;

[0054] Step 8: Analyze the local features l(F) of the multi-focus image sequence obtained in Step 6. l The global features g(F) of the multifocus image sequence obtained in step 7 and step 7. g According to equation (6), the feature joint operation is performed to obtain the local-global feature set F.cou ,

[0055] F cou =f(l(F) l ),g(F g (6)

[0056] Where f(·,·) represents the joint function of local and global features;

[0057] Step 9: Process the local-global features F obtained in Step 8 cou The depth map D of the object to be measured is obtained iteratively through the feature mapping module of equation (7).

[0058]

[0059] Where conv4, conv5, and conv6 represent the convolution parameters in the neural network layers. This is the deconvolution operator.

Claims

1. A method for three-dimensional topography reconstruction that combines local and global focusing features, comprising the following steps: Step 1: By vertically adjusting the relative distance between the object under test and the camera's photosensitive imaging surface, focused images of different regions are generated, thus obtaining a multi-focus image sequence of the object under test. n represents the number of multifocus image sequences, and its value ranges from 1 to n to N. Step 2: Process the multifocus image sequence obtained in Step 1 The image sequence feature set is obtained by performing a convolution operation according to equation (1). n′ represents the number of feature sets, and its value ranges from 1 ≤ n′ ≤ N′. where conv1 and conv2 represent neural network convolutional layer parameters, is a convolution operator; Step 3: Process the image sequence feature set obtained in Step 2 According to equation (2), I′ n′ The initial focused feature I is obtained by performing deformable convolution on the feature at position P. low , wherein W k and P k denote the pre-specified convolution weights and convolution offsets for the kth position, and K denotes the total number of deformable convolution kernels; Step 4: initial focusing features I obtained in step 3 low performing a max-pooling operation on it according to formula (3) to obtain a focusing feature set with a value range of 1≤k≤K, Where F k This indicates that the k-th feature map is in the feature matrix I. loc Max pooling output value, x kpq Represents the characteristic matrix I loc The k-th eigenvalue located at (p, q); Step 5: Apply the focusing feature set obtained in Step 4 New features {F} are obtained through the feature separation module. l ,F g }, where F l F represents the input feature of the local feature modeling branch. g This represents the input features of the global feature modeling branch; Step 6: Input features F for the local feature modeling branch in step 5 l Local features l(F l ) of the multi-focus image sequence are obtained by deep convolution according to formula (4). Where conv3 represents the depth convolution parameter in the neural network layer; Step 7: Input feature F for the global feature modeling branch in Step 5 g Self-attention correction is performed according to equation (5) to obtain the global features g(F) of the multi-focus image sequence. g ), Where g i (F g ) represents the input feature F at position i. g The association between θ and other location features is constructed. i (F g ) represents the input feature F at position i. g In the feature embedding process in the θ space, φ j (F g )and These represent the input features F at position j, respectively. g In φ and The feature embedding process in space, where N represents the number of feature points, T represents the transpose of the feature vector, exp represents the exponential function with the natural constant e as the base, and ∑(·) represents the accumulation of feature values; Step 8: Perform feature combination operation on the local features l(F l ) of the multi-focus image sequence obtained in Step 6 and the global features g(F g ) of the multi-focus image sequence obtained in Step 7 according to formula (6) to obtain a local-global feature set F cou , F cou = f(l(F l ), g(F g )) (6) Where f(·,·) represents the joint function of local and global features; Step 9: Partial-global features F obtained in step 8 are concatenated to form the global feature F cou The depth map D of the object under test is obtained by iterating the feature mapping module of formula (7). where conv4, conv5 and conv6 represent convolution parameters in a neural network layer, is an anti-convolution operator.

Images