Component multi-dimensional information measurement method and system based on multi-source information fusion

Abstract

The invention discloses a component multi-dimensional information measurement method and system based on multi-source information fusion, and the method comprises the steps: 1, obtaining a fluorescence speckle image and a texture image before deformation, and decomposing the two images into low-frequency features and high-frequency features through discrete wavelet transform; 2, fusing low-frequency features of the fluorescence speckle image and the texture image before deformation; fusing the high-frequency features of the fluorescent speckle image and the texture image before deformation; 3, reconstructing the fused low-frequency features and high-frequency features through inverse wavelet transform to obtain a fused image before deformation; 4, acquiring a fused image after deformation; 5, the three-dimensional shape of the component is obtained through an FPP method; then calculating a sub-pixel-level displacement field, and constructing a three-dimensional displacement field according to the three-dimensional morphology and the sub-pixel-level displacement field; and extracting a partial derivative of the three-dimensional displacement field to the pixel coordinate, and calculating the Green strain tensor. According to the invention, the interference of environmental factors on displacement and strain measurement is effectively overcome, and the measurement precision and robustness are improved.

Description

Technical Field

[0001] This invention relates to the fields of optical measurement and computer vision technology, and in particular to a method and system for measuring multi-dimensional information of components based on multi-source information fusion. Background Technology

[0002] With the development of intelligent manufacturing and industrial inspection technologies, the measurement of morphology, strain, and displacement of complex components has become a key technology. Currently, commonly used measurement methods include Fringe Projection Profilometry (FPP) and Digital Image Correlation (DIC). Fringe Projection Profilometry, by projecting fringe patterns and analyzing reflected images, can obtain high-precision surface morphology information of components, but it cannot directly provide displacement and strain information. Digital Image Correlation is an image-matching-based measurement technique that measures displacement and strain information by comparing the texture images of the component's surface before and after deformation. However, traditional DIC heavily relies on the surface texture information of the component, and its measurement accuracy is easily affected by various complex interferences. Especially when dealing with highly reflective or weakly textured complex component surfaces, its measurement accuracy and robustness will significantly decrease. Moreover, DIC generally only provides displacement and strain information and cannot directly obtain the three-dimensional morphology data of the component. In summary, existing measurement technologies struggle to achieve simultaneous measurement of morphology, displacement, and strain of complex components, and the measurement accuracy is highly susceptible to environmental factors. Therefore, there is an urgent need for a new measurement method that can simultaneously acquire multi-dimensional information and improve measurement accuracy and robustness. Summary of the Invention

[0003] This invention provides a method and system for measuring multi-dimensional information of components based on multi-source information fusion, in order to solve the technical problems mentioned in the background art.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows: This invention provides a method for measuring multi-dimensional information of components based on multi-source information fusion, comprising the following steps: S1. Obtain the fluorescence speckle image and texture image before deformation, and decompose both images into low-frequency features and high-frequency features through discrete wavelet transform; S2. Local variance dynamic weight fusion is applied to the low-frequency features of the fluorescence speckle image and the texture image before deformation; local energy weight fusion is applied to the high-frequency features of the fluorescence speckle image and the texture image before deformation; thus, the fused low-frequency and high-frequency features before deformation are obtained. S3. Reconstruct the fused low-frequency and high-frequency features before deformation using inverse wavelet transform to obtain the fused image before deformation. S4. Obtain the deformed fused image using the same methods as S1 to S3; S5. The three-dimensional shape of the component is obtained by using the fringe projection profilometry (FPP); the subpixel level displacement field is calculated, and the three-dimensional displacement field is constructed based on the three-dimensional shape and the subpixel level displacement field; the partial derivatives of the three-dimensional displacement field with respect to the pixel coordinates are extracted, and the Green strain tensor is calculated based on the differential chain rule using the partial derivatives. The three-dimensional shape, the three-dimensional displacement field, and the Green strain tensor constitute the multi-dimensional information of the component.

[0005] Furthermore, step S1 specifically includes the following steps: S11. Uniformly spray fluorescent material onto the surface of the component before deformation, and acquire a fluorescent speckle image of the component before deformation using a camera. Texture images ; These are the pixel coordinates of the image before deformation; S12. Obtain the fluorescence speckle image before deformation. The fluorescence speckle image is decomposed into low-frequency features and high-frequency features in the horizontal, vertical and diagonal directions by discrete wavelet transform. S13, Transfer the texture image before deformation. The texture image is decomposed into low-frequency features and three high-frequency features in the horizontal, vertical and diagonal directions through discrete wavelet transform.

[0006] Furthermore, step S2 specifically includes the following steps: S21. The low-frequency features of the fluorescence speckle image and the texture image before deformation are fused using local variance dynamic weighting to obtain the fused low-frequency features. The specific calculation formula is as follows: ; in, This indicates the low-frequency characteristics after fusion; , These represent the low-frequency features of the fluorescence speckle image and the texture image before deformation, respectively; , These represent the weights corresponding to the low-frequency features of the fluorescence speckle image and the texture image before deformation, respectively. S22. The high-frequency features of the fluorescence speckle image and the texture image before deformation are fused using local energy weighting to obtain three fused high-frequency features in the horizontal, vertical, and diagonal directions. The specific calculation formula is as follows: ; ; ; in, , , These are the three high-frequency features after fusion in the horizontal, vertical, and diagonal directions; , , The fluorescence speckle image before deformation features three high-frequency characteristics in the horizontal, vertical, and diagonal directions; , , These are the three high-frequency features of the texture image before deformation in the horizontal, vertical, and diagonal directions; , , These are the weights corresponding to three high-frequency features in the horizontal, vertical, and diagonal directions of the fluorescence speckle image before deformation; , , These are the weights corresponding to three high-frequency features in the horizontal, vertical, and diagonal directions of the texture image before deformation.

[0007] Furthermore, the weights in S21 and weight The specific calculation formula is as follows: ; ; In the formula, , These represent the local variances of the fluorescence speckle image and texture image before deformation within the specified window, respectively. Weights in S22 and weight The specific calculation formula is as follows: ; ; in, , These are the local energies of the fluorescence speckle image and texture image before deformation within a specified window, respectively.

[0008] Furthermore, the calculation formula for the fused image before deformation in S3 is: ; in, This represents the fused image before deformation; IDWT represents the inverse discrete wavelet transform.

[0009] Furthermore, step S5 specifically includes the following steps: S51. A measurement system is used and the fringe projection profilometry (FPP) is employed to obtain the three-dimensional shape of the component; S52. Divide the surface of the component into gridded units, and use the zero-mean normalized squared difference correlation algorithm to calculate the matching similarity of the fused images before and after deformation within each gridded unit; S53. Minimize the matching similarity of the fused images before and after deformation within each gridded unit, and filter out high-confidence matching points with similarity values ​​lower than the preset similarity threshold. S54. For high-confidence matching points, the initial displacement field distribution is obtained using a least-squares-based surface fitting algorithm, and the displacement gradient is iteratively optimized using the Gauss-Newton method to obtain the sub-pixel-level displacement field. ; S55. Utilizing the three-dimensional topography in S51, the sub-pixel displacement field The two-dimensional pixel displacements in the image are mapped to three-dimensional space, and the three-dimensional displacement vectors of all pixels in the image are calculated and fused after deformation to obtain a three-dimensional displacement field. S56. The partial derivatives of the three-dimensional displacement field with respect to the pixel coordinates are extracted by a differentiable convolution kernel, and the Green strain tensor is calculated based on the differential chain rule. The Green strain tensor includes the normal strain components and shear strain components in the three axial directions of x, y, and z. The three-dimensional morphology, the three-dimensional displacement field, and the Green strain tensor constitute the multi-dimensional information of the component.

[0010] Furthermore, the formula for calculating the matching similarity in S52 is: ; in, This represents the currently calculated matching similarity. , These are the average gray values ​​of the local window frames within the fused images before and after deformation; , These are the standard deviations of the local window frames within the fused images before and after deformation; , These represent the local grayscale values ​​of the fused images before and after deformation, respectively. , These represent the horizontal and vertical offsets used to traverse all pixel positions within the matching window, respectively. It is a matching window.

[0011] Furthermore, the formula for calculating the three-dimensional displacement vector in S55 is: ; in, These are the pixel coordinates of the deformed image. , , This represents the three-dimensional shape corresponding to the pixel coordinates. , , This represents the three-dimensional displacement corresponding to the pixel coordinates.

[0012] Furthermore, the formula for calculating the Green strain tensor in S56 is: ; in, , , These represent the normal strain components in the x, y, and z axial directions, respectively. , , These are the shear strain components in the x, y, and z axial directions, respectively; This indicates the partial derivative sign.

[0013] In another aspect, the present invention provides a component multi-dimensional information measurement system based on multi-source information fusion, including a measurement system that performs multi-dimensional information measurement according to the above-described component multi-dimensional information measurement method.

[0014] The beneficial effects of this invention are: 1. This invention utilizes discrete wavelet transform to fuse fluorescent speckle images and texture images, fully combining the contrast information in the high-frequency features of the fluorescent speckle image with the structural information in the low-frequency features of the texture image. This effectively overcomes the interference of environmental factors on displacement and strain measurements, improving measurement accuracy and robustness.

[0015] 2. This invention combines fringe projection profilometry, zero-mean normalized squared difference correlation algorithm, and differential chain rule to calculate Green's strain tensor (including normal strain components and shear strain components in the x, y, and z axial directions), realizing the simultaneous measurement of three-dimensional morphology, displacement, and strain of complex components, and significantly improving the robustness and accuracy of multi-dimensional information measurement. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the generation process of the fused image in this invention; Figure 2 This is a flowchart of the multi-dimensional information measurement process in this invention. Detailed Implementation

[0017] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0018] Reference Figure 1 and Figure 2 This application provides a method for measuring multi-dimensional information of components based on multi-source information fusion, which is suitable for the simultaneous high-precision measurement of surface morphology, displacement and strain of complex components; specifically, it includes the following steps: S1. Obtain the fluorescence speckle image and texture image before deformation, and decompose both images into low-frequency features and high-frequency features through discrete wavelet transform; S2. Local variance dynamic weight fusion is applied to the low-frequency features of the fluorescence speckle image and the texture image before deformation; local energy weight fusion is applied to the high-frequency features of the fluorescence speckle image and the texture image before deformation; thus, the fused low-frequency and high-frequency features before deformation are obtained. S3. Reconstruct the fused low-frequency and high-frequency features before deformation using inverse wavelet transform to obtain the fused image before deformation. S4. Obtain the deformed fused image using the same methods as S1 to S3; S5. The three-dimensional shape of the component is obtained by using the fringe projection profilometry (FPP); the subpixel level displacement field is calculated, and the three-dimensional displacement field is constructed based on the three-dimensional shape and the subpixel level displacement field; the partial derivatives of the three-dimensional displacement field with respect to the pixel coordinates are extracted, and the Green strain tensor is calculated based on the differential chain rule using the partial derivatives. The three-dimensional shape, the three-dimensional displacement field, and the Green strain tensor constitute the multi-dimensional information of the component.

[0019] This invention utilizes discrete wavelet transform to fuse fluorescent speckle images and texture images, fully combining the contrast information in the high-frequency features of the fluorescent speckle image with the structural information in the low-frequency features of the texture image. This effectively overcomes the interference of environmental factors on displacement and strain measurements, improving measurement accuracy and robustness.

[0020] In some embodiments, refer to Figure 1 S1 specifically includes the following steps: S11. A colorless, odorless, and non-toxic fluorescent material is uniformly sprayed onto the surface of the component before deformation, utilizing the material's self-hidden properties to overcome the problem of large reflectivity differences in traditional speckle patterns; and fluorescent speckle images before deformation are acquired using a camera. Texture images ; These are the pixel coordinates of the image before deformation; S12. Using Haar wavelet basis, extract the fluorescence speckle image before deformation. Discrete wavelet transform is used to decompose the fluorescent speckle image into low-frequency features and high-frequency features in the horizontal, vertical, and diagonal directions, thereby achieving information decomposition of multi-source images (i.e., fluorescent speckle images and texture images). Specifically: ; Where DWT represents Discrete Wavelet Transform; S13, Transfer the texture image before deformation. The texture image is decomposed into low-frequency features and three high-frequency features in the horizontal, vertical, and diagonal directions through discrete wavelet transform. Specifically: .

[0021] In some embodiments, refer to Figure 1 S2 specifically includes the following steps: S21. The low-frequency features of the fluorescence speckle image and the texture image before deformation are fused using local variance dynamic weighting to obtain the fused low-frequency features. The specific calculation formula is as follows: ; in, This indicates the low-frequency characteristics after fusion; , These represent the low-frequency features of the fluorescence speckle image and the texture image before deformation, respectively; , These represent the weights corresponding to the low-frequency features of the fluorescence speckle image and the texture image before deformation, respectively. S22. The high-frequency features of the fluorescence speckle image and the texture image before deformation are fused using local energy weighting to obtain three fused high-frequency features in the horizontal, vertical, and diagonal directions. The specific calculation formula is as follows: ; ; ; in, , , These are the three high-frequency features after fusion in the horizontal, vertical, and diagonal directions; , , The fluorescence speckle image before deformation features three high-frequency characteristics in the horizontal, vertical, and diagonal directions; , , These are the three high-frequency features of the texture image before deformation in the horizontal, vertical, and diagonal directions; , , These are the weights corresponding to three high-frequency features in the horizontal, vertical, and diagonal directions of the fluorescence speckle image before deformation; , , These are the weights corresponding to three high-frequency features in the horizontal, vertical, and diagonal directions of the texture image before deformation.

[0022] In some embodiments, the weights in S21 and weight The specific calculation formula is as follows: ; ; In the formula, , These represent the local variances of the fluorescence speckle image and texture image before deformation within the specified window, respectively. Weights in S22 and weight The specific calculation formula is as follows: ; ; in, , These are the local energies of the fluorescence speckle image and texture image before deformation within a specified window, respectively.

[0023] In some embodiments, the calculation formula for the fused image before deformation in S3 is: ; in, This represents the fused image before deformation; IDWT represents the inverse discrete wavelet transform.

[0024] In some embodiments, refer to Figure 2 S5 specifically includes the following steps: S51. A measurement system is used and fringe projection profilometry (FPP) is employed to obtain the three-dimensional shape of the component; the measurement system includes a projection optical engine and a camera; S52. Divide the surface of the component into gridded units, and use the zero-mean normalized squared difference correlation algorithm to calculate the matching similarity of the fused images before and after deformation within each gridded unit; S53. Minimize the matching similarity of the fused images before and after deformation within each gridded unit, and filter out high-confidence matching points with similarity values ​​lower than the preset similarity threshold. S54. For high-confidence matching points, the initial displacement field distribution is obtained using a least-squares-based surface fitting algorithm, and the displacement gradient is iteratively optimized using the Gauss-Newton method to obtain the sub-pixel-level displacement field. ; S55. Utilizing the three-dimensional topography in S51, the sub-pixel displacement field The two-dimensional pixel displacements in the image are mapped to three-dimensional space, and the three-dimensional displacement vectors of all pixels in the image are calculated and fused after deformation to obtain a three-dimensional displacement field. S56. The partial derivatives of the three-dimensional displacement field with respect to the pixel coordinates are extracted by a differentiable convolution kernel, and the Green strain tensor is calculated based on the differential chain rule. The Green strain tensor includes the normal strain components and shear strain components in the three axial directions of x, y, and z. The three-dimensional morphology, the three-dimensional displacement field, and the Green strain tensor constitute the multi-dimensional information of the component.

[0025] In some embodiments, the formula for calculating the matching similarity in S52 is: ; in, This represents the currently calculated matching similarity. , These are the average gray values ​​of the local window frames within the fused images before and after deformation; , These are the standard deviations of the local window frames within the fused images before and after deformation; , These represent the local grayscale values ​​of the fused images before and after deformation, respectively. , These represent the horizontal and vertical offsets used to traverse all pixel positions within the matching window, respectively. It is a matching window.

[0026] In some embodiments, the formula for calculating the three-dimensional displacement vector in S55 is: ; in, These are the pixel coordinates of the deformed image. , , This represents the three-dimensional shape corresponding to the pixel coordinates. , , This represents the three-dimensional displacement corresponding to the pixel coordinates.

[0027] In some embodiments, the formula for calculating the Green strain tensor in S56 is: ; in, , , These represent the normal strain components in the x, y, and z axial directions, respectively. , , These are the shear strain components in the x, y, and z axial directions, respectively; This indicates the partial derivative sign.

[0028] This invention combines Fringe Projection Profilometry (FPP), zero-mean normalized squared difference correlation algorithm, and differential chain rule to calculate Green's strain tensor (including normal strain components and shear strain components in the x, y, and z axial directions), realizing the simultaneous measurement of three-dimensional morphology, displacement, and strain of complex components, and significantly improving the robustness and accuracy of multi-dimensional information measurement.

[0029] In another aspect, the present invention provides a component multi-dimensional information measurement system based on multi-source information fusion, including a measurement system that performs multi-dimensional information measurement according to the above-described component multi-dimensional information measurement method.

[0030] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for measuring multi-dimensional information of components based on multi-source information fusion, characterized in that, Includes the following steps: S1. Obtain the fluorescence speckle image and texture image before deformation, and decompose both images into low-frequency features and high-frequency features through discrete wavelet transform; S2. Local variance dynamic weight fusion is applied to the low-frequency features of the fluorescence speckle image and the texture image before deformation; local energy weight fusion is applied to the high-frequency features of the fluorescence speckle image and the texture image before deformation; thus, the fused low-frequency and high-frequency features before deformation are obtained. S3. Reconstruct the fused low-frequency and high-frequency features before deformation using inverse wavelet transform to obtain the fused image before deformation. S4. Obtain the deformed fused image using the same methods as S1 to S3; S5. The three-dimensional shape of the component is obtained by using the fringe projection profilometry (FPP); the subpixel level displacement field is calculated, and the three-dimensional displacement field is constructed based on the three-dimensional shape and the subpixel level displacement field; the partial derivatives of the three-dimensional displacement field with respect to the pixel coordinates are extracted, and the Green strain tensor is calculated based on the differential chain rule using the partial derivatives. The three-dimensional shape, the three-dimensional displacement field, and the Green strain tensor constitute the multi-dimensional information of the component.

2. The component multi-dimensional information measurement method based on multi-source information fusion according to claim 1, characterized in that, S1 specifically includes the following steps: S11. Uniformly spray fluorescent material onto the surface of the component before deformation, and acquire a fluorescent speckle image of the component before deformation using a camera. Texture images ; These are the pixel coordinates of the image before deformation; S12. Obtain the fluorescence speckle image before deformation. The fluorescence speckle image is decomposed into low-frequency features and high-frequency features in the horizontal, vertical and diagonal directions by discrete wavelet transform. S13, Transfer the texture image before deformation. The texture image is decomposed into low-frequency features and three high-frequency features in the horizontal, vertical and diagonal directions through discrete wavelet transform.

3. The method for measuring multi-dimensional information of components based on multi-source information fusion according to claim 2, characterized in that, S2 specifically includes the following steps: S21. The low-frequency features of the fluorescence speckle image and the texture image before deformation are fused using local variance dynamic weighting to obtain the fused low-frequency features. The specific calculation formula is as follows: ； in, This indicates the low-frequency characteristics after fusion; , These represent the low-frequency features of the fluorescence speckle image and the texture image before deformation, respectively; , These represent the weights corresponding to the low-frequency features of the fluorescence speckle image and the texture image before deformation, respectively. S22. The high-frequency features of the fluorescence speckle image and the texture image before deformation are fused using local energy weighting to obtain three fused high-frequency features in the horizontal, vertical, and diagonal directions. The specific calculation formula is as follows: ； ； ； in, , , These are the three high-frequency features after fusion in the horizontal, vertical, and diagonal directions; , , The fluorescence speckle image before deformation features three high-frequency characteristics in the horizontal, vertical, and diagonal directions; , , These are the three high-frequency features of the texture image before deformation in the horizontal, vertical, and diagonal directions; , , These are the weights corresponding to three high-frequency features in the horizontal, vertical, and diagonal directions of the fluorescence speckle image before deformation; , , These are the weights corresponding to three high-frequency features in the horizontal, vertical, and diagonal directions of the texture image before deformation.

4. The method for measuring multi-dimensional information of components based on multi-source information fusion according to claim 3, characterized in that, The weights in S21 and weight The specific calculation formula is as follows: ； ； in, , These represent the local variances of the fluorescence speckle image and texture image before deformation within the specified window, respectively. Weights in S22 and weight The specific calculation formula is as follows: ； ； in, , These are the local energies of the fluorescence speckle image and texture image before deformation within a specified window, respectively.

5. The method for measuring multi-dimensional information of components based on multi-source information fusion according to claim 4, characterized in that, The calculation formula for the fused image before deformation in S3 is: ； in, This represents the fused image before deformation. This represents the inverse discrete wavelet transform.

6. The method for measuring multi-dimensional information of components based on multi-source information fusion according to claim 5, characterized in that, S5 specifically includes the following steps: S51. A measurement system is used and the fringe projection profilometry (FPP) is employed to obtain the three-dimensional shape of the component; S52. Divide the surface of the component into gridded units, and use the zero-mean normalized squared difference correlation algorithm to calculate the matching similarity of the fused images before and after deformation within each gridded unit; S53. Minimize the matching similarity of the fused images before and after deformation within each gridded unit, and filter out high-confidence matching points with similarity values ​​lower than the preset similarity threshold. S54. For high-confidence matching points, the initial displacement field distribution is obtained using a least-squares-based surface fitting algorithm, and the displacement gradient is iteratively optimized using the Gauss-Newton method to obtain the sub-pixel-level displacement field. ; S55. Utilizing the three-dimensional topography in S51, the sub-pixel displacement field The two-dimensional pixel displacements in the image are mapped to three-dimensional space, and the three-dimensional displacement vectors of all pixels in the fused image after deformation are calculated to obtain the three-dimensional displacement field. S56. The partial derivatives of the three-dimensional displacement field with respect to the pixel coordinates are extracted by a differentiable convolution kernel, and the Green strain tensor is calculated based on the differential chain rule. The Green strain tensor includes the normal strain components and shear strain components in the three axial directions of x, y, and z. The three-dimensional morphology, the three-dimensional displacement field, and the Green strain tensor constitute the multi-dimensional information of the component.

7. The method for measuring multi-dimensional information of components based on multi-source information fusion according to claim 6, characterized in that, The formula for calculating the matching similarity in S52 is as follows: ； in, This represents the currently calculated matching similarity. , These are the average gray values ​​of the local window frames within the fused images before and after deformation; , These are the standard deviations of the local window frames within the fused images before and after deformation; , These represent the local grayscale values ​​of the fused images before and after deformation, respectively. , This represents the horizontal and vertical offsets used to iterate through all pixel positions within the matching window. It is a matching window.

8. The method for measuring multi-dimensional information of components based on multi-source information fusion according to claim 7, characterized in that, The formula for calculating the three-dimensional displacement vector in S55 is: ； in, These are the pixel coordinates of the deformed image. , , This represents the three-dimensional shape corresponding to the pixel coordinates. , , This represents the three-dimensional displacement corresponding to the pixel coordinates.

9. The method for measuring multi-dimensional information of components based on multi-source information fusion according to claim 8, characterized in that, The formula for calculating the Green strain tensor in S56 is: ； in, , , These represent the normal strain components in the x, y, and z axial directions, respectively. , , These are the shear strain components in the x, y, and z axial directions, respectively; This indicates the partial derivative sign.

10. A component multi-dimensional information measurement system based on multi-source information fusion, characterized in that, It includes a measurement system that performs multi-dimensional information measurement according to the component multi-dimensional information measurement method according to any one of claims 1 to 9.

Images