Structured light fringe enhancement method based on neural network and Hessian matrix

Abstract

This invention discloses a structured light fringe enhancement method based on a neural network and a Hessian matrix. First, an initial window is constructed on the input image. The grayscale values of the pixels within the window are then input into a trained neural network to obtain enhancement parameters. The window is then moved across the entire image to construct a table of enhancement parameter correspondences. Finally, Hessian enhancement is performed on the image based on the table, and an enhanced image is output. This structured light fringe enhancement method, based on a neural network and a Hessian matrix, achieves the same enhancement effect across different contrast regions of structured light fringe under low illumination conditions, serving as a preprocessing method for subsequent image processing.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision images, and in particular relates to a structured light stripe enhancement method based on a neural network and a Hessian matrix. Background Art

[0002] Due to the influence of various factors such as imaging equipment, complex background noise, and uneven illumination intensity, the captured images inevitably exhibit blur and low contrast. This lack of image detail complicates subsequent processing of structural stripes. To more accurately extract the center information of stripes in structured light images, it is necessary to enhance the stripe structure and suppress non-striped structures before image segmentation. Image enhancement is the process of processing images using a series of techniques to highlight information meaningful to humans or machines and suppress useless information, making the image more suitable for specific applications than the original image.

[0003] The Hessian matrix is widely used for image enhancement because its eigenvalues and eigenvectors form an ellipse that reflects image anisotropy. However, due to environmental factors, the contrast of the fringe patterns projected by structured light can be inconsistent. Therefore, Hessian enhancement based on global grayscale distribution characteristics suppresses fringe information in low-contrast areas.

[0004] Neural networks have great advantages in processing image tasks. They learn corresponding features from samples and avoid the complex feature extraction process. They have been applied to various computer vision tasks and have achieved great success. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem of inconsistent enhancement effect of Hessian enhancement in low-contrast areas under the existing technology through neural network training, and to provide a structured light fringe enhancement method based on neural network and Hessian matrix.

[0006] To achieve the above object, the present invention adopts a structured light fringe enhancement method based on a neural network and a Hessian matrix, comprising the following steps:

[0007] S1, construct an initial window on the input image;

[0008] S2. Preprocessing of pixels within the initial window

[0009] S2-1, counting the grayscale values of all pixels in the initial window;

[0010] S2-2, the obtained pixel grayscale value is used as input to the trained neural network to obtain the enhancement parameters;

[0011] S2-3, the initial window automatically shifts along the row direction to traverse the entire image;

[0012] S2-4, statistically obtaining the enhancement parameters and constructing an enhancement parameter correspondence table;

[0013] S3. Perform Hessian enhancement on the image according to the enhancement parameter correspondence table and output the enhanced image.

[0014] Preferably, the initial window is a square with a side length of 5, and the pixels in the viewing window are initial pixels.

[0015] Preferably, the neural network model includes a positional encoder, an encoder and a fully connected neural network, and the encoder includes a multi-head attention mechanism and a feedforward neural network.

[0016] Preferably, the training method of the neural network model is: the grayscale value is used as input data in the neural network model, and is processed by position encoding, multi-head attention mechanism and feedforward neural network, and then the enhancement parameters are predicted by a fully connected neural network.

[0017] Preferably, the position code calculation formula is as follows:

[0018]

[0019]

[0020] Where pos represents the position, i represents the i-th dimension, and d model Represents the dimension of the feature vector.

[0021] Preferably, the encoder comprises two parts: a multi-head self-attention mechanism and a feedforward neural network. The calculation formula of the self-attention mechanism is as follows:

[0022]

[0023] Q=XW Q

[0024] K=XW K

[0025] V=XW V

[0026] Among them, X represents the input feature matrix, Q is the query matrix, K is the key matrix, and V is the value matrix, which are respectively composed of the input feature matrix X and the corresponding weight matrix W Q 、W K 、W V By multiplying them together, the weight matrix parameters can be obtained through model training and learning.

[0027] Preferably, step S3 includes the following sub-steps:

[0028] S3-1. Calculate the Hessian matrix of the input image and obtain the eigenvalue;

[0029] S3-2, constructing the stripe structure similarity function from the eigenvalues;

[0030] S3-3, according to the enhancement parameter correspondence table, substituting the characteristic value and the enhancement parameter into the stripe structure similarity function to obtain a response;

[0031] S3-4. Enhance the image according to the response.

[0032] Preferably, the Hessian matrix is H(x,y), and the Hessian matrix H(x,y) is defined as follows:

[0033]

[0034] Wherein, f(x,y) represents the input image, g(x,y) represents a two-dimensional Gaussian function, and g(x,y) is shown as follows:

[0035]

[0036] Among them, I xx is the second-order partial differential of the input image f(x,y) in the x direction, I yy is the second-order partial differential of the input image f(x,y) in the y direction, I xy =I yx is the mixed partial differential of the input image f(x,y) in the x-direction and the y-direction.

[0037] Preferably, the two eigenvalues of the Hessian matrix are λ1 and λ2, and the specific calculation method is:

[0038]

[0039] Preferably, the stripe structure similarity function is defined as follows:

[0040] other

[0041] in, Used to distinguish structural shapes, β is used to adjust the sensitivity of distinguishing block areas from strip areas, and c affects the smoothness of the filtered image. β is set to 0.5, and c is set according to the enhancement parameter correspondence table.

[0042] The present invention has the following characteristics and beneficial effects

[0043] The structured light stripe enhancement method based on neural network and Hessian matrix in the present invention can extract complete and coherent structured light stripes with rich details, thereby improving the inconsistent enhancement effect of Hessian matrix in low-contrast areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of a structured light fringe enhancement method based on a neural network and a Hessian matrix according to an embodiment of the present invention.

[0045] Figure 2 This is a neural network architecture diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] On the contrary, the present invention covers any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention as defined by the claims. Furthermore, to facilitate a better understanding of the present invention, certain specific details are described in detail below in the detailed description of the present invention. Those skilled in the art will be able to fully understand the present invention without these details.

[0048] Figure 1 The embodiment of the present invention provides a structured light stripe enhancement method based on a neural network and a Hessian matrix. Figure 1 As shown, the following steps are included:

[0049] S1. Construct an initial window on the input image. The initial window is a square with a side length of 5. The pixels in the window are considered as initial pixels. Count the grayscale values of all pixels in the 5×5 initial window.

[0050] S2. Preprocessing of pixels within the initial window

[0051] S2-1, counting the grayscale values of all pixels in the initial window;

[0052] S2-2, the obtained pixel grayscale value is used as input to the trained neural network to obtain the enhancement parameter;

[0053] S2-3, the initial window automatically shifts along the row direction, traverses the entire image, and after the neural network calculates the enhancement parameters of the central pixel of the initial window, the initial window automatically shifts along the row direction, traverses the entire image, and calculates the enhancement parameters at each pixel;

[0054] S2-4, statistically obtaining the enhancement parameters and constructing an enhancement parameter correspondence table;

[0055] S3. Perform Hessian enhancement on the image according to the enhancement parameter correspondence table and output the enhanced image.

[0056] The present invention is further configured as follows: Figure 2 As shown, the neural network model includes a position encoder, an encoder and a fully connected neural network, and the encoder includes a multi-head attention mechanism and a feedforward neural network.

[0057] Specifically, the training method of the neural network model is as follows: the grayscale value is used as input data in the neural network model, and is processed by position encoding, multi-head attention mechanism and feedforward neural network, and then the enhancement parameters are predicted by a fully connected neural network.

[0058] Specifically, this neural network model is an improved model based on Transformer.

[0059] It should be noted that a feedforward neural network is the simplest type of neural network. Neurons are arranged in layers, and each neuron is connected only to neurons in the previous layer. The layers receive the output of the previous layer and output it to the next layer, with no feedback between layers. Therefore, in this embodiment, a detailed explanation and description of feedforward neural networks is not provided.

[0060] Similarly, a fully connected neural network, also known as a multilayer perceptron, is used to find the most reasonable and robust hyperplane between categories. The most representative perceptron is the support vector machine algorithm. This is a conventional technical method and is not specifically explained or illustrated in this embodiment.

[0061] Specifically, the position code calculation formula is as follows:

[0062]

[0063]

[0064] Where pos represents the position, i represents the i-th dimension, and d model Represents the dimension of the feature vector.

[0065] Furthermore, the calculation formula of the multi-head attention mechanism is as follows:

[0066]

[0067] Q=XW Q

[0068] K=XW K

[0069] V=XW V

[0070] Among them, X represents the input feature matrix, Q is the query matrix, K is the key matrix, and V is the value matrix, which are respectively composed of the input feature matrix X and the corresponding weight matrix W Q 、W K 、W V By multiplying them together, the weight matrix parameters can be obtained through model training and learning.

[0071] A further configuration of the present invention is that in the neural network model, layer regularization is provided between the multi-head attention mechanism and the feedforward neural network, and between the feedforward neural network and the fully connected neural network.

[0072] As you can imagine, layer regularization can prevent overfitting and improve the generalization ability of the model. The multi-head attention mechanism can enable the model to better extract the grayscale value features of the pixel points and the relationship between the positions of different pixel points, thereby improving the accuracy of the enhancement parameter prediction.

[0073] The last fully connected layer outputs the predicted value of the enhanced parameter. Under the approximation of the MSE loss function, the predicted value of the enhanced parameter gradually approaches the true value.

[0074] According to a further configuration of the present invention, step S3 includes the following sub-steps:

[0075] S3-1. Calculate the Hessian matrix of the input image and obtain the eigenvalue;

[0076] Specifically, the Hessian matrix is H(x, y), and the Hessian matrix H(x, y) is defined as follows:

[0077]

[0078] Among them, f(x,y) represents the input image, g(x,y) represents the two-dimensional Gaussian function, I xx is the second-order partial differential of the input image f(x,y) in the x direction, I yy is the second-order partial differential of the input image f(x,y) in the y direction, I xy =I yx is the mixed partial differential of the input image f(x,y) in the x-direction and the y-direction.

[0079] Where g(x,y) is as follows:

[0080]

[0081] The two eigenvalues of the Hessian matrix are λ1 and λ2, and the specific calculation method is:

[0082]

[0083] S3-2, constructing the stripe structure similarity function from the eigenvalues;

[0084] Among them, the stripe structure similarity function is defined as follows:

[0085] other

[0086] in, Used to distinguish structural shapes,

[0087] β is used to adjust the sensitivity of distinguishing blocky and stripy areas, and c affects the smoothness of the filtered image. β is set to 0.5, and c is set according to the enhancement parameter table.

[0088] That is, when the pixel's local structure is a striped structure, Rb = 0, so P0 is close to 1. When the pixel is noise, the eigenvalues λ1 and λ2 are very small, and thus P0 is also very small. Therefore, P0 takes the value of (0, 1). Only when the pixel's local structure is a striped structure is P0 maximum. When it is other local structures or noise, P0 is close to 0, thus achieving the purpose of stripe enhancement.

[0089] S3-3, according to the enhancement parameter correspondence table, substituting the characteristic value and the enhancement parameter into the stripe structure similarity function to obtain a response;

[0090] S3-4. Enhance the image according to the response.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A structured light fringe enhancement method based on neural network and Hessian matrix, characterized in that: The following steps are involved: S1, construct an initial window on the input image; S2. Preprocessing of pixels within the initial window S2-1, counting the grayscale values of all pixels in the initial window; S2-2, the obtained pixel grayscale value is used as input to the trained neural network to obtain the enhancement parameter; S2-3, the initial window automatically shifts along the row direction to traverse the entire image; S2-4, statistically obtaining the enhancement parameters and constructing an enhancement parameter correspondence table; S3, perform Hessian enhancement on the image according to the enhancement parameter correspondence table, and output the enhanced image. S3-1. Calculate the Hessian matrix of the input image and obtain the eigenvalue; The Hessian matrix is H(x,y), and the Hessian matrix H(x,y) is defined as follows: Wherein, f(x,y) represents the input image, g(x,y) represents a two-dimensional Gaussian function, and g(x,y) is shown as follows: Among them, I xx is the second-order partial differential of the input image f(x,y) in the x direction, I yy is the second-order partial differential of the input image f(x,y) in the y direction, I xy =I yx is the mixed partial differential of the input image f(x,y) in the x-direction and y-direction; The two eigenvalues of the Hessian matrix are λ1 and λ2, and the specific calculation method is: S3-2, constructing the stripe structure similarity function from the eigenvalues; The stripe structure similarity function is defined as follows: in, Used to distinguish structural shapes, β is used to adjust the sensitivity of distinguishing block areas from strip areas, and c affects the smoothness of the filtered image. β is set to 0.5, and c is set according to the enhancement parameter correspondence table; S3-3, according to the enhancement parameter correspondence table, substituting the characteristic value and the enhancement parameter into the stripe structure similarity function to obtain a response; S3-4. Enhance the image according to the response.

2. The structured light fringe enhancement method based on a neural network and a Hessian matrix according to claim 1, characterized in that: The initial window is a square with a side length of 5, and the pixels in the viewing window are initial pixels.

3. The structured light fringe enhancement method based on a neural network and a Hessian matrix according to claim 1, characterized in that: The neural network model includes a position encoder, an encoder and a fully connected neural network, and the encoder includes a multi-head attention mechanism and a feedforward neural network.

4. The structured light fringe enhancement method based on a neural network and a Hessian matrix according to claim 3, characterized in that: The training method of the neural network model is as follows: the grayscale value is used as input data in the neural network model, and then processed by position encoding, multi-head attention mechanism and feedforward neural network, and then the enhancement parameters are predicted by a fully connected neural network.

5. The structured light fringe enhancement method based on neural network and Hessian matrix according to claim 3, characterized in that: The position coding calculation formula is as follows: Where pos represents the position, i represents the i-th dimension, and d model Represents the dimension of the feature vector.

6. The structured light fringe enhancement method based on neural network and Hessian matrix according to claim 4, characterized in that: The calculation formula of the multi-head attention mechanism is as follows: Q=XW Q K=XW K V=XW V Among them, X represents the input feature matrix, Q is the query matrix, K is the key matrix, and V is the value matrix, which are respectively composed of the input feature matrix X and the corresponding weight matrix W Q 、W K 、W V By multiplying them together, the weight matrix parameters can be obtained through model training and learning.

Images