Method for automatic recognition of measurement error data of a structured light system based on neural networks

Abstract

This invention relates to the fields of deep learning, optical measurement, and computer vision, and particularly to an automatic identification method for measurement error data of a structured light system based on a neural network. The method includes the following steps: S1, capturing images of structured light stripes during the three-dimensional measurement process to obtain the structured light stripe cross-section; S2, converting the structured light stripe cross-section into a grayscale distribution image as a dataset, and dividing the dataset into two types: defective and non-defective; S3, training a 50-layer residual neural network to obtain a neural network model for structured light stripe quality detection; S4, designing a program for structured light stripe quality detection based on the model trained in S3; S5, directly inputting the measured stripe cross-section into the prediction program to obtain a structured light stripe quality evaluation. The method utilizes a residual neural network to train a structured light stripe defect detection model, which, along with the prediction algorithm, directly detects defects in the input structured light stripe cross-section. The process is convenient, fast, and highly accurate.

Description

Technical Field

[0001] This invention relates to the fields of deep learning, optical measurement, and computer vision, and in particular to an automatic identification method for measurement error data of structured light systems based on neural networks. Background Technology

[0002] The foundation of structured light 3D measurement is the principle of triangulation. Structured light methods use a light source to project a structured light pattern onto the object being measured. The system typically consists of a camera, a light source, and the object. Based on the different projected patterns, it can be divided into three types: point structured light, line structured light, and surface structured light. All three types of structured light operate on the same principle: the light source projects a pattern containing specific modulation information onto the surface of the object. The pattern deforms due to the modulation of the object's surface shape. After the camera captures the deformed pattern, the 3D information of the object is calculated using the principle of triangulation.

[0003] In structured light systems, the accuracy of fringe localization is one of the decisive factors in measurement accuracy. Since 3D information is derived from modulated fringes, the position of the fringes in the image is crucial to the accuracy of structured light 3D measurements. Most fringe center localization methods are based on the grayscale distribution of the fringes. However, many factors, such as image noise, the color and shape of the object being measured, projection angle, and shooting angle, can degrade the grayscale distribution of the fringes, thereby reducing the accuracy of fringe center extraction. These factors lead to low signal-to-noise ratios and asymmetrical distributions of fringe grayscale, thus reducing the accuracy of fringe localization and introducing significant error data into the measurement results. Therefore, a method is needed to quickly identify and detect defective structured light fringes with severely degraded grayscale distributions, providing a basis for improving subsequent measurement accuracy.

[0004] Current structured light stripe defect detection technologies based on image processing are complex to operate and not fast enough. The setting of thresholds relies heavily on the user's experience, which leads to significant inaccuracies in practical use. Summary of the Invention

[0005] The purpose of this invention is to overcome the problems of complex operation and reliance on user experience in existing image processing-based structured light stripe defect detection technologies. Instead, it provides an automatic identification method for measurement error data of structured light systems based on neural networks. This method uses a model trained on a residual neural network to directly predict the cross-section of the input structured light stripe and returns a quality evaluation of the structured light stripe, providing a basis for noise elimination and stripe center extraction in the next step of structured light three-dimensional measurement.

[0006] The objective of this invention is achieved through the following measures: an automatic identification method for measurement error data of a structured light system based on a neural network, comprising the following steps:

[0007] S1. Capture structured light stripe images during the structured light 3D measurement process to obtain the structured light stripe cross section;

[0008] S2. Convert the structured light stripe cross-section into a grayscale distribution image as a dataset, and divide the dataset into two types: defective and non-defective.

[0009] S3. A neural network model for structured light stripe quality detection is obtained by training a fifty-layer residual neural network.

[0010] S4. Based on the model trained in S3, design a program for structured light stripe quality detection;

[0011] S5. Directly input the cross section of the stripe to be measured into the prediction program to obtain the quality evaluation of the structured light stripe.

[0012] Preferably, in step S1, the measurement basis of the structured light method is the principle of triangulation. The structured light method uses a light source to project a structured light pattern onto the object to be measured. The system consists of a camera, a light source, and the object to be measured. According to the different patterns projected, it is divided into three types: point structured light, line structured light, and surface structured light. The principle of all three types of structured light is that the light source projects a pattern containing specific modulation information onto the surface of the object to be measured. The pattern is deformed by the shape modulation of the surface of the object to be measured. After the camera captures the deformed pattern, the three-dimensional information of the object is calculated according to the principle of triangulation.

[0013] Preferably, in step S2, the grayscale distribution characteristics of the cross-section of the stripe without defects in the dataset require that all intermediate points must fall on the center line of the stripe and be symmetrically distributed. Stripes that meet the above conditions are defined as defect-free stripes; otherwise, the stripes have defects. 6450 grayscale distribution images of the stripe cross-section are selected as the dataset, in which the ratio of defective stripes to defect-free stripes is 2:1.

[0014] Preferably, in step S3, the residual neural network used is composed of a series of residual blocks (ResNet blocks), and a residual block (ResNet block) can be represented as:

[0015] x l+1 =f(x) l +F(x l W l ))

[0016] Where x1 and x l+1 F represents the input and output of the residual block, f is the ReLU activation function, and W represents all the weights in the residual block.

[0017] The features learned from shallow layer 1 to deep layer L are represented as follows:

[0018]

[0019] That is, for any unit L of arbitrary depth, the characteristic x L This can be expressed as a feature x1 of shallow unit 1 plus a form like The residual function, where F represents the residual function, x i W represents the input, and W represents all the weights within the residual block.

[0020] There are two types of residual blocks: one is a two-layer structure for 34-layer residual neural networks, and the other is a three-layer structure for 50 / 101 / 152-layer residual neural networks, which replaces two 3*3 convolutional layers with 1*1+3*3+1*1. The ResNet network is based on the VGG19 (VGG refers to Visual Geometry Group) network, with modifications, and residual units are added through a short-circuit mechanism. The main changes are that ResNet (residual neural network) directly uses convolutions with stride=2 for downsampling and replaces fully connected layers with GAP (global average pooling) layers. Wherein: the input image size is (224, 224, 3); after the first 7x7 convolutional layer, the output channel = 64, stride = 2, pad = 3; after a 3x3 max pooling layer, stride = 2, pad = 1; except for the 3x3 max pooling layer, all other downsampling is implemented using convolutional layers; we call conv2_x, conv3_x, conv4_x, and conv5_x four convolutional groups. The downsampling of conv2_x is implemented by the max pooling layer, and the downsampling of the other three convolutional groups is implemented by the residual blocks adjacent to the previous convolutional group.

[0021] Preferably, in step S3, the training uses a 50-layer residual neural network. The system and hardware use a Windows 10 operating system, and the CPU model is an 11th Gen Intel(R) Core(TM) i5-11300H@3.10GHz 3.11GHz. The software and dependent environment are as follows: the training framework is ResNet50, and the training neural network software and development tools include Python 3.6, Keras 2.2.0, TensorFlow 1.9.0, and NumPyL. 13.3. The training uses the Adam optimizer, which has the advantages of simple implementation, high computational efficiency, and low memory requirements. The default settings are an initial learning rate of 10, an exponential decay rate of 0.9, a batch size of 20, and an epoch of 60. The training stops when the loss no longer decreases.

[0022] Preferably, in step S4, the flow of the structured light stripe defect detection program is as follows: first, input a cross-sectional image of the structured light stripe; convert the cross-sectional image of the structured light stripe into a grayscale distribution image; change the size of the grayscale distribution image to meet the optimal input format of the trained neural network; input the distribution image into the prediction model to obtain a determination of whether the stripe quality is defective, providing a basis for improving the accuracy of subsequent measurements.

[0023] The beneficial effects of this invention are as follows: This invention overcomes the problems of existing image processing-based structured light stripe defect detection technologies, which are complex to operate, not fast enough, and rely heavily on user experience for threshold setting, leading to significant inaccuracies in practical use. By leveraging the end-to-end and convenient characteristics of deep learning, a residual neural network is used to train a structured light stripe defect detection model, and this model and prediction algorithm are used to directly detect defects in the input structured light stripe cross-section. This process is convenient, fast, and has high accuracy. Attached Figure Description

[0024] Figure 1 This is a flowchart of a method for automatic identification of measurement error data in a structured light system based on neural networks;

[0025] Figure 2 A schematic diagram of a structured light 3D measurement method for an automatic identification of measurement error data in a structured light system based on a neural network;

[0026] Figure 3 This is a schematic diagram of a dataset sample for an automatic identification method of measurement error data in a structured light system based on neural networks.

[0027] Figure 4 This is a schematic diagram of the ResNetblock, a method for automatic identification of measurement error data in structured light systems based on neural networks.

[0028] Figure 5 This is a schematic diagram of the residual neural network structure for an automatic identification method of measurement error data in a structured light system based on a neural network. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Example 1: As Figures 1-5 As shown, an automatic identification method for measurement error data of a structured light system based on a neural network includes the following steps:

[0031] S1. Capture structured light stripe images during the structured light 3D measurement process to obtain the structured light stripe cross section;

[0032] S2. Convert the structured light stripe cross-section into a grayscale distribution image as a dataset, and divide the dataset into two types: defective and non-defective.

[0033] S3. A neural network model for structured light stripe quality detection is obtained by training a fifty-layer residual neural network.

[0034] S4. Based on the model trained in S3, design a program for structured light stripe quality detection;

[0035] S5. Directly input the cross section of the stripe to be measured into the prediction program to obtain the quality evaluation of the structured light stripe.

[0036] In step S1, the measurement basis of the structured light method is the principle of triangulation. The structured light method uses a light source to project a structured light pattern onto the object to be measured. The system consists of a camera, a light source, and the object to be measured. According to the different patterns projected, there are three types: point structured light, line structured light, and surface structured light. The principle of all three types of structured light is that the light source projects a pattern containing specific modulation information onto the surface of the object to be measured. The pattern is deformed by the shape modulation of the surface of the object to be measured. After the camera captures the deformed pattern, the three-dimensional information of the object is calculated according to the principle of triangulation.

[0037] In step S2, the grayscale distribution characteristics of the cross-section of the stripe without defects in the dataset require that all midpoints must fall on the center line of the stripe and be symmetrically distributed. Stripes that meet the above conditions are defined as defect-free stripes; otherwise, the stripes have defects. A dataset of 6450 grayscale distribution images of stripe cross-sections is selected, with a ratio of defective to defect-free stripes of 2:1. The dataset samples, i.e., the grayscale distribution images of the stripe cross-sections, are shown below. Figure 3 .

[0038] In step S3, the residual neural network used consists of a series of residual blocks (ResNet blocks are listed in the appendix). Figure 4 A residual block in a ResNet can be represented as:

[0039] x l+1 =f(x) l +F(x l W l ))

[0040] Where x1 and x l+1 F represents the input and output of the residual block, f represents the residual function, f is the ReLU activation function, and W represents all the weights within the residual block.

[0041] The features learned from the shallow layer l to the deep layer L are represented as follows:

[0042]

[0043] That is, for any unit L of arbitrary depth, the characteristic x L This can be expressed as a feature x1 of shallow unit 1 plus a form like The residual function, where F represents the residual function, x i represents the input, and W represents all the weights within the residual block.

[0044] There are two types of residual blocks: one is a two-layer structure for 34-layer residual neural networks, and the other is a three-layer structure for 50 / 101 / 152-layer residual neural networks, which replaces two 3*3 convolutional layers with 1*1+3*3+1*1. The ResNet network is based on the VGG19 (VGG refers to Visual Geometry Group) network, with modifications, and residual units are added through a short-circuit mechanism. The main changes are that ResNet (residual neural network) directly uses convolutions with stride=2 for downsampling, and replaces fully connected layers with GAP (global average pooling) layers. Wherein: the input image size is (224, 224, 3); after the first 7x7 convolutional layer, the output channel = 64, stride = 2, pad = 3; after a 3x3 max pooling layer, stride = 2, pad = 1; except for the 3x3 max pooling layer, all other downsampling is implemented using convolutional layers; we call conv2_x, conv3_x, conv4_x, and conv5_x four convolutional groups. The downsampling of conv2_x is implemented by the max pooling layer, and the downsampling of the other three convolutional groups is implemented by the residual blocks adjacent to the previous convolutional group.

[0045] In step S3, a 50-layer residual neural network is used for training. The system and hardware are based on Windows 10 operating system, with a CPU model of 11th Gen Intel(R) Core(TM) i5-11300H @ 3.10GHz 3.11GHz. The software and dependencies include the ResNet50 training framework, and software and development tools such as Python 3.6, Keras 2.2.0, TensorFlow 1.9.0, and NumPy 1.13.3. The Adam optimizer is used for training, as it is simple to implement, computationally efficient, and requires little memory. The default initial learning rate is set to 10. -3 The exponential decay rate is 0.9, the batch size is set to 20, the epoch is set to 60, and training in the current stage stops when the loss no longer decreases.

[0046] In step S4, the procedure for detecting defects in structured light stripes is as follows: First, input a cross-sectional image of the structured light stripes; convert the cross-sectional image of the structured light stripes into a grayscale distribution image; change the size of the grayscale distribution image to meet the optimal input format of the trained neural network; input the distribution image into the prediction model to obtain a determination of whether the stripe quality is defective, providing a basis for improving the accuracy of subsequent measurements.

[0047] Deep learning is an end-to-end learning method; data is input and results are obtained immediately, making it convenient and fast. Combining deep learning technology with structured light stripe defect detection technology leverages the characteristics of deep learning algorithms to avoid discussions about threshold settings, making detection even more convenient and efficient.

[0048] Deep learning is a type of machine learning in artificial intelligence. It emerged primarily to solve tasks that are difficult for artificial intelligence to formally describe. For these problems, deep learning allows computers to learn from experience and understand the world based on a hierarchical conceptual framework. Through multiple layers of "learning," computers can express complex and abstract representations using simple and visual representations, solving the core problem of representation learning. Today, the research results of deep learning have been successfully applied in fields such as speech recognition, pattern recognition, object recognition, natural language programming, and artificial intelligence.

[0049] Residual Neural Networks (ResNet) are created by adding a concatenation mechanism to a regular convolutional model, resulting in a residual convolutional neural network model. This effectively reduces the training difficulty of the network parameters without causing a decrease in accuracy. The residual block of the model (ResNet block) is shown in the attached image. Figure 4 As shown. These residual block units are connected in series to form residual modules, and multiple residual modules constitute the smallest unit of the residual neural network model. The residual neural network model has slightly different unit structures depending on the number of network layers, with four model structures: 18 layers, 34 layers, 50 layers, and 101 layers. See the appendix for specific network structure configurations. Figure 5 .

[0050] In S5, the cross-section of the fringe to be measured is directly input into the prediction program to obtain whether there are defects in the structured light fringes at the cross-section. A table showing the measured fringe defect quality inspection is shown in Table 1.

[0051] Table 1. Experimental Results of Stripe Defect Quality Detection

[0052]

[0053]

[0054] The measurement error refers to the relative distance between each point in the structured light 3D point cloud and the corresponding point in the actual measured object. The calculation formula is as follows:

[0055] Error = |d-d'|

[0056] Where d-d' represents the relative distance between a point in the 3D point cloud and the corresponding point on the actual measured object.

[0057] The proposed automatic identification method for large error data in structured light system measurements based on residual neural networks, along with existing image processing methods, were used to optimize the structured light 3D point cloud. The comparison of the effects before and after processing is shown in Table 2.

[0058] Table 2. Experimental results of optimization using the two methods respectively.

[0059]

[0060] As can be seen, compared with existing methods, the automatic identification technology for large error data in structured light system measurements based on residual neural networks proposed in this study has the following improvements: First, the method proposed in this study can directly obtain the detection results by inputting the structured light fringe cross-section, avoiding discussions on thresholds, and is simple and convenient to operate; in addition, the average measurement error after removing defective fringe data using the method proposed in this study is lower, further improving the detection accuracy. Therefore, it can be considered that the method proposed in this paper can quickly and accurately detect defective fringes in structured light three-dimensional measurements.

[0061] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An automatic identification method for measurement error data of a structured light system based on a neural network, characterized in that: Includes the following steps: S1. Capture structured light stripe images during the structured light 3D measurement process to obtain the structured light stripe cross section; S2. Convert the structured light stripe cross-section into a grayscale distribution image as a dataset, and divide the dataset into two types: defective and non-defective. S3. A neural network model for structured light stripe quality detection is obtained by training a fifty-layer residual neural network. S4. Based on the model trained in S3, design a program for structured light stripe quality detection; S5. Directly input the cross section of the stripe to be measured into the prediction program to obtain the quality evaluation of the structured light stripe; In step S1, the measurement basis of the structured light method is the principle of triangulation. The structured light method uses a light source to project a structured light pattern onto the object to be measured. The system consists of a camera, a light source, and the object to be measured. According to the different patterns projected, there are three types: point structured light, line structured light, and surface structured light. The principle of all three types of structured light is that the light source projects a pattern containing specific modulation information onto the surface of the object to be measured. The pattern is deformed by the shape modulation of the surface of the object to be measured. After the camera captures the deformed pattern, the three-dimensional information of the object is calculated according to the principle of triangulation.

2. The method for automatic identification of measurement error data of a structured light system based on a neural network according to claim 1, characterized in that: In step S2, the grayscale distribution characteristics of the cross-section of the stripe without defects in the dataset require that all intermediate points must fall on the center line of the stripe and be symmetrically distributed. Stripes that meet the above conditions are defined as defect-free stripes; otherwise, the stripes are defective. 6450 grayscale distribution images of stripe cross-sections are selected as the dataset, in which the ratio of defective stripes to defect-free stripes is 2:

1.

3. The method for automatic identification of measurement error data of a structured light system based on a neural network according to claim 1, characterized in that: In step S3, the residual neural network used is composed of a series of residual blocks, and a residual block can be represented as: ， in x l and x l+1 Represents the input and output of the residual block. F Represents the residual function. f It is the ReLU activation function. W Represents all weights within the residual block; From the shallow layer l to the deep layers L The learned features are represented as follows: ， That is, for any depth of element L Features x L It can be expressed as a shallow unit. l Features x l Add a shape like The residual function, where F Represents the residual function. x i Represents input, W Represents all weights within the residual block; There are two types of residual blocks: one is a two-layer structure for 34-layer residual neural networks, and the other is a three-layer structure for 50 / 101 / 152-layer residual neural networks.

4. The method for automatic identification of measurement error data of a structured light system based on a neural network according to claim 1, characterized in that: In step S3, a 50-layer residual neural network is used for training. The system and hardware use Windows 10 operating system and an 11th Gen Intel(R) Core(TM) i5-11300H @ 3.10GHz 3.11GHz CPU. The software and dependent environment are as follows: the training framework is ResNet50, and the training neural network software and development tools include Python 3.6, Keras 2.2.0, TensorFlow 1.9.0, and NumPy 1.13.

3.

5. The method for automatic identification of measurement error data of a structured light system based on a neural network according to claim 1, characterized in that: In step S4, the flow of the structured light stripe defect detection program is as follows: First, input the cross-sectional image of the structured light stripe; The structured light stripe cross-sectional image is converted into a grayscale distribution map; the size of the grayscale distribution map is changed to meet the optimal input format of the trained neural network; the distribution map is input into the prediction model to obtain a judgment on whether the stripe quality is defective, providing a basis for improving the accuracy of subsequent measurements.

Images