Parameter calibration method and device based on deep learning and high-reflection profile detector
Through the parameter calibration method based on deep learning, the neural network model is constructed and optimized, and the problem of low calibration efficiency of traditional high-reverse shape detectors is solved, achieving a more efficient and accurate calibration process.
Patent Information
- Application Number
- CN202411849859.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-16
- Publication Date
- 2025-04-25
AI Technical Summary
The calibration method of traditional high-reverse shape detectors is not efficient, resulting in limited accuracy and repeatability of measurement results.
The parameter calibration method based on deep learning is adopted, and the initial neural network model is constructed, the model is trained using the training data set, and the model parameters of the target neural network model are obtained as calibration results.
The calibration efficiency of the high-reverse shape detector is improved, labor and time costs are reduced during the calibration process, and the accuracy and repeatability of the measurement results are enhanced.
Smart Images
Figure CN119879775A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of intelligent control of contour measurement and visual three-dimensional reconstruction equipment, and in particular to a parameter calibration method and equipment based on deep learning and a high-reflection surface shape detector. Background Art
[0002] A high-reflectivity surface detector is a precision instrument used to detect surface topography (such as flatness, curvature, ripples, etc.). Before the experiment, calibration plays a vital role in this equipment because it ensures the accuracy and repeatability of the measurement results. Calibration can reduce the systematic errors of the instrument itself, which may come from sensors, optical components, mechanical parts, etc. Through calibration, these errors can be minimized, thereby improving measurement accuracy. However, the traditional calibration method of high-reflectivity surface detectors is not efficient. Summary of the invention
[0003] The present application aims to propose a parameter calibration method, device and high-reflection surface shape detector based on deep learning, which can improve the calibration efficiency of the high-reflection surface shape detector.
[0004] The embodiment of the present application provides a parameter calibration method based on deep learning, which is applied to an electronic device, wherein the electronic device is arranged in a high-reflective surface shape detector, and the method comprises:
[0005] Acquire a training data set, the training data set includes a plurality of sample data, the plurality of sample data all include first height data of the object to be measured and phase data of the object to be measured, the first height data represents the height of the object-carrying plane of the high-reflection surface shape detector, the object-carrying plane is used to place the object to be measured, the phase data represents the phase of the object to be measured, and the first height data corresponds to the phase data;
[0006] An initial neural network model is constructed using the parameters to be calibrated of the high-reflective surface shape detector as model parameters, wherein the objective function of the initial neural network model is used to output second height data based on the input phase data, wherein the second height data represents the predicted height of the high-reflective surface shape detector;
[0007] Using the training data set to train a preset initial neural network model to obtain a target neural network model;
[0008] The model parameters of the target neural network model are used as target calibration parameters.
[0009] According to some embodiments of the present application, obtaining a training data set includes:
[0010] Acquire a plurality of different first height data of the object to be measured;
[0011] Acquire a plurality of fringe projection data of the object to be measured, wherein the plurality of fringe projection data correspond to a plurality of the first height data;
[0012] Performing phase unwrapping processing on the plurality of fringe projection data to obtain a plurality of phase data;
[0013] The training data set is obtained according to a plurality of the first height data and a plurality of the phase data.
[0014] According to some embodiments of the present application, the sample data includes first sample data and second sample data, the first sample data includes the first height data when the object under test is in a reference plane, and the phase data when the object under test is in the reference plane, and the second sample data includes the first height data when the object under test is in a non-reference plane, and the phase data when the object under test is in the non-reference plane;
[0015] The initial neural network model includes:
[0016] A first feature extraction module, the first feature extraction module is used to extract feature information of the first sample data to obtain first feature data;
[0017] A second feature extraction module, the second feature extraction module is used to extract feature information of the second sample data to obtain second feature data;
[0018] A feature splicing module, the feature splicing module is connected to the first feature extraction module, the feature splicing module is connected to the second feature extraction module, and the feature splicing module is used to splice the first feature data and the second feature data to obtain feature fusion data;
[0019] The encoding and decoding module, the feature splicing module is connected to the encoding and decoding module, and the encoding and decoding module is used to perform reconstruction processing according to the feature fusion data to obtain reconstructed feature data;
[0020] A result output module is used to obtain and output the second height data according to the feature fusion data and the reconstructed feature data.
[0021] According to some embodiments of the present application, the first feature extraction module includes a first residual module, a first convolutional layer, and a first spatial pyramid pooling module connected in sequence, and the first spatial pyramid pooling module is connected to the feature splicing module.
[0022] According to some embodiments of the present application, the second feature extraction module includes a second residual module, a second convolutional layer, a third convolutional layer, and a second spatial pyramid pooling module connected in sequence, the second spatial pyramid pooling module is connected to the feature splicing module, and the third convolutional layer is jump-connected to the feature splicing module.
[0023] According to some embodiments of the present application, the objective function of the initial neural network model includes:
[0024]
[0025] Among them, θ is the model parameter, N is the number of pixels, C i is the image point of the object being measured, Returns the projection of the point in the world coordinate system onto the image plane, Θ c is the intrinsic parameter of the first camera, R cs is the relative position parameter of the first camera, t cs is the relative position parameter of the projection screen of the high-reflection surface detector, is the coordinate of point B on the surface of the object under test in the projection screen coordinate system, is the coordinate of the image point C in the plane of the projection screen, Θ s is the intrinsic parameter of the projection screen, is the coordinate of point B on the surface of the object in the camera coordinate system.
[0026] According to some embodiments of the present application, the physical constraints of the initial neural network model include:
[0027] F=[e r ] × S s (S c ) + =[e r ] × K s R cs (K c ) -1 =(K s ) -T [t cs ] × R cs (K c ) -1 ,
[0028] Wherein, F is the intrinsic matrix of the parameter to be calibrated, e r is the pole vector, e r Characterizes the projection of the optical center of the first camera of the high-reflection surface detector on the imaging plane of the second camera of the high-reflection surface detector, S sis the projection matrix of the projection screen of the high-reflection surface detector, S c is the imaging matrix of the first camera, K s is the internal parameter matrix of the projection screen, K c is the intrinsic parameter matrix of the first camera, R cs is the relative position parameter of the first camera, t cs It is the relative position parameter of the projection screen of the high-reflection surface detector.
[0029] According to some embodiments of the present application,
[0030]
[0031] Among them, Loss(θ,φ0,φ C ,h B ) is the total loss function, θ is the model parameter, φ0 is the first phase data, the first phase data is the phase data of the first sample data, φ C is the second phase data, the second phase data is the phase data of the second sample data, and h B is the first height data of the second sample data, λ1 is the first weight, λ2 is the second weight, is: the imaging horizontal coordinate of a point on the object under test on the second camera, The imaging horizontal coordinate of a point on the object under test on the second camera is calculated based on the phase data: is the difference between the second phase data and the first phase data corresponding to the first camera, is the difference between the second phase data corresponding to the second camera and φ0, is the first height data, h i is the second height data, and N is the number of pixels.
[0032] In a second aspect, an embodiment of the present application provides an electronic device, including:
[0033] at least one processor;
[0034] at least one memory for storing at least one program;
[0035] When at least one of the programs is executed by at least one of the processors, the parameter calibration method based on deep learning as described above is implemented.
[0036] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the deep learning-based parameter calibration method as described above when executed by the processor.
[0037] In a fourth aspect, an embodiment of the present application provides a high-reflective surface shape detector, which is provided with the above-mentioned electronic device.
[0038] In an embodiment of the present application, the parameters to be calibrated of the high-reflection surface shape detector are used as model parameters to construct an initial neural network model, and the preset initial neural network model is trained using a training data set. The relationship between the height and phase of the object to be measured is established through deep learning of the initial neural network model, that is, the model parameters of the initial neural network model are iteratively optimized to obtain a target neural network model. The model parameters of the target neural network model are the calibration results of the parameters to be calibrated, which can reduce the labor cost and time cost in the calibration process and has high calibration efficiency.
[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The present application is further described below with reference to the accompanying drawings and embodiments, wherein:
[0041] Figure 1 A flowchart of an embodiment of a parameter calibration method based on deep learning provided in this application;
[0042] Figure 2 A positional relationship diagram of the object-carrying plane, the first camera, the second camera and the projection screen in the embodiment of the high-reflection surface shape detector provided in the present application;
[0043] Figure 3 A schematic diagram of the structure of an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in this application;
[0044] Figure 4 A result diagram of the PV value of the object under test before and after optimization of the initial neural network model of the deep learning-based parameter calibration method embodiment provided in the present application;
[0045] Figure 5 A schematic diagram of sample data of a first type of object under test input into an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0046] Figure 6 A schematic diagram of second height data of a first type of object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0047] Figure 7 A schematic diagram of sample data of a second object under test input to an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0048] Figure 8 A schematic diagram of second height data of a second object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0049] Fig. 9 A schematic diagram of sample data of a third object under test inputted into the initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0050] Fig.10 A schematic diagram of second height data of a third object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0051] Fig.11 A schematic diagram of sample data of a fourth object under test input to the initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0052] Fig.12 A schematic diagram of second height data of a fourth object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0053] Fig.13 A schematic diagram of sample data of a fifth type of object under test inputted into the initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0054] Fig.14 A schematic diagram of second height data of a fifth type of object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0055] Fig.15 A schematic diagram of sample data of the sixth type of object under test inputted into the initial neural network model of the embodiment of the parameter calibration method based on deep learning provided in the present application;
[0056] Fig.16 A schematic diagram of second height data of a sixth type of object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0057] Fig.17 A schematic diagram of sample data of the seventh object under test inputted into the initial neural network model of the embodiment of the parameter calibration method based on deep learning provided in the present application;
[0058] Fig.18 A schematic diagram of second height data of a seventh object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0059] Fig.19 A schematic diagram of sample data of an eighth type of object under test inputted into the initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0060] Fig. 20 A schematic diagram of second height data of an eighth type of object under test output by an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0061] Fig.21 A schematic diagram of sample data of a ninth type of object under test input into an initial neural network model of an embodiment of a parameter calibration method based on deep learning provided in the present application;
[0062] Fig. 22 A schematic diagram of the second height data of the ninth type of object under test output by the initial neural network model of the deep learning-based parameter calibration method embodiment provided in the present application.
[0063] Reference numerals:
[0064] The first camera 100 , the second camera 200 , the projection screen 300 , the object plane 400 , the first sample data 500 , the second sample data 510 , the second spatial pyramid pooling module 520 , the result output module 530 , the encoding and decoding module 540 , and the second height data 550 . DETAILED DESCRIPTION
[0065] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.
[0066] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0067] In the description of this application, "a plurality" means more than two. If there is a description of "first" or "second", it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0068] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.
[0069] Refer to the following Figures 1 to 3 A parameter calibration method, device and high-reflection surface shape detector based on deep learning according to an embodiment of the present application are described.
[0070] The embodiment of the present application provides a parameter calibration method based on deep learning, which is applied to an electronic device, wherein the electronic device is arranged in a high-reflection surface shape detector, such as Figure 2 As shown, the high reflection surface shape detector has a first camera 100, a second camera 200, a loading plane 400 and a projection screen 300, as shown in FIG. Figure 1 As shown, the method includes:
[0071] Step S100: obtaining a training data set, the training data set includes a plurality of sample data, the plurality of sample data all include first height data of the object to be measured and phase data of the object to be measured, the first height data represents the height of the object-carrying plane 400 of the high-reflection surface shape detector, the object-carrying plane 400 is used to place the object to be measured, the phase data represents the phase of the object to be measured, and the first height data corresponds to the phase data;
[0072] Step S200: using the parameters to be calibrated of the high-reflection surface shape detector as model parameters, constructing an initial neural network model, the objective function of the initial neural network model is used to output second height data 550 based on the input phase data, and the second height data 550 represents the predicted height of the high-reflection surface shape detector;
[0073] Step S300: training a preset initial neural network model using a training data set to obtain a target neural network model;
[0074] Step S400: using the model parameters of the target neural network model as target calibration parameters.
[0075] In an embodiment of the present application, the parameters to be calibrated of the high-reflection surface shape detector are used as model parameters to construct an initial neural network model, and the preset initial neural network model is trained using a training data set. The relationship between the height and phase of the object to be measured is established through deep learning of the initial neural network model, that is, the model parameters of the initial neural network model are iteratively optimized to obtain a target neural network model. The model parameters of the target neural network model are the calibration results of the parameters to be calibrated, which can reduce the labor cost and time cost in the calibration process and has high calibration efficiency.
[0076] like Figure 4 As shown, Figure 4The PV value result diagram of the object under test before and after the optimization of the initial neural network model, where the smaller points represent the PV values before optimization, and the larger points represent the PV values after optimization. The PV value reflects the algorithm error of the initial neural network model.
[0077] like Figures 5 to 22 As shown, Figures 5 to 22 The sample data of nine different objects under test input into the initial neural network model and the corresponding output second height data are shown.
[0078] In some embodiments of the present application, the “obtaining a training data set” in the above step S100 is further described, and step S100 includes:
[0079] Step S110: Acquire a plurality of different first height data of the object to be measured;
[0080] Step S120: Acquire a plurality of fringe projection data of the object to be measured, where the plurality of fringe projection data correspond to a plurality of first height data;
[0081] Step S130: performing phase unwrapping processing on the plurality of fringe projection data to obtain a plurality of phase data;
[0082] Step S140: obtaining a training data set according to the plurality of first height data and the plurality of phase data.
[0083] In this embodiment, the object to be measured is collected at each height according to the experimental steps of height calibration. For example, the plane when the height of the object plane 400 is 138.5mm is used as the reference plane, that is, the height of the object point of the object to be measured is set to 0mm at this time, and then 3mm is used as the moving step length, and the plane at the object point height of the object to be measured is ±24mm as the maximum absolute height, and multiple sets of plane fringe projection data and first height data are collected, and the shooting area resolution of the first camera 100 and the second camera 200 is 4096 pixels × 2048 pixels. Phase unwrapping processing is performed on multiple fringe projection data to obtain multiple phase data, and multiple sets of phase data and first height data data pairs are obtained, that is, a training data set is obtained.
[0084] In some embodiments of the present application, the objects to be measured include different types of commonly used highly reflective devices such as spherical mirrors, HUDs, and rearview mirrors. The objects to be measured are collected and unpacked at each height according to the experimental steps of height calibration to obtain a training data set. Therefore, when the training data set is used to train the initial neural network model, part of the training set is used as supervision during the deep learning process of the initial neural network.
[0085] In some embodiments of the present application, the sample data includes first sample data 500 and second sample data 510, the first sample data 500 includes first height data when the object under test is in a reference plane, and phase data when the object under test is in the reference plane, and the second sample data 510 includes first height data when the object under test is in a non-reference plane, and phase data when the object under test is in the non-reference plane;
[0086] like Figure 3 As shown ( Figure 3 In the figure, Dense is the fully connected layer, Conv is the convolutional layer, Encoder is the encoder, Decoder is the decoder, Concatenate is the feature concatenation module, Max Pooling is the maximum pooling module, Reshape is the shaping module, Residual is the residual module, and Spatial Pyramid Pooling is the first spatial pyramid pooling module). The initial neural network model includes:
[0087] A first feature extraction module, the first feature extraction module is used to extract feature information of the first sample data 500 to obtain first feature data;
[0088] A second feature extraction module, the second feature extraction module is used to extract feature information of the second sample data 510 to obtain second feature data;
[0089] A feature splicing module, the feature splicing module is connected to the first feature extraction module, the feature splicing module is connected to the second feature extraction module, and the feature splicing module is used to splice the first feature data and the second feature data to obtain feature fusion data;
[0090] The encoding and decoding module 540, the feature splicing module is connected to the encoding and decoding module 540, and the encoding and decoding module 540 is used to perform reconstruction processing according to the feature fusion data to obtain reconstructed feature data;
[0091] The result output module 530 is used to obtain and output the second height data 550 according to the feature fusion data and the reconstructed feature data.
[0092] In this embodiment, the feature information of the first sample data 500 is extracted by the first feature extraction module to obtain the first feature data, the feature information of the second sample data 510 is extracted by the second feature extraction module to obtain the second feature data, the first feature data and the second feature data are spliced by the feature splicing module to obtain feature fusion data, and the feature fusion data is a richer feature representation. The encoding and decoding module 540 performs reconstruction processing based on the feature fusion data to obtain reconstructed feature data, and the result output module 530 obtains and outputs the second height data 550 based on the feature fusion data and the reconstructed feature data.
[0093] In some embodiments of the present application, the encoding and decoding module 540 is a U-shaped multi-layer encoder-decoder structure, and the encoding and decoding module 540 converts the input feature fusion data into a height map output. The encoder gradually downsamples the input feature fusion data, extracts high-level features in the low-dimensional space, and compresses the high-dimensional features into a low-dimensional hidden layer representation. The decoder gradually upsamples these features, reconstructs the input spatial information, and re-decodes the low-dimensional hidden layer representation into the required height map output, which can well handle high-precision reconstruction problems.
[0094] In some embodiments of the present application, Figure 3 As shown, the result output module 530 includes a convolutional layer and a fully connected layer. The multi-layer perceptron in the fully connected layer is used to capture the nonlinear relationship between height and phase, and several hidden layers are added to capture complex nonlinear relationships.
[0095] In some embodiments of the present application, Figure 3 As shown, the first feature extraction module includes a first residual module, a first convolutional layer and a first spatial pyramid pooling module which are connected in sequence, and the first spatial pyramid pooling module is connected to the feature splicing module.
[0096] In this embodiment, the first sample data 500 is corrected by the first residual module. The first residual module is a deep neural network that solves the common gradient vacancies and height jump problems in deep network training by introducing jump connections. The deep neural network learns the residual, that is, the difference between the target output and the input, rather than directly learning the output. Through this design, the deep neural network can more easily retain the original features of high detail, and the neural network with greater depth can also maintain good performance. The first residual module contains two transmission paths. The main path is mainly used to learn the mapping relationship from the input end to the output end. The branch path is a jump connection, which is not calculated by the main path and is used to fuse the features of the input end and the output end. During the training process, the first residual module continuously updates the weights through gradient back propagation, while the jump connection maintains the stable propagation of the gradient, avoiding gradient vacancies and height jumps. The first convolutional layer extracts the features of the first sample data 500, and then captures the upper and lower information of different scales through the first spatial pyramid pooling module. The upper information describes the macroscopic characteristics in the global scope, capturing the large-scale patterns and trends of the input data. In this embodiment, the global characteristics refer to the changes in the phase data that can reflect the macroscopic characteristics such as the shape and contour of the entire object. The lower information focuses on local details and small-scale features, reflecting small but important changes in the data. In this embodiment, the local characteristics refer to the local changes (such as small fluctuations or detail differences) contained in the phase data corresponding to the texture or tiny bumps on the surface of the object. The introduction of the first spatial pyramid pooling module can better learn cross-scale features, thereby improving the reconstruction accuracy of the encoding and decoding module 540.
[0097] In some embodiments of the present application, Figure 3 As shown, the second feature extraction module includes a second residual module, a second convolutional layer, a third convolutional layer and a second spatial pyramid pooling module 520 connected in sequence, the second spatial pyramid pooling module 520 is connected to the feature splicing module, and the third convolutional layer is jump-connected to the feature splicing module.
[0098] In this embodiment, the second sample data 510 is corrected by the second residual module. The second residual module is a deep neural network that solves the common gradient vacancies and height jump problems in deep network training by introducing jump connections. The deep neural network learns the residual, that is, the difference between the target output and the input, rather than directly learning the output. Through this design, the deep neural network can more easily retain the original features of high detail, and the neural network with greater depth can also maintain good performance. The first residual module includes two transmission paths. The main path is mainly used to learn the mapping relationship from the input end to the output end. The branch path is a jump connection, which is not calculated by the main path and is used to fuse the features of the input end and the output end. During the training process, the first residual module continuously updates the weights through gradient back propagation, while the jump connection maintains the stable propagation of the gradient, avoiding gradient vacancies and height jumps. The first convolutional layer extracts the features of the first sample data 500, and then captures the upper and lower information of different scales through the first spatial pyramid pooling module. The upper information describes the macroscopic characteristics in the global scope, capturing the large-scale patterns and trends of the input data. In this embodiment, the global characteristics refer to the changes in the phase data, which can reflect the macroscopic characteristics such as the shape and contour of the entire object. The lower information focuses on local details and small-scale features, reflecting small but important changes in the data. In this embodiment, the local characteristics refer to the texture or tiny bumps on the surface of the object corresponding to the local changes (such as small fluctuations or detail differences) contained in the phase data. The introduction of the second spatial pyramid pooling module 520 can better learn cross-scale features, thereby improving the reconstruction accuracy of the encoding and decoding module 540.
[0099] In some embodiments of the present application, Figure 3 As shown, the second spatial pyramid pooling module 520 includes four pooling units, each of which includes a maximum pooling module, a fourth convolutional layer, a shaping module and a fifth convolutional layer connected in sequence.
[0100] In this embodiment, the second spatial pyramid pooling module 520 extracts features through different pooling scales, performs multi-scale pooling, and combines global information with local details. The first pooling unit in the second spatial pyramid pooling module 520 is a large window pooling to extract global information. The second pooling unit is a medium-sized pooling window to capture medium-scale regional information. The third pooling unit is a smaller pooling window to retain more local details. The fourth pooling unit is a minimum window pooling, focusing on the smallest scale of detail changes.
[0101] In some embodiments of the present application, Figure 2 In the world coordinate system in , point B on the object to be measured is taken. This point reflects the image projected by the projection screen 300 and forms an image at the coordinate (x c1 ,yc1 ), the phase unwrapping algorithm based on Fourier profilometry can calculate its unfolded phase, that is, phase data φC. Because each highly reflective device can be obtained through the homography h of the plane where it is located. B Mapped to its phase data φC in the image plane, based on which a loss function is established. The loss function of the initial neural network model includes:
[0102]
[0103] Among them, Loss(θ,φ0,φ C ,h B ) is the total loss function, θ is the model parameter, φ0 is the first phase data, the first phase data is the phase data of the first sample data 500, φ C is the second phase data, which is the phase data of the second sample data 510, and h B is the first height data of the second sample data 510, λ1 is the first weight, λ2 is the second weight, is: the imaging horizontal coordinate of a point on the object under test on the second camera 200, The imaging horizontal coordinate of a point on the object under test on the second camera 200 is calculated based on the phase data: is the difference between the second phase data and the first phase data corresponding to the first camera 100, is the difference between the second phase data corresponding to the second camera 200 and φ0, is the first height data, h i is the second height data 550, and N is the number of pixels.
[0104] In some embodiments of the present application, the phase information matrix in the first sample data 500 establishes point correspondence between the first camera 100, the second camera 200 and the screen. The same point on the plane is associated with the corresponding image point in another coordinate system through the plane homography, so Figure 2 The points B and B′ on the object to be measured are on the same polar line. Multiple image points C of the object to be measured are collected from the camera coordinate system of the first camera and the second camera, and their points C in the plane of the projection screen 300 are obtained from the phase data. s The coordinates of point B on the surface of the object to be measured in the camera coordinate system and the projection screen 300 coordinate system are B c and B s . Return the projection of the midpoint in the world coordinate system on the image plane. Therefore, the objective function imaging error minimization is constructed as the objective function of the initial neural network model, which can be expressed as:
[0105]
[0106] Among them, θ is the model parameter, N is the number of pixels, C i is the image point of the object being measured, Returns the projection of the point in the world coordinate system onto the image plane, Θ c is the intrinsic parameter of the first camera 100, R cs is the relative position parameter of the first camera 100, t cs is the relative position parameter of the projection screen 300 of the high-reflection surface detector, is the coordinate of point B on the surface of the object under test in the projection screen 300 coordinate system, is the coordinate of the image point C in the plane of the projection screen 300, Θ s is the internal parameter of the projection screen 300, is the coordinate of point B on the surface of the object in the camera coordinate system.
[0107] In some embodiments of the present application, the physical constraints of the initial neural network model include:
[0108] F=[e r ] × S s (S c ) + =[e r ] × K s R cs (K c ) -1 =(K s ) -T [t cs ] × R cs (K c ) -1 ,
[0109] Among them, F is the intrinsic matrix of the parameters to be calibrated, e r is the pole vector, e r The projection of the optical center of the first camera 100 of the high-reflection surface shape detector on the imaging plane of the second camera 200 of the high-reflection surface shape detector, S s is the projection matrix of the projection screen 300 of the high-reflection surface detector, S c is the imaging matrix of the first camera 100, K s is the internal parameter matrix of the projection screen 300, K c is the intrinsic parameter matrix of the first camera 100, R cs is the relative position parameter of the first camera 100, t cs It is the relative position parameter of the projection screen 300 of the high reflection surface detector.
[0110] In this embodiment, multiple loss functions can respectively measure different objectives or characteristics of the model, so that the training process can optimize multiple aspects of performance simultaneously. The loss function is used to capture global characteristics or trends. The loss is based on physical geometric relationships to ensure that the initial neural network model has a good learning ability for the physical constraints or rules in the data. The Loss2 loss function is used to target local characteristics, focusing on pixel-level errors or local details to improve the accuracy of local predictions. A single loss function may cause the model to fall into a local optimal state during training, making it difficult to further improve accuracy. Introducing multiple loss functions can increase the diversity of optimization paths, provide additional gradient directions for the neural network model, and help it escape the local optimal state to find a better solution. The last regression layer The loss function directly penalizes the deviation between the initial neural network model's prediction and the true value.
[0111] In addition, an embodiment of the present application provides an electronic device, including:
[0112] at least one processor;
[0113] at least one memory for storing at least one program;
[0114] When at least one program is executed by at least one processor, the parameter calibration method based on deep learning as described above is implemented.
[0115] In an embodiment of the present application, the parameters to be calibrated of the high-reflection surface shape detector are used as model parameters to construct an initial neural network model, and the preset initial neural network model is trained using a training data set. The relationship between the height and phase of the object to be measured is established through deep learning of the initial neural network model, that is, the model parameters of the initial neural network model are iteratively optimized to obtain a target neural network model. The model parameters of the target neural network model are the calibration results of the parameters to be calibrated, which can reduce the labor cost and time cost in the calibration process and has high calibration efficiency.
[0116] In addition, an embodiment of the present application provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the parameter calibration method based on deep learning as described above.
[0117] In an embodiment of the present application, the parameters to be calibrated of the high-reflection surface shape detector are used as model parameters to construct an initial neural network model, and the preset initial neural network model is trained using a training data set. The relationship between the height and phase of the object to be measured is established through deep learning of the initial neural network model, that is, the model parameters of the initial neural network model are iteratively optimized to obtain a target neural network model. The model parameters of the target neural network model are the calibration results of the parameters to be calibrated, which can reduce the labor cost and time cost in the calibration process and has high calibration efficiency.
[0118] In addition, an embodiment of the present application provides a high-reflection surface shape detector, which is provided with the above-mentioned electronic device.
[0119] In an embodiment of the present application, the parameters to be calibrated of the high-reflection surface shape detector are used as model parameters to construct an initial neural network model, and the preset initial neural network model is trained using a training data set. The relationship between the height and phase of the object to be measured is established through deep learning of the initial neural network model, that is, the model parameters of the initial neural network model are iteratively optimized to obtain a target neural network model. The model parameters of the target neural network model are the calibration results of the parameters to be calibrated, which can reduce the labor cost and time cost in the calibration process and has high calibration efficiency.
[0120] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0121] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.
Claims
1. A parameter calibration method based on deep learning, characterized in that: Applied to electronic equipment, the electronic equipment is arranged in a high-reflection surface shape detector, and the method comprises: Acquire a training data set, the training data set includes a plurality of sample data, the plurality of sample data all include first height data of the object to be measured and phase data of the object to be measured, the first height data represents the height of the object-carrying plane of the high-reflection surface shape detector, the object-carrying plane is used to place the object to be measured, the phase data represents the phase of the object to be measured, and the first height data corresponds to the phase data; An initial neural network model is constructed using the parameters to be calibrated of the high-reflective surface shape detector as model parameters, wherein the objective function of the initial neural network model is used to output second height data based on the input phase data, wherein the second height data represents the predicted height of the high-reflective surface shape detector; Using the training data set to train a preset initial neural network model to obtain a target neural network model; The model parameters of the target neural network model are used as target calibration parameters.
2. The parameter calibration method based on deep learning according to claim 1, characterized in that: The step of obtaining a training data set includes: Acquire a plurality of different first height data of the object to be measured; Acquire a plurality of fringe projection data of the object to be measured, wherein the plurality of fringe projection data correspond to a plurality of the first height data; Performing phase unwrapping processing on the plurality of fringe projection data to obtain a plurality of phase data; The training data set is obtained according to a plurality of the first height data and a plurality of the phase data.
3. The parameter calibration method based on deep learning according to claim 1, characterized in that: The sample data includes first sample data and second sample data, the first sample data includes the first height data when the object under test is in a reference plane, and the phase data when the object under test is in the reference plane, and the second sample data includes the first height data when the object under test is in a non-reference plane, and the phase data when the object under test is in the non-reference plane; The initial neural network model includes: A first feature extraction module, the first feature extraction module is used to extract feature information of the first sample data to obtain first feature data; A second feature extraction module, the second feature extraction module is used to extract feature information of the second sample data to obtain second feature data; A feature splicing module, the feature splicing module is connected to the first feature extraction module, the feature splicing module is connected to the second feature extraction module, and the feature splicing module is used to splice the first feature data and the second feature data to obtain feature fusion data; The encoding and decoding module, the feature splicing module is connected to the encoding and decoding module, and the encoding and decoding module is used to perform reconstruction processing according to the feature fusion data to obtain reconstructed feature data; A result output module is used to obtain and output the second height data according to the feature fusion data and the reconstructed feature data.
4. The parameter calibration method based on deep learning according to claim 3, characterized in that: The first feature extraction module includes a first residual module, a first convolutional layer and a first spatial pyramid pooling module which are connected in sequence, and the first spatial pyramid pooling module is connected to the feature splicing module.
5. The parameter calibration method based on deep learning according to claim 3, characterized in that: The second feature extraction module includes a second residual module, a second convolutional layer, a third convolutional layer and a second spatial pyramid pooling module which are connected in sequence, the second spatial pyramid pooling module is connected to the feature splicing module, and the third convolutional layer is jump-connected to the feature splicing module.
6. The parameter calibration method based on deep learning according to claim 3, characterized in that: The loss function of the initial neural network model includes: Among them, Loss(θ,φ0,φ C ,h B ) is the total loss function, θ is the model parameter, φ0 is the first phase data, the first phase data is the phase data of the first sample data, φ C is the second phase data, the second phase data is the phase data of the second sample data, and h B is the first height data of the second sample data, λ1 is the first weight, λ2 is the second weight, is: the imaging horizontal coordinate of a point on the object under test on the second camera, The imaging horizontal coordinate of a point on the object under test on the second camera is calculated based on the phase data: is the difference between the second phase data and the first phase data corresponding to the first camera, is the difference between the second phase data corresponding to the second camera and φ0, is the first height data, h i is the second height data, and N is the number of pixels.
7. The parameter calibration method based on deep learning according to claim 1, characterized in that: The objective function of the initial neural network model includes: Among them, θ is the model parameter, N is the number of pixels, C i is the image point of the object being measured, Returns the projection of the point in the world coordinate system onto the image plane, Θ c is the intrinsic parameter of the first camera, R cs is the relative position parameter of the first camera, t cs is the relative position parameter of the projection screen of the high-reflection surface detector, is the coordinate of point B on the surface of the object under test in the projection screen coordinate system, is the coordinate of the image point C in the plane of the projection screen, Θ s is the intrinsic parameter of the projection screen, is the coordinate of point B on the surface of the object in the camera coordinate system.
8. The parameter calibration method based on deep learning according to claim 1, characterized in that: The physical constraints of the initial neural network model include: F=[e r ] × S s (S c ) + =[e r ] × K s R cs (K c ) -1 =(K s ) -T [t cs ] × R cs (K c ) -1 , Wherein, F is the intrinsic matrix of the parameter to be calibrated, e r is the pole vector, e r Characterizes the projection of the optical center of the first camera of the high-reflection surface detector on the imaging plane of the second camera of the high-reflection surface detector, S s is the projection matrix of the projection screen of the high-reflection surface detector, S c is the imaging matrix of the first camera, K s is the internal parameter matrix of the projection screen, K c is the intrinsic parameter matrix of the first camera, R cs is the relative position parameter of the first camera, t cs It is the relative position parameter of the projection screen of the high-reflection surface detector.
9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the deep learning-based parameter calibration method as described in any one of claims 1 to 8 is implemented.
10. A high-reflection surface detector, characterized in that: Comprising the electronic device as claimed in claim 9.