Detection image distortion correction method and system based on structured light

Abstract

The invention discloses a detection image distortion correction method and system based on structured light, and belongs to the technical field of image processing. Latticed structured light is projected to the surface of a to-be-detected object, an image is collected, a grid contour map is extracted by using pixel values of grid pixel points, and the grid contour map is further separated into a plurality of grid skeleton sub-diagrams. The method comprises the following steps of: firstly, identifying an edge with a minimum distortion degree value, constructing an ideal grid model, estimating the distortion amount of each pixel point based on the coordinate difference between the model and a grid framework sub-graph, and correcting the corresponding pixel points on an object surface image so as to obtain a clear and accurate corrected image. The correction duration is shortened, camera parameters are not needed, the parameters needing to be adjusted and considered in the correction process are reduced, and the complexity of the correction process is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for correcting detected image distortion based on structured light. Background Art

[0002] In the modern industrial field, accurate 3D measurement and surface inspection technology play a vital role. Among them, the 3D measurement method based on structured light is widely used in many application scenarios due to its high precision, fast speed and non-contact characteristics. However, in the actual application process, due to the influence of various factors, the acquired images often have different degrees of distortion, which seriously affects the accuracy and reliability of the measurement.

[0003] Image distortion refers to the distortion of images due to the optical characteristics of the imaging system and the characteristics of environmental factors. This distortion may appear as image distortion, deformation or local stretching and compression, which makes the geometric relationship in the image inconsistent with the actual object. This distortion will directly affect the extraction of depth information and the accuracy of 3D reconstruction.

[0004] Traditional image distortion correction methods usually rely on a complex camera calibration process, which requires the use of a specific calibration plate and the combination of the camera's intrinsic parameters (such as focal length, principal point coordinates, distortion coefficient) and extrinsic parameters (camera posture). The image of the calibration plate is taken multiple times to obtain reliable calibration results. However, the correction process is time-consuming and highly complex. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and system for correcting detected image distortion based on structured light, which solves the problems of the prior art in that the correction process is time-consuming and highly complex.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a detection image distortion correction method based on structured light, comprising the following steps:

[0007] S1, projecting a grid-shaped structured light onto the surface of the object to be measured, and collecting the projected surface image of the object;

[0008] S2, extracting a grid contour map according to the pixel values ​​of the grid pixel points on the object surface image;

[0009] S3, separating the mesh contour image into multiple mesh skeleton sub-images;

[0010] S4, finding the horizontal edge and the vertical edge with the smallest distortion value in multiple mesh skeleton sub-graphs, and constructing an ideal mesh model;

[0011] S5, estimating the distortion amount of each pixel point inside the grid skeleton sub-image according to the difference in coordinates between the ideal grid model and the grid skeleton sub-image;

[0012] S6. According to the pixel distortion amount on the grid skeleton sub-image, the same pixel on the object surface image is corrected to obtain a corrected image.

[0013] Furthermore, the S2 comprises the following sub-steps:

[0014] S21, taking the pixel values ​​of the grid pixels on the surface image of the object as comparison pixel values;

[0015] S22, calculating the similarity between the comparison pixel value and the pixel values ​​of other pixels;

[0016] S23. When the similarity is greater than a similarity threshold, the corresponding pixel points are used as grid contour pixel points, and all grid contour pixel points constitute a grid contour map.

[0017] Furthermore, S3 is specifically as follows: in each grid area of ​​the grid contour map, only grid edge pixel points inside the grid are retained, and all grid edge pixel points inside a grid constitute a grid skeleton subgraph.

[0018] Furthermore, the S4 comprises the following sub-steps:

[0019] S41, calculating the curvature of each pixel in each grid skeleton sub-image;

[0020] S42, obtaining four corner points of each grid skeleton sub-image according to the curvature of each pixel point;

[0021] S43, using the four corner points as segmentation points, segmenting the corresponding grid skeleton subgraph into four edges, wherein the four edges include two horizontal edges and two vertical edges;

[0022] S44, calculating the distortion degree value of each horizontal edge and each vertical edge, wherein the distortion degree value is the sum of the curvatures of each pixel point on an edge;

[0023] S45, finding a horizontal edge with the smallest distortion value from all horizontal edges to obtain a horizontal reference edge;

[0024] S46, finding a longitudinal edge with the smallest distortion value from all longitudinal edges to obtain a longitudinal reference edge;

[0025] S47. Construct an ideal grid model based on the horizontal reference edge and the vertical reference edge.

[0026] Furthermore, the S5 comprises the following sub-steps:

[0027] S51, taking the geometric center of the ideal grid model as the coordinate origin, performing coordinate translation on each network skeleton subgraph, wherein the geometric center of each network skeleton subgraph coincides with the coordinate origin after translation;

[0028] S52, subtracting the upper lateral edge in the translated network skeleton sub-image from the upper lateral edge in the ideal grid model to obtain a coordinate distortion sequence of the upper lateral edge;

[0029] S53, subtracting the lower lateral edge in the translated network skeleton sub-image from the lower lateral edge in the ideal grid model to obtain a coordinate distortion sequence of the lower lateral edge;

[0030] S54, subtracting the left longitudinal edge in the translated network skeleton sub-image from the left longitudinal edge in the ideal grid model to obtain a coordinate distortion sequence of the left longitudinal edge;

[0031] S55, subtracting the right longitudinal edge in the translated network skeleton sub-image from the right longitudinal edge in the ideal grid model to obtain a right longitudinal edge coordinate distortion sequence;

[0032] S56, according to the upper horizontal edge coordinate distortion sequence, the lower horizontal edge coordinate distortion sequence, the left vertical edge coordinate distortion sequence and the right vertical edge coordinate distortion sequence, based on the horizontal coordinate distortion estimation model and the vertical coordinate distortion estimation model, estimate the distortion of each pixel point in the internal area of ​​the grid skeleton sub-image.

[0033] Furthermore, the S56 comprises the following sub-steps:

[0034] S561, separating the abscissa distortion amounts and ordinate distortion amounts in the upper lateral edge coordinate distortion sequence, the lower lateral edge coordinate distortion sequence, the left longitudinal edge coordinate distortion sequence, and the right longitudinal edge coordinate distortion sequence to obtain an upper lateral edge abscissa distortion sequence, a lower lateral edge abscissa distortion sequence, a left longitudinal edge abscissa distortion sequence, and a right longitudinal edge abscissa distortion sequence, and an upper lateral edge ordinate distortion sequence, a lower lateral edge ordinate distortion sequence, a left longitudinal edge ordinate distortion sequence, and a right longitudinal edge ordinate distortion sequence;

[0035] S562, using a horizontal coordinate distortion estimation model to process the upper horizontal side horizontal coordinate distortion sequence, the lower horizontal side horizontal coordinate distortion sequence, the left longitudinal side horizontal coordinate distortion sequence, the right longitudinal side horizontal coordinate distortion sequence, and the coordinates of the internal pixel points to be predicted, to obtain the horizontal coordinate distortion values ​​of the internal pixel points;

[0036] S563, using a ordinate distortion estimation model to process the upper lateral side ordinate distortion sequence, the lower lateral side ordinate distortion sequence, the left longitudinal side ordinate distortion sequence, the right longitudinal side ordinate distortion sequence and the coordinates of the internal pixel points to be predicted, to obtain the ordinate distortion values ​​of the internal pixel points.

[0037] Further, the abscissa distortion estimation model in S562 and the ordinate distortion estimation model in S563 both include: a first estimation unit, a second estimation unit, a third estimation unit, a fourth estimation unit and a fully connected layer;

[0038] The input end of the fully connected layer is connected to the output end of the first estimation unit, the output end of the second estimation unit, the output end of the third estimation unit and the output end of the fourth estimation unit respectively, and the output end thereof serves as the output end of the abscissa distortion amount estimation model or the ordinate distortion amount estimation model;

[0039] In the horizontal coordinate distortion estimation model, the first estimation unit is used to process the horizontal coordinate distortion sequence of the upper horizontal side and the coordinates of the internal pixel points to be predicted, the second estimation model is used to process the horizontal coordinate distortion sequence of the lower horizontal side and the coordinates of the internal pixel points to be predicted, the third estimation model is used to process the horizontal coordinate distortion sequence of the left longitudinal side and the coordinates of the internal pixel points to be predicted, and the fourth estimation model is used to process the horizontal coordinate distortion sequence of the right longitudinal side and the coordinates of the internal pixel points to be predicted;

[0040] In the vertical coordinate distortion estimation model, the first estimation unit is used to process the vertical coordinate distortion sequence of the upper horizontal side and the coordinates of the internal pixel points to be predicted, the second estimation unit is used to process the vertical coordinate distortion sequence of the lower horizontal side and the coordinates of the internal pixel points to be predicted, the third estimation unit is used to process the vertical coordinate distortion sequence of the left longitudinal side and the coordinates of the internal pixel points to be predicted, and the fourth estimation unit is used to process the vertical coordinate distortion sequence of the right longitudinal side and the coordinates of the internal pixel points to be predicted.

[0041] Furthermore, the first estimation unit, the second estimation unit, the third estimation unit and the fourth estimation unit have the same structure, and all include: an LSTM layer, a Concat layer and an activation function layer; the LSTM layer is used to process the corresponding distortion sequence, the Concat layer is used to splice the LSTM layer output and the coordinates of the internal pixel point to be predicted to obtain a spliced ​​sequence, and the activation function layer is used to process the spliced ​​sequence using an activation function to obtain an estimated value.

[0042] Furthermore, the expression of the activation function layer is:

[0043] , where y is the estimated value of the activation function layer output, s i is the i-th element in the concatenated sequence, w i For i The weight of b i For i , f is the activation function, N is the number of elements in the concatenated sequence, and i is a positive integer.

[0044] A detection image distortion correction system based on structured light, comprising: an image acquisition subsystem, a contour extraction subsystem, a separation subsystem, an ideal grid model construction subsystem, a distortion estimation subsystem and a correction subsystem;

[0045] The image acquisition subsystem is used to project grid-shaped structured light onto the surface of the object to be measured and acquire the projected surface image of the object;

[0046] The contour extraction subsystem is used to extract a grid contour map according to the pixel values ​​of the grid pixel points on the surface image of the object;

[0047] The separation subsystem is used to separate the mesh contour graph into a plurality of mesh skeleton subgraphs;

[0048] The ideal grid model construction subsystem is used to find the horizontal edge and the vertical edge with the smallest distortion value in multiple grid skeleton subgraphs to construct an ideal grid model;

[0049] The distortion estimation subsystem is used to estimate the distortion of each pixel point inside the mesh skeleton sub-image according to the difference in coordinates between the ideal mesh model and the mesh skeleton sub-image;

[0050] The correction subsystem is used to correct the same pixel point on the object surface image according to the pixel point distortion amount on the grid skeleton sub-image to obtain a corrected image.

[0051] The beneficial effects of the present invention are as follows: the present invention projects a grid-shaped structured light onto the surface of the object to be measured and collects images, uses the pixel values ​​of the grid pixels to extract the grid contour map, and further separates it into multiple grid skeleton sub-graphs. Then, the edge with the smallest distortion value is identified, an ideal grid model is constructed, and based on the difference in coordinates between the model and the skeleton sub-graph, the distortion amount of each pixel is estimated, and the corresponding pixel points on the surface image of the object are corrected, thereby obtaining a clear and accurate corrected image. In the present invention, there is no need to capture images multiple times, which reduces the correction time, and does not require camera parameters, which reduces the parameters that need to be adjusted and considered in the correction process, and reduces the complexity of the correction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of a detection image distortion correction method based on structured light;

[0053] Figure 2 Schematic diagram of grid-shaped structured light;

[0054] Figure 3 It is a structural diagram of the horizontal axis distortion estimation model and the vertical axis distortion estimation model. DETAILED DESCRIPTION

[0055] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0056] Embodiment 1, as Figure 1 As shown, a detection image distortion correction method based on structured light includes the following steps:

[0057] S1, projecting a grid-shaped structured light onto the surface of the object to be measured, and collecting the projected surface image of the object;

[0058] S2, extracting a grid contour map according to the pixel values ​​of the grid pixel points on the object surface image;

[0059] S3, separating the mesh contour image into multiple mesh skeleton sub-images;

[0060] S4, finding the horizontal edge and the vertical edge with the smallest distortion value in multiple mesh skeleton sub-graphs, and constructing an ideal mesh model;

[0061] S5, estimating the distortion amount of each pixel point inside the grid skeleton sub-image according to the difference in coordinates between the ideal grid model and the grid skeleton sub-image;

[0062] S6. According to the pixel distortion amount on the grid skeleton sub-image, the same pixel on the object surface image is corrected to obtain a corrected image.

[0063] In this embodiment, the sizes of the grids in the grid-shaped structured light are the same. Figure 2 shown.

[0064] In this embodiment, the projected object is an object with a flat surface (such as a wall, a table, or a piece of paper), thereby reducing the influence of the shape of the object itself.

[0065] In this embodiment, S2 includes the following sub-steps:

[0066] S21, taking the pixel values ​​of the grid pixels on the surface image of the object as comparison pixel values;

[0067] S22, calculating the similarity between the comparison pixel value and the pixel values ​​of other pixels;

[0068] S23. When the similarity is greater than a similarity threshold, the corresponding pixel points are used as grid contour pixel points, and all grid contour pixel points constitute a grid contour map.

[0069] In step S21, the grid pixel points, i.e., the imaging pixel points of the structured light in the surface image of the object, are clustered to obtain a grid contour map.

[0070] Specifically, S3 is as follows: in each grid area of ​​the grid outline graph, only the grid edge pixel points inside the grid are retained, and all the grid edge pixel points inside a grid constitute a grid skeleton subgraph.

[0071] Since the grid contour image is extracted in step S23, the internal edge pixels of the grid are retained to refine the contour and obtain a grid skeleton sub-image with a line width of 1 pixel.

[0072] In this embodiment, S4 includes the following sub-steps:

[0073] S41, calculating the curvature of each pixel in each grid skeleton sub-image;

[0074] S42, obtaining four corner points of each grid skeleton sub-image according to the curvature of each pixel point;

[0075] S43, using the four corner points as segmentation points, segmenting the corresponding grid skeleton subgraph into four edges, wherein the four edges include two horizontal edges and two vertical edges;

[0076] S44, calculating the distortion degree value of each horizontal edge and each vertical edge, wherein the distortion degree value is the sum of the curvatures of each pixel point on an edge;

[0077] S45, finding a horizontal edge with the smallest distortion value from all horizontal edges to obtain a horizontal reference edge;

[0078] S46, finding a longitudinal edge with the smallest distortion value from all longitudinal edges to obtain a longitudinal reference edge;

[0079] S47. Construct an ideal grid model based on the horizontal reference edge and the vertical reference edge.

[0080] In the ideal grid model, the two horizontal edges are horizontal reference edges, and the two vertical edges are vertical reference edges.

[0081] In this embodiment, step S42 is implemented by arranging the curvature of each pixel point in each mesh skeleton sub-image in descending order, and extracting the first four pixel points as the corner points of each mesh skeleton sub-image.

[0082] Since a grid-shaped structured light is used, the grid skeleton sub-image is in a grid shape, and the positions with the largest curvature belong to the corners.

[0083] Since the grid-shaped structured light is used in the present invention, the corner points can be found through the curvature, the grid skeleton sub-image can be divided into four edges, the horizontal edge with the smallest distortion value is found among all the horizontal edges, and the vertical edge with the smallest distortion value is found among all the vertical edges, that is, the two edges with the smallest distortion effect, and an ideal grid model is constructed based on these two edges.

[0084] In this embodiment, S5 includes the following sub-steps:

[0085] S51, taking the geometric center of the ideal grid model as the coordinate origin, performing coordinate translation on each network skeleton subgraph, wherein the geometric center of each network skeleton subgraph coincides with the coordinate origin after translation;

[0086] S52, subtracting the upper lateral edge in the translated network skeleton sub-image from the upper lateral edge in the ideal grid model to obtain a coordinate distortion sequence of the upper lateral edge;

[0087] S53, subtracting the lower lateral edge in the translated network skeleton sub-image from the lower lateral edge in the ideal grid model to obtain a coordinate distortion sequence of the lower lateral edge;

[0088] S54, subtracting the left longitudinal edge in the translated network skeleton sub-image from the left longitudinal edge in the ideal grid model to obtain a coordinate distortion sequence of the left longitudinal edge;

[0089] S55, subtracting the right longitudinal edge in the translated network skeleton sub-image from the right longitudinal edge in the ideal grid model to obtain a right longitudinal edge coordinate distortion sequence;

[0090] S56, according to the upper horizontal edge coordinate distortion sequence, the lower horizontal edge coordinate distortion sequence, the left vertical edge coordinate distortion sequence and the right vertical edge coordinate distortion sequence, based on the horizontal coordinate distortion estimation model and the vertical coordinate distortion estimation model, estimate the distortion of each pixel point in the internal area of ​​the grid skeleton sub-image.

[0091] In steps S52, S53, S54 and S55, the horizontal coordinates and the vertical coordinates of the pixel points of the two sides are subtracted.

[0092] The ideal grid model and each network skeleton subgraph are translated to the same range of the same coordinate system to facilitate the comparison between the ideal grid model and each network skeleton subgraph, and the coordinates of the same orientation are subtracted to obtain the distortion amount of each pixel.

[0093] In this embodiment, the S56 includes the following sub-steps:

[0094] S561, separating the abscissa distortion amounts and ordinate distortion amounts in the upper lateral edge coordinate distortion sequence, the lower lateral edge coordinate distortion sequence, the left longitudinal edge coordinate distortion sequence, and the right longitudinal edge coordinate distortion sequence to obtain an upper lateral edge abscissa distortion sequence, a lower lateral edge abscissa distortion sequence, a left longitudinal edge abscissa distortion sequence, and a right longitudinal edge abscissa distortion sequence, and an upper lateral edge ordinate distortion sequence, a lower lateral edge ordinate distortion sequence, a left longitudinal edge ordinate distortion sequence, and a right longitudinal edge ordinate distortion sequence;

[0095] S562, using a horizontal coordinate distortion estimation model to process the upper horizontal side horizontal coordinate distortion sequence, the lower horizontal side horizontal coordinate distortion sequence, the left longitudinal side horizontal coordinate distortion sequence, the right longitudinal side horizontal coordinate distortion sequence, and the coordinates of the internal pixel points to be predicted, to obtain the horizontal coordinate distortion values ​​of the internal pixel points;

[0096] S563, using a ordinate distortion estimation model to process the upper lateral side ordinate distortion sequence, the lower lateral side ordinate distortion sequence, the left longitudinal side ordinate distortion sequence, the right longitudinal side ordinate distortion sequence and the coordinates of the internal pixel points to be predicted, to obtain the ordinate distortion values ​​of the internal pixel points.

[0097] Since the distortion sequence in step S56 includes the abscissa distortion amount and the ordinate distortion amount, step S651 is adopted to separate them and predict them separately.

[0098] According to step S3, the mesh skeleton sub-image only has edge pixels, and the internal pixel in steps S56, S562, and S563 refers to a pixel position inside the mesh skeleton sub-image.

[0099] like Figure 3 As shown, the abscissa distortion estimation model in S562 and the ordinate distortion estimation model in S563 both include: a first estimation unit, a second estimation unit, a third estimation unit, a fourth estimation unit and a fully connected layer;

[0100] The input end of the fully connected layer is connected to the output end of the first estimation unit, the output end of the second estimation unit, the output end of the third estimation unit and the output end of the fourth estimation unit respectively, and the output end thereof serves as the output end of the abscissa distortion amount estimation model or the ordinate distortion amount estimation model;

[0101] In the horizontal coordinate distortion estimation model, the first estimation unit is used to process the horizontal coordinate distortion sequence of the upper horizontal side and the coordinates of the internal pixel points to be predicted, the second estimation model is used to process the horizontal coordinate distortion sequence of the lower horizontal side and the coordinates of the internal pixel points to be predicted, the third estimation model is used to process the horizontal coordinate distortion sequence of the left longitudinal side and the coordinates of the internal pixel points to be predicted, and the fourth estimation model is used to process the horizontal coordinate distortion sequence of the right longitudinal side and the coordinates of the internal pixel points to be predicted;

[0102] In the vertical coordinate distortion estimation model, the first estimation unit is used to process the vertical coordinate distortion sequence of the upper horizontal side and the coordinates of the internal pixel points to be predicted, the second estimation unit is used to process the vertical coordinate distortion sequence of the lower horizontal side and the coordinates of the internal pixel points to be predicted, the third estimation unit is used to process the vertical coordinate distortion sequence of the left longitudinal side and the coordinates of the internal pixel points to be predicted, and the fourth estimation unit is used to process the vertical coordinate distortion sequence of the right longitudinal side and the coordinates of the internal pixel points to be predicted.

[0103] In the horizontal coordinate distortion estimation model and the vertical coordinate distortion estimation model, the distortion of each internal pixel is predicted based on the distortion in four directions and the coordinates of the internal pixels. Each estimation unit combines the distortion sequence in one direction and the coordinates of the internal pixels to obtain an estimated value. The horizontal coordinate distortion and the vertical coordinate distortion of each internal pixel are obtained by combining the estimated values ​​in four directions.

[0104] The first estimation unit, the second estimation unit, the third estimation unit and the fourth estimation unit have the same structure, and all include: an LSTM layer, a Concat layer and an activation function layer; the LSTM layer is used to process the corresponding distortion sequence, the Concat layer is used to splice the LSTM layer output and the coordinates of the internal pixel points to be predicted to obtain a spliced ​​sequence, and the activation function layer is used to process the spliced ​​sequence using an activation function to obtain an estimated value.

[0105] The fully connected layer is used to weight the four estimated values ​​and predict the distortion.

[0106] The expression of the activation function layer is:

[0107] , where y is the estimated value of the activation function layer output, s i is the i-th element in the concatenated sequence, w i For i The weight of b i For i , f is the activation function, N is the number of elements in the concatenated sequence, and i is a positive integer.

[0108] The present invention adopts the LSTM layer to capture the distortion relationship of each element in the distortion sequence, and then splices the output features of the LSTM layer and the coordinates of the internal pixel points to be predicted through the Concat layer, so that the estimation unit considers the position of the internal pixel points and uses the activation function layer to predict the estimated value.

[0109] In this embodiment, the abscissa distortion estimation model and the ordinate distortion estimation model may be trained by a gradient descent method.

[0110] After step S5, the distortion amounts of the grid edge pixels and internal pixel positions on the grid skeleton sub-image are known, so the same pixel on the object surface image can be corrected. However, due to the process of step S3 (in each grid area of ​​the grid contour image, only the grid edge pixels inside the grid are retained), there are a few pixels on the boundary of the grid area whose distortion amounts are not predicted. In this embodiment, in order to make the object surface image correction more accurate, the distortion amounts of the pixels on the boundary can be kept consistent with the known distortion amounts of the nearest pixels in the neighborhood.

[0111] Embodiment 2, a detection image distortion correction system based on structured light, comprising: an image acquisition subsystem, a contour extraction subsystem, a separation subsystem, an ideal grid model construction subsystem, a distortion estimation subsystem and a correction subsystem;

[0112] The image acquisition subsystem is used to project grid-shaped structured light onto the surface of the object to be measured and acquire the projected surface image of the object;

[0113] The contour extraction subsystem is used to extract a grid contour map according to the pixel values ​​of the grid pixel points on the surface image of the object;

[0114] The separation subsystem is used to separate the mesh contour graph into a plurality of mesh skeleton subgraphs;

[0115] The ideal grid model construction subsystem is used to find the horizontal edge and the vertical edge with the smallest distortion value in multiple grid skeleton subgraphs to construct an ideal grid model;

[0116] The distortion estimation subsystem is used to estimate the distortion of each pixel point inside the mesh skeleton sub-image according to the difference in coordinates between the ideal mesh model and the mesh skeleton sub-image;

[0117] The correction subsystem is used to correct the same pixel point on the object surface image according to the pixel point distortion amount on the grid skeleton sub-image to obtain a corrected image.

[0118] The specific implementation process of Example 2 is the same as that of Example 1.

[0119] The present invention projects a grid-shaped structured light onto the surface of the object to be measured and collects images, extracts a grid outline using the pixel values ​​of the grid pixels, and further separates it into multiple grid skeleton sub-graphs. Then, the edge with the smallest distortion value is identified, an ideal grid model is constructed, and based on the difference in coordinates between the model and the grid skeleton sub-graph, the distortion amount of each pixel is estimated, and the corresponding pixel points on the surface image of the object are corrected, thereby obtaining a clear and accurate corrected image. In the present invention, there is no need to take images multiple times, which reduces the correction time, and does not require camera parameters, which reduces the parameters that need to be adjusted and considered in the correction process, and reduces the complexity of the correction process.

[0120] The present invention replaces the process of shooting calibration plates multiple times in the traditional method by projecting structured light and collecting images in a single time, greatly reducing the data collection time, eliminating the need for complex camera calibration process, eliminating the step of using a specific calibration plate, and not relying on the camera's internal parameters (such as focal length, principal point coordinates, distortion coefficient) and external parameters (camera posture), simplifying the parameters required for correction. By constructing an ideal grid model and comparing it with the actual grid, the distortion amount is directly estimated, avoiding the derivation of complex mathematical models. Structured light projection provides high-density, evenly distributed reference points, which can more accurately estimate the distortion of the entire image compared to the limited calibration points in the traditional method.

[0121] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for correcting detection image distortion based on structured light, characterized in that: The following steps are involved: S1, projecting a grid-shaped structured light onto the surface of the object to be measured, and collecting the projected surface image of the object; S2, extracting a grid contour map according to the pixel values ​​of the grid pixel points on the object surface image; S3, separating the mesh contour image into multiple mesh skeleton sub-images; S4, finding the horizontal edge and the vertical edge with the smallest distortion value in multiple mesh skeleton subgraphs, and constructing an ideal mesh model; S5, estimating the distortion amount of each pixel point inside the grid skeleton sub-image according to the difference in coordinates between the ideal grid model and the grid skeleton sub-image; S6. According to the pixel distortion amount on the grid skeleton sub-image, the same pixel on the object surface image is corrected to obtain a corrected image.

2. The method for correcting detected image distortion based on structured light according to claim 1, characterized in that: The S2 comprises the following sub-steps: S21, taking the pixel values ​​of the grid pixels on the surface image of the object as comparison pixel values; S22, calculating the similarity between the comparison pixel value and the pixel values ​​of other pixels; S23. When the similarity is greater than a similarity threshold, the corresponding pixel points are used as grid contour pixel points, and all grid contour pixel points constitute a grid contour map.

3. The method for correcting detected image distortion based on structured light according to claim 1, characterized in that: Specifically, S3 is as follows: in each grid area of ​​the grid outline graph, only the grid edge pixel points inside the grid are retained, and all the grid edge pixel points inside a grid constitute a grid skeleton subgraph.

4. The method for correcting detected image distortion based on structured light according to claim 1, characterized in that: The S4 comprises the following sub-steps: S41, calculating the curvature of each pixel in each grid skeleton sub-image; S42, obtaining four corner points of each grid skeleton sub-image according to the curvature of each pixel point; S43, using the four corner points as segmentation points, segmenting the corresponding grid skeleton subgraph into four edges, wherein the four edges include two horizontal edges and two vertical edges; S44, calculating the distortion degree value of each horizontal edge and each vertical edge, wherein the distortion degree value is the sum of the curvatures of each pixel point on an edge; S45, finding a horizontal edge with the smallest distortion value from all horizontal edges to obtain a horizontal reference edge; S46, finding a longitudinal edge with the smallest distortion value from all longitudinal edges to obtain a longitudinal reference edge; S47. Construct an ideal grid model based on the horizontal reference edge and the vertical reference edge.

5. The method for correcting detected image distortion based on structured light according to claim 1, characterized in that: The S5 comprises the following sub-steps: S51, taking the geometric center of the ideal grid model as the coordinate origin, performing coordinate translation on each network skeleton subgraph, wherein the geometric center of each network skeleton subgraph coincides with the coordinate origin after translation; S52, subtracting the upper lateral edge in the translated network skeleton sub-image from the upper lateral edge in the ideal grid model to obtain a coordinate distortion sequence of the upper lateral edge; S53, subtracting the lower lateral edge in the translated network skeleton sub-image from the lower lateral edge in the ideal grid model to obtain a coordinate distortion sequence of the lower lateral edge; S54, subtracting the left longitudinal edge in the translated network skeleton sub-image from the left longitudinal edge in the ideal grid model to obtain a coordinate distortion sequence of the left longitudinal edge; S55, subtracting the right longitudinal edge in the translated network skeleton sub-image from the right longitudinal edge in the ideal grid model to obtain a right longitudinal edge coordinate distortion sequence; S56, according to the upper horizontal edge coordinate distortion sequence, the lower horizontal edge coordinate distortion sequence, the left vertical edge coordinate distortion sequence and the right vertical edge coordinate distortion sequence, based on the horizontal coordinate distortion estimation model and the vertical coordinate distortion estimation model, estimate the distortion of each pixel point in the internal area of ​​the grid skeleton sub-image.

6. The method for correcting detected image distortion based on structured light according to claim 5, characterized in that: The S56 comprises the following sub-steps: S561, separating the abscissa distortion amounts and ordinate distortion amounts in the upper lateral edge coordinate distortion sequence, the lower lateral edge coordinate distortion sequence, the left longitudinal edge coordinate distortion sequence, and the right longitudinal edge coordinate distortion sequence to obtain an upper lateral edge abscissa distortion sequence, a lower lateral edge abscissa distortion sequence, a left longitudinal edge abscissa distortion sequence, and a right longitudinal edge abscissa distortion sequence, and an upper lateral edge ordinate distortion sequence, a lower lateral edge ordinate distortion sequence, a left longitudinal edge ordinate distortion sequence, and a right longitudinal edge ordinate distortion sequence; S562, using a horizontal coordinate distortion estimation model to process the upper horizontal side horizontal coordinate distortion sequence, the lower horizontal side horizontal coordinate distortion sequence, the left longitudinal side horizontal coordinate distortion sequence, the right longitudinal side horizontal coordinate distortion sequence, and the coordinates of the internal pixel points to be predicted, to obtain the horizontal coordinate distortion values ​​of the internal pixel points; S563, using a ordinate distortion estimation model to process the upper lateral side ordinate distortion sequence, the lower lateral side ordinate distortion sequence, the left longitudinal side ordinate distortion sequence, the right longitudinal side ordinate distortion sequence and the coordinates of the internal pixel points to be predicted, to obtain the ordinate distortion values ​​of the internal pixel points.

7. The method for correcting detected image distortion based on structured light according to claim 6, characterized in that: The abscissa distortion estimation model in S562 and the ordinate distortion estimation model in S563 both include: a first estimation unit, a second estimation unit, a third estimation unit, a fourth estimation unit and a fully connected layer; The input end of the fully connected layer is connected to the output end of the first estimation unit, the output end of the second estimation unit, the output end of the third estimation unit and the output end of the fourth estimation unit respectively, and the output end thereof serves as the output end of the abscissa distortion amount estimation model or the ordinate distortion amount estimation model; In the horizontal coordinate distortion estimation model, the first estimation unit is used to process the horizontal coordinate distortion sequence of the upper horizontal side and the coordinates of the internal pixel points to be predicted, the second estimation model is used to process the horizontal coordinate distortion sequence of the lower horizontal side and the coordinates of the internal pixel points to be predicted, the third estimation model is used to process the horizontal coordinate distortion sequence of the left longitudinal side and the coordinates of the internal pixel points to be predicted, and the fourth estimation model is used to process the horizontal coordinate distortion sequence of the right longitudinal side and the coordinates of the internal pixel points to be predicted; In the vertical coordinate distortion estimation model, the first estimation unit is used to process the vertical coordinate distortion sequence of the upper horizontal side and the coordinates of the internal pixel points to be predicted, the second estimation unit is used to process the vertical coordinate distortion sequence of the lower horizontal side and the coordinates of the internal pixel points to be predicted, the third estimation unit is used to process the vertical coordinate distortion sequence of the left longitudinal side and the coordinates of the internal pixel points to be predicted, and the fourth estimation unit is used to process the vertical coordinate distortion sequence of the right longitudinal side and the coordinates of the internal pixel points to be predicted.

8. The method for correcting detected image distortion based on structured light according to claim 7, characterized in that: The first estimation unit, the second estimation unit, the third estimation unit and the fourth estimation unit have the same structure, and all include: an LSTM layer, a Concat layer and an activation function layer; the LSTM layer is used to process the corresponding distortion sequence, the Concat layer is used to splice the LSTM layer output and the coordinates of the internal pixel points to be predicted to obtain a spliced ​​sequence, and the activation function layer is used to process the spliced ​​sequence using an activation function to obtain an estimated value.

9. The method for correcting detected image distortion based on structured light according to claim 8, characterized in that: The expression of the activation function layer is: , where y is the estimated value of the activation function layer output, s i is the i-th element in the concatenated sequence, w i For i The weight of b i For i , f is the activation function, N is the number of elements in the concatenated sequence, and i is a positive integer.

10. A detection image distortion correction system based on structured light, implemented based on the detection image distortion correction method based on structured light according to any one of claims 1 to 9, characterized in that: include: Image acquisition subsystem, contour extraction subsystem, separation subsystem, ideal grid model construction subsystem, distortion estimation subsystem and correction subsystem; The image acquisition subsystem is used to project grid-shaped structured light onto the surface of the object to be measured and acquire the projected surface image of the object; The contour extraction subsystem is used to extract a grid contour map according to the pixel values ​​of the grid pixel points on the surface image of the object; The separation subsystem is used to separate the mesh contour graph into a plurality of mesh skeleton subgraphs; The ideal grid model construction subsystem is used to find the horizontal edge and the vertical edge with the smallest distortion value in multiple grid skeleton subgraphs to construct an ideal grid model; The distortion estimation subsystem is used to estimate the distortion of each pixel point inside the mesh skeleton sub-image according to the difference in coordinates between the ideal mesh model and the mesh skeleton sub-image; The correction subsystem is used to correct the same pixel point on the object surface image according to the pixel point distortion amount on the grid skeleton sub-image to obtain a corrected image.

Images