A three-dimensional measurement method based on spatial coding

Abstract

A three-dimensional measurement method based on spatial coding includes the following steps: S1. Designing a spatially coded projection pattern in the form of a three-dimensional honeycomb structure and saving a set of intersection coding results; S2. Calibrating the intrinsic and extrinsic parameters of the camera and projector in a structured light system, projecting the spatially coded projection pattern using the projector, and synchronously capturing images using the camera; S3. Extracting an image skeleton from the captured image, extracting intersections based on the image skeleton, and classifying the intersections into two categories of intersection sets; S4. Drawing a bipartite graph based on the captured image and the intersection set using a connecting line traversal algorithm, and calculating an intersection neighborhood decoding result set based on the bipartite graph; S5. Using a matching algorithm, calculating an intersection pairing set based on the intersection neighborhood decoding result set and the intersection coding result set; and performing three-dimensional measurement of the intersection pairing set using a triangulation method, combined with the intrinsic and extrinsic parameters of the camera and projector, to generate a point cloud. This method solves the problem of non-uniform intersections caused by distortion in traditional grid coding and accelerates decoding speed.

Description

Technical Field

[0001] The present invention relates to three-dimensional measurement technology, and in particular to a three-dimensional measurement method based on space coding. Background Art

[0002] Thoracic and abdominal respiratory motion tracking technology typically uses a structured light system—comprising a projector and camera—as its core equipment. This system measures point clouds of the thoracic and abdominal surfaces in real time and tracks respiratory motion, thereby enabling the perception tasks of surgical navigation robots. Respiratory motion tracking systems typically utilize a projector to project a high-speed (typically greater than 120Hz) structured light pattern. A camera simultaneously captures the image, decodes the captured image to obtain the structured light encoding information, and uses this encoding information to rapidly perform 3D measurements in real time, thus enabling respiratory motion tracking.

[0003] For this respiratory tracking three-dimensional measurement system, considering that breathing causes irregular deformation of the human chest and abdominal surface and changes rapidly over time, the system can only use structured light patterns with spatial encoding; furthermore, projecting complex coding patterns will lead to high complexity of structured light decoding, which will also affect the real-time performance of respiratory motion tracking.

[0004] Spatially encoded structured light 3D measurement algorithms project a coded pattern and, during decoding, determine the encoding result based on the features within the feature point range. This makes them more suitable for 3D measurement of moving objects. Current methods for structured light spatial encoding include informal coding, DeBruijn sequence coding, and M-array coding. These different encoding methods use different calculation formulas to generate random sequences to ensure the uniqueness of each position in the spatial encoding. DeBruijn sequence coding is the most commonly used scheme. It generates a unique pseudo-random sequence based on a calculation formula and uses color as the encoding element to achieve uniqueness. Color coding can be further categorized into stripe coding, grid coding, and dense coding. Stripe-based and grid-based coding methods utilize one-dimensional and two-dimensional DeBruijn sequences, respectively, to design the corresponding patterns. Dense coding uses multiple projectors and cameras to achieve dense reconstruction of the entire surface based on the overlapping effects of the projected patterns on the object surface.

[0005] Existing methods for 3D reconstruction based on stripe coding only utilize spatial properties in one dimension, requiring coarser stripes to ensure code length and employing extreme constraints to complete 3D measurement, resulting in generally low accuracy. Existing grid-based spatial coding schemes, on the other hand, introduce two-dimensional spatial coding, ensuring keypoint detection and decoding, improving the accuracy of the decoded point cloud. However, these algorithms require longer code sequences, reducing decoding accuracy. Furthermore, existing decoding algorithms suffer from slow decoding speeds, making them unsuitable for real-time measurement.

[0006] Existing technical solutions can achieve good results for subjects with relatively small fluctuations. However, when measuring respiratory motion on the chest and abdominal surfaces, breathing-induced deformation of the ribs, flanks, and other areas of the body introduces significant noise and height variations. This can lead to significant errors in the decoding results of the aforementioned methods, resulting in significant errors in the resulting point cloud. This, in turn, affects the system's ability to track chest and abdominal breathing and subsequent algorithm processing.

[0007] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0008] The main purpose of the present invention is to overcome the defects of the above-mentioned background technology and provide a three-dimensional measurement method based on spatial coding.

[0009] To achieve the above object, the present invention adopts the following technical solutions:

[0010] A three-dimensional measurement method based on spatial coding comprises the following steps:

[0011] S1. Design a spatial coding projection pattern of a three-dimensional honeycomb structure grid and save the intersection coding result set E ref ;

[0012] S2. Calibrate the internal and external parameters of the camera and projector in the structured light system, use the projector to project the spatially coded projection pattern, and use the camera to achieve synchronous image acquisition I raw ;

[0013] S3. Collected image I raw Extracting image skeleton I skeleton , extract the intersection points according to the image skeleton and classify the intersection points into two categories to obtain two types of intersection point sets V1 and V2;

[0014] S4. Based on Image I raw With the intersection set V1, V2, a bipartite graph is drawn using the connection line traversal algorithm, and the intersection neighborhood decoding result set E is calculated based on the bipartite graph. dec ;

[0015] S5. Using the matching algorithm, decode the result set E according to the intersection neighborhood dec and the intersection coding result set E ref The intersection point pairing set P is obtained by calculation; and a triangulation method is used to perform three-dimensional measurement on the intersection point pairing set P in combination with the internal and external parameters of the camera and the projector to generate a point cloud.

[0016] Further:

[0017] Step S1 includes:

[0018] S11. Add one dimension to the two-dimensional color grid encoding method to complete the three-dimensional DeBruijn pseudo-random sequence encoding generation;

[0019] S12. Pseudo-random stripes are placed at intervals of 0°, 45°, and 135° from the positive x-axis according to a pseudo-random sequence. The stripe colors are generated according to the coded colors. Some intersections and adjacent edges are deleted according to predetermined rules to generate a honeycomb grid spatial coding pattern.

[0020] S13. Traverse the three connecting edges of each intersection and count the code of the intersection itself; for each intersection, traverse the intersections connected on its three edges and count the neighborhood coding results of the intersection; count all the intersections to obtain the intersection coding result set E ref .

[0021] Step S3 includes:

[0022] S31. Extracting image skeleton using image thinning algorithm I skeleton ;

[0023] S32. Traversing the image skeleton I skeleton , take its neighborhood for each point, if a predetermined number of pixels in the neighborhood have non-zero values ​​greater than the set threshold and the number of changes from 0 to 1 clockwise reaches the preset value, then it is determined as a candidate intersection point, and the candidate point set V is determined. Intersection ;

[0024] S33. Traverse the candidate point set V Intersection For each point in , calculate the angular position from the non-zero value in the neighborhood to the center point, and use the unsupervised clustering method to obtain the cluster center. Calculate the error between the cluster center and the reference result and classify them to obtain two types of intersection point sets V1 and V2.

[0025] In step S33, an intersection classification method is performed using an unsupervised learning-based or labeled supervised machine learning method instead of an unsupervised clustering method.

[0026] Step S4 includes:

[0027] S41. Initialize the bipartite graph G = {V, E}, define the adjacency matrix A(G) and the color matrix C(G), initialize the queue l for iteration, and randomly select a point v initial As the starting point of the breadth-first search algorithm BFS;

[0028] S42. Queue l pops the intersection point as the starting point v for this iteration i , use the connection line traversal algorithm to find and combine all iAdd the connected intersection points to queue l, record the corresponding edge number and color simultaneously, and update the matrix A(G) and C(G);

[0029] S43. Iterate step S42 until the queue l is empty, and output the matrices A(G) and C(G) of the bipartite graph.

[0030] S44. Traverse all intersections and use matrices A(G) and C(G) to calculate the intersection decoding result set D.

[0031] S45. Traverse all intersections and use the intersection decoding result set D and matrix A(G) to construct the decoding result set E dec .

[0032] In step S41 , a depth-first search or a set-based traversal algorithm is used instead of a breadth-first search algorithm.

[0033] Step S5 includes:

[0034] S51. Decoding result set E based on intersection neighborhood dec and the intersection neighborhood coding result set E ref Complete the intersection codeword matching and obtain the intersection pair set P, E dec With E ref The remaining intersection points are the set E dec,res and set E ref,res ;

[0035] S52. For the remaining intersection set E dec,res A point in the set E is selected ref,res If the Hamming distance is less than or equal to 2, the point pair is added to the set P, otherwise the intersection point is discarded;

[0036] S53. Repeat the steps of S52 until E dec,res is an empty set, and the intersection pair set P at this time is the decoding result;

[0037] S54. For each element in the intersection pair set P obtained in step S53, introduce the parameters of the calibrated camera and projector, calculate its position in the three-dimensional space, and establish point cloud data.

[0038] In step S52 , the vector similarity calculation is completed using cosine similarity or Euclidean distance instead of Hamming distance.

[0039] The method is used for thoracic and abdominal respiratory motion tracking, environmental detection of sweeping robots, or three-dimensional measurement of workpiece detection.

[0040] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the three-dimensional measurement method based on spatial coding.

[0041] The present invention has the following beneficial effects:

[0042] Starting with coded structured light, this paper proposes a novel structured light spatial encoding and decoding scheme. By redesigning the grid-like projection pattern and improving the decoding method to accelerate projection, the paper then finds the correspondence between the decoded results of key points and the encoded results in the structured light pattern. It then uses triangulation to calculate the three-dimensional measurement results, quickly and accurately generating a point cloud. In applications such as chest and abdominal respiratory motion tracking, this method can quickly and accurately generate chest and abdominal point clouds, enabling respiratory tracking.

[0043] This invention proposes for the first time the use of honeycomb grid-type structured light spatial coding as a projection pattern and designs a corresponding decoding algorithm. The main advantages include solving the problem of non-uniform intersections caused by distortion in traditional grid coding and accelerating the decoding speed. In addition, the invention uses a bipartite graph for decoding, and the coding itself contains redundant information, which has a certain error correction capability for erroneous decoding, making it more practical and robust than previous methods.

[0044] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of a three-dimensional measurement method based on spatial coding according to an embodiment of the present invention.

[0046] Figure 2 In the embodiment of the present invention, some intersection points and their adjacent edges are deleted to generate a honeycomb grid-type spatial coding map, and the effect diagrams before and after deletion are shown.

[0047] Figure 3 The present invention improves the intersection detection algorithm process and adds an intersection classification step in the embodiment of the present invention, which is a schematic diagram of the process flow for intersection detection of honeycomb grid patterns.

[0048] Figure 4 This is an example diagram of 7*7 neighborhood pixels in two intersection image skeletons in an embodiment of the present invention.

[0049] Figure 5 Schematic diagram of path numbering for two types of intersections in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0051] See Figure 1The present invention provides a three-dimensional measurement method based on spatial coding, which can be applied to applications such as rapid three-dimensional measurement during respiratory tracking. The method includes the following steps:

[0052] S1. Design a spatial coding projection pattern of a three-dimensional honeycomb structure grid and save the intersection coding result set E ref ;

[0053] S2. Calibrate the internal and external parameters of the camera and projector in the structured light system, use the projector to project the spatially coded projection pattern, and use the camera to achieve synchronous image acquisition I raw ;

[0054] S3. Collected image I raw Extracting image skeleton I skeleton , according to the image skeleton I skeleton Extract the intersection points and classify them into two categories to obtain two types of intersection point sets V1 and V2;

[0055] S4. Based on Image I raw With the intersection set V1, V2, a bipartite graph is drawn using the connection line traversal algorithm, and the intersection neighborhood decoding result set E is calculated based on the bipartite graph. dec This step uses the connection line traversal algorithm and bipartite graph characteristics to calculate the intersection neighborhood decoding result set E based on the image in step S2 and the intersection point set V1, V2 in step S3 dec ;

[0056] S5. Using the matching algorithm, decode the result set E according to the intersection neighborhood dec and the intersection coding result set E ref A set of intersection pairs P is calculated and then three-dimensionally measured using triangulation techniques, combined with the internal and external parameters of the camera and projector, to generate a point cloud. This can be used, for example (but not limited to), to create point cloud data for a human chest and abdomen model during chest and abdominal respiratory motion tracking.

[0057] Specific embodiments of the present invention are further described below.

[0058] This embodiment uses the DeBruijn sequence encoding of the B(k,n) table, where the k element is used to form a cyclic sequence. All sequences consisting of k elements of length n are included in the DeBruijn sequence and appear only once. Considering that when designing spatially coded structured light projection patterns based on DeBrujin sequence encoding, the number of projection colors and the length of the coding neighborhood are usually selected based on the projection image resolution and the object being measured. For this embodiment, the pseudo-random encoding uses k=5 and n=3 to achieve a sequence length that meets the requirements for sequence generation:

[0059] L = k n +n-1=127

[0060] Since the existing grid coding methods are all squares, that is, the intersections are generated by the intersection of four lines, the intersection decoding will not be unique due to the problem of collecting images during decoding. This example designs a honeycomb space coding, and its generation method is not the same. On the basis of the original horizontal and vertical grids, the vertical stripes are tilted at an angle, and then stripes in another direction are added at a symmetrical angle. It should be noted that they are all encoded using the DeBruijn sequence. At this time, there are connecting lines in six directions for each intersection, and there is a lot of redundant information, which is not conducive to the design of the decoding algorithm. Therefore, a heuristic algorithm is used to delete some intersections and their adjacent edges to generate a honeycomb grid space coding map. The effects before and after deletion are shown as follows. Figure 2 Obviously, the result after deletion still has the uniqueness of spatial coding, and can be used as a spatial coding pattern projection and decoded normally.

[0061] The step of designing a three-dimensional honeycomb structure grid-type spatial coding projection pattern in step S1 includes:

[0062] S11. Select the number of projection colors and the length of the coding neighborhood to complete the three-dimensional DeBruijn pseudo-random sequence code generation.

[0063] S12. Based on the pseudo-random sequence, pseudo-random lines are placed at intervals of 0°, 45°, and 135°. The line colors are generated according to the coded colors, generating a honeycomb grid spatial coding pattern. Specifically, for the coding pattern, there are two types of intersections, with the angles between the connection and the intersection being [0°, 135°, 225°] and [45°, 180°, 315°], respectively. There are no directly connected edges within the two types of intersections, and for each intersection, there are three edges between the two types (excluding the edges).

[0064] S13. Traverse the three connected edges of each intersection and count the code of the intersection itself (length is 3); for each intersection, traverse the intersections connected on its three edges and count the neighborhood coding results of the intersection (length is 9). Count all the intersections and get the coding result E ref .

[0065] After determining the spatially coded projection pattern, this example first considers that if spatially coded 3D measurement is required, the camera and projector system must be calibrated in the specific implementation. In this embodiment, Zhang Zhengyou's checkerboard calibration method is used. The calibration parameters are used to restore the point cloud using triangulation. After calibration, the position of the camera and projector remains unchanged to complete the projection shooting. The captured image is I raw The steps in step S2 include:

[0066] S21. Fix the relative position of the camera and projector, and use the calibration method to obtain the internal and external parameters of the camera and projector.

[0067] S22. The projector projects the spatial coding pattern designed in S1, and the camera synchronously captures and saves the image I raw .

[0068] When decoding spatially coded images, image processing is usually used to complete the detection and decoding of feature points. raw When directly counting the generated intersections, multiple intersections may be detected due to the width of the projected stripes, resulting in ambiguity and affecting the measurement accuracy. This implementation improves the intersection detection algorithm process and adds an intersection classification step for intersection detection of honeycomb grid patterns. The process and a brief schematic diagram are shown below. Figure 3 First, choosing to extract the image skeleton of the captured image and then perform intersection detection will improve the accuracy. The image skeleton is first binarized and then iteratively eroded at non-zero positions until only skeleton lines with a width of 1 remain. This implementation uses the Zhang-Suen algorithm to complete the skeleton extraction and obtain the image skeleton I skeleton Then, the intersection point is extracted, and each pixel in the image skeleton is traversed. If the non-zero value exists and the number of changes meets the requirements, it is determined as a candidate point. In the implementation, the candidate points include two types of intersection points. This implementation considers using the statistical learning method to complete the intersection classification: take the 7*7 neighborhood pixels of the intersection point (the 7*7 neighborhood pixels in the two intersection image skeletons are examples of Figure 4 ), count the number and location of non-zero values ​​(excluding the center point), and use the K-means clustering algorithm to divide them into three clusters in an unsupervised manner. Then calculate the errors between the x-axis angles of the three clusters and the reference angles [0°, 135°, 225°] and [45°, 180°, 315°], respectively. The smaller error is used as the classification result and updated to the intersection sets V1 and V2 respectively. In summary, the method for classifying intersections in the image in step S3 includes the following steps:

[0069] S31. Extracting image skeleton using image thinning algorithm I skeleton ;

[0070] S32. Traverse i skeleton For each point, take its neighborhood. If there are 8 pixels in the neighborhood with a non-zero value greater than the set 3 and the number of changes from 0 to 1 clockwise is 3, it is defined as a candidate intersection point. The candidate point set is defined as V Intersection ;

[0071] S33. Traverse V Intersection For each point in the dataset, calculate the angular position of the non-zero value in the neighborhood to the center point and use an unsupervised clustering method to obtain the cluster center. Calculate the error between the cluster center and the reference result and classify them, finally obtaining two types of intersection point sets V1 and V2;

[0072] After obtaining two different sets of intersection points V1 and V2, due to the characteristics of the pattern design in step S1, the two types of intersection points together form a graph. Since the two associated vertices i and j in the graph belong to these two different intersection points V1 and V2, respectively, the graph is a bipartite graph. To complete the neighborhood decoding of the point cloud, we need to first construct a bipartite graph using image processing methods.

[0073] There is no unique algorithm for constructing a bipartite graph. This implementation uses a breadth-first search (BFS) algorithm to construct a bipartite graph G = {V, E}, where V = V1 ∪ V2. Furthermore, this implementation uses an adjacency matrix to represent the graph. Assume that the two intersection sets V1 and V2 have n and m intersections, respectively. Therefore, there are n + m vertices in the graph. Therefore, the adjacency matrix can be initialized as:

[0074]

[0075] Similarly, initialize a color matrix of the same size to record the colors of the edges in the bipartite graph, denoted as C(G).

[0076]

[0077] After initialization, the relationship between the intersection points is obtained by image processing. At the beginning of the iteration, a point v is randomly selected from the intersection point set V obtained in step S3. initial Add to queue l. Each time you traverse, take the first element in the queue as the starting point of this traversal, denoted as v i For v i , find all non-zero points in the 3*3 neighborhood of the skeleton image as the starting point of the iteration; take three adjacent pixels in the 3*3 neighborhood of the skeleton image each time in the direction away from the intersection according to the corresponding angle, and let the non-zero pixels among them be the starting point of the next iteration; loop the iterative steps until the intersection is found or the number of iterations is exhausted, if the intersection v found j Located at the intersection with the starting point v i In the set V1 (or V2), the traversal result is abandoned; if the intersection point v i ,v j If they are in the intersection set V1 and V2 respectively, they are added to the traversal list; at the same time, v is calculated j With v i The angle between them is expressed as the path number using the connecting line with the smallest angle difference. If the path numbers of the points at both ends of the edge are the same, it is expressed as:

[0078] a i,j =a j,i ,a i,j ∈{1,2,3},

[0079] For two intersection point sets V1, V2, the path number is as follows Figure 5 As shown. According to all the pixels in the path, raw The average value of the pixel color in decodes the color information of the edge and updates the matrix C(G):

[0080] c i,j =c j,i ,c i,j ∈{0,1,2,3,4}.

[0081] When the queue l is empty, it means that the generation of the bipartite graph is completed. At this time, for each intersection v i , find v according to the order of the edges connected in the adjacency matrix i Connect the three intersection points v j ,v k ,v l And record, and take the corresponding results in the decoding matrix in order as the decoding result of the intersection itself, record the intersection v i Self-decoding result d i :

[0082] d i =(c i,j ,c i,k ,c i,l ),

[0083] All d i The set of is denoted as D.

[0084] Traverse all the intersection points again, and for the intersection point v i Connect the three intersection points v j ,v k ,v l The decoding result d j ,d k ,d l Splice as the intersection v i Neighborhood decoding result e i :

[0085] e i =(d j ,d k ,d l ).

[0086] The neighborhood decoding results of all intersection points after traversal are made into a set, which is recorded as the decoding result set E dec Using the steps described above, the decoding process of the intersection points V1 and V2 in step S3 in step S4 is implemented in this embodiment and includes the following steps:

[0087] S41. Initialize the bipartite graph G = {V, E}, define the adjacency matrix A(G), the color matrix C(G), initialize the queue l for iteration, and randomly select a point v initial As the starting point of the breadth-first search algorithm (BFS).

[0088] S42. Queue l pops the intersection point as the starting point v for this iteration i , use the connection line traversal algorithm to find and combine all i The connected intersection points are added to the queue l, the number and color of the corresponding edges are recorded synchronously, and the matrices A(G) and C(G) are updated.

[0089] S43. Iterate step S42 until queue l is empty, and output the A(G) and C(G) matrices of the bipartite graph.

[0090] S44. Traverse all intersections and use A(G) and C(G) to calculate the intersection decoding result set D.

[0091] S45. Traverse all intersections and use the intersection's own decoding result set D and A(G) to construct the decoding result set E dec .

[0092] Based on the intersection neighborhood decoding result set E in step S4 dec The intersection neighborhood coding result set E in step S1 ref , the intersection matching can be completed. First, for each E dec Find the points in E ref Match the codewords that are exactly the same in E, and get the intersection point set recorded as P; then dec The remaining points E dec,res Find it in E ref The remaining point E ref,res The closest Hamming distance is taken as the matching intersection. If the closest coding Hamming distance is less than 2, the set P is iteratively updated, otherwise the intersection is abandoned. The iterative algorithm is repeated until E dec,res After completion, the set P is used to combine the internal and external parameters of the camera and projector to obtain the point cloud data of all intersections, and the point cloud of the human chest and abdomen model in three-dimensional space is established, thereby achieving accurate respiratory motion tracking. The steps of the method are as follows:

[0093] S51. The intersection neighborhood decoding result set E according to step S5 dec The intersection neighborhood coding result set E in step S1 ref Complete the intersection codeword matching and obtain the intersection pair set P, E dec With E ref The remaining intersection points are E dec,res , E ref,res .

[0094] S52. The remaining intersection set E in step S51 dec,res Select an intersection point in E ref,res If the Hamming distance is less than or equal to 2, the point pair can be added to the set P, otherwise the intersection point is discarded.

[0095] S53. Repeat the steps of S52 until E dec,res is an empty set, and the intersection pair set P at this time is the decoding result.

[0096] S54. For each element in the intersection pair set P obtained in step S53, introduce the camera projector calibration parameters in step S2, calculate its position in three-dimensional space, and establish point cloud data of the human chest and abdomen model.

[0097] The coding and decoding scheme of the embodiment of the present invention can be mainly applied to the visible light structured light spatial coding method.

[0098] In an alternative embodiment, there are multiple optional schemes for the intersection classification method in step S33, such as unsupervised learning, labeled supervised machine learning methods, etc. The present invention adopts the unsupervised clustering algorithm K-means clustering to achieve classification.

[0099] In an alternative embodiment, the traversal algorithm used in step S42 may be a variety of traversal methods, such as depth-first search, breadth-first search, set-based traversal, etc. The breadth-first search method is the most intuitive and effective, and the traversal can be completed in a multi-threaded manner, which is faster.

[0100] In an alternative embodiment, the vector similarity calculation algorithm used in step S52 includes multiple algorithms, such as cosine similarity, Euclidean distance, Hamming distance, etc. The Hamming distance is used to complete distance calculation and matching, and the noise margin of the code is calculated.

[0101] In some embodiments of the present invention, a structured light device is used to complete pattern projection and camera acquisition, key points are detected in the acquired image, and image processing methods are used to decode the neighborhood information of each key point and pair it with a reference codeword, and then the triangulation principle is used to complete point cloud reconstruction.

[0102] In some embodiments of the present invention, improvements are made starting from the coding pattern, and a new dimension is added to the existing two-dimensional grid space coding pattern. In this way, the multiple intersection problem existing in the original intersection detection can be solved through projection pattern design; in addition, in the connection line traversal algorithm in the decoding algorithm, a bipartite graph network is naturally formed according to the designed projection pattern, which avoids the problem of incorrect identification of intersection connection lines to a certain extent.

[0103] This invention can be applied in any scenario requiring three-dimensional measurement, such as rapid tracking of respiratory motion, environmental monitoring by sweeping robots, and factory workpiece detection. It addresses the problems of existing three-dimensional measurement algorithms based on spatially coded structured light, such as slow algorithm execution, poor robustness, and frequent decoding errors.

[0104] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0105] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0106] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the method described above.

[0107] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0108] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0109] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0111] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0112] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0113] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0114] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0115] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0116] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that, without departing from the scope of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use of the same should be considered to fall within the scope of protection of the present invention.

Claims

1. A three-dimensional measurement method based on spatial coding, characterized in that: The steps include: S1. Design a spatial coding projection pattern in the form of a three-dimensional honeycomb structure grid and save the intersection coding result set ; S2. Calibrate the internal and external parameters of the camera and projector in the structured light system, use the projector to project the spatially coded projection pattern, and use the camera to synchronously capture the image ; S3. Collected images Extract image skeleton , extract the intersection points according to the image skeleton and classify the intersection points into two categories to obtain two types of intersection points ; S4. Image-based and intersection set , use the connection line traversal algorithm to draw a bipartite graph, and calculate the intersection neighborhood decoding result set based on the bipartite graph ; S5. Using the matching algorithm, decode the result set according to the intersection neighborhood Intersection coding result set Calculate the intersection pairing set ; Using the triangulation method, combined with the internal and external parameters of the camera and the projector, the intersection point pairing set Perform three-dimensional measurement and generate point clouds; Step S1 includes: S11. Add one dimension to the two-dimensional color grid encoding method to complete the three-dimensional DeBruijn pseudo-random sequence encoding generation; S12. According to the pseudo-random sequence, the angle with the positive direction of the x-axis Pseudo-random stripes are placed at intervals in three directions. The stripe colors are generated according to the coded colors. Some intersections and adjacent edges are deleted according to predetermined rules to generate a honeycomb grid space coding pattern. S13. Traverse the three connected edges of each intersection and count the code of the intersection itself; for each intersection, traverse the intersections connected on its three edges and count the neighborhood coding results of the intersection; count all the intersections to obtain the intersection coding result set .

2. The three-dimensional measurement method based on spatial coding according to claim 1, characterized in that: Step S3 includes: S31. Extracting image skeleton using image thinning algorithm ; S32. Traverse the image skeleton , take its neighborhood for each point, if a predetermined number of pixels in the neighborhood have non-zero values ​​greater than the set threshold and the number of changes from 0 to 1 clockwise reaches the preset value, then it is determined as a candidate intersection point, and the candidate point set is determined ; S33. Traverse the candidate point set For each point in the neighborhood, calculate the angle position from the non-zero value to the center point, and use the unsupervised clustering method to obtain the cluster center, calculate the error between the cluster center and the reference result and classify them to obtain two types of intersection points. .

3. The three-dimensional measurement method based on spatial coding according to claim 2, characterized in that: In step S33, an unsupervised learning-based or labeled supervised machine learning method is used instead of an unsupervised clustering method to perform intersection classification.

4. The three-dimensional measurement method based on spatial coding according to any one of claims 1 to 3, characterized in that: Step S4 includes: S41. Initialize the bipartite graph , define the adjacency matrix and the color matrix , initialize the queue For iteration, randomly select a point As the starting point of the breadth-first search algorithm BFS; S42. Queue The intersection of the stack is used as the starting point of this iteration , use the connection line traversal algorithm to find and combine all Connected intersections are added to the queue In the process, the corresponding edge number and color are recorded synchronously, and the matrix is ​​updated. ; S43. Iterate step S42 until The queue is empty, output the matrix of the bipartite graph ; S44. Traverse all intersections and use the matrix Calculate the intersection point's own decoding result set ; S45. Traverse all intersections and use the intersections themselves to decode the result set With the matrix Constructing the intersection neighborhood decoding result set .

5. The three-dimensional measurement method based on spatial coding according to claim 4, characterized in that: In step S41 , a depth-first search or a set-based traversal algorithm is used instead of a breadth-first search algorithm.

6. The three-dimensional measurement method based on spatial coding according to any one of claims 1 to 3, characterized in that: Step S5 includes: S51. Decoding result set based on intersection neighborhood Intersection coding result set Complete the intersection codeword matching and obtain the intersection pair set and The remaining intersection points in are the set and collection ; S52. For the remaining intersection points A point in the set If the Hamming distance is less than or equal to 2, the point pair is added to the set. , otherwise the intersection point is discarded; S53. Repeat the steps of S52 until Is an empty set, the intersection pair set That is the decoding result; S54. For the intersection point pair set obtained in step S53 For each element in the image, the parameters of the calibrated camera and projector are introduced, its position in the three-dimensional space is calculated, and point cloud data is established.

7. The three-dimensional measurement method based on spatial coding according to claim 6, characterized in that: In step S52 , cosine similarity or Euclidean distance is used instead of Hamming distance to complete vector similarity calculation.

8. The three-dimensional measurement method based on spatial coding according to any one of claims 1 to 3, characterized in that: Used for 3D measurement of chest and abdominal respiratory motion tracking, sweeping robot environment detection, or workpiece detection.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the three-dimensional measurement method based on spatial coding as claimed in any one of claims 1 to 8 is implemented.

Images