Processing method and device of three-dimensional scanning and three-dimensional scanning equipment

Abstract

The application discloses a three-dimensional scanning processing method and device and a three-dimensional scanning device. The method comprises the following steps: projecting a plurality of lines to the surface of a measured object by a pattern projector; acquiring two-dimensional images of the surface of the measured object by three cameras, so as to obtain three two-dimensional images; determining matching point pairs between the three two-dimensional images, so as to obtain three groups of matching point pairs; verifying matching consistency between the matching point pairs; and performing three-dimensional reconstruction on the matching point pairs with matching consistency, so as to obtain three-dimensional points of the surface of the measured object. By the application, the scanning efficiency of binocular laser scanning in the related art cannot meet some scenes with high scanning efficiency requirements. The method for improving the scanning efficiency is to increase the number of scanning lines. However, in the binocular scanning system, increasing the number of scanning lines will result in a sharp decline in matching accuracy.

Description

Technical Field

[0001] This application relates to the field of 3D scanning technology, and more specifically, to a 3D scanning processing method, apparatus, and 3D scanning device. Background Technology

[0002] With the increasing maturity of handheld 3D scanning technology, its application in industry is becoming more and more widespread. In practical scanning scenarios, some industrial objects being measured are often large-scale industrial parts, requiring high scanning efficiency. The most common existing scanning solution is binocular laser scanning, but the laser line count of binocular laser scanning is less than 20 lines, which cannot meet the scanning efficiency requirements of some scenarios. To improve scanning efficiency, the number of scan lines needs to be increased, but in a binocular stereo vision system, increasing the number of scan lines leads to a decrease in matching accuracy.

[0003] For scenarios where the scanning efficiency of binocular laser scanning in related technologies cannot meet the requirements of high scanning efficiency, the method to improve scanning efficiency is to increase the number of scan lines. However, increasing the number of scan lines in a binocular scanning system will lead to a sharp drop in matching accuracy, and no effective solution has been proposed yet. Summary of the Invention

[0004] The main objective of this application is to provide a three-dimensional scanning processing method, apparatus, and three-dimensional scanning device to solve the problem that the scanning efficiency of binocular laser scanning in related technologies cannot meet the requirements of some high scanning efficiency scenarios. The method to improve scanning efficiency is to increase the number of scanning lines. However, in a binocular scanning system, increasing the number of scanning lines will lead to a sharp decrease in matching accuracy.

[0005] To achieve the above objectives, according to one aspect of this application, a method for processing three-dimensional scanning is provided. The method includes: projecting multiple lines onto the surface of a test object using a pattern projector; acquiring two-dimensional images of the test object surface using three cameras to obtain three corresponding two-dimensional images; determining matching point pairs between each pair of the three two-dimensional images to obtain three sets of matching point pairs; verifying the matching consistency between the matching point pairs; and performing three-dimensional reconstruction on the matching point pairs with matching consistency to obtain three-dimensional points on the test object surface.

[0006] Further, determining the pairwise matching point pairs between the three frames of two-dimensional images yields three sets of matching point pairs, including: acquiring multiple lines in the three frames of two-dimensional images, wherein each line is composed of multiple pixels; selecting a pixel on a line in one frame of two-dimensional images as a selected point; determining multiple candidate matching points in another frame of two-dimensional images that match the selected point; performing three-dimensional reconstruction on the selected point and the multiple candidate matching points based on the triangulation principle to obtain multiple first candidate three-dimensional points; determining the first candidate three-dimensional points that meet preset conditions as second candidate three-dimensional points, wherein there are multiple second candidate three-dimensional points; and forming the matching point pairs with the selected points and the candidate matching points corresponding to the second candidate three-dimensional points.

[0007] Furthermore, before verifying the matching consistency among the three sets of matching point pairs, the method further includes: obtaining multiple light planes corresponding to the second candidate three-dimensional point, wherein the light plane is the light plane corresponding to the selected point; and determining the target light plane corresponding to the selected point from the multiple light planes.

[0008] Further, determining the target light plane corresponding to the selected point from the plurality of light planes includes: acquiring a plurality of light planes corresponding to a plurality of pixels on the same line as the selected point; calculating the number of times each light plane appears, and taking the light plane with the most appearances as the target light plane.

[0009] Further, verifying the matching consistency among the three sets of matching point pairs includes: taking the matching point corresponding to the target light plane as the target matching point; forming a target matching point pair with the selected point and the target matching point to obtain three sets of target matching point pairs; and verifying the consistency among the three sets of target matching point pairs.

[0010] Further, the three two-dimensional images are a first two-dimensional image, a second two-dimensional image, and a third two-dimensional image, respectively. Verifying the matching consistency among the three sets of target matching point pairs includes: obtaining a selected point in the first two-dimensional image, and a target light plane one determined by matching the selected point with the first two-dimensional image and the second two-dimensional image; obtaining a selected point in the first two-dimensional image, and a target light plane two determined by matching the selected point with the first two-dimensional image and the third two-dimensional image; obtaining the target matching point of the selected point in the first two-dimensional image in the second two-dimensional image; obtaining the target light plane three determined by matching the target matching point in the second two-dimensional image with the second two-dimensional image and the third two-dimensional image; determining whether the target light plane one, the target light plane two, and the target light plane three are the same light plane; when the target light plane one, the target light plane two, and the target light plane three are the same light plane, then the three sets of target matching point pairs have matching consistency.

[0011] Further, taking a pixel on a line in a frame of a two-dimensional image as a selected point, and determining multiple candidate matching points in another frame of a two-dimensional image that match the selected point, includes: obtaining the epipolar equation of the selected point to the other frame of the two-dimensional image; and taking the intersection of the epipolar equation with multiple lines in the other frame of the two-dimensional image as the multiple candidate matching points.

[0012] To achieve the above objectives, according to another aspect of this application, a three-dimensional scanning device is provided. The device includes three cameras, which are combined in pairs to form three binocular systems. These three binocular systems are used to acquire two-dimensional images of the surface of an object under test, resulting in three frames of two-dimensional images. Matching point pairs are determined between each pair of the three frames of two-dimensional images, resulting in three sets of matching point pairs. The matching consistency between the matching point pairs is verified. Three-dimensional reconstruction is performed on the matching point pairs with matching consistency to obtain three-dimensional points on the surface of the object under test.

[0013] To achieve the above objectives, according to another aspect of this application, a three-dimensional scanning processing apparatus is provided. The apparatus includes: a projection unit for projecting multiple lines onto the surface of a measured object using a pattern projector; an acquisition unit for acquiring two-dimensional images of the surface of the measured object using three cameras, corresponding to three frames of two-dimensional images; a first determination unit for determining matching point pairs between each pair of the three frames of two-dimensional images, corresponding to three sets of matching point pairs; a verification unit for verifying the matching consistency between the matching point pairs; and a reconstruction unit for performing three-dimensional reconstruction on the matching point pairs with matching consistency to obtain three-dimensional points on the surface of the measured object.

[0014] Further, the determining unit includes: a first acquisition subunit, used to acquire multiple lines in the three frames of two-dimensional images, wherein the lines are composed of multiple pixels; a first determining subunit, used to select a pixel on a line in one frame of two-dimensional image as a selected point, and determine multiple candidate matching points in another frame of two-dimensional image that match the selected point; a reconstruction subunit, used to perform three-dimensional reconstruction of the selected point and the multiple candidate matching points based on the triangulation principle to obtain multiple first candidate three-dimensional points; a second determining subunit, used to determine the first candidate three-dimensional points that meet preset conditions as second candidate three-dimensional points, wherein there are multiple second candidate three-dimensional points; and a first composition subunit, used to form the matching point pair by combining the candidate matching points corresponding to the second candidate three-dimensional points with the selected point.

[0015] Furthermore, the device further includes: an acquisition unit, configured to acquire multiple light planes corresponding to the second candidate three-dimensional point before verifying the matching consistency between the three sets of matching point pairs, wherein the light plane is the light plane corresponding to the selected point; and a second determination unit, configured to determine the target light plane corresponding to the selected point from the multiple light planes.

[0016] Furthermore, the second determining unit includes: a second acquisition subunit, used to acquire multiple light planes corresponding to multiple pixels on the same line as the selected point; and a calculation subunit, used to calculate the number of times each light plane appears, and take the light plane with the most appearances as the target light plane.

[0017] Furthermore, the verification unit includes: taking the matching point corresponding to the target light plane as the target matching point; a second composition subunit, used to form target matching point pairs with the selected point and the target matching point to obtain three sets of target matching point pairs; and a verification subunit, used to verify the consistency between the three sets of target matching point pairs.

[0018] Further, the three two-dimensional images are a first two-dimensional image, a second two-dimensional image, and a third two-dimensional image, respectively. The verification subunit includes: a first acquisition module, used to acquire a selected point in the first two-dimensional image, and a target light plane one determined by matching the selected point with the first two-dimensional image and the second two-dimensional image; a second acquisition module, used to acquire a selected point in the first two-dimensional image, and a target light plane two determined by matching the selected point with the first two-dimensional image and the third two-dimensional image; a third acquisition module, used to acquire a target matching point of the selected point in the first two-dimensional image in the second two-dimensional image; a fourth acquisition module, used to acquire a target light plane three determined by matching the target matching point in the second two-dimensional image with the second two-dimensional image and the third two-dimensional image; and a judgment module, used to judge whether the target light plane one, the target light plane two, and the target light plane three are the same light plane; when the target light plane one, the target light plane two, and the target light plane three are the same light plane, then the three sets of target matching point pairs have matching consistency.

[0019] Further, the first determining subunit includes: a fifth acquisition unit, used to acquire the epipolar equation of the selected point to the other frame of the two-dimensional image; and to use the intersection of the epipolar equation with multiple lines in the other frame of the two-dimensional image as the multiple candidate matching points.

[0020] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is provided, the storage medium including a stored program, wherein the program executes the three-dimensional scanning processing method described in any one of the preceding claims.

[0021] To achieve the above objectives, according to another aspect of this application, a processor is provided for running a program, wherein the program executes the three-dimensional scanning processing method described in any one of the preceding claims.

[0022] This application employs the following steps: projecting multiple lines onto the surface of the object under test using a pattern projector; acquiring two-dimensional images of the object's surface using three cameras, resulting in three frames of two-dimensional images; determining matching point pairs between each pair of the three frames of two-dimensional images, resulting in three sets of matching point pairs; verifying the matching consistency between the matching point pairs; and performing three-dimensional reconstruction on the matching point pairs with matching consistency to obtain the three-dimensional points of the object's surface. This addresses the issue in related technologies where the scanning efficiency of binocular laser scanning cannot meet the requirements of some high-scanning-efficiency scenarios. While increasing the number of scan lines can improve scanning efficiency, in binocular scanning systems, increasing the number of scan lines leads to a sharp decrease in matching accuracy. By acquiring two-dimensional images of the object's surface using three cameras, obtaining three frames of two-dimensional images, matching each pair of the three frames of two-dimensional images to obtain three sets of matching point pairs, and performing three-dimensional reconstruction on the matching point pairs with matching consistency to obtain the three-dimensional points of the object's surface, the accuracy of matching is improved. Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1 This is a flowchart of a three-dimensional scanning processing method provided according to an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of an optional three-frame two-dimensional image provided according to an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of an optional 3D scanning device provided according to an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of a three-dimensional scanning processing apparatus provided according to an embodiment of this application. Detailed Implementation

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of a three-dimensional scanning processing method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0032] Step S101: Project multiple lines onto the surface of the object being measured using a pattern projector.

[0033] For example, multiple scan lines (e.g., laser scan lines) are projected onto the surface of the object whose 3D model needs to be built using a pattern projector of a 3D scanning device.

[0034] Step S102: Three two-dimensional images of the surface of the object being measured are acquired by three cameras, resulting in three corresponding two-dimensional images.

[0035] Three 2D images of the object's surface to be modeled using three cameras (e.g., Cam L, Cam M, and Cam R) of a 3D scanning device are acquired, resulting in three corresponding 2D image frames (e.g., ...). Figure 2(The three two-dimensional images shown are the first, second, and third two-dimensional images.) In the embodiments of this application, the three-dimensional scanning device is a handheld three-dimensional scanner that can move relative to the object being measured for scanning. Preferably, the three cameras acquire data synchronously, that is, the three cameras acquire data synchronously at the first time, synchronously at the second time, until the scanning is completed, ensuring the correspondence of the images acquired by the three cameras. The three cameras can also be asynchronous, but it is necessary to ensure that the acquisition time interval is extremely small and the position of the three-dimensional scanning device relative to the object being measured remains almost unchanged. If the three-dimensional scanning device is used in a fixed position each time the object being measured is acquired, that is, the three-dimensional scanning device is fixed in the first position to acquire one part of the object being measured, and then fixed in the second position to acquire another part of the object being measured, until the object being measured is scanned, then it is not limited whether the acquisition of the three cameras is synchronous.

[0036] Step S103: Determine the matching point pairs between each pair of the three two-dimensional images, resulting in three sets of matching point pairs.

[0037] Perform pairwise matching on three frames of two-dimensional images to obtain three sets of matching point pairs. For example, Figure 2 As shown, point A(1,2) on the first frame of the two-dimensional image, and the matching point coordinates of A(1,2) in the second frame of the two-dimensional image are B1(2,3); then the matching point pair between the first and second frames of the two-dimensional image is A(1,2) and B1(2,3); the matching point coordinates of A(1,2) in the third frame of the two-dimensional image are C(1,3); then the matching point pair between the first and third frames of the two-dimensional image is A(1,2) and C(1,3); the matching point of B1(2,3) on the second frame of the two-dimensional image is C(1,3); then the matching point pair between the second and third frames of the two-dimensional image is B1(2,3) and C(1,3).

[0038] Step S104: Verify the matching consistency between matching point pairs.

[0039] Verify the consistency of the three sets of matching points obtained.

[0040] Step S105: Perform three-dimensional reconstruction on the matching point pairs with matching consistency to obtain three-dimensional points on the surface of the object being measured.

[0041] When three sets of matching point pairs have consistent matching, the point is reconstructed in three dimensions to obtain the three-dimensional points of the object whose three-dimensional model needs to be built. The three-dimensional model of the object is then constructed using these three-dimensional points.

[0042] Through the above steps, two-dimensional images of an object are collected by three cameras to obtain three frames of two-dimensional images. The three frames of two-dimensional images are matched in pairs to obtain matching point pairs. Three-dimensional reconstruction is performed on matching point pairs with matching consistency, thereby obtaining a three-dimensional model of the object, which improves the accuracy of three-dimensional reconstruction matching.

[0043] Optionally, in the three-dimensional scanning processing method provided in the embodiments of the present application, determining matching point pairs between every two of the three frames of two-dimensional images to correspondingly obtain three groups of matching point pairs comprises: acquiring a plurality of lines in the three frames of two-dimensional images, wherein a line is composed of a plurality of pixel points; taking one pixel point on a line in one frame of two-dimensional image as a selected point, and determining a plurality of candidate matching points matching the selected point in another frame of two-dimensional image; performing three-dimensional reconstruction on the selected point and the plurality of candidate matching points based on the triangulation principle to obtain a plurality of first candidate three-dimensional points; determining a first candidate three-dimensional point satisfying a preset condition as a second candidate three-dimensional point, wherein there are a plurality of second candidate three-dimensional points; and forming a matching point pair from the candidate matching point corresponding to the second candidate three-dimensional point and the selected point.

[0044] For example, as Figure 2 shown, the three frames of two-dimensional images comprise a plurality of lines (that is, a plurality of scanning lines emitted by a pattern projector), and the lines are composed of a plurality of pixel points, Figure 2 the image corresponding to Cam L is a first frame of two-dimensional image, that corresponding to Cam M is a second frame of two-dimensional image, and that corresponding to Cam H is a third frame of two-dimensional image. Taking point A in the first frame of two-dimensional image as a selected point, a plurality of candidate matching points matching A are found in the second frame of two-dimensional image. For example, the candidate matching points in the second frame of two-dimensional image are B1, B2, B3, B4, B5 and B6. Three-dimensional reconstruction is performed on point A and the candidate matching points B1, B2, B3, B4, B5 and B6 by triangulation to obtain a plurality of first candidate three-dimensional points O1, O2, O3, O4, O5 and O6. The first candidate three-dimensional points are primarily screened, and those satisfying the preset condition are taken as second candidate three-dimensional points. The screening method comprises calculating the distance Distance(k) from a first candidate three-dimensional point to the optical plane Plane(k) corresponding to all scanning lines. If the distance value of a certain first candidate three-dimensional point is within a set distance threshold, that is, Distance(k)<dist TH, it indicates that the first candidate three-dimensional point is within the No. k optical plane (the k-th projection line), that is, the first candidate three-dimensional point satisfies the preset requirement. Assuming that the second candidate three-dimensional points are O1, O2 and O3, then the candidate matching points B1, B2 and B3 corresponding to O1, O2 and O3 form matching point pairs with point A. Through the above method, matching points of each pixel point on the first frame of two-dimensional image on the second frame of two-dimensional image are obtained. The same method is also used to obtain matching point pairs between the first frame of two-dimensional image and the third frame of two-dimensional image, and between the second frame of two-dimensional image and the third frame of two-dimensional image.

[0045] By obtaining a plurality of lines in three frames of two-dimensional images, it is convenient to determine the coordinates of each pixel in the three frames of two-dimensional images. Pairwise matching is performed on the three frames of two-dimensional images to obtain three sets of matching point pairs, which can effectively improve the accuracy of matching.

[0046] Optionally, in the three-dimensional scanning processing method provided in the embodiments of the present application, before verifying the matching consistency among the three sets of matching point pairs, the method further includes: acquiring a plurality of light planes corresponding to second candidate three-dimensional points, wherein the light plane is a light plane corresponding to a selected point; and determining a target light plane corresponding to the selected point from the plurality of light planes.

[0047] Before verifying the three sets of matching point pairs, it is necessary to determine the optimal matching point pair. Theoretically, when two frames of two-dimensional images are matched, only one pair among the obtained multiple matching point pairs is correct, so it is necessary to determine the optimal matching point pair. First, a second candidate three-dimensional point is determined by calculating the distance Distance(k) between the point and the light plane Plane(k) corresponding to all scanning lines. When the distance value of a certain second candidate three-dimensional point is within the set distance threshold, that is, Distance(k)<distTH, it indicates that the point is within the No. k light plane (the k-th projection line). That is, one second candidate three-dimensional point corresponds to one light plane. For example, the light planes corresponding to the second candidate three-dimensional points O1, O2 and O3 are K1, K2 and K3 respectively. The optimal light plane (that is, the above-mentioned target light plane) is selected from these light planes.

[0048] Further screening the obtained three sets of matching point pairs enables more accurate acquisition of reconstructed three-dimensional points on the object surface.

[0049] Optionally, in the three-dimensional scanning processing method provided in the embodiments of the present application, determining the target light plane corresponding to the selected point from the plurality of light planes includes: acquiring a plurality of light planes corresponding to a plurality of pixel points on the same line as the selected point; calculating the occurrence frequency of each light plane, and taking the light plane with the highest occurrence frequency as the target light plane.

[0050] For example, the method for selecting the optimal light plane from the light planes K1, K2 and K3 is as follows: first obtain a plurality of light planes corresponding to a plurality of pixel points on the same line as the selected point, such as Figure 2As shown, there are multiple pixels A1, A2, A3, and A4 on the same line as point A. The multiple light planes corresponding to A1 are K1 and K2; A2 is K1 and K4; A3 is K1, K2, and K3; and A4 is K1 and K2. The frequency of each light plane is calculated, and the light plane with the most occurrences is taken as the optimal light plane. Therefore, the optimal light plane corresponding to point A is K1. In practical applications, the lines in the three frames of a 2D image may be discontinuous. Therefore, the optimal light plane corresponding to the selected point is determined by searching for multiple light planes corresponding to multiple pixels in the neighborhood of the selected point along the line.

[0051] The optimal light plane corresponding to the selected point is determined by using multiple light planes corresponding to multiple pixels on the same line as the selected point, which improves the accuracy of the light plane corresponding to the pixel.

[0052] Optionally, in the three-dimensional scanning processing method provided in the embodiments of this application, verifying the matching consistency between the three sets of matching point pairs includes: taking the matching point corresponding to the target light plane as the target matching point; forming a target matching point pair with the selected point and the target matching point to obtain three sets of target matching point pairs; and verifying the consistency between the three sets of target matching point pairs.

[0053] To verify the consistency of matching among the three sets of matching point pairs, it is only necessary to verify the consistency among the best matching point pairs (i.e., the target matching point pairs mentioned above). The matching point corresponding to the best lighting plane is the best matching point. For example, the best matching point pairs between the first and second 2D images are A(1,2) and B1(2,3), and the corresponding best lighting plane is K1; the best matching point pairs between the first and third 2D images are A(1,2) and C(1,3), and the corresponding best lighting plane is K1; the best matching point pairs between the second and third 2D images are B1(2,3) and C(1,3), and the corresponding best lighting plane is K1.

[0054] Only the 3D points reconstructed through the best matching point pairs can be the 3D points on the surface of the object whose 3D model needs to be built. Therefore, it is necessary to verify the consistency between the best matching point pairs.

[0055] Optionally, in the three-dimensional scanning processing method provided in this application embodiment, the three frames of two-dimensional images are a first frame of two-dimensional images, a second frame of two-dimensional images, and a third frame of two-dimensional images. Verifying the matching consistency between the three sets of target matching point pairs includes: obtaining a selected point in the first frame of two-dimensional images, and a target light plane one determined by matching the selected point with the first frame of two-dimensional images and the second frame of two-dimensional images; obtaining a selected point in the first frame of two-dimensional images, and a target light plane two determined by matching the selected point with the first frame of two-dimensional images and the third frame of two-dimensional images; obtaining the target matching point of the selected point in the first frame of two-dimensional images in the second frame of two-dimensional images; obtaining a target light plane three determined by matching the target matching point in the second frame of two-dimensional images with the second frame of two-dimensional images and the third frame of two-dimensional images; determining whether the target light plane one, the target light plane two, and the target light plane three are the same light plane; when the target light plane one, the target light plane two, and the target light plane three are the same light plane, then the three sets of target matching point pairs have matching consistency.

[0056] For example, if a point A(1,2) is in the first frame of a 2D image, and the best matching point coordinates of A(1,2) in the second frame of a 2D image are B1(2,3), then the best matching point pair between the first and second frames of the 2D image is A(1,2) and B1(2,3), and the corresponding best lighting plane is K1. Similarly, if A(1,2) is in the third frame of a 2D image, and the best matching point coordinates of A(1,2) are C(1,3), then the best matching point pair between the first and third frames of the 2D image is A(1,2) and C(1,3), and the corresponding best lighting plane is K1. Likewise, if B1(2,3) in the second frame of a 2D image is in the third frame of a 2D image, then the best matching point pair between the second and third frames of the 2D image is B1(2,3) and C(1,3), and the corresponding best lighting plane is K1. Based on these examples, the best lighting plane obtained from pairwise matching is the same, K1, indicating that the three sets of best matching point pairs have matching consistency.

[0057] The matching consistency is determined by verifying the light plane corresponding to the matching point pair. This is because only if the light plane is the same can it be said that the matching point pair is the same three-dimensional point. Therefore, the above steps further improve the accuracy of the matching.

[0058] Optionally, in the three-dimensional scanning processing method provided in the embodiments of this application, a pixel on a line in a frame of two-dimensional image is selected as a selected point, and multiple candidate matching points in another frame of two-dimensional image are determined to match the selected point, including: obtaining the epipolar equation of the selected point to the other frame of two-dimensional image; and taking the intersection of the epipolar equation with multiple lines in the other frame of two-dimensional image as multiple candidate matching points.

[0059] For example, when determining multiple candidate matching points for point A in the first two-dimensional image within the second two-dimensional image, this needs to be achieved using the epipolar equation. First, the epipolar equation is calculated based on the relative positions of Cam L and Cam M. For example... Figure 2 The schematic epipolar equation is marked in the image. The intersections of the epipolar equation with multiple lines in another frame of the 2D image are used as multiple candidate matching points.

[0060] The 3D scanning processing method provided in this application involves projecting multiple lines onto the surface of a measured object using a pattern projector; acquiring 2D images of the object's surface using three cameras to obtain three 2D image frames; determining matching point pairs between each pair of the three 2D image frames to obtain three sets of matching point pairs; verifying the matching consistency between the matching point pairs; and performing 3D reconstruction on the matching point pairs with matching consistency to obtain the 3D points on the surface of the measured object. This method solves the problem that the scanning efficiency of binocular laser scanning in related technologies cannot meet the requirements of some high scanning efficiency scenarios. While increasing the number of scanning lines can improve scanning efficiency, in binocular scanning systems, increasing the number of scanning lines can lead to a sharp decrease in matching accuracy. By acquiring 2D images of the object's surface using three cameras to obtain three 2D image frames, matching each pair of the three 2D image frames to obtain three sets of matching point pairs, and performing 3D reconstruction on the matching point pairs with matching consistency to obtain the 3D points on the surface of the measured object, the method achieves the effect of improving matching accuracy.

[0061] This application also provides a three-dimensional scanning device, which includes three cameras. The three cameras are combined in pairs to obtain three binocular systems. The three binocular systems are used to acquire two-dimensional images of the surface of the object being measured, resulting in three frames of two-dimensional images. Matching point pairs between each pair of the three frames of two-dimensional images are determined, resulting in three sets of matching point pairs. The matching consistency between the matching point pairs is verified. Three-dimensional reconstruction is performed on the matching point pairs with matching consistency to obtain three-dimensional points on the surface of the object being measured.

[0062] like Figure 3 The diagram shown is an optional 3D scanning device provided according to an embodiment of this application, wherein CamL, CamM, and CamR are three cameras, and the three cameras form three binocular systems. The projector is a pattern projector.

[0063] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0064] This application also provides a three-dimensional scanning processing apparatus. It should be noted that the three-dimensional scanning processing apparatus of this application can be used to execute the three-dimensional scanning processing method provided in this application. The three-dimensional scanning processing apparatus provided in this application will be described below.

[0065] Figure 4 This is a schematic diagram of a three-dimensional scanning processing apparatus according to an embodiment of this application. Figure 4 As shown, the device includes: a projection unit 401, an acquisition unit 402, a first determination unit 403, a verification unit 404, and a reconstruction unit 405.

[0066] The projection unit 401 is used to project multiple lines onto the surface of the object being measured via a pattern projector.

[0067] The acquisition unit 402 is used to acquire two-dimensional images of the surface of the object under test through three cameras, thereby obtaining three frames of two-dimensional images.

[0068] The first determining unit 403 is used to determine the matching point pairs between each pair of the three two-dimensional images, thereby obtaining three sets of matching point pairs.

[0069] Verification unit 404 is used to verify the matching consistency between matching point pairs.

[0070] The reconstruction unit 405 is used to perform three-dimensional reconstruction on the matching point pairs with matching consistency to obtain three-dimensional points on the surface of the object being measured.

[0071] The 3D scanning processing apparatus provided in this application embodiment projects multiple lines onto the surface of the object to be measured through a pattern projector via a projection unit 401; an acquisition unit 402 acquires 2D images of the surface of the object to be measured through three cameras, resulting in three frames of 2D images; a first determination unit 403 determines the matching point pairs between each pair of the three frames of 2D images, resulting in three sets of matching point pairs; a verification unit 404 verifies the matching consistency between the matching point pairs; and a reconstruction unit 405 performs 3D reconstruction on the matching point pairs with matching consistency to obtain 3D points on the surface of the object to be measured. This solves the problem that the scanning efficiency of binocular laser scanning in related technologies cannot meet the requirements of some high scanning efficiency scenarios. The method to improve scanning efficiency is to increase the number of scanning lines. However, in a binocular scanning system, increasing the number of scanning lines will lead to a sharp decrease in matching accuracy. By acquiring 2D images of the surface of the object to be measured through three cameras, obtaining three frames of 2D images, matching each pair of the three frames of 2D images to obtain three sets of matching point pairs, and performing 3D reconstruction on the matching point pairs with matching consistency to obtain 3D points on the surface of the object to be measured, the accuracy of matching is improved.

[0072] Optionally, in the three-dimensional scanning processing apparatus provided in this application embodiment, the determining unit includes: a first acquisition subunit, used to acquire multiple lines in the three two-dimensional images before determining the matching point pairs between each pair of the three two-dimensional images and obtaining three sets of matching point pairs, wherein the lines are composed of multiple pixels; a first determining subunit, used to take one pixel on a line in one two-dimensional image as a selected point and determine multiple candidate matching points in another two-dimensional image that match the selected point; a reconstruction subunit, used to perform three-dimensional reconstruction of the selected point and multiple candidate matching points based on the triangulation principle to obtain multiple first candidate three-dimensional points; a second determining subunit, used to determine the first candidate three-dimensional points that meet preset conditions as second candidate three-dimensional points, wherein there are multiple second candidate three-dimensional points; and a first forming subunit, used to form matching point pairs between the candidate matching points corresponding to the second candidate three-dimensional points and the selected points.

[0073] Optionally, in the three-dimensional scanning processing apparatus provided in the embodiments of this application, the apparatus further includes: an acquisition unit, configured to acquire multiple light planes corresponding to a second candidate three-dimensional point before verifying the matching consistency between three sets of matching point pairs, wherein the light plane is the light plane corresponding to the selected point; and a second determination unit, configured to determine the target light plane corresponding to the selected point from the multiple light planes.

[0074] Optionally, in the three-dimensional scanning processing apparatus provided in the embodiments of this application, the second determining unit includes: a second acquiring subunit, used to acquire multiple light planes corresponding to multiple pixel points on the same line as the selected point; and a calculation subunit, used to calculate the number of times each light plane appears, and take the light plane with the most appearances as the target light plane.

[0075] Optionally, in the three-dimensional scanning processing apparatus provided in the embodiments of this application, the verification unit includes: taking the matching point corresponding to the target light plane as the target matching point; a second composition subunit, used to form target matching point pairs with the selected point and the target matching point to obtain three sets of target matching point pairs; and a verification subunit, used to verify the consistency between the three sets of target matching point pairs.

[0076] Optionally, in the three-dimensional scanning processing device provided in this application embodiment, the three two-dimensional images are a first two-dimensional image, a second two-dimensional image, and a third two-dimensional image. The verification subunit includes: a first acquisition module, used to acquire a selected point in the first two-dimensional image, and a target light plane one determined by matching the selected point with the first two-dimensional image and the second two-dimensional image; a second acquisition module, used to acquire a selected point in the first two-dimensional image, and a target light plane two determined by matching the selected point with the first two-dimensional image and the third two-dimensional image; a third acquisition module, used to acquire a target matching point of the selected point in the first two-dimensional image in the second two-dimensional image; a fourth acquisition module, used to acquire a target light plane three determined by matching the target matching point in the second two-dimensional image with the second two-dimensional image and the third two-dimensional image; and a judgment module, used to judge whether the target light plane one, the target light plane two, and the target light plane three are the same light plane; when the target light plane one, the target light plane two, and the target light plane three are the same light plane, then the three sets of target matching point pairs have matching consistency.

[0077] Optionally, in the three-dimensional scanning processing apparatus provided in the embodiments of this application, the first determining subunit includes: a fifth obtaining unit, used to obtain the epipolar equation of the selected point to another frame of two-dimensional image; and to take the intersection of the epipolar equation with multiple lines in the other frame of two-dimensional image as multiple candidate matching points.

[0078] The three-dimensional scanning processing device includes a processor and a memory. The projection unit, acquisition unit, first determination unit, verification unit and reconstruction unit are all stored as program units in the memory. The processor executes the program units stored in the memory to realize the corresponding functions.

[0079] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the reconstruction of the three-dimensional points of the object being measured can be achieved by adjusting the kernel parameters.

[0080] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0081] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements a three-dimensional scanning processing method.

[0082] This invention provides a processor for running a program, wherein the program executes a three-dimensional scanning processing method during runtime.

[0083] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: projecting multiple lines onto the surface of a measured object using a pattern projector; acquiring two-dimensional images of the surface of the measured object using three cameras to obtain three frames of two-dimensional images; determining matching point pairs between each pair of the three frames of two-dimensional images to obtain three sets of matching point pairs; verifying the matching consistency between the matching point pairs; and performing three-dimensional reconstruction on the matching point pairs with matching consistency to obtain three-dimensional points on the surface of the measured object.

[0084] Optionally, matching point pairs are determined between each pair of the three 2D images, resulting in three sets of matching point pairs. This includes: acquiring multiple lines in the three 2D images, where each line is composed of multiple pixels; selecting a pixel on a line in one 2D image as a selected point; determining multiple candidate matching points in another 2D image that match the selected point; performing 3D reconstruction on the selected point and multiple candidate matching points based on the triangulation principle to obtain multiple first candidate 3D points; determining the first candidate 3D points that meet preset conditions as second candidate 3D points, where there are multiple second candidate 3D points; and forming matching point pairs between the candidate matching points corresponding to the second candidate 3D points and the selected points.

[0085] Optionally, before verifying the matching consistency among the three sets of matching point pairs, the method further includes: obtaining multiple light planes corresponding to the second candidate 3D point, wherein the light plane is the light plane corresponding to the selected point; and determining the target light plane corresponding to the selected point from the multiple light planes.

[0086] Optionally, determining the target light plane corresponding to the selected point from multiple light planes includes: acquiring multiple light planes corresponding to multiple pixels on the same line as the selected point; calculating the number of times each light plane appears, and taking the light plane with the most appearances as the target light plane.

[0087] Optionally, verifying the matching consistency among the three sets of matching point pairs includes: taking the matching point corresponding to the target light plane as the target matching point; forming a target matching point pair with the selected point and the target matching point to obtain three sets of target matching point pairs; and verifying the consistency among the three sets of target matching point pairs.

[0088] Optionally, the three two-dimensional images are a first two-dimensional image, a second two-dimensional image, and a third two-dimensional image, respectively. Verifying the matching consistency between the three sets of target matching point pairs includes: obtaining a selected point in the first two-dimensional image, and a target light plane one determined by matching the selected point with the first two-dimensional image and the second two-dimensional image; obtaining a selected point in the first two-dimensional image, and a target light plane two determined by matching the selected point with the first two-dimensional image and the third two-dimensional image; obtaining the target matching point of the selected point in the first two-dimensional image in the second two-dimensional image; obtaining the target light plane three determined by matching the target matching point in the second two-dimensional image with the second two-dimensional image and the third two-dimensional image; determining whether target light plane one, target light plane two, and target light plane three are the same light plane; when target light plane one, target light plane two, and target light plane three are the same light plane, then the three sets of target matching point pairs have matching consistency.

[0089] Optionally, a pixel on a line in one frame of a two-dimensional image is selected as the chosen point, and multiple candidate matching points in another frame of a two-dimensional image are determined. This includes: obtaining the epipolar equation of the selected point in the other frame of the two-dimensional image; and using the intersections of the epipolar equation with multiple lines in the other frame of the two-dimensional image as multiple candidate matching points. The device in this article can be a server, PC, PAD, mobile phone, etc.

[0090] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: projecting multiple lines onto the surface of a measured object through a pattern projector; acquiring two-dimensional images of the surface of the measured object through three cameras, corresponding to three frames of two-dimensional images; determining matching point pairs between each pair of the three frames of two-dimensional images, corresponding to three sets of matching point pairs; verifying the matching consistency between the matching point pairs; and performing three-dimensional reconstruction on the matching point pairs with matching consistency to obtain three-dimensional points on the surface of the measured object.

[0091] Optionally, matching point pairs are determined between each pair of the three 2D images, resulting in three sets of matching point pairs. This includes: acquiring multiple lines in the three 2D images, where each line is composed of multiple pixels; selecting a pixel on a line in one 2D image as a selected point; determining multiple candidate matching points in another 2D image that match the selected point; performing 3D reconstruction on the selected point and multiple candidate matching points based on the triangulation principle to obtain multiple first candidate 3D points; determining the first candidate 3D points that meet preset conditions as second candidate 3D points, where there are multiple second candidate 3D points; and forming matching point pairs between the candidate matching points corresponding to the second candidate 3D points and the selected points.

[0092] Optionally, before verifying the matching consistency among the three sets of matching point pairs, the method further includes: obtaining multiple light planes corresponding to the second candidate 3D point, wherein the light plane is the light plane corresponding to the selected point; and determining the target light plane corresponding to the selected point from the multiple light planes.

[0093] Optionally, determining the target light plane corresponding to the selected point from multiple light planes includes: acquiring multiple light planes corresponding to multiple pixels on the same line as the selected point; calculating the number of times each light plane appears, and taking the light plane with the most appearances as the target light plane.

[0094] Optionally, verifying the matching consistency among the three sets of matching point pairs includes: taking the matching point corresponding to the target light plane as the target matching point; forming a target matching point pair with the selected point and the target matching point to obtain three sets of target matching point pairs; and verifying the consistency among the three sets of target matching point pairs.

[0095] Optionally, the three two-dimensional images are a first two-dimensional image, a second two-dimensional image, and a third two-dimensional image, respectively. Verifying the matching consistency between the three sets of target matching point pairs includes: obtaining a selected point in the first two-dimensional image, and a target light plane one determined by matching the selected point with the first two-dimensional image and the second two-dimensional image; obtaining a selected point in the first two-dimensional image, and a target light plane two determined by matching the selected point with the first two-dimensional image and the third two-dimensional image; obtaining the target matching point of the selected point in the first two-dimensional image in the second two-dimensional image; obtaining the target light plane three determined by matching the target matching point in the second two-dimensional image with the second two-dimensional image and the third two-dimensional image; determining whether target light plane one, target light plane two, and target light plane three are the same light plane; when target light plane one, target light plane two, and target light plane three are the same light plane, then the three sets of target matching point pairs have matching consistency.

[0096] Optionally, a pixel on a line in a frame of a two-dimensional image is selected as the selected point, and multiple candidate matching points in another frame of a two-dimensional image are determined, including: obtaining the epipolar equation of the selected point in the other frame of a two-dimensional image; and taking the intersection of the epipolar equation with multiple lines in the other frame of a two-dimensional image as multiple candidate matching points.

[0097] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0099] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0100] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0101] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0102] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0103] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0104] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0105] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for processing three-dimensional scanning, characterized in that, include: Multiple lines are projected onto the surface of the object being measured using a pattern projector; Three cameras are used to capture two-dimensional images of the surface of the object under test, resulting in three corresponding two-dimensional images; By determining the pairwise matching point pairs between the three frames of two-dimensional images, three sets of matching point pairs are obtained. The matching consistency between the matching point pairs is verified by verifying whether the light planes corresponding to the matching point pairs are consistent. Three-dimensional reconstruction is performed on the matching point pairs with consistent matching to obtain the three-dimensional points on the surface of the object under test.

2. The method according to claim 1, characterized in that, The matching point pairs between each pair of the three 2D images are determined, resulting in three sets of matching point pairs, including: Obtain multiple lines from the three frames of the two-dimensional image, wherein each line is composed of multiple pixels. Using a pixel on a line in one frame of a two-dimensional image as a selected point, determine multiple candidate matching points in another frame of a two-dimensional image that match the selected point; Based on the principle of triangulation, the selected point and the multiple candidate matching points are reconstructed in three dimensions to obtain multiple first candidate three-dimensional points; The first candidate 3D point that meets the preset conditions is determined as the second candidate 3D point, wherein there are multiple second candidate 3D points; The candidate matching point corresponding to the second candidate 3D point is combined with the selected point to form the matching point pair.

3. The method according to claim 2, characterized in that, Before verifying the matching consistency among the three sets of matching point pairs, the method further includes: Obtain multiple light planes corresponding to the second candidate 3D point, wherein the light plane is the light plane corresponding to the selected point; The target light plane corresponding to the selected point is determined from the plurality of light planes.

4. The method according to claim 3, characterized in that, Determining the target light plane corresponding to the selected point from the plurality of light planes includes: Obtain multiple light planes corresponding to multiple pixels on the same line as the selected point; Calculate the number of times each light plane appears, and select the light plane that appears the most times as the target light plane.

5. The method according to claim 4, characterized in that, Verifying the matching consistency among the three sets of matching point pairs includes: The matching point corresponding to the target light plane is taken as the target matching point; The selected point and the target matching point are combined to form a target matching point pair, resulting in three sets of target matching point pairs; Verify the consistency among the three sets of target matching point pairs.

6. The method according to claim 5, characterized in that, in, The three 2D images are the first 2D image, the second 2D image, and the third 2D image, respectively. Verifying the matching consistency among the three sets of target matching point pairs includes: Obtain a selected point in the first frame of the two-dimensional image, and a target light plane determined by matching the selected point with the first frame of the two-dimensional image and the second frame of the two-dimensional image; Obtain a selected point in the first frame of the two-dimensional image, and a target light plane two determined by matching the selected point with the first frame of the two-dimensional image and the third frame of the two-dimensional image; Obtain the target matching point of the selected point in the first frame of the two-dimensional image in the second frame; Obtain the target light plane three determined by matching the target matching point in the second frame two-dimensional image with the third frame two-dimensional image; Determine whether the target light plane one, the target light plane two, and the target light plane three are the same light plane; When the target light plane one, the target light plane two, and the target light plane three are the same light plane, then the three sets of target matching point pairs have matching consistency.

7. The method according to claim 2, characterized in that, Using a pixel on a line in one frame of a 2D image as a selected point, determine multiple candidate matching points in another frame of a 2D image that match the selected point, including: Obtain the epipolar equation of the selected point in the other frame of the two-dimensional image; The intersection points of the epipolar equation with multiple lines in the other frame of the two-dimensional image are used as the multiple candidate matching points.

8. A three-dimensional scanning device, characterized in that, The 3D scanning device includes three cameras, which are combined in pairs to form three binocular systems. These three binocular systems are used to acquire 2D images of the surface of the object being measured, resulting in three frames of 2D images. Matching point pairs are determined between each pair of the three 2D images, resulting in three sets of matching point pairs. The matching consistency between the matching point pairs is verified by checking whether the light planes corresponding to the matching point pairs are consistent. 3D reconstruction is performed on the matching point pairs with matching consistency to obtain 3D points on the surface of the object being measured.

9. A three-dimensional scanning processing device, characterized in that, include: The projection unit is used to project multiple lines onto the surface of the object being measured via a pattern projector. The acquisition unit is used to acquire two-dimensional images of the surface of the object under test through three cameras, thereby obtaining three frames of two-dimensional images. The first determining unit is used to determine the pair of matching points between each pair of the three frames of two-dimensional images, thereby obtaining three sets of matching point pairs. The verification unit is used to verify the matching consistency between the matching point pairs by verifying whether the light planes corresponding to the matching point pairs are consistent. The reconstruction unit is used to perform three-dimensional reconstruction on matching point pairs with matching consistency to obtain three-dimensional points on the surface of the object under test.

10. A computer-readable storage medium, characterized in that, The storage medium includes a stored program, wherein the program executes the three-dimensional scanning processing method according to any one of claims 1 to 7.

11. A processor, characterized in that, The processor is used to run a program, wherein the program executes the three-dimensional scanning processing method according to any one of claims 1 to 7.

Images