Three-dimensional reconstruction method and device

Abstract

The invention provides a three-dimensional reconstruction method and device, and the method comprises the steps: obtaining a first image and a second image, the first image and the second image are obtained through the collection of a target image through a pre-calibrated binocular camera, and the target image is formed by a projection pattern projected to the surface of a measured object, the projection pattern is at least composed of binary stripes containing feature points; the positions of feature points in the first image and the second image are determined respectively, multiple first adjacent domains corresponding to the feature points in the first image and multiple second adjacent domains corresponding to the feature points in the second image are determined respectively, the feature points in the first image are in one-to-one correspondence with the first adjacent domains, and the feature points in the second image are in one-to-one correspondence with the second adjacent domains; the feature points in the second image are in one-to-one correspondence with the second adjacent domains; matching the feature points in the first image with the feature points in the second image one by one according to the matching degree between the first adjacent domain and the second adjacent domain; and completing three-dimensional reconstruction of the measured object according to the mutually matched feature points in the first image and the second image.

Description

[0001] Technology Neighborhood

[0002] The present application relates to the field of three-dimensional reconstruction, and more specifically, to a three-dimensional reconstruction method and device. Background Art

[0003] At present, the means of obtaining dental model data in the field of dental diagnosis and treatment has gradually shifted from impression 3D scanning to intraoral 3D scanning technology. This technology abandons the method of obtaining dental model data from impressions, molds, and 3D scanning, and can directly obtain dental 3D data by scanning. Oral digital impression instrument, also known as intraoral 3D scanner, is a device that uses an invasive optical scanning head to directly scan the patient's oral cavity to obtain the 3D morphology and color texture information of the soft and hard tissue surfaces such as teeth, gums, and mucosa in the oral cavity. One method of this device is to use the active structured light triangulation imaging principle, use a digital projection system to project active light patterns, and after the camera acquisition system obtains the pattern, it performs 3D reconstruction and splicing through algorithm processing. That is, the integrity of the data in this area is reduced. Due to the cramped environment inside the oral cavity, the volume of the invasive optical scanning head is inevitably limited, so each scan can only obtain images of two or fewer teeth, and the scanning efficiency is very low. Summary of the invention

[0004] The main purpose of the present application is to provide a three-dimensional reconstruction method and device to at least solve the technical problem of low image scanning efficiency in the related art.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a three-dimensional reconstruction method is provided, the method comprising: acquiring a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated binocular camera, the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is composed of at least binary stripes containing feature points; determining the positions of the feature points in the first image and the second image respectively, and determining a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image respectively, wherein the feature points in the first image correspond one-to-one to the first adjacent domains, and the feature points in the second image correspond one-to-one to the second adjacent domains; matching the feature points in the first image with the feature points in the second image one-to-one according to the matching degree between the first adjacent domains and the second adjacent domains; and completing the three-dimensional reconstruction of the object to be measured according to the mutually matching feature points in the first image and the second image.

[0006] Optionally, the projection pattern is determined in the following manner, including: determining the type of binary stripes, wherein the type of binary stripes includes at least one of the following: line segment stripes with endpoints as feature points and binary stripes with feature points randomly distributed on the stripes; arranging the binary stripes according to a predetermined distribution rule to form a projection pattern, wherein the length of the line segment stripes is random and the feature points include at least one of the following shapes: a horizontal line of preset length, a sphere, a cross and a square.

[0007] Optionally, the binary stripes are formed into a projection pattern according to a predetermined distribution rule, including: determining a first distribution rule of the binary stripes from a plurality of horizontal distribution rules in the predetermined distribution rules, wherein the plurality of horizontal distribution rules include at least one of the following: the horizontal spacing between horizontally adjacent binary stripes is equal, the horizontal spacing between horizontally adjacent binary stripes increases in sequence with equal intervals, and the horizontal spacing between horizontally adjacent binary stripes is randomly selected in a preset distance interval; determining a second distribution rule of the binary stripes from the predetermined distribution rules according to the type of the binary stripes; and arranging the binary stripes according to the first distribution rule and the second distribution rule to form a projection pattern.

[0008] Optionally, a second distribution rule for the binary stripes is determined from predetermined distribution rules according to the type of the binary stripes, including: when the binary stripes in the projection pattern are line segment stripes, the second distribution rule is selected from multiple vertical distribution rules in the predetermined distribution rules, wherein the multiple vertical distribution rules include at least one of the following: the vertical distances between longitudinally adjacent line segment stripes are equal, the vertical distances between longitudinally adjacent line segment stripes increase in sequence at equal intervals, and the vertical distances between longitudinally adjacent line segment stripes are randomly selected in a preset distance interval; when the binary stripes in the projection pattern are binary stripes with feature points randomly distributed on the stripes, the second distribution rule is selected from multiple feature point distribution rules in the predetermined distribution rules, wherein the multiple feature point distribution rules include at least: the spacing between multiple feature points is equal, and the multiple feature points are randomly distributed on the stripes.

[0009] Optionally, determining multiple first adjacent domains corresponding to feature points in the first image includes: determining feature points to be matched from the first image; determining a sub-image window of a preset size in the first image with the feature points to be matched as the center; and determining the area framed by the sub-image window as the first adjacent domain.

[0010] Optionally, the feature point to be matched is determined in the second image in the following manner, including: determining the first adjacent domain corresponding to the feature point to be matched in the first image; determining the second adjacent domain corresponding to each feature point in the second image in turn; determining the degree of matching between the feature point to be matched and each feature point in the second image according to the grayscale value at the feature point to be matched, the grayscale value of each feature point in the second image, the grayscale average of the first adjacent domain corresponding to the feature point to be matched, and the grayscale average of the second adjacent domain corresponding to each feature point in the second image, and selecting the feature point with the highest matching degree and satisfying the polar line constraint from the second image as the feature point in the second image that matches the feature point to be matched.

[0011] Optionally, the intrinsic and extrinsic parameters of the binocular camera may be determined by: extracting coordinates of feature points of an image in a calibration plate captured by the binocular camera; and calculating the intrinsic and extrinsic parameters of the binocular camera according to the coordinates of feature points of the image in the calibration plate.

[0012] Optionally, the internal and external parameters of the binocular camera are calculated according to the coordinates of the feature points of the image in the calibration plate, including: determining the target parameters that need to be calibrated from the multiple internal and external parameters of the binocular camera, the target parameters including: the rotation angle and translation displacement of the multiple feature points of the image in the calibration plate from the world coordinate system to the camera coordinate system, the camera focal length and the radial distortion coefficient; substituting the three-dimensional coordinates of the multiple feature points of the image in the calibration plate and the corresponding image coordinates into a preset linear equation group to determine the optimal solution of the preset linear equation group; and determining the target parameters based on the optimal solution.

[0013] Optionally, determining the target parameters based on the optimal solution includes: determining the horizontal axis component and the vertical axis component in the rotation matrix corresponding to the rotation angle and the translation matrix corresponding to the translation displacement based on the optimal solution; and determining the camera focal length and the radial distortion coefficient in turn based on the horizontal axis component and the vertical axis component in the rotation matrix and the translation matrix.

[0014] According to another aspect of the present application, a three-dimensional reconstruction method is also provided, including: acquiring a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated monocular camera, and the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is at least composed of a random distribution of binary stripes containing feature points; determining the positions of the feature points in the first image and the second image respectively, and determining a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image respectively, wherein the feature points in the first image correspond one-to-one to the first adjacent domains, and the feature points in the second image correspond one-to-one to the second adjacent domains; matching the feature points in the first image with the feature points in the second image one-to-one according to the matching degree between the first adjacent domains and the second adjacent domains; and completing the three-dimensional reconstruction of the object to be measured based on the mutually matching feature points in the first image and the second image.

[0015] Optionally, acquiring the first image and the second image includes: acquiring the target image at different times through a monocular camera to obtain the first image and the second image; or acquiring the target image through a monocular camera to obtain the first image, and modulating the first image to obtain the second image.

[0016] According to another aspect of the present application, a method for three-dimensional reconstruction of teeth inside an oral cavity is also provided, including: acquiring a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated binocular camera, the target image is formed by a projection pattern projected onto the surface of the tooth to be measured, and the projection pattern is at least composed of a random distribution of binary stripes containing feature points; determining the positions of the feature points in the first image and the second image respectively, and determining a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image respectively, wherein the feature points in the first image correspond one-to-one to the first adjacent domains, and the feature points in the second image correspond one-to-one to the second adjacent domains; matching the feature points in the first image with the feature points in the second image one-to-one according to the matching degree between the first adjacent domains and the second adjacent domains; and completing the three-dimensional reconstruction of the tooth to be measured according to the mutually matching feature points in the first image and the second image.

[0017] According to another aspect of the present application, a three-dimensional reconstruction device is also provided, including: an acquisition module, used to acquire a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated binocular camera, and the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is at least composed of binary stripes containing feature points; a determination module, used to respectively determine the positions of the feature points in the first image and the second image, and respectively determine a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image, wherein the feature points in the first image correspond one-to-one to the first adjacent domains, and the feature points in the second image correspond one-to-one to the second adjacent domains; a matching module, used to match the feature points in the first image with the feature points in the second image one-to-one according to the matching degree between the first adjacent domains and the second adjacent domains; and a reconstruction module, used to complete the three-dimensional reconstruction of the object to be measured based on the mutually matching feature points in the first image and the second image.

[0018] According to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned three-dimensional reconstruction method.

[0019] According to another aspect of the present application, an oral scanner is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing the above-mentioned three-dimensional reconstruction method.

[0020] By applying the technical solution of the present application, the object to be measured is scanned by randomly distributing a projection image composed of binary stripes containing at least feature points, thereby achieving the purpose of completing encoding and decoding of the object to be measured with fewer sequence images, thereby achieving the technical effect of shortening the time for encoding and decoding the object to be measured, and further solving the technical problem of low efficiency in scanning images such as teeth in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A hardware structure block diagram of a mobile terminal for performing a three-dimensional reconstruction method provided in an embodiment of the present application is shown;

[0023] Figure 2 A schematic diagram of a process of a three-dimensional reconstruction method provided according to an embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram of a projection pattern provided according to an embodiment of the present application is shown;

[0025] Figure 4 Another schematic diagram of a projection pattern provided according to an embodiment of the present application is shown;

[0026] Figure 5 A schematic diagram of another projection pattern provided according to an embodiment of the present application is shown;

[0027] Figure 6 A schematic diagram of a calibration plate pattern provided according to an embodiment of the present application is shown;

[0028] Figure 7 A schematic diagram of a process of another three-dimensional reconstruction method provided according to an embodiment of the present application is shown;

[0029] Figure 8 A flow chart of a method for three-dimensional reconstruction of teeth inside an oral cavity provided according to an embodiment of the present application is shown;

[0030] Fig. 9 A schematic diagram of a three-dimensional reconstruction device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0032] In order to enable people in the technical field to better understand the solution of this application, the technical solution in the embodiment of this application will be clearly and completely described below in conjunction with the drawings in the embodiment of this application. Obviously, the described embodiment is only a part of the embodiment of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of this application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] As introduced in the background technology, the technical problem of low efficiency of tooth image scanning in the related art is solved. In order to solve the above problem, an embodiment of the present application provides a three-dimensional reconstruction method.

[0035] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0036] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a three-dimensional reconstruction method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. A person skilled in the art can understand that Figure 1The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0037] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the three-dimensional reconstruction method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific network example may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] In this embodiment, a three-dimensional reconstruction method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 2 FIG. 1 is a flow chart of a three-dimensional reconstruction method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0040] Step S202, acquiring a first image and a second image, wherein the first image and the second image are obtained by respectively capturing a target image by a pre-calibrated binocular camera, the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is composed of at least binary stripes containing feature points;

[0041] Step S204, respectively determining positions of feature points in the first image and the second image, and respectively determining a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image, wherein the feature points in the first image correspond to the first adjacent domains in a one-to-one manner, and the feature points in the second image correspond to the second adjacent domains in a one-to-one manner;

[0042] Step S206, matching the feature points in the first image with the feature points in the second image one by one according to the matching degree between the first adjacent domain and the second adjacent domain;

[0043] Step S208: completing three-dimensional reconstruction of the object to be measured based on the matching feature points in the first image and the second image.

[0044] Specifically, in the process of oral scanning, the factors that determine the single scan time are often related to the structured light encoding and decoding time. When designing the structured light encoding pattern, it is usually considered to decode the entire image, such as time phase unwrapping and spatial phase unwrapping. On the basis of obtaining the folded phase, phase unwrapping is also required to obtain the real absolute phase to solve the periodicity problem of the folded phase. In order to globally unfold the phase, a large number of image sequences or a more complex spatial encoding and decoding process are usually required. To shorten the single scan time, the number of sequence images can be reduced and the decoding algorithm time can be shortened. The reliance on the image neighborhood pixel information in the structured light encoding and decoding algorithm is the key to the integrity of the single scan data. In the process of seeking the single pixel coding value, the more neighborhood pixel information is considered, the more the decoding accuracy will be improved. Therefore, sufficient coding information is required within the neighborhood pixel range. However, due to the rich contour information on the tooth surface, including gingival sulci, tooth gaps, adjacent tooth occlusions, and small tooth spaces, the intraoral image will have a large amount of discontinuous surface information and structured light patterns, resulting in the loss of neighborhood pixel information and affecting the failure of decoding of the pixel position, reducing the integrity of the data in the area. The structured light decoding algorithm relies on less neighborhood information, which will greatly improve the difficulty of incomplete single-scan data and significantly improve scanning efficiency.

[0045] The optical properties of the tooth surface determine that general optical 3D scanning technology is not applicable, such as the phase shift method used by dental model 3D scanners. The unique enamel on the tooth surface poses a great challenge to optical measurement methods. The enamel has high reflection, transmitted light and diffused light. High reflection usually reduces the dynamic range of object imaging, and different viewing angles are needed to avoid the influence of reflection on 3D reconstruction. Transmitted light and diffused light will seriously reduce the overall signal-to-noise ratio of object imaging and reduce the intensity of light, and change the incident point of light, thereby affecting the depth information of 3D reconstruction. There is an attempt to change the optical properties of teeth by spraying solid powder or liquid coating on teeth and gums, using powder or coating to block the incident light, and controlling the thickness of the powder or coating can restore the 3D information of the tooth surface under the powder layer. However, both powder and coating have their inconveniences, such as patients are allergic to or difficult to accept, prolonging the entire scanning time, and the thickness of powder or coating directly affects the accuracy of 3D data, covering up the defects on the tooth surface, and failing to obtain the true tooth color. The method can realize dynamic real-time three-dimensional scanning with color texture directly on materials such as teeth and gums in the mouth that have the characteristics of high reflection, transmission and diffusion of light.

[0046] In step S202, the projection pattern uses randomly truncated binary stripes with start and end features as the active light pattern. The advantage of this pattern is that even with the diffusion of enamel, the characteristics of the stripes (extracting the center of the stripes or the edge of the stripes) can still be maintained. At the same time, the randomly truncated pattern is used as decoding information, and the spatial coding scheme is used to reduce the complexity of the optical structure to achieve the purpose of three-dimensional reconstruction by single-frame projection acquisition, which is easy to identify and improves recognition accuracy.

[0047] In actual application scenarios, an oral scanning system can be used, including: a projection optical system, an image acquisition optical system, a timing control circuit, a heat dissipation system, a heating and anti-fogging system, a software algorithm system, etc. The timing control circuit triggers the projection optical system to project white light or a designed pattern onto the object to be measured (such as teeth or gums), and at the same time controls the image acquisition optical system (such as one or more CCD chips) to quickly obtain the image of the object to be measured with the projection pattern with coded information. The collected image is processed by a software algorithm to obtain three-dimensional data of the object to be measured with color texture. The heating and anti-fogging system can eliminate the atomization defects caused by the temperature difference between the mouth and the surrounding environment, and is equipped with a heat dissipation system to maintain the overall temperature of the system. This system realizes the direct acquisition of three-dimensional surface data of teeth in the mouth without powder spraying, and solves the problem of image clarity and three-dimensional measurement affected by the high reflectivity and scattered light characteristics of tooth enamel. The projection optical system in this system can use a customized DLP projector or a transmissive projection lens. For low cost considerations, this system uses a transmissive projection lens, which greatly reduces the manufacturing cost of the entire system.

[0048] In step S204, the center of the binary stripe or the edge of the stripe can be extracted, and the position of the feature point in the image can be found by using template matching. Specifically, the center of the stripe or the edge of the stripe is extracted, and the starting and ending points of the stripe are located (features such as a short horizontal line, a sphere, a cross, a square, etc.); at the same time, the feature positions in the image, such as a short horizontal line, a sphere, a cross, a square, etc., need to be found by using template matching, and marked, sorted, and recorded.

[0049] In step S206, before matching the feature points in the first image and the second image, the two original images actually captured by the camera can be corrected to image array row alignment using calibration parameters to improve the efficiency of image matching search, and the originally required two-dimensional search can be changed to a one-dimensional search to achieve binocular stereoscopic image distortion correction. In the actual matching process, the matching relationship between the start and end points can be checked by the epipolar constraint of binocular stereoscopic vision, and points and line segments with high correlation matching but no epipolar constraint can be eliminated; the positions of the feature points can also be arranged in a random number manner to achieve a feature position that does not repeat in each column of stripes in the array pattern. Using the relationship of the limit constraint, point pairs with high correlation matching but unable to meet the limit constraint can be eliminated to reduce invalid data.

[0050] In step S208, the principle of triangulated imaging can be used to perform three-dimensional reconstruction of the sub-pixel stripe center or stripe edge for points on the qualified line segments; in actual application scenarios, taking the reconstruction of teeth in the oral cavity as an example, the three-dimensional morphology of multiple scanned teeth is combined with real-time stitching and fusion technology to generate complete three-dimensional tooth morphology data and texture information.

[0051] In this embodiment, by scanning the object to be measured by randomly distributing binary stripes containing at least feature points to form a projection image, the purpose of using fewer sequence images to complete encoding and decoding of the object to be measured is achieved, thereby achieving the technical effect of shortening the time for encoding and decoding the object to be measured, and further solving the technical problem of low image scanning efficiency in related technologies.

[0052] At the same time, the method provided by the present application utilizes binary stripes with random bit truncation of the projection pattern and start and end features as active light pattern encoding and decoding technology, and adopts a stripe extraction algorithm based on spatial coding. Only one frame of two-dimensional image is needed for three-dimensional reconstruction, which greatly reduces the camera frame rate and the computing cost of the algorithm. Therefore, compared with the three-dimensional reconstruction method based on time coding in the prior art, which is difficult to realize real-time handheld scanning and requires high frame rate camera and high-speed algorithm support, the present method is easier to realize real-time handheld scanning, and the requirements for camera frame rate and algorithm are significantly reduced; the method provided by the present application eliminates the need for dynamic projection, simplifies the original projection system (such as: DLP, LCOS, etc.) into a transmission projection design, greatly reduces the hardware cost, so compared with the method based on the principle of microscopic confocal three-dimensional imaging in the prior art, the hardware processing cost is significantly reduced.

[0053] In order to enable technical personnel in this field to more clearly understand the technical solution of the present application, the implementation process of the three-dimensional reconstruction method of the present application will be described in detail below in combination with specific embodiments.

[0054] In step S202, the binary stripes in the projection image include at least the following forms: line segment stripes with endpoints as feature points and binary stripes with feature points randomly distributed on the stripes. The binary stripes are arranged according to a predetermined distribution rule to form the projection pattern, wherein the length of the line segment stripes is random, and the feature points include at least one of the following shapes: a horizontal line of preset length, a sphere, a cross and a square.

[0055] It should be noted that the shape features of the above-mentioned feature points are only exemplary and are not limited to the above-mentioned shape features, such as: triangle, ellipse, circle, hexagon, pentagon, T-shape, irregular shape, etc.

[0056] In another optional embodiment, the projected image may also include a pattern using a combination of binary stripes in various forms, for example, combining vertical line stripes of a first preset length in the vertical direction and horizontal line stripes of a second preset length in the horizontal direction to form a grid of line stripes; in yet another optional embodiment, adjusting the inclination angle of the line stripes to form a projection image in a manner that is not completely vertical and horizontal, for example, using multiple line stripes inclined at preset angles.

[0057] The predetermined distribution rules include but are not limited to the following methods: determining a first distribution rule of the binary stripes from a plurality of horizontal distribution rules in the predetermined distribution rules, wherein the plurality of horizontal distribution rules include at least one of the following: the horizontal spacing between adjacent binary stripes in the horizontal direction is equal, the horizontal spacing between adjacent binary stripes in the horizontal direction increases in sequence at equal intervals, and the horizontal spacing between adjacent binary stripes in the horizontal direction is randomly selected in a preset distance interval; determining a second distribution rule of the binary stripes from the predetermined distribution rules according to the type of the binary stripes; and arranging the binary stripes according to the first distribution rule and the second distribution rule to form the projection pattern.

[0058] Among them, the second distribution rule includes but is not limited to the following methods: when the binary stripes in the projection pattern are the line segment stripes, the second distribution rule is selected from multiple vertical distribution rules in the predetermined distribution rules, wherein the multiple vertical distribution rules include at least one of the following: the vertical distances between longitudinally adjacent line segment stripes are equal, the vertical distances between the longitudinally adjacent line segment stripes increase in sequence and at equal intervals, and the vertical distances between the longitudinally adjacent line segment stripes are randomly selected in a preset distance interval; when the binary stripes in the projection pattern are binary stripes with feature points randomly distributed on the stripes, the second distribution rule is selected from multiple feature point distribution rules in the predetermined distribution rules, wherein the multiple feature point distribution rules include at least: the spacing between the multiple feature points is equal, and the multiple feature points are randomly distributed on the stripes.

[0059] Figures 3 to 5 Schematic diagrams of a projection pattern are shown respectively, such as Figure 3 As shown, the projection pattern is composed of a plurality of randomly distributed line segments with random lengths, and the endpoint (feature point) of each line segment is a horizontal line of a preset length; Figure 4 As shown, the projection pattern is composed of multiple line segments with random lengths and randomly distributed stripes; Figure 5 As shown, the projection pattern is composed of a plurality of randomly distributed binary stripes, and a plurality of spherical feature points are randomly distributed on each binary stripe.

[0060] It should be noted that Figures 3 to 5 Only three projected images are shown as examples, not all patterns, so the projected images of this application are not limited to Figures 3 to 5 For example, a first preset length of vertical line stripes and a second preset length of horizontal line stripes are combined to form a grid of line stripes; in another optional manner, the line stripes are tilted at an angle that is not completely vertical or horizontal, for example, a plurality of line stripes tilted at a preset angle are used to form a projected image.

[0061] It should be understood that the projected image can be randomly or regularly composed according to the characteristics of the binary stripes, for example: replacing the feature points into a cross and a square, replacing the feature points into an ellipse and a hexagon, adjusting the length of the line stripes, changing the inclination angle, changing the combination method, changing the distribution rules and other features. In addition, the projected image of the present application also includes the ability to add coding information to the characteristics of the binary stripes.

[0062] In step S204, the corresponding adjacent domains of the feature points in the first image and the second image can be determined in the following manner: taking the feature points in the first image as an example, determining the feature points to be matched from the first image; determining a sub-image window of a preset size in the first image with the feature points to be matched as the center; and determining the area framed by the sub-image window as the first adjacent domain.

[0063] Specifically, a sub-image window of size M×N with a feature point in the first image as the center is called a neighborhood window of the feature point, and the area framed by the neighborhood window is determined as a neighboring domain. The method for determining the second neighboring domain corresponding to the second image is similar to that of the first image, and will not be repeated here.

[0064] There are many methods for matching feature points in the first image and the second image, such as: mean absolute difference algorithm (MAD), absolute error sum algorithm (SAD), error square sum algorithm (SSD), mean error square sum algorithm (MSD), normalized product correlation algorithm (NCC), sequential similarity detection algorithm (SSDA), Hadamard transform algorithm (SATD),

[0065] In an optional manner, a normalized cross-correlation method may be used to perform pixel-level fast matching. Taking determining a feature point in the second image that matches a feature point in the first image as an example, the first adjacent domain corresponding to the feature point to be matched in the first image is determined; the second adjacent domain corresponding to each feature point in the second image is determined in turn; the degree of matching between the feature point to be matched and each feature point in the second image is determined based on the grayscale value at the feature point to be matched, the grayscale value of each feature point in the second image, the grayscale average of the first adjacent domain corresponding to the feature point to be matched, and the grayscale average of the second adjacent domain corresponding to each feature point in the second image; the feature point with the highest matching degree and meeting the epipolar constraint condition is selected from the second image as the feature point in the second image that matches the feature point to be matched.

[0066] Specifically, a sub-image window of size m×n, centered on a feature in the image, is called the neighborhood window of the feature point. Let (x, y) be the horizontal and vertical coordinate positions of the image, S(x, y) be the similarity of the two neighborhood windows corresponding to (x, y), I(x, y) be the grayscale value at (x, y) of the target image, and I be the grayscale average of the neighborhood window of the target image; T(x, y) be the grayscale value at (x, y) of the source image, and T be the grayscale average of the neighborhood window of the source image. When searching for the correspondence of feature points in the left image (first image) in the right image (second image), the right image is the target image, and the left image is the source image, and vice versa, the similarity calculation is expressed using the following formula:

[0067]

[0068] Wherein, point (x′, y′) and point (x+x′, y+y′) are points in the neighborhood window corresponding to point (x, y), 0<x′<m, 0<y′<n.

[0069] It can be understood that the internal and external parameters of the binocular camera can be determined in the following way: extracting the coordinates of the feature points in the image in the calibration plate collected by the binocular camera; calculating the internal and external parameters of the binocular camera according to the coordinates of the feature points in the image in the calibration plate, and the specific steps of the above steps include: determining the target parameters that need to be calibrated from the multiple internal and external parameters of the binocular camera, and the target parameters include: the rotation angle and translation displacement of multiple feature points in the image on the calibration plate from the world coordinate system to the camera coordinate system, the camera focal length and the radial distortion coefficient; substituting the three-dimensional coordinates of the multiple feature points in the image on the calibration plate and the corresponding image coordinates into a preset linear equation group to determine the optimal solution of the preset linear equation group; determining the target parameters based on the optimal solution, and the process of determining the target parameters based on the optimal solution includes: determining the horizontal axis component and the vertical axis component in the rotation matrix corresponding to the rotation angle and the translation matrix corresponding to the translation displacement based on the optimal solution; determining the camera focal length and the radial distortion coefficient in turn based on the horizontal axis component and the vertical axis component in the rotation matrix and the translation matrix.

[0070] In actual application scenarios, the process of binocular calibration and optical plane calibration is that the calibration plate is connected to the motor through an axis and the plane of the calibration plate is kept at a certain angle (such as 30°) to the optical axis of the camera. The motor is controlled to drive the calibration plate to rotate parallel to the axis, and the camera is synchronously controlled to collect the image of the calibration plate and extract the coordinates of the feature points. The calibration is iteratively optimized using multiple feature point coordinates to calculate the internal and external parameters of the camera. In this process, you only need to click the start button to synchronize the movement of the calibration plate and the acquisition of images, and extract them at the same time. After the acquisition is completed, the calculation can be automatically optimized until the calibration result is obtained. There are six external parameters that need to be calibrated in the camera calibration work, namely the rotation angle around the three coordinate axes in the rotation matrix and the displacement along the three coordinate axes in the translation matrix; there are six internal parameters, namely C x , Cy , S x , S y , f, and k, where (C x ,C y ) is the coordinate of the center of the collected image, S x , S y are the number of pixels per unit distance in the X and Y directions in the image plane (pixels / mm), i.e., the proportionality coefficient. S x / S y =β, β is the scale factor, C x , C y , S y has been determined by pre-calibration. Therefore, only S among the six external parameters needs to be solved here x , effective focal length f and radial distortion coefficient k. In the process of scanner calibration, the binocular stereo vision imaging principle is used to extract the center coordinates of the circle as each feature point for stereo matching, and the triangulation imaging principle is used to reconstruct the three-dimensional coordinates of each feature point, and then the difference and gradient related information are used to obtain the dense point cloud. The pattern of the calibration plate is as follows: Figure 6 As shown in the figure, the center diameter and spacing are standard values. The center coordinates of the circle are extracted by center fitting as shown in the figure, and the center coordinates of the two-dimensional calibration plate are obtained by using multiple calibration plate images. Then, the internal and external parameters of the camera are calculated to complete the calibration algorithm process.

[0071] Optionally, the present invention adopts a camera model based on first-order radial distortion, and uses a step-by-step decomposition camera linear calibration method to gradually decompose each parameter, and by solving a set of linear equations, first calculate the rotation matrix, then calculate the translation matrix, and finally calculate the internal parameters focal length and radial distortion coefficient. The specific derivation is as follows:

[0072] According to the projection and perspective transformation theory and matrix transformation knowledge, the complete transformation from the three-dimensional world coordinate system to the computer image coordinate system can be divided into four steps:

[0073] (1) The transformation from the 3D space coordinate system to the camera coordinate system, i.e., from (x w ,y w , z w ) to (x, y, z).

[0074] in,

[0075] Among them, R and T are the rotation and translation transformation from the world coordinate system to the camera coordinate system, R is a 3×3 orthogonal matrix, T is a 3×1 translation vector, and there are six independent variables, namely, the three rotation angles reflecting the rotation transformation and the three translation components reflecting the translation transformation.

[0076] in, T=[T x T y T z ],ψ,θ, Represent the three angles of rotation transformation respectively.

[0077] (2) Ideal perspective projection transformation under the pinhole camera model, that is, from (x, y, z) to (X u , Y u )’s transformation.

[0078] in,

[0079] (3) Distortion model: describes the actual image coordinates (X d , Y d ) and the ideal image coordinates (X u , Y u ). Most experiments have shown that the distortion at the center of the image is small, while the distortion at the edge of the image is large, so As the distortion factor, the following distortion model is established:

[0080] Among them, X d =(1+kR d 2 )X u , formula (2.6); Y d =(1+kR d 2 )Y u , formula (2.7); R d 2 =X d 2 +Y d 2 , Rd is the square of the radial radius, and k is the radial distortion coefficient.

[0081] (4) Transformation from actual image coordinates to computer image coordinates, i.e., from (X d , Y d ) to (X f , Y f ).

[0082] Among them, X f =C x +S x X d , formula (2.8); Y f =C y +S y Y d , formula (2.9), (C x , C y ) is the coordinate of the center of the computer frame storage image (captured image), Sx , S y They are the number of pixels per unit distance in the X and Y directions in the image plane (pixels / mm), that is, the proportional coefficient.

[0083] Let S x / S y =β;X d '=(X f -C x ) / S y ; then X d =X d ' / β.

[0084] It can be seen that the parameters that need to be calibrated include:

[0085] External parameters: R and T have a total of six independent variables;

[0086] Internal parameters: f is the effective focal length, k is the radial distortion coefficient, S x ,S y are the proportional coefficients in the X and Y directions respectively, β is the scale factor, (C x , C y ) are the center coordinates of the image stored in the computer frame.

[0087] Among them, C x ,C y ,S y It can be obtained through pre-calibration.

[0088] Let the rotation matrix Translation Matrix Calculate the rotation matrix R and T of T x , T y Quantity.

[0089] From formula (2.3) we get From equations (2.4), (2.5), (2.6), and (2.7), we get

[0090] so Transpose the formula and we get:

[0091]

[0092] Optionally, for each calibration point, when its 3D coordinates and corresponding image coordinates are known, the above X d 'Equation. In the above X dIn the equation, the seven elements of the column vector are unknowns. Take seven calibration points and solve the linear equations to find the seven unknowns. Considering the random errors in the three-dimensional coordinate points and image coordinates during the calibration process, more calibration points (greater than 7) should be taken to find the optimal solution with the minimum total error of each calibration point according to the principle of least squares. Then, according to the orthogonal property of the rotation matrix R, the elements of the rotation matrix R and T can be further calculated. x and T y .

[0093] Optionally, the least square method is used to solve the above overdetermined equations (N>=7) to obtain the following variables:

[0094]

[0095] Alternatively, the orthogonality of R (standard orthogonal matrix) can be used to calculate β, T y ,T x and r1, r2, as follows:

[0096] (1) Calculate |T y |, where |T y |=(α5 2 +α6 2 +α7 2 ) 1 / 2 .

[0097] (2) Calculation of β and S x , where β=(α1 2 +α2 2 +α3 2 ) 1 / 2 |T y |, then S x =βS y .

[0098] (3) Obtain |T y |After that, there is T y The sign of X needs to be determined. In fact, from the imaging geometry, we know that d has the same sign as x, Y d and y have the same sign, which can be used to determine T y The sign of . Then we can get |T y | Then select a feature point P k (x k ,y k ,z k ), first assume that T y is positive, and using the variables solved by the above overdetermined equations (N>=7), r1-r6 and T can be calculated. x .

[0099] Optionally, the x and y corresponding to the feature point can be calculated. d , y and Y d The same sign, then T y The sign is positive, otherwise T y The sign is negative.

[0100] (4) Known T y and β, r1-r6, T can be directly calculated using the variables solved by the above overdetermined equations (N>=7) x Using the orthogonality and right-handedness of R (corresponding to the world coordinate system being a right-handed system), we can see that r7-r9 is obtained by the cross product of the first two rows:

[0101] r7=r2r6-r3r5; r8=r3r4-r1r6; r9=r1r5-r2r4.

[0102] Through the above process, the entire rotation matrix R and T in T have been solved x , T y components and the image scale factor β.

[0103] Optionally, calculate the effective focal length f, T z The components and the lens distortion coefficient k can be calculated:

[0104]

[0105] Assumptions:

[0106] H x =r1x w +r2y w +r3z w +T x ,

[0107] H y =r4x w +r5y w +r6z w +T y ,

[0108] W=r7x w +r8y w +r9z w , f k =f·k,

[0109] It can be concluded that:

[0110] H x ·f+H x r d 2 ·f k -X d·T z =X d ·W, H y ·f+H x r d 2 ·f k -Y d ·T z =Y d ·W.

[0111] Optionally, for N feature points, the least squares method is used to estimate the joint optimal parameters of the above two equations, and f, f k , T z , and then obtain f, k, T z .

[0112] According to another aspect of the present application, a three-dimensional reconstruction method is also provided. Figure 7 As shown, including:

[0113] Step S702, acquiring a first image and a second image, wherein the first image and the second image are obtained by respectively capturing a target image with a pre-calibrated monocular camera, the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is composed of at least a random distribution of binary stripes containing feature points;

[0114] Step S704, respectively determining positions of feature points in the first image and the second image, and respectively determining a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image, wherein the feature points in the first image correspond to the first adjacent domains in a one-to-one manner, and the feature points in the second image correspond to the second adjacent domains in a one-to-one manner;

[0115] Step S706, matching the feature points in the first image with the feature points in the second image one by one according to the matching degree between the first adjacent domain and the second adjacent domain;

[0116] Step S708: completing three-dimensional reconstruction of the object to be measured based on the matching feature points in the first image and the second image.

[0117] In an optional manner, the target image can be captured by a monocular camera at different times to obtain a first image and a second image; or the target image can be captured by a monocular camera to obtain a first image, and the first image can be modulated to obtain a second image.

[0118] According to another aspect of the present application, a method for three-dimensional reconstruction of teeth in an oral cavity is also provided. Figure 8 As shown, including:

[0119] Step S802, acquiring a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated binocular camera, the target image is formed by a projection pattern projected onto the surface of the tooth to be measured, and the projection pattern is at least composed of randomly distributed binary stripes containing feature points;

[0120] Step S804, respectively determining positions of feature points in the first image and the second image, and respectively determining a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image, wherein the feature points in the first image correspond to the first adjacent domains in one-to-one correspondence, and the feature points in the second image correspond to the second adjacent domains in one-to-one correspondence;

[0121] Step S806, matching the feature points in the first image with the feature points in the second image one by one according to the matching degree between the first adjacent domain and the second adjacent domain;

[0122] Step S808: completing the three-dimensional reconstruction of the tooth to be measured according to the matching feature points in the first image and the second image.

[0123] In actual application scenarios, the matching relationship between the start and end points is checked through the polar line constraint of binocular stereo vision, and the points and line segments with high correlation matching but no polar line constraint are eliminated; because the repeatability of the feature start and end points is taken into account when designing the structured light pattern, the position of the feature points is arranged in a random number manner, so that each column of stripes in the array pattern will not have repeated feature positions. Using the relationship of the limit constraint, the point pairs with high correlation matching but unable to meet the limit constraint can be eliminated to reduce the generation of mixed data.

[0124] It should be noted that the method and projection pattern based on any of the above embodiments can be applied to medical scanners such as oral scanners, as well as industrial scanners, professional scanners, facial scanners, desktop scanners, etc. There is no limitation on the scanning scene, and it can be used for scanning teeth, industrial products, cultural relics, racing cars, aerospace, clothing, human faces, and other scenes.

[0125] The embodiment of the present application also provides a three-dimensional reconstruction device. It should be noted that the three-dimensional reconstruction device of the embodiment of the present application can be used to execute the three-dimensional reconstruction method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0126] The following is an introduction to the three-dimensional reconstruction device provided in the embodiments of the present application.

[0127] Fig. 9 is a schematic diagram of a three-dimensional reconstruction device according to an embodiment of the present application, such as Fig. 9 As shown, the device includes: an acquisition module 90, used to acquire a first image and a second image, wherein the first image and the second image are obtained by respectively collecting target images by a pre-calibrated binocular camera, and the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is at least composed of binary stripes containing feature points; a determination module 92, used to respectively determine the positions of the feature points in the first image and the second image, and respectively determine a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image, wherein the feature points in the first image correspond to the first adjacent domains one-to-one, and the feature points in the second image correspond to the second adjacent domains one-to-one; a matching module 94, used to match the feature points in the first image with the feature points in the second image one-to-one according to the matching degree between the first adjacent domains and the second adjacent domains; and a reconstruction module 96, used to complete the three-dimensional reconstruction of the object to be measured according to the mutually matching feature points in the first image and the second image.

[0128] The acquisition module 90 includes: a projection submodule, which is used to determine the type of binary stripes, wherein the type of binary stripes includes at least one of the following: line stripes with endpoints as feature points and binary stripes with feature points randomly distributed on the stripes; the binary stripes are arranged according to a predetermined distribution rule to form a projection pattern, wherein the length of the line stripes is random, and the feature points include at least one of the following shapes: a horizontal line of preset length, a sphere, a cross and a square.

[0129] The projection submodule includes: a first determining unit and a second determining unit, wherein the second determining unit is used to determine a first distribution rule of binary stripes from a plurality of horizontal distribution rules in a predetermined distribution rule, wherein the plurality of horizontal distribution rules include at least one of the following: the horizontal spacings between adjacent binary stripes in the horizontal direction are equal, the horizontal spacings between adjacent binary stripes in the horizontal direction increase in sequence at equal intervals, and the horizontal spacings between adjacent binary stripes in the horizontal direction are randomly selected in a preset distance interval; determine a second distribution rule of the binary stripes from the predetermined distribution rule according to the type of the binary stripes; and arrange the binary stripes according to the first distribution rule and the second distribution rule to form a projection pattern.

[0130] The second determination unit is used to select a second distribution rule from a plurality of vertical distribution rules in the predetermined distribution rules when the binary stripes in the projection pattern are line stripes, wherein the plurality of vertical distribution rules include at least one of the following: the vertical distances between longitudinally adjacent line stripes are equal, the vertical distances between longitudinally adjacent line stripes increase in sequence at equal intervals, and the vertical distances between longitudinally adjacent line stripes are randomly selected in a preset distance interval; and when the binary stripes in the projection pattern are binary stripes with feature points randomly distributed on the stripes, select the second distribution rule from a plurality of feature point distribution rules in the predetermined distribution rules, wherein the plurality of feature point distribution rules include at least: the spacings between the plurality of feature points are equal, and the plurality of feature points are randomly distributed on the stripes.

[0131] Determine module 92, neighborhood submodule and feature point submodule, the neighborhood submodule is used to determine the feature point to be matched from the first image; determine a subimage window of a preset size in the first image with the feature point to be matched as the center; and determine the area framed by the subimage window as the first neighboring domain.

[0132] The feature point submodule is used to determine the first adjacent domain corresponding to the feature point to be matched in the first image; determine the second adjacent domain corresponding to each feature point in the second image in turn; determine the matching degree between the feature point to be matched and each feature point in the second image according to the gray value at the feature point to be matched, the gray value of each feature point in the second image, the gray average of the first adjacent domain corresponding to the feature point to be matched, and the gray average of the second adjacent domain corresponding to each feature point in the second image, and select the feature point with the highest matching degree and meeting the epipolar constraint condition from the second image as the feature point in the second image that matches the feature point to be matched.

[0133] The reconstruction module 96 includes an internal and external parameter submodule, which is used to extract the coordinates of feature points in the image in the calibration plate collected by the binocular camera; and calculate the internal and external parameters of the binocular camera according to the coordinates of the feature points in the image in the calibration plate.

[0134] The internal and external parameter submodule includes: a calibration unit and a solution unit. The calibration unit is used to determine the target parameters that need to be calibrated from multiple internal and external parameters of the binocular camera. The target parameters include: the rotation angle and translation displacement of multiple feature points of the image in the calibration plate from the world coordinate system to the camera coordinate system, the camera focal length and the radial distortion coefficient; the three-dimensional coordinates of the multiple feature points of the image in the calibration plate and the corresponding image coordinates are substituted into a preset linear equation group to determine the optimal solution of the preset linear equation group; the target parameters are determined based on the optimal solution.

[0135] The solving unit is used to determine the horizontal axis component and the vertical axis component in the rotation matrix corresponding to the rotation angle and the translation matrix corresponding to the translation displacement based on the optimal solution; and determine the camera focal length and radial distortion coefficient in turn based on the horizontal axis component and the vertical axis component in the rotation matrix and the translation matrix.

[0136] The three-dimensional reconstruction device includes a processor and a memory, and the above units are stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions. The above modules are all located in the same processor; or, the above modules are located in different processors in any combination.

[0137] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters are adjusted to scan the object under test by randomly distributing binary stripes containing feature points to form a projection image, so as to achieve the purpose of completing the encoding and decoding of the object under test with fewer sequence images, thereby achieving the technical effect of shortening the encoding and decoding time of the object under test, and thus solving the technical problem of low image scanning efficiency in related technologies.

[0138] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0139] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, a device where the computer-readable storage medium is located is controlled to execute a three-dimensional reconstruction method.

[0140] An embodiment of the present invention provides an oral scanner, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented: acquiring a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by pre-calibrated cameras (binocular cameras or monocular cameras), and the target image is formed by a projection pattern projected onto the surface of a measured object, and the projection pattern is composed of at least binary stripes containing feature points; determining positions of feature points in the first image and the second image respectively, and determining a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image respectively, wherein the feature points in the first image correspond one-to-one to the first adjacent domains, and the feature points in the second image correspond one-to-one to the second adjacent domains; matching the feature points in the first image with the feature points in the second image one-to-one according to the matching degree between the first adjacent domains and the second adjacent domains; and completing three-dimensional reconstruction of the measured object according to the mutually matching feature points in the first image and the second image.

[0141] Among them, the oral scanner can be an intraoral scanner or an extraoral scanner, which refers to a scanning device that can be used to obtain oral data.

[0142] It should be noted that the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located can execute Figure 2 The method shown can also be used to perform Figure 7 and Figure 8 The above-mentioned oral scanner can be used to perform Figure 2 The method shown can also be used to perform Figure 7 and Figure 8 The method shown.

[0143] Obviously, those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0144] The technicians in this field should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes.

[0145] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0149] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0150] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0151] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0152] The above are only preferred embodiments of the present application and are not intended to limit the present application. For technicians in this field, the present application may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A three-dimensional reconstruction method, It is characterized in that include: Acquire a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated binocular camera, the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is composed of at least binary stripes containing feature points; Determine positions of feature points in the first image and the second image respectively, and determine a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image respectively, wherein the feature points in the first image correspond to the first adjacent domains in a one-to-one manner, and the feature points in the second image correspond to the second adjacent domains in a one-to-one manner; Matching feature points in the first image with feature points in the second image one by one according to the matching degree between the first adjacent domain and the second adjacent domain; The three-dimensional reconstruction of the object to be measured is completed according to the mutually matching feature points in the first image and the second image.

2. The method according to claim 1, It is characterized in that The projection pattern is determined by: Determine the type of the binary stripes, wherein the type of the binary stripes includes at least one of the following: line segment stripes with endpoints as feature points and binary stripes with feature points randomly distributed on the stripes; The binary stripes are arranged according to a predetermined distribution rule to form the projection pattern, wherein the length of the line segment stripes is random, and the feature points include at least one of the following shapes: a horizontal line of preset length, a sphere, a cross, and a square.

3. The method according to claim 2, It is characterized in that The step of forming the projection pattern from the binary stripes according to a predetermined distribution rule comprises: Determining a first distribution rule of the binary stripes from a plurality of horizontal distribution rules in the predetermined distribution rules, wherein the plurality of horizontal distribution rules include at least one of the following: the horizontal spacings between the horizontally adjacent binary stripes are equal, the horizontal spacings between the horizontally adjacent binary stripes are increased in sequence and at equal intervals, and the horizontal spacings between the horizontally adjacent binary stripes are randomly selected in a preset distance interval; Determining a second distribution rule of the binary stripes from the predetermined distribution rules according to the type of the binary stripes; The binary stripes are arranged according to the first distribution rule and the second distribution rule to form the projection pattern.

4. The method according to claim 3, It is characterized in that Determining a second distribution rule of the binary stripes from the predetermined distribution rule according to the type of the binary stripes comprises: In the case where the binary stripes in the projection pattern are the line segment stripes, a second distribution rule is selected from a plurality of vertical distribution rules in the predetermined distribution rules, wherein the plurality of vertical distribution rules include at least one of the following: the vertical distances between longitudinally adjacent line segment stripes are equal, the vertical distances between the longitudinally adjacent line segment stripes increase in sequence at equal intervals, and the vertical distances between the longitudinally adjacent line segment stripes are randomly selected in a preset distance interval; in the case where the binary stripes in the projection pattern are binary stripes with feature points randomly distributed on the stripes, a second distribution rule is selected from a plurality of feature point distribution rules in the predetermined distribution rules, wherein the plurality of feature point distribution rules include at least: the spacings between the plurality of feature points are equal, and the plurality of feature points are randomly distributed on the stripes.

5. The method according to claim 1, It is characterized in that Determining a plurality of first adjacent domains corresponding to feature points in the first image includes: Determining feature points to be matched from the first image; Determining a sub-image window of a preset size in the first image with the feature point to be matched as the center; The area framed by the sub-image window is determined as the first adjacent domain.

6. The method according to claim 5, It is characterized in that The feature point to be matched is determined in the second image by the following method, which matches the feature point to be matched, including: Determine a first adjacent domain corresponding to the feature point to be matched in the first image; sequentially determining a second adjacent domain corresponding to each feature point in the second image; The degree of matching between the feature point to be matched and each feature point in the second image is determined based on the grayscale value at the feature point to be matched, the grayscale value of each feature point in the second image, the grayscale average of the first adjacent domain corresponding to the feature point to be matched, and the grayscale average of the second adjacent domain corresponding to each feature point in the second image, and the feature point with the highest matching degree and satisfying the epipolar constraint condition is selected from the second image as the feature point in the second image that matches the feature point to be matched.

7. The method according to claim 1, It is characterized in that The internal and external parameters of the binocular camera can be determined by the following methods, including: Extracting the coordinates of feature points of the image in the calibration plate captured by the binocular camera; The internal and external parameters of the binocular camera are calculated according to the coordinates of the feature points of the image in the calibration plate.

8. The method according to claim 7, It is characterized in that The step of calculating the internal and external parameters of the binocular camera according to the coordinates of the feature points of the image in the calibration plate includes: Determining target parameters that need to be calibrated from multiple internal and external parameters of the binocular camera, the target parameters include: rotation angles and translation displacements of multiple feature points of the image in the calibration plate from the world coordinate system to the camera coordinate system, the camera focal length, and the radial distortion coefficient; Substituting the three-dimensional coordinates of the plurality of feature points of the image in the calibration plate and the corresponding image coordinates into a preset linear equation group to determine the optimal solution of the preset linear equation group; The target parameter is determined based on the optimal solution.

9. The method according to claim 8, It is characterized in that The determining the target parameter based on the optimal solution includes: Determine, based on the optimal solution, a horizontal axis component and a vertical axis component in a rotation matrix corresponding to the rotation angle and a translation matrix corresponding to the translation displacement; The camera focal length and the radial distortion coefficient are determined in sequence based on the horizontal axis component and the vertical axis component in the rotation matrix and the translation matrix.

10. A three-dimensional reconstruction method, It is characterized in that include: Acquire a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated monocular camera, the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is composed of at least randomly distributed binary stripes containing feature points; Determine positions of feature points in the first image and the second image respectively, and determine a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image respectively, wherein the feature points in the first image correspond to the first adjacent domains in a one-to-one manner, and the feature points in the second image correspond to the second adjacent domains in a one-to-one manner; Matching feature points in the first image with feature points in the second image one by one according to the matching degree between the first adjacent domain and the second adjacent domain; The three-dimensional reconstruction of the object to be measured is completed according to the mutually matching feature points in the first image and the second image.

11. The method according to claim 10, It is characterized in that Acquiring a first image and a second image, comprising: The target image is captured by the monocular camera at different times to obtain the first image and the second image; Or the target image is captured by the monocular camera to obtain the first image, and the first image is modulated to obtain the second image.

12. A method for three-dimensional reconstruction of teeth inside the oral cavity, It is characterized in that include: Acquire a first image and a second image, wherein the first image and the second image are obtained by respectively capturing target images by a pre-calibrated binocular camera, the target image is formed by a projection pattern projected onto the surface of the tooth to be measured, and the projection pattern is composed of at least a random distribution of binary stripes containing feature points; Determine positions of feature points in the first image and the second image respectively, and determine a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image respectively, wherein the feature points in the first image correspond to the first adjacent domains in a one-to-one manner, and the feature points in the second image correspond to the second adjacent domains in a one-to-one manner; Matching feature points in the first image with feature points in the second image one by one according to the matching degree between the first adjacent domain and the second adjacent domain; The three-dimensional reconstruction of the tooth to be measured is completed according to the mutually matching feature points in the first image and the second image.

13. A three-dimensional reconstruction device, It is characterized in that include: An acquisition module, used to acquire a first image and a second image, wherein the first image and the second image are obtained by respectively acquiring a target image by a pre-calibrated binocular camera, the target image is formed by a projection pattern projected onto the surface of the object to be measured, and the projection pattern is composed of at least binary stripes containing feature points; a determination module, configured to respectively determine positions of feature points in the first image and the second image, and respectively determine a plurality of first adjacent domains corresponding to the feature points in the first image and a plurality of second adjacent domains corresponding to the feature points in the second image, wherein the feature points in the first image correspond to the first adjacent domains in a one-to-one manner, and the feature points in the second image correspond to the second adjacent domains in a one-to-one manner; a matching module, configured to match feature points in the first image with feature points in the second image one by one according to a matching degree between the first adjacent domain and the second adjacent domain; A reconstruction module is used to complete three-dimensional reconstruction of the object under test based on the matching feature points in the first image and the second image.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the three-dimensional reconstruction method according to any one of claims 1 to 11.

15. An oral scanner, It is characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the three-dimensional reconstruction method described in any one of claims 1 to 11.

Images