Three-dimensional reconstruction method and device, system

Abstract

The application discloses a three-dimensional reconstruction method and device and system. The method comprises the following steps: acquiring a first image and a second image, the first image and the second image being obtained by different image acquisition devices collecting a predetermined code image projected onto a measured object surface, the predetermined code image containing a plurality of target codes randomly distributed according to a preset direction, and the target code being a line segment stripe; acquiring a first target pixel point of each code in the first image, and determining a matching pixel point of the first target pixel point of each code in the first image in the second image; and determining a three-dimensional coordinate of the first target pixel point according to at least a pixel coordinate of the first target pixel point in the first image and a pixel coordinate of the matching pixel point in the second image, so as to complete three-dimensional reconstruction. The application solves the technical problem of low matching accuracy of codes according to code feature points due to unclear images.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction, and more specifically, to a three-dimensional reconstruction method, apparatus, and system. Background Technology

[0002] Structured light 3D reconstruction technology is a 3D reconstruction technique that projects an optically encoded pattern onto the surface of a measured object and recovers the object's 3D surface data through the acquired deformed pattern. It boasts high efficiency and anti-interference capabilities, making it widely applicable in various 3D reconstruction scenarios. The core issue in structured light 3D reconstruction technology is matching corresponding pixels. Different matching strategies rely on different encoding methods. Based on the encoding method of the projected pattern, structured light technology can be divided into temporal encoding and spatial encoding. Temporal encoding requires projecting multiple frames of patterns sequentially into the measurement scene, typically requiring the measured object and the projector to be relatively stationary. Therefore, high frame rate scanning is not possible, limiting its applicability to static scanning scenarios. Spatial encoding usually only requires projecting one pattern into the measured scene to complete 3D reconstruction. Currently, related technologies employ two methods: one uses circular symbol encoding, utilizing the relative displacement relationship of neighboring symbols for decoding; the other uses matching between image blocks to obtain 3D data. The former method has a smaller symbol encoding capacity, reconstructing only sparse symbol points, resulting in less data per frame and lower scanning efficiency; the latter method uses image blocks for matching, leading to lower accuracy.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a three-dimensional reconstruction method, apparatus, and system to at least solve the technical problem of low accuracy in matching symbols based on symbol feature points due to unclear images.

[0005] According to one aspect of the embodiments of this application, a three-dimensional reconstruction method is provided, comprising: acquiring a first image and a second image, wherein the first image and the second image are respectively acquired by different image acquisition devices and are projected onto the surface of a measured object, wherein the predetermined symbol images contain a plurality of target symbols randomly distributed in a preset direction, and the target symbols are line segments or stripes; acquiring a first target pixel of each symbol in the first image, and determining a matching pixel of the first target pixel of each symbol in the first image in the second image; determining the three-dimensional coordinates of the first target pixel at least based on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction.

[0006] Optionally, determining the matching pixel of the first target pixel of each symbol in the first image in the second image includes: determining the pixel coordinates and grayscale value of the first target pixel; determining a first region of a preset area in the first image and taking the center point of the first region as the first target pixel; determining a second region of a preset area in the second image, wherein the vertical coordinate of the center of the second region in the second image is the same as the vertical coordinate of the center of the first region in the first image; and determining the matching pixel that matches the first target pixel in the second region.

[0007] Optionally, determining matching pixels from the second region that match the first target pixel includes: determining the correlation coefficient of each pixel in the second region one by one; the correlation coefficient is used to characterize the correlation between the pixels in the second region and the first target pixel; and determining the pixel with the largest correlation coefficient in the second region as the matching pixel.

[0008] Optionally, the correlation coefficient of each pixel in the second region is determined one by one, including: determining the gray-scale mean of all pixels in the first region as the first gray-scale mean, and determining the gray-scale mean of all pixels in the second region as the second gray-scale mean; and determining the correlation coefficient between each pixel in the second region and the first target pixel based on the difference between the gray-scale value of the first target pixel in the first region and the first gray-scale mean, and the difference between the gray-scale value of each pixel in the second region and the second gray-scale mean.

[0009] Optionally, determining the pixel with the highest correlation coefficient in the second region as the matching pixel includes: determining the pixel with the highest correlation coefficient in the second region as a candidate matching point; if the candidate matching point coincides with the second target pixel in the second image, determining the candidate matching point as the matching pixel; if the candidate matching point does not coincide with the second target pixel in the second image, determining the second target pixel within a preset range around the candidate matching point as the matching pixel.

[0010] Optionally, the predetermined symbol image contains multiple target symbols randomly distributed in a preset direction. The target symbols are line segments or stripes. The process includes: determining the target region where the target symbol is located based on the length of the target symbol, the width of the target symbol, the spacing between the target symbols, and the pixel coordinates of the center pixel of the target symbol, wherein the pixel coordinates of the center pixel of the target symbol are randomly generated within the region of the symbol image; traversing all pixels within the target region, and generating a target symbol in the target region if no target symbol exists in the target region, wherein the target symbol includes at least a line segment of a preset length and two endpoints corresponding to the line segment of the preset length; and generating target symbols in all target regions within the symbol image region.

[0011] Optionally, the above method further includes: determining a first neighboring code set for any code in the first image and multiple second neighboring code sets for multiple candidate code elements in the second image; determining the number of matches between neighboring code elements in the multiple second neighboring code sets and neighboring code elements in the first neighboring code set; determining the second neighboring code set with the most matches among the multiple second neighboring code sets as the target second neighboring code set; and determining the candidate code element corresponding to the target second neighboring code set as the target code element.

[0012] Optionally, the method further includes: determining a plurality of first target pixels in the first image and a plurality of second target pixels in the second image; and matching the plurality of first target pixels with the plurality of second target pixels in the second image one by one.

[0013] According to another aspect of the embodiments of this application, a three-dimensional reconstruction apparatus is also provided, comprising: an acquisition module for acquiring a first image and a second image, wherein the first image and the second image are respectively acquired by different image acquisition devices from predetermined symbol images projected onto the surface of a measured object, the predetermined symbol images containing a plurality of target symbols randomly distributed in a preset direction, the target symbols being line segments or stripes; a matching module for acquiring a first target pixel of each symbol in the first image and determining a matching pixel of the first target pixel of each symbol in the first image in the second image; and a reconstruction module for determining the three-dimensional coordinates of the first target pixel at least based on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction.

[0014] According to another aspect of the embodiments of this application, a three-dimensional reconstruction system is also provided, applied to a three-dimensional reconstruction method, characterized in that it includes: at least two image acquisition devices, a projection device, and a first processor; the projection device is used to project a predetermined symbol image onto the surface of a measured object; at least two image acquisition modules are used to acquire the predetermined symbol image from the surface of the measured object to obtain a first image and a second image; the first processor is used to acquire a first target pixel of each symbol in the first image, and determine a matching pixel of the first target pixel of each symbol in the first image in the second image; and is also used to determine the three-dimensional coordinates of the first target pixel at least based on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction.

[0015] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute the above-described three-dimensional reconstruction method.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor; the processor is used to run a program, wherein the program executes the above-described three-dimensional reconstruction method when it runs.

[0017] In this embodiment, a first image and a second image are acquired. The first image and the second image are respectively acquired by different image acquisition devices, which acquire predetermined symbol images projected onto the surface of the object being measured. The predetermined symbol images contain multiple target symbols randomly distributed in a preset direction, and the target symbols are line segments or stripes. The first target pixel of each symbol in the first image is acquired, and the matching pixel of the first target pixel of each symbol in the first image is determined in the second image. The three-dimensional coordinates of the first target pixel are determined based at least on the pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image to complete the three-dimensional reconstruction. By determining the encoding method of the matching pixel of the first target pixel of each symbol in the second image, the purpose of replacing the symbol feature point encoding method is achieved, thereby improving the technical effect of symbol matching accuracy and solving the technical problem of low accuracy of symbol matching based on symbol feature points due to unclear images. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a computer terminal (or mobile device) for a three-dimensional reconstruction method according to an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of a three-dimensional reconstruction method according to this application;

[0021] Figure 3 This is a schematic diagram of an optional encoding pattern according to an embodiment of this application;

[0022] Figure 4 These are schematic diagrams illustrating five optional symbol shapes according to embodiments of this application;

[0023] Figure 5 This is an optional three-dimensional reconstruction system according to an embodiment of this application;

[0024] Figure 6 This is an optional three-dimensional reconstruction apparatus according to an embodiment of this application;

[0025] The above figures include the following reference numerals:

[0026] 501. The object being measured; 502. The first processor; 503. The image acquisition device; 504. The projection device. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] According to an embodiment of this application, an embodiment of a model training method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, cloud servers, or similar computing devices. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a 3D reconstruction method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0031] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the 3D reconstruction method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the 3D reconstruction method of the application described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0033] The transmission module 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission module 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0034] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0035] According to an embodiment of this application, an embodiment of a three-dimensional reconstruction method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0036] Figure 2 This is a flowchart of a three-dimensional reconstruction method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0037] Step S202: Acquire a first image and a second image. The first image and the second image are respectively acquired by different image acquisition devices and are obtained by acquiring predetermined symbol images projected onto the surface of the object being measured. The predetermined symbol images contain multiple target symbols randomly distributed in a preset direction. The target symbols are line segments and stripes.

[0038] Step S204: Obtain the first target pixel of each symbol in the first image, and determine the matching pixel of the first target pixel of each symbol in the first image in the second image;

[0039] Step S206: Determine the three-dimensional coordinates of the first target pixel based at least on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction.

[0040] Through the above steps, the encoding method of the first target pixel of each code element in the first image and the matching pixel in the second image can be determined, thereby achieving the purpose of replacing the code element feature point encoding method, thus realizing the technical effect of improving the code element matching accuracy, and solving the technical problem of low accuracy of code element matching based on code element feature points due to unclear images.

[0041] It should be noted that 3D reconstruction refers to establishing a mathematical model of a 3D object suitable for computer representation and processing. It forms the basis for processing, manipulating, and analyzing the object's properties in a computer environment, and is also a virtual reality technology for representing objective events within a computer. 3D reconstruction technology typically uses structured light temporal coding and spatial coding techniques. The challenge of spatial coding structured light lies in using pixel spatial grayscale information to stably and reliably encode and decode each pixel. One approach involves encoding specific pixels using certain symbolic information. This is achieved through a coding method using circular symbols of varying sizes, utilizing epipolar constraints and the relative displacement relationship between neighboring symbols and symbols in the image to encode and decode each symbol, thus reconstructing each symbol point. The drawbacks of this method are limited symbolic encoding capacity, resulting in only sparse symbol points that can be reconstructed, small data sets per frame, low scanning efficiency, and significant dependence on object surface texture. Another approach uses random speckle patterns to match corresponding points based on the correlation between pixel blocks. For example, a pseudo-random speckle image is projected onto the surface of the object being measured using a projection device, and an improved SGM (semi-global matching) algorithm is used to match image blocks, thereby obtaining the 3D data of the object's surface. The drawback of this method is that it applies to a large image block space, resulting in poor accuracy and detail in the reconstructed data, making it difficult to reconstruct complex objects. Related technologies generally use symbol feature point matching to complete symbol matching; however, when image clarity is poor or feature points are highly blurred, the accuracy of feature point extraction decreases, leading to a reduction in the accuracy of symbol matching.

[0042] The method proposed in this application utilizes the encoding method of line segment stripe symbols to obtain the first target pixel point in the line segment stripes, such as the midpoint of the line segment stripes. Matching is performed using the first target feature point of the symbol, replacing the method of matching using symbol feature points, thereby improving the accuracy of symbol matching even in unclear image conditions. Furthermore, the use of line segment stripe symbols increases the encoding capacity, improving the data volume of a single frame image and thus enhancing scanning efficiency. Additionally, the method proposed in this application, with its line segment stripe distribution and high symbol density, further improves the accuracy of the acquired reconstructed data.

[0043] In step S202, the first image is an image acquired by the first image acquisition device, and the second image is an image acquired by the second image acquisition device. Multiple target symbols in the first image have the same orientation, and the spacing between the symbols is randomly determined. For example, the spacing between symbol 1 and symbol 2 is 100 μm, and the spacing between symbol 2 and symbol 3 is 80 μm. The symbols are randomly distributed vertically along the horizontal direction of the image (camera epipolar direction). Each symbol's feature contains at least two extractable grayscale feature point patterns, and the feature points are distributed vertically at a preset spacing. Figure 3 An optional symbol image is shown, in which symbols are randomly distributed in the vertical direction.

[0044] In step S204, the first target pixel of each symbol in the first image is acquired, and the matching pixel of the first target pixel of each symbol in the first image is determined in the second image. It can be understood that the first image and the second image are projected images onto the surface of the object being measured, acquired by different image acquisition devices. Therefore, the symbol structure in the first image is the same as the symbol structure in the second image. Thus, taking the first target pixel as the midpoint of a line segment stripe in the first image as an example, the matching pixel in the second image is the midpoint of a line segment stripe in the second image.

[0045] In step S206, the first target pixel in the first image corresponds one-to-one with the matching pixel in the second image. The three-dimensional coordinates of the first target pixel are determined based on at least the pixel coordinates of the first target pixel in the first image and the pixel coordinates of the corresponding matching pixel in the second image, thus completing the reconstruction.

[0046] The following detailed embodiments illustrate steps S202 to S206.

[0047] In step S204, the matching pixel of the first target pixel in the second image can be determined using the pixel coordinates and the gray value of the pixel. Specifically, the pixel coordinates and gray value of the first target pixel are determined; a first region of a preset area is determined in the first image, and the center point of the first region is taken as the first target pixel; a second region of a preset area is determined from the second image, wherein the vertical coordinate of the center of the second region in the second image is the same as the vertical coordinate of the center of the first region in the first image; and a matching pixel that matches the first target pixel is determined from the second region.

[0048] In one alternative approach, the first region and the second region can be image regions with an image size of m×n, where m and n are pixel values.

[0049] It should be noted that the above vertical coordinates are pixel coordinates. It can be understood that the vertical coordinate of the center of the second region in the second image is the same as the vertical coordinate of the center of the first region in the first image. This means that the distance of the center of the first region from the boundary of the first image in the preset direction is the same as the distance of the center of the second region from the same boundary of the second image in the preset direction.

[0050] In one alternative approach, the correlation coefficient between the first target pixel and each pixel in the second region can be compared. For example, the correlation coefficient of each pixel in the second region can be determined one by one. The correlation coefficient is used to characterize the correlation between the pixels in the second region and the first target pixel. The pixel with the largest correlation coefficient in the second region is determined as the matching pixel.

[0051] The correlation coefficient between each pixel in the second region and the first target pixel is determined as follows: First, the average gray level of all pixels in the first region is determined to be the first average gray level, and the average gray level of all pixels in the second region is determined to be the second average gray level. Based on the difference between the gray level of the first target pixel in the first region and the first average gray level, and the difference between the gray level of each pixel in the second region and the second average gray level, the correlation coefficient between each pixel in the second region and the first target pixel is determined.

[0052] Specifically, for the first target pixel (u) in the first image 1 v 1 Its grayscale value is I. 1 (u 1 v 1 Using any pixel as the center of a first region with image size m×n, a second region with image size m×n is determined in the second image, located at the same pixel row position as the first region. The correlation coefficient ω of each pixel in the second region is then calculated. i Let i = 0, 1, W-1, where W is the pixel width of the first image. The formula for calculating the correlation coefficient is as follows:

[0053]

[0054] Where m is the pixel width of the first region, and n is the pixel height of the first region. The average grayscale value of each pixel in the first region. The grayscale average of each pixel in the second region, (u 2 v 2 ) represents the pixel coordinates of the pixels in the second image that are involved in the correlation coefficient calculation, I 2 (u 2 )v 2 ) represents the grayscale value of the pixel in the second image that is involved in the correlation coefficient calculation.

[0055] It should be noted that the correlation coefficient can be the zero-mean normalized correlation coefficient determined by matching using the zero-mean normalized correlation coefficient (ZNCC) method. It describes the degree of similarity between two different image patches. The higher the correlation coefficient, the higher the similarity between the image patches, and the higher the probability that they are matching points.

[0056] Taking the first target pixel as the midpoint of a line segment stripe in the first image as an example, through the above steps, for each midpoint of a line segment stripe in the first image, a candidate matching point (the pixel with the highest correlation coefficient) can be found in the second image. It is obvious that since the first target pixel is the midpoint of the line segment stripe, the matching pixel in the second image that matches the first target pixel can only be a midpoint of a line segment stripe in the second image. Based on this principle, all candidate matching points are traversed. If a candidate matching point coincides with the second target pixel in the second image, the candidate matching point is determined as the matching pixel. If a candidate matching point does not coincide with the second target pixel in the second image, the second target pixels within a preset range around the candidate matching point are determined as the matching pixels.

[0057] In another alternative approach, after determining the matching pixel, the first target pixel can be used as a feature point to determine the first neighboring symbol set in the second image that matches any symbol in the first image, and the multiple second neighboring symbol sets in the second image that match multiple candidate symbols. The number of matches between neighboring symbols in the multiple second neighboring symbol sets and neighboring symbols in the first neighboring symbol set is determined, and the second neighboring symbol set with the most matches is determined as the target second neighboring symbol set. The candidate symbols corresponding to the target second neighboring symbol set are determined as symbols that match any of the aforementioned symbols.

[0058] Specifically, the first neighboring symbol set is the set of neighboring symbols of a symbol in the second image. For example, taking symbol p in the second image as a line segment of a set length and its two corresponding endpoints, the neighboring symbol set of symbol p is {p 1 ,p 2 ,p 3 ,p 4 The second neighborhood symbol set is the set of neighboring symbols of any symbol in the second image, which is the set of candidate symbols. For example, if the candidate symbol of symbol p is symbol q, the set of neighboring symbols of symbol q is {q}. 1 ,q 2 ,q 3 ,q 4}, in the case where there are three candidate codewords for codeword p, for example: q i If i = 1, 2, 3, then there exist three sets of second neighboring symbols. When there are multiple code elements p, the situation is similar to the one described above, and will not be repeated here.

[0059] It should be further explained that, taking symbol p as an example, for each candidate symbol q of symbol p... i If i is a positive integer, first determine the code element p. 1 Is it related to the first neighboring symbol of candidate symbol q1? Matching is performed, and then all candidate symbols of symbol p are searched sequentially to determine the neighboring symbols of all candidate symbols and symbol p. 1 Whether it matches, with symbol p 1 For each matched candidate code, increment the number of matched data by 1. For example: code With code element p 1 If a match is found, then the number of matches for candidate code q1 is 1. If the code... With code element p 2 If a match is found, the number of matches for candidate codeword q1 is incremented by 1 to 2. The candidate codeword with the largest number of matches is determined as the target codeword. Taking codeword p as an example, if candidate codeword q1 has the largest number of matches with codeword p, then candidate codeword q1 is determined as the target codeword and matches with codeword p.

[0060] In step S202, the predetermined symbol image is generated as follows: The target region containing the target symbol is determined based on the length, width, spacing, and pixel coordinates of the center pixel of the target symbol. The pixel coordinates of the center pixel are randomly generated within the region of the symbol image. All pixels within the target region are traversed. If no target symbol exists in the target region, a target symbol is generated within that region. Each target symbol includes at least a line segment of a preset length and its two corresponding endpoints. Target symbols are generated in all target regions within the symbol image region. Figure 4 As shown, five target symbols are illustrated. Symbol 1 consists of a line segment and two circular endpoints corresponding to the line segment. Symbol 2 consists of a line segment of a first preset length and two line segments of a second preset length as endpoints, where the first preset length is greater than the second preset length. Symbol 3 consists of a line segment of a first preset length and three line segments of a second preset length as endpoints. Symbol 4 consists of a line segment and two endpoints of the line segment itself. Symbol 5 consists of a line segment intersecting with another line segment.

[0061] Once the projection module and image acquisition module of the reconstruction system are determined, the magnification of the projection equipment... Magnification of the image acquisition module These are inherent parameters of the system. The unit pixel length l of the symbol image. p The unit pixel length l of the first image cThere is a relationship between them: Since the minimum length of a unit pixel in a symbolic image that a projection device can project is determined to be l min According to this formula, the minimum length L of the stripes in the first image can be determined. mim To ensure the randomness of the projected pattern with width W and height H, the maximum stripe length L in the pattern is... max If the length of each stripe does not exceed H / 2, then the length of each stripe is L. i ∈[L min L max ].

[0062] Specifically, the stripe length L in the projected pattern can be a fixed value or a random value. If it is a random value, the length of each stripe can be determined by a pseudo-random sequence {L}. i It is determined that the range of values ​​for the pseudo-random sequence is L. i ∈[L min L max ].

[0063] During the generation of the symbol image, the target region where the target symbol is located is determined based on the coordinates of the center pixel of the target symbol, the length of the target symbol, and the width of the target symbol. Only one target symbol exists within this target region. For example... Figure 5 The diagram shows the distribution of the first line segment stripe 201 in an image color patch. The stripe length is L, the stripe width is S, and the stripe spacing is G / 2. The area occupied by the first line segment stripe 201 in the image is (S+G)×(L+G). An image color patch can contain only one stripe, or none at all. In the image of W×H stripes to be filled, a random coordinate position (u, v) is generated as the center position of a candidate image color patch. Then, each pixel in the image to be filled corresponding to this color patch is traversed, checking whether the candidate image color patch already contains stripes. If no stripes are present, a stripe is generated at the candidate color patch position; otherwise, no stripe is generated. Next, the generation of the next random coordinate, the search for the color patch, and the generation of the stripe are performed. This process is repeated until no more stripes can be generated in the entire image to be filled.

[0064] When the first target pixel is the midpoint of a line segment stripe in the first image, the only matching point can be the midpoint of a line segment stripe in the second image. The midpoint of the line segment stripe in the second image is then designated as the second target pixel. The correlation coefficient between each second target pixel and the first target pixel is directly determined, and the second target pixel with the highest correlation coefficient is selected as the matching pixel for the first target pixel. This reduces the number of pixel calculations and thus speeds up the matching process.

[0065] In some embodiments of this application, before determining multiple candidate symbols corresponding to the target symbol in the second image in the first image, the method further includes: adjusting the first image and the second image to the same plane.

[0066] It should be noted that in the field of stereo vision, there is an epipolar constraint relationship between the first and second images. That is, feature points on the same epipolar line in the projected image will also be on the same epipolar line in the image acquired by the image acquisition device. Using the principle of epipolar correction, the corresponding epipolar lines in the first and second images can be corrected to the same horizontal direction. After correction, the first and second images are filtered and smoothed, including but not limited to Gaussian filtering.

[0067] The first and second images are generated by projecting the image to be projected onto the surface of the object being measured. The projection parameters of each symbol can be determined based on the preset calibration results.

[0068] Based on the above method, this application can achieve rapid and accurate reconstruction of the three-dimensional data of the surface of the measured object. The stripe center point reconstruction method in this application can achieve accurate three-dimensional data acquisition; the random stripe method increases the number of encoded points (any pixel at the center of the stripe is an encoded point), improves data redundancy, and thus improves scanning efficiency.

[0069] This application also provides a three-dimensional reconstruction system, such as... Figure 5 As shown, it includes: at least two image acquisition devices 503, a projection device 504, and a first processor 502; the projection device 504 is used to project a predetermined symbol image onto the surface of the object under test; the at least two image acquisition modules 503 are used to acquire predetermined symbol images from the surface of the object under test 501 to obtain a first image and a second image; the first processor 502 is used to acquire a first target pixel of each symbol in the first image, and determine the matching pixel of the first target pixel of each symbol in the first image in the second image; it is also used to determine the three-dimensional coordinates of the first target pixel based on the predetermined image parameters of the first image and the second image to complete the three-dimensional reconstruction.

[0070] The image acquisition device 503 includes, but is not limited to, grayscale and color cameras, and the projection device 504 can project structures using methods including, but not limited to, DLP (Digital Light Processing), MASK (mask projection), and DOE (Diffraction Projection), capable of projecting structured light patterns. Alternatively, multiple image acquisition devices may be used.

[0071] In one alternative approach, the first processor 502 can pre-generate a symbol image, which is then transmitted to the projection device 504. The projection device 504 projects the symbol image onto the surface of the object under test 501. Two image acquisition devices 503 then acquire the first image and the second image respectively and transmit them to the first processor 502 for symbol matching. After the symbol matching is completed, the three-dimensional coordinates of each symbol in the first image are determined, and the three-dimensional reconstruction is completed.

[0072] In some embodiments of this application, a symbol image can be displayed on a display interface by inputting generation instructions on an interactive device connected to the first processor 502, and the displayed symbol image can also be modified; at the same time, various data during the reconstruction process can be displayed on the interactive device for real-time monitoring.

[0073] This application also provides a model training device, such as... Figure 6 As shown, it includes: an acquisition module 60, used to acquire a first image and a second image, the first image and the second image being acquired by different image acquisition devices from predetermined symbol images projected onto the surface of the object being measured, the predetermined symbol images containing multiple target symbols randomly distributed according to a preset direction, the target symbols being line segments and stripes; a matching module 62, used to acquire the first target pixel of each symbol in the first image and determine the matching pixel of the first target pixel of each symbol in the first image in the second image; and a reconstruction module 64, used to determine the three-dimensional coordinates of the first target pixel based on the predetermined image parameters of the first image and the second image, so as to complete the three-dimensional reconstruction.

[0074] Matching module 62 includes: a first determining submodule, configured to determine the pixel coordinates and grayscale value of a first target pixel; determine a first region of a preset area in a first image, and use the center point of the first region as the first target pixel; determine a second region of a preset area from a second image, wherein the vertical coordinate of the center of the second region in the second image is the same as the vertical coordinate of the center of the first region in the first image; and determine a matching pixel from the second region that matches the first target pixel.

[0075] The first determining submodule includes: a first determining unit and a second determining unit; the first determining unit is used to determine the correlation coefficient of each pixel in the second region one by one; the correlation coefficient is used to characterize the correlation between the pixel in the second region and the first target pixel; the pixel with the largest correlation coefficient in the second region is determined as the matching pixel; the second determining unit is used to determine that the gray-scale mean of all pixels in the first region is the first gray-scale mean, and to determine that the gray-scale mean of all pixels in the second region is the second gray-scale mean; based on the difference between the gray-scale value of the first target pixel in the first region and the first gray-scale mean, and the difference between the gray-scale value of each pixel in the second region and the second gray-scale mean, the correlation coefficient between each pixel in the second region and the first target pixel is determined.

[0076] The first determining unit includes: a determining subunit, which is used to determine the pixel with the largest correlation coefficient in the second region as a candidate matching point; if the candidate matching point coincides with the second target pixel in the second image, the candidate matching point is determined as a matching pixel. If the candidate matching point does not coincide with the second target pixel in the second image, the second target pixel within a preset range around the candidate matching point is determined as a matching pixel.

[0077] The matching module 62 further includes a matching submodule, which is used to determine a plurality of first target pixels in the first image and a plurality of second target pixels in the second image; and to match the plurality of first target pixels with the plurality of second target pixels in the second image one by one.

[0078] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, including a stored program, wherein the program controls the device where the non-volatile storage medium is located to execute the above-described three-dimensional reconstruction method when it is running.

[0079] According to another aspect of the embodiments of this application, a processor is also provided, which is used to run a program, wherein the program executes the above-described three-dimensional reconstruction method when it runs.

[0080] The processor described above is used to run a program that performs the following functions: acquiring a first image and a second image, wherein the first image and the second image are respectively acquired by different image acquisition devices and projected onto the surface of the object being measured, wherein the predetermined symbol images contain multiple target symbols randomly distributed in a preset direction, and the target symbols are line segments or stripes; acquiring the first target pixel of each symbol in the first image, and determining the matching pixel of the first target pixel of each symbol in the first image in the second image; determining the three-dimensional coordinates of the first target pixel based at least on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction.

[0081] The processor executes the three-dimensional reconstruction method described above. By determining the encoding method of the first target pixel of each symbol in the first image and the matching pixel in the second image, it achieves the purpose of replacing the symbol feature point encoding method, thereby improving the technical effect of symbol matching accuracy. This solves the technical problem of low accuracy of symbol matching based on symbol feature points caused by unclear images.

[0082] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0084] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0085] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0086] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0087] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A three-dimensional reconstruction method, characterized in that, include: A first image and a second image are acquired. The first image and the second image are respectively acquired by different image acquisition devices and are projected onto the surface of the object being measured. The predetermined symbol images contain multiple target symbols randomly distributed in a preset direction. The target symbols are line segments or stripes. Each target symbol includes at least one line segment of a preset length and two endpoints corresponding to the line segment of the preset length. The endpoints include at least one of the following: circular endpoints, square endpoints, or line segment endpoints. Obtain the first target pixel of each symbol in the first image, and determine the matching pixel of the first target pixel of each symbol in the first image in the second image; The three-dimensional coordinates of the first target pixel are determined based at least on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction; Determining the matching pixel of the first target pixel of each symbol in the first image in the second image includes: determining the pixel coordinates and grayscale value of the first target pixel; determining a first region of a preset area in the first image, wherein the first target pixel is the center point of the first region; determining a second region of the preset area in the second image, wherein the vertical coordinate of the center of the second region in the second image is the same as the vertical coordinate of the center of the first region in the first image; and determining a matching pixel that matches the first target pixel in the second region, wherein the pixel coordinates of the center pixel of the target symbol are randomly generated within the region where the symbol image is located.

2. The method according to claim 1, characterized in that, Determining a matching pixel in the second region that matches the first target pixel includes: The correlation coefficient of each pixel in the second region is determined one by one; the correlation coefficient is used to characterize the correlation between the pixels in the second region and the first target pixel. The pixel with the highest correlation coefficient in the second region is determined as the matching pixel.

3. The method according to claim 2, characterized in that, Determine the correlation coefficient of each pixel in the second region one by one, including: The average gray level of all pixels in the first region is determined to be the first average gray level, and the average gray level of all pixels in the second region is determined to be the second average gray level. Based on the difference between the gray value of the first target pixel in the first region and the first gray value mean, and the difference between the gray value of each pixel in the second region and the second gray value mean, the correlation coefficient between each pixel in the second region and the first target pixel is determined.

4. The method according to claim 2, characterized in that, The pixel with the highest correlation coefficient in the second region is determined as the matching pixel, including: The pixel with the highest correlation coefficient in the second region is determined as the candidate matching point; If the candidate matching point coincides with the second target pixel in the second image, the candidate matching point is determined as the matching pixel, wherein the second target pixel is the midpoint of the line segment stripe in the second image; If the candidate matching point does not coincide with the second target pixel in the second image, the second target pixel within a preset range around the candidate matching point is determined as the matching pixel.

5. The method according to claim 1, characterized in that, The predetermined symbol image contains multiple target symbols randomly distributed according to a preset direction. The target symbols are line segments or stripes, including: The target region where the target code is located is determined based on the length of the target code, the width of the target code, the spacing between the target code and the pixel coordinates of the center pixel of the target code, wherein the pixel coordinates of the center pixel of the target code are randomly generated within the region where the code image is located; Traverse all pixels within the target region, and if the target symbol does not exist in the target region, generate the target symbol in the target region. The target symbol is generated in all target regions within the symbol image region.

6. The method according to claim 1, characterized in that, The method further includes: Determine the first neighboring symbol set of any symbol in the first image and the multiple second neighboring symbol sets of multiple candidate symbols in the second image; Determine the number of neighboring symbols in the plurality of second neighboring symbol sets that match the neighboring symbols in the first neighboring symbol set, and determine the second neighboring symbol set with the largest number of matches in the plurality of second neighboring symbol sets as the target second neighboring symbol set; The candidate code corresponding to the target second neighborhood code set is determined as the code that matches any of the above code.

7. The method according to claim 1, characterized in that, The method further includes: Identify multiple first target pixels in the first image and multiple second target pixels in the second image; The plurality of first target pixels are matched one by one with the plurality of second target pixels in the second image.

8. A three-dimensional reconstruction device, characterized in that, include: The acquisition module is used to acquire a first image and a second image. The first image and the second image are respectively acquired by different image acquisition devices from predetermined symbol images projected onto the surface of the object being measured. The predetermined symbol images contain multiple target symbols randomly distributed in a preset direction. The target symbols are line segments or stripes. Each target symbol includes at least one line segment of a preset length and two endpoints corresponding to the line segment of the preset length. The endpoints include at least one of the following: a circular endpoint, a square endpoint, or a line segment-shaped endpoint. The matching module is used to obtain the first target pixel of each symbol in the first image and determine the matching pixel of the first target pixel of each symbol in the first image in the second image; The reconstruction module is used to determine the three-dimensional coordinates of the first target pixel based at least on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction. Determining the matching pixel of the first target pixel of each symbol in the first image in the second image includes: determining the pixel coordinates and grayscale value of the first target pixel; determining a first region of a preset area in the first image, wherein the first target pixel is the center point of the first region; determining a second region of the preset area in the second image, wherein the vertical coordinate of the center of the second region in the second image is the same as the vertical coordinate of the center of the first region in the first image; and determining a matching pixel that matches the first target pixel in the second region, wherein the pixel coordinates of the center pixel of the target symbol are randomly generated within the region where the symbol image is located.

9. A three-dimensional reconstruction system, applied to a three-dimensional reconstruction method, characterized in that, include: At least two image acquisition devices, a projection device, and a first processor; The projection device is used to project a predetermined symbol image onto the surface of the object being measured. The target symbol includes at least a line segment of a preset length and two endpoints corresponding to the line segment of the preset length. The endpoints include at least one of a circular endpoint, a square endpoint, or a line segment endpoint. The at least two image acquisition modules are used to acquire the predetermined symbol images from the surface of the object under test to obtain a first image and a second image; The first processor is configured to acquire a first target pixel of each symbol in the first image and determine a matching pixel of the first target pixel of each symbol in the first image in the second image; it is also configured to determine the three-dimensional coordinates of the first target pixel based at least on the predetermined pixel coordinates of the first target pixel in the first image and the pixel coordinates of the matching pixel in the second image, so as to complete the three-dimensional reconstruction. Determining the matching pixel of the first target pixel of each symbol in the first image in the second image includes: determining the pixel coordinates and grayscale value of the first target pixel; determining a first region of a preset area in the first image, wherein the first target pixel is the center point of the first region; determining a second region of the preset area in the second image, wherein the vertical coordinate of the center of the second region in the second image is the same as the vertical coordinate of the center of the first region in the first image; and determining a matching pixel that matches the first target pixel in the second region, wherein the pixel coordinates of the center pixel of the target symbol are randomly generated within the region where the symbol image is located.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the non-volatile storage medium to perform the three-dimensional reconstruction method according to any one of claims 1 to 7.

11. An electronic device, characterized in that, include: Memory and processor; The processor is used to run a program, wherein the program executes the three-dimensional reconstruction method according to any one of claims 1 to 7.

Images