A 3D target reconstruction method and device based on binocular structured light

Abstract

The present invention discloses a method and device for 3D object reconstruction based on binocular structured light. The method comprises: constructing a binocular structured light system comprising a projector, a first camera, and a second right camera, and determining an optimal working distance that satisfies a first error function; using the first camera and the second camera to respectively acquire a first point cloud and a second point cloud corresponding to the 3D target object under projection by the projector; and registering and fusing the first point cloud and the second point cloud using the determined overall position transformation relationship between the point clouds, wherein the overall position transformation relationship between the point clouds is obtained by coarse and fine registration of standard parts. The present invention has the advantages of high accuracy, high speed, and high reconstructed image integrity.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and more specifically, to a 3D target reconstruction method and device based on binocular structured light. Background Art

[0002] 3D object reconstruction can be applied in a variety of fields. For example, fingerprints, as a human biometric, are unique and difficult to forge. Among various biometric technologies such as facial recognition and iris recognition, fingerprint recognition is the most widely used due to its convenience, harmlessness, and ease of use. In the judicial field, police can identify suspects by extracting fingerprints from crime scenes. In information security, fingerprint recognition can be used in applications such as personal household registration and financial transactions that strictly require confidentiality. In daily life, fingerprint recognition can also be used to lock doors and unlock mobile phones, providing a high level of protection. Given the wide range of applications for fingerprint recognition, obtaining complete, high-quality fingerprints and improving fingerprint recognition accuracy are particularly important.

[0003] Currently, commonly used 3D fingerprint reconstruction techniques include methods based on stereo vision, photometry, and structured light. Stereo vision-based fingerprint reconstruction typically uses multiple cameras to simultaneously capture the fingerprint from different angles. Disparity is then calculated based on stereo matching to derive the fingerprint's depth information. This method suffers from a lack of image feature points and their uneven distribution, making it difficult to reconstruct a complete fingerprint. Photometry-based 3D fingerprint reconstruction often requires using multiple light sources to provide varying illumination, capturing multiple 2D fingerprint images, calculating the directional gradient of the object's surface, and estimating the surface normal. While this method can well preserve detail information, it only captures the normal integral, not the absolute height. Furthermore, fingerprint reconstruction is slow, and it's difficult to keep the finger completely still during the capture process, which can introduce errors. Structured light-based 3D fingerprint reconstruction typically uses monocular structured light, which limits the captured fingerprint area and the number of features and minutiae, impacting fingerprint recognition accuracy. For example, existing solutions include patent application CN113570699A (a method and device for three-dimensional fingerprint reconstruction) and patent application CN113505626A (a method and system for rapid three-dimensional fingerprint acquisition).

[0004] In summary, existing 3D fingerprint reconstruction solutions have the following main problems:

[0005] 1) The photometric method cannot obtain the absolute distance of the fingerprint height, and the reconstruction speed is slow; the stereo vision method has a fast reconstruction speed, but the reconstruction effect is often poor.

[0006] 2) Existing 3D fingerprint reconstruction methods (such as monocular structured light) only obtain a limited fingerprint area, making it difficult to reconstruct a complete 3D fingerprint.

[0007] 3) The existing binocular structured light technology method for collecting 3D fingerprints uses sinusoidal stripes combined with Gray code, which has no advantage in reconstruction speed (such as patent application CN113505626A).

[0008] 4) The sizes of different fingers of an individual and the same finger of different individuals vary, making it difficult to find the optimal working distance to ensure accurate reconstruction of surface fingerprints of different individuals and different fingers. In addition, after the height of the equipment is adjusted, there is a certain degree of settling. If the optimal working distance cannot be accurately determined, reconstruction errors will occur. Summary of the Invention

[0009] The purpose of the present invention is to overcome the above-mentioned defects of the prior art and provide a 3D target reconstruction method and device based on binocular structured light.

[0010] According to a first aspect of the present invention, a method for 3D object reconstruction based on binocular structured light is provided. The method comprises the following steps:

[0011] Constructing a binocular structured light system and determining an optimal working distance that satisfies a first error function, wherein the binocular structured light system includes a projector, a first camera, and a second right camera;

[0012] Under projection by the projector, using the first camera and the second camera to respectively acquire a first point cloud and a second point cloud corresponding to the 3D target object;

[0013] Using the determined overall position transformation relationship between the point clouds, the first point cloud and the second point cloud are registered and fused;

[0014] The overall position transformation relationship between the point clouds is determined according to the following sub-steps:

[0015] Perform a rough registration on the two point clouds corresponding to the standard part to obtain a first position transformation relationship between the point clouds, which includes a first rotation matrix and the first translation vector ;

[0016] For the two point clouds corresponding to the standard parts, the second error function is used as the registration target for precise registration to obtain the second position transformation relationship between the point clouds. The second position transformation relationship includes the second rotation matrix and the second translation vector ;

[0017] The first position transformation relationship and the second position relationship are combined to obtain an overall position transformation relationship between the point clouds.

[0018] According to a second aspect of the present invention, a 3D object reconstruction device based on binocular structured light is provided. The device comprises:

[0019] A binocular structured light system, comprising a projector, a first camera, and a second camera, and having an optimal working distance that satisfies a first error function;

[0020] Point cloud acquisition unit: used for acquiring a first point cloud and a second point cloud corresponding to the 3D target object using a first camera and a second camera respectively under projection of the projector;

[0021] Registration and fusion unit: used to register and fuse the first point cloud and the second point cloud using the determined overall position transformation relationship between the point clouds;

[0022] The overall position transformation relationship between the point clouds is determined according to the following sub-steps:

[0023] Perform a rough registration on the two point clouds corresponding to the standard part to obtain a first position transformation relationship between the point clouds, which includes a first rotation matrix and the first translation vector ;

[0024] For the two point clouds corresponding to the standard parts, the second error function is used as the registration target for precise registration to obtain the second position transformation relationship between the point clouds. The second position transformation relationship includes the second rotation matrix and the second translation vector ;

[0025] The first position transformation relationship and the second position relationship are combined to obtain an overall position transformation relationship between the point clouds.

[0026] Compared with the existing technology, the advantages of the present invention are that, in order to address the shortcomings of incomplete stereoscopic fingerprint reconstruction and highly inaccurate and slow photometric fingerprint reconstruction, the present invention adopts a binocular structured light system for fingerprint collection to achieve high-precision and high-speed 3D fingerprint collection; by using two cameras for fingerprint reconstruction, combined with the point cloud registration and fusion process, a more complete large-area 3D fingerprint can be obtained; for biometric features such as fingerprints that vary from individual to individual, the present invention proposes a method for determining the optimal working distance. The optimal working distance determined by this method can collect fingerprints with high accuracy for most people.

[0027] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0029] Figure 1 is a flowchart of a 3D target reconstruction method based on binocular structured light according to an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a binocular structured light system according to one embodiment of the present invention;

[0031] Figure 3 This is a plane fitting effect diagram according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of a coin reconstruction effect according to an embodiment of the present invention;

[0033] Figure 5 FIG. 4 is a schematic diagram of fingerprint reconstruction effect according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention.

[0035] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0036] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0037] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0038] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0039] The present invention provides a 3D object reconstruction method based on binocular structured light. It can be applied to the reconstruction of various types of 3D objects, with high accuracy and high speed, and can reconstruct objects of different sizes. For clarity, the following explanation uses fingerprint reconstruction as an example.

[0040] In general, the present invention builds a binocular structured light system to perform real-time fingerprint data acquisition and obtain two high-precision point clouds, marked as high-precision point cloud 1 and high-precision point cloud 2; then, the two fingerprint point clouds are coarsely aligned using the N-point SVD decomposition algorithm; then, the two fingerprint point clouds are finely aligned using the ICP (Iterative Closest Point) algorithm; and then, the position / region-based point cloud fusion strategy is used to complete the fusion of the two point clouds, obtaining a fingerprint point cloud with high precision and complete characteristics.

[0041] Specifically, see Figure 1 As shown, the provided 3D target reconstruction method based on binocular structured light includes the following steps:

[0042] Step S110: Build a binocular structured light system and determine the optimal working distance of the system for the 3D target.

[0043] See also Figure 2 As shown, the binocular structured light system utilizes a blue light projector and two 3.2-megapixel resolution industrial cameras (left and right). A synchronization trigger device enables simultaneous capture of the camera modules, enabling contactless, complete, and high-precision 3D fingerprint imaging. It should be understood that the cameras, optical machine models, and device distances used are all adjustable. Furthermore, the cameras can be positioned arbitrarily, not necessarily on either side of the projector, as long as they can simultaneously see the projected area. Furthermore, the number of cameras can be increased to achieve comprehensive fingerprint capture. In addition to using a structured light system as a 3D imaging device, other 3D imaging sensors, such as infrared cameras and MEMS galvanometers, can also be used, provided that 3D imaging accuracy and speed are maintained.

[0044] In one embodiment, a stripe edge encoding-based structured light 3D reconstruction technique is employed (Song, Z., Chung, R., & Zhang, XT (2012). An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3-D. IEEE Transactions on Industrial Electronics, 60(3), 1023-1032.). By employing this stripe edge encoding method, the anti-reflection capability of the structured light can be improved, while the introduction of traditional Gray coding can eliminate the periodic blurring of the stripes. Furthermore, by simultaneously using both positive and negative patterns, the stripe edges can be accurately detected; and by using an improved zero-crossing edge detector, sub-pixel precision stripe edge positioning can be achieved, thereby achieving high-precision 3D reconstruction.

[0045] Specifically, the coordinates of a point M in the three-dimensional space in the world coordinate system, camera coordinate system, and projector coordinate system are respectively , projected by the projector and photographed by the camera, the coordinates of the point M in the camera pixel coordinate system and the projector pixel coordinate system are By calibrating the camera and projector separately, the internal and external parameters of the camera and projector can be obtained. ,in is the external parameter of the camera and projector relative to the same world coordinate system. Then the coordinate of point M in the camera pixel coordinate system is and its coordinates in the camera coordinate system There is the following relationship, where for Identity matrix:

[0046]

[0047] Similarly, the coordinates of point M in the projector pixel coordinate system are and its coordinates in the projector coordinate system There is the following relationship, where for Identity matrix:

[0048]

[0049] In addition, the coordinates of point M in the camera coordinate system are and its coordinates in the world coordinate system There is the following relationship, and The rotation matrix and translation vector from the world coordinate system to the camera coordinate system are:

[0050]

[0051] Similarly, the coordinates of point M in the projector coordinate system are and its coordinates in the world coordinate system There is the following relationship, and The rotation matrix and translation vector from the world coordinate system to the camera coordinate system are:

[0052]

[0053] Assume that the transformation relationship between the camera coordinate system and the projector coordinate system is Indicates that the coordinates of point M in the camera coordinate system are and its coordinates in the projector coordinate system There is the following relationship, where and The rotation matrix and translation vector from the projector coordinate system to the camera coordinate system are:

[0054]

[0055] After camera calibration, we can get , substituting into (3) and (4), we can get Then, according to and The depth of point M can be obtained by the corresponding relationship and triangulation.

[0056] Preferably, to account for the individual characteristics of a finger, the optimal working distance for fingerprint measurement using a binocular structured light system can be determined. In a binocular structured light system, the two cameras are typically positioned in a fixed, horizontal position. Therefore, it is generally sufficient to record the distance from one camera to a reference plane when the imaging state is optimal. This distance is recorded as the optimal working distance. The reference plane refers to the plane where the target object resides.

[0057] For example, let the target optimal working distance be , the current system working distance is , the point cloud collected at the current working distance is , the point cloud collected by other devices (i.e. other high-precision 3D imaging devices with determined optimal working distances) at their optimal working distances is , establish the error function , then it can be transformed into an optimization problem:

[0058] Where n represents the number of points in the point cloud. Solving the above optimization problem can get the optimal working distance of the system Through continuous optimization, a fixed-height plane is superimposed on the reference plane (the reference plane is the base of the device after it is fixed, such as the surface where your fingers rest) based on the updated d, and this plane is used as the new reference plane. After the optimal working distance of the system is determined, the current device height is updated to the optimal working distance. This avoids the problem of inaccurate optimal working distance caused by multiple device height adjustments.

[0059] In step S120 , a coarse registration is performed on the two point clouds corresponding to the standard component to preliminarily obtain a position transformation relationship between the coarsely registered point clouds.

[0060] In one embodiment, the coarse registration uses the N-point SVD decomposition algorithm to select at least three feature points on each of the two point clouds corresponding to the standard part. The feature points selected on the two point clouds correspond to each other in the same order. After SVD decomposition, the rotation matrix is ​​calculated successively. and translation vectors .

[0061] To facilitate subsequent fingerprint reconstruction, the standard component can be a small flat object with fine surface features, such as a coin or a small component marked with a certain product trademark. The small flat object with fine features can be placed on an object with the average thickness of the finger as the reference.

[0062] After the coarse registration stage, we initially obtained the position transformation relationship between the two point clouds. , the two standards can be roughly aligned.

[0063] Step S130 : performing precise registration on the two point clouds of the standard component to obtain a position transformation relationship between the precisely registered point clouds.

[0064] For example, use the ICP algorithm to precisely align the two point clouds corresponding to the standard parts to obtain the precisely aligned rotation matrix and translation vectors .

[0065] The basic principle of the ICP algorithm is to find the nearest neighbor points in the source point cloud q and the target point cloud p according to certain constraints. , and then calculate the optimal matching parameters and , so that the error function is minimized, the error function is

[0066]

[0067] in, is the number of nearest neighbor pairs, Target point cloud One point in Source point cloud Zhongyu The corresponding nearest point, is the rotation matrix, is the translation vector.

[0068] Step S140 , combining the coarse registration and fine registration results for the standard part to obtain the position transformation relationship between the overall point clouds, and applying it to the point cloud registration and fusion of the target 3D object.

[0069] For example, by combining the coarse registration results obtained using standard parts with the fine registration results, we can obtain the overall rotation matrix and translation vector between the two point clouds. The process of combining the coarse registration calculation results with the fine registration calculation results is as follows:

[0070] The parameters obtained in the coarse registration stage are The parameters obtained in the fine registration stage are , then the final RT parameter It can be expressed as:

[0071]

[0072] .

[0073] Next, this RT can be used to directly register the target 3D object, such as two fingerprint point clouds. For example, using a binocular structured light system to collect fingerprints can obtain two 3D fingerprint point clouds. Using coarse and fine registration to determine the overall position transformation relationship, a matched and complete 3D fingerprint point cloud can be obtained.

[0074] After completing the registration of the fingerprint point cloud, in order to maximize the quality of the retained fingerprint point cloud while reducing the amount of data and facilitating subsequent storage and calculation, a certain fusion strategy is selected to fuse the fingerprint point cloud. The specific process is as follows:

[0075] Find the fingerprint's mid-axis plane (the plane perpendicular to the fingernail plane and bisecting the left and right fingerprints) and adjust the fingerprint point cloud's pose so that the mid-axis plane is aligned with a reference plane in the current coordinate system.

[0076] The point cloud retention strategy is determined based on the distance between the points in the point cloud and the medial axis plane. For example, the following strategy is adopted:

[0077] The retention ratio is proportional to the distance between the point in the point cloud and the median axis. The farther away from the median axis, the more points are retained, which helps to reduce the number of overlapping points in the two pieces.

[0078] Adopt a fusion strategy with different ratios in different areas but fixed ratio values. For example, the point cloud can be divided into several different areas based on the distance from the point to the median axis, and different strategies of retaining 100% or 50% can be adopted for each area.

[0079] According to the density of the registered point cloud, the retention ratio of point clouds in different areas is determined. For example, if the point cloud density in the central area is the highest, 50% will be retained, while the density on both sides is the lowest, 100% will be retained.

[0080] After the fusion is completed, the point cloud pose can be adjusted back to the state before fusion.

[0081] Accordingly, the present invention also provides a 3D target reconstruction device based on binocular structured light, which is used to implement one or more aspects of the above-mentioned method. For example, the device includes: a binocular structured light system: the binocular structured light system includes a projector, a first camera, and a second camera, and has an optimal working distance that satisfies a first error function; a point cloud acquisition unit: used to use the first camera and the second camera to respectively acquire a first point cloud and a second point cloud corresponding to the 3D target object under projection by the projector; and a registration and fusion unit: used to register and fuse the first point cloud and the second point cloud using the determined overall position transformation relationship between the point clouds.

[0082] It should be noted that those skilled in the art may make appropriate changes or modifications to the above embodiments without departing from the spirit and scope of the present invention. For example, the coarse registration algorithm and the fine registration algorithm are not limited to the N-point SVD algorithm and the ICP algorithm. Other methods for coarse and fine registration, such as PCA-based coarse registration methods, RANSAC, and various variations of the ICP algorithm, may also be used. Furthermore, the method for determining the optimal working distance may also employ measurements of other objects that do not have a uniform standard size.

[0083] In order to further verify the effect of the present invention, an experimental verification was conducted. The relevant parameters of the system equipment used in the experiment are shown in Table 1. The relevant parameters such as resolution and field of view meet the requirements of reconstructing fingerprints. The system reconstructs the standard plane with good results, such as Figure 3 As shown, the standard plane uses the calibration plate with the marking dots removed. Figure 3 (a) is the plane fitting effect of the standard plane photographed by camera 1 at the determined optimal working distance. Figure 3 (b) is the plane fitting effect of the standard plane photographed by camera 2 at the determined optimal working distance, and the plane fitting accuracy is high, see Table 2. In addition, the reconstruction effect of standard reference objects (such as markers of a certain type of goods) and reflective objects such as coins was verified, see Figure 4 As shown, Figure 4 (a) is the reconstruction effect of the coin photographed by camera 1 at the determined optimal working distance. Figure 4(b) is the reconstruction effect of the coin photographed by camera 2 at the determined optimal working distance, and both achieved good results. The reconstructed 3D fingerprint can be seen in Figure 5 As shown, Figure 5 (a) is the reconstruction effect of the fingerprint captured by camera 1 at the determined optimal working distance, Figure 5 (b) is the reconstruction effect of the fingerprint taken by camera 2 at the determined optimal working distance. Figure 5 (c) is the merged point cloud packaging effect diagram, where the ridge and valley structure can be clearly seen, and the finger contour is consistent with the real finger.

[0084] Table 1: Relevant technical parameters of acquisition equipment

[0085]

[0086] Table 2: Reconstruction accuracy of acquisition equipment

[0087]

[0088] In summary, current fingerprint reconstruction technologies struggle to achieve both high precision and high speed, and few are designed for large-area fingerprint reconstruction. However, both precision and area size affect fingerprint recognition accuracy. To improve fingerprint recognition accuracy, the present invention employs a binocular structured light system for fingerprint reconstruction, enabling high-precision, high-speed, and large-area 3D fingerprint reconstruction. Compared to existing technologies, the present invention has the following advantages:

[0089] 1) A method for completely reconstructing fingerprints using point clouds acquired by a binocular structured light system is proposed. This method involves coarse registration, fine registration, and a fusion strategy for two fingerprint point clouds. The reconstructed 3D fingerprint is highly accurate and fast. The finger point cloud obtained by registering the two point clouds has a wide coverage area and high integrity.

[0090] 2) This invention utilizes an iterative optimization method to determine the optimal working distance for fingerprint collection equipment, ensuring clear imaging and accurate reconstruction of the fingerprints of most individuals. The optimal working distance is determined not by imaging results, but by placing an object with fine features on an object with the average thickness of a finger. This ensures that, at the determined optimal working distance, the fingerprints of most individuals can be accurately reconstructed. Furthermore, a single height adjustment from the initialization height allows the device to be precisely positioned at the optimal working distance, effectively eliminating errors associated with repeated system height adjustments during debugging (primarily the difficulty of adjusting the device height with the same step size each time and the difficulty of measuring the degree of drop after adjustment).

[0091] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0092] A computer-readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure within a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0093] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0094] The computer program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the state information of the computer-readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), so that the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present invention.

[0095] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0096] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0097] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0098] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of an instruction, and the module, program segment or part of the instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

[0099] While various embodiments of the present invention have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A 3D target reconstruction method based on binocular structured light, comprising the following steps: Constructing a binocular structured light system and determining an optimal working distance that satisfies a first error function, wherein the binocular structured light system includes a projector, a first camera, and a second camera; Under projection by the projector, using the first camera and the second camera to respectively acquire a first point cloud and a second point cloud corresponding to the 3D target object; Using the determined overall position transformation relationship between the point clouds, the first point cloud and the second point cloud are registered and fused; The overall position transformation relationship between the point clouds is determined according to the following sub-steps: Performing coarse registration on two point clouds corresponding to the 3D target object to obtain a first position transformation relationship between the point clouds, where the first position transformation relationship includes a first rotation matrix R1 and a first translation vector T1; For the two point clouds corresponding to the 3D target object, perform fine registration using the set second error function as the registration target to obtain a second position transformation relationship between the point clouds, where the second position transformation relationship includes a second rotation matrix R2 and a second translation vector T2; Combining the first position transformation relationship and the second position transformation relationship to obtain an overall position transformation relationship between the point clouds; Wherein, the second error function is set to: Among them, the second error function represents finding the nearest neighbor point (p) in the source point cloud q and the target point cloud p according to certain constraints. i ,q i ), and then calculate the optimal matching parameters R2 and T2 to minimize the second error function, n is the number of nearest neighboring point pairs, p i is a point in the target point cloud p, q i is the source point cloud q and p i The corresponding nearest point, E represents the expectation; Wherein, the first error function is set to: Among them, d * Indicates the optimal working distance of the target, P d is the point cloud collected at the current working distance d, P is the point cloud collected by the device with the optimal working distance determined, D is the optimal working distance of the binocular structured light system, and M is the number of points in the point cloud.

2. The method according to claim 1, characterized in that The coarse registration of two point clouds corresponding to the 3D target object includes: At least three feature points are selected from the two point clouds corresponding to the 3D target object, and the feature points selected from the two point clouds correspond to each other in the same order. After SVD decomposition, the first rotation matrix R1 and the first translation vector T1 are calculated successively.

3. The method according to claim 1, characterized in that The 3D target object is a fingerprint point cloud, and point cloud fusion is performed according to the following steps: Find the fingerprint's mid-axis plane and adjust the fingerprint point cloud's pose so that the mid-axis plane is aligned with a reference plane in the current coordinate system. The mid-axis plane is a plane perpendicular to the fingernail plane and bisects the left and right fingerprints. The point cloud retention strategy is determined based on the distance from the point in the point cloud to the medial plane, including: According to the distance between the point in the point cloud and the medial axis, the point cloud is divided into multiple different areas, and different retention ratios are set for each area; According to the density of the point cloud after registration, the retention ratio of point clouds in different areas is determined.

4. The method according to claim 1, wherein The overall position transformation relationship between the point clouds is expressed as: R3=R2*R1 T3=T2-R2 / R1*T1 Among them, R3 is the overall rotation matrix between point clouds, and T3 is the overall translation vector between point clouds.

5. The method according to claim 1, wherein The 3D target object is a coin or a product tag.

6. A 3D target reconstruction device based on binocular structured light, comprising: A binocular structured light system, comprising a projector, a first camera, and a second camera, and having an optimal working distance that satisfies a first error function; Point cloud acquisition unit: used for acquiring a first point cloud and a second point cloud corresponding to the 3D target object using a first camera and a second camera respectively under projection of the projector; Registration and fusion unit: used to register and fuse the first point cloud and the second point cloud using the determined overall position transformation relationship between the point clouds; The overall position transformation relationship between the point clouds is determined according to the following sub-steps: Performing coarse registration on two point clouds corresponding to the 3D target object to obtain a first position transformation relationship between the point clouds, where the first position transformation relationship includes a first rotation matrix R1 and a first translation vector T1; For the two point clouds corresponding to the 3D target object, perform fine registration using the set second error function as the registration target to obtain a second position transformation relationship between the point clouds, where the second position transformation relationship includes a second rotation matrix R2 and a second translation vector T2; Combining the first position transformation relationship and the second position transformation relationship to obtain an overall position transformation relationship between the point clouds; Wherein, the second error function is set to: Among them, the second error function represents finding the nearest neighbor point (p) in the source point cloud q and the target point cloud p according to certain constraints. i ,q i ), and then calculate the optimal matching parameters R2 and T2 to minimize the second error function, n is the number of nearest neighboring point pairs, p i is a point in the target point cloud p, q i is the source point cloud q and p i The corresponding nearest point, E represents the expectation; Wherein, the first error function is set to: Among them, d * Indicates the optimal working distance of the target, P d is the point cloud collected at the current working distance d, P is the point cloud collected by the device with the optimal working distance determined, D is the optimal working distance of the binocular structured light system, and M is the number of points in the point cloud.

7. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory and a processor, wherein a computer program capable of being run on the processor is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

Images