Multi-camera structured light three-dimensional imaging method, device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a multi-camera structured light three-dimensional imaging method, device, electronic equipment and storage medium, the method acquires original point cloud data collected by a three-dimensional structured light imaging system, the three-dimensional structured light imaging system includes multiple cameras and at least one projector; the original point cloud data collected by each camera is screened to obtain corresponding effective point cloud data; the effective point cloud data corresponding to each camera is mapped into a reference coordinate system, and the space where the reference coordinate system is located is divided according to a preset size of a pixel block; the effective point cloud data in each pixel block is weighted and fused to obtain the point cloud coordinates corresponding to each pixel block. Through the screening of the point cloud and the division according to the preset pixel block, the processing efficiency of the point cloud data can be improved while ensuring the integrity of the effective point cloud data, and the fused point cloud is smoother.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of three-dimensional imaging technology, and in particular to multi-camera structured light three-dimensional imaging methods, devices, electronic devices and storage media. Background Technology

[0002] In structured light 3D measurement sensors designed for microscopic targets on complex optical surfaces, high precision, robustness, and interference resistance are fundamental requirements. Fringe projection and phase-shifting algorithms, due to their high precision, high resolution, and high robustness, are finding wider application in structured light 3D sensors.

[0003] The inventors found that when using an existing camera-projector-based structured light imaging system for 3D acquisition, the image quality of images acquired from different viewpoints varied due to factors such as occlusion, shadows, and large changes in surface reflectivity, and there were even some missing data. Summary of the Invention

[0004] This invention provides a multi-camera structured light 3D imaging method, device, electronic device, and storage medium to solve the technical problem that there are differences in the imaging quality of images acquired from different perspectives or even the phenomenon of missing data in the 3D imaging process of the prior art.

[0005] In a first aspect, embodiments of the present invention provide a multi-camera structured light three-dimensional imaging method, comprising:

[0006] Acquire raw point cloud data collected by a three-dimensional structured light imaging system, which includes multiple cameras and at least one projector;

[0007] The raw point cloud data collected by each camera is filtered to obtain the corresponding valid point cloud data;

[0008] The effective point cloud data corresponding to each camera is mapped to a reference coordinate system, and the space where the reference coordinate system is located is divided into pixel blocks of a preset size.

[0009] The effective point cloud data in each pixel block are weighted and fused to obtain the point cloud coordinates corresponding to each pixel block.

[0010] Secondly, embodiments of the present invention provide a multi-camera structured light three-dimensional imaging device, comprising:

[0011] The data acquisition unit is used to acquire raw point cloud data collected by the three-dimensional structured light imaging system, which includes multiple cameras and at least one projector.

[0012] The data filtering unit is used to filter the raw point cloud data collected by each camera to obtain the corresponding valid point cloud data.

[0013] The pixel block division unit is used to map the effective point cloud data corresponding to each camera to a reference coordinate system, wherein the space of the reference coordinate system is divided into pixel blocks of a preset size.

[0014] The data fusion unit is used to perform weighted fusion of the effective point cloud data in each pixel block to obtain the point cloud coordinates corresponding to each pixel block.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0016] One or more processors;

[0017] Memory, used to store one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the electronic device enables the multi-camera structured light three-dimensional imaging method as described in the first aspect.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-camera structured light three-dimensional imaging method as described in the first aspect.

[0020] The aforementioned multi-camera structured light 3D imaging method, apparatus, electronic device, and storage medium involve acquiring raw point cloud data from a 3D structured light imaging system, which includes multiple cameras and at least one projector. The method filters the raw point cloud data acquired by each camera to obtain corresponding valid point cloud data. The valid point cloud data corresponding to each camera is mapped to a reference coordinate system, the space of which is divided into pixel blocks of a preset size. The valid point cloud data in each pixel block is then weighted and fused to obtain the point cloud coordinates corresponding to each pixel block. By filtering the point cloud and dividing it into preset pixel blocks, the processing efficiency of the point cloud data can be improved while ensuring the integrity of the valid point cloud data, resulting in a smoother fused point cloud. Attached Figure Description

[0021] Figure 1 A flowchart of a multi-camera structured light three-dimensional imaging method provided in an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of the imaging model of a camera and a projector;

[0023] Figure 3 This is a schematic diagram of a multi-view imaging model;

[0024] Figure 4 This is a phase-shifted fringe pattern;

[0025] Figure 5 For the wrapper phase diagram;

[0026] Figure 6 This is a schematic diagram of epipolar constraints in three-dimensional space.

[0027] Figure 7 This is a schematic diagram of pixel block division provided in an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of the mapping relationship of point clouds in pixel blocks provided in an embodiment of the present invention;

[0029] Figure 9 This is a schematic diagram of the single-point cloud generation process;

[0030] Figure 10 This is a schematic diagram of point cloud fusion provided in an embodiment of the present invention;

[0031] Figure 11 This is a schematic diagram of the structure of a multi-camera structured light three-dimensional imaging device provided in an embodiment of the present invention;

[0032] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and not for limiting the invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention and not the entire structure.

[0034] It should be noted that, due to space limitations, this application specification does not exhaustively list all possible implementation methods. Those skilled in the art should be able to conceive after reading this application specification that, as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method.

[0035] For example, in one embodiment, a technical feature is described: the first substrate region is obtained by segmenting the depth map using the depth information of the depth map. In another embodiment, another technical feature is described: the second substrate region in the two-dimensional image is confirmed by statistical analysis of the color channels in the two-dimensional image. Those skilled in the art should be able to conceive of this application specification after reading it. An implementation that has both of these features is also an optional implementation. That is, in the specific implementation process, the first substrate region is obtained based on the depth information, and the second substrate region is obtained based on the statistical analysis of the color channels, thereby confirming the substrate region of the PCB to be tested.

[0036] The embodiments are described in detail below.

[0037] Figure 1 This is a flowchart of a multi-camera structured light 3D imaging method provided in an embodiment of the present invention. This multi-camera structured light 3D imaging method is used in electronic devices. As shown in the figure, the multi-camera structured light 3D imaging method includes:

[0038] Step S110: Acquire the raw point cloud data collected by the three-dimensional structured light imaging system, which includes multiple cameras and at least one projector.

[0039] In a complete 3D structured light imaging system, multiple cameras are arranged around a projector as the center, and both the projector and the cameras satisfy the following requirements: Figure 2 The diagram shows a pinhole imaging model. Figure 2 In this model, the subscript 'c' represents the camera, and the pinhole imaging model describes a point X in space. w At camera image coordinates m c The correspondence, S c K is the scaling factor. c R is the camera intrinsic parameter matrix; c Let t be a rotation matrix. c The translation vector, rotation matrix, and translation vector together form the camera extrinsic parameters, which represent the transformation relationship from the reference coordinate system to the camera coordinate system.

[0040] Further reference Figure 3 It is a schematic diagram of a multi-view imaging model of a camera and a projector, where R s and t s The relative extrinsic parameters of the camera and projector are obtained through calibration. Specific calibration methods are widely implemented in existing 3D structured light imaging systems and will not be repeated here. Based on... Figure 3 The imaging relationship shown can be applied to the camera pixel coordinates m. c and the coordinates m of the projector pixels p Construct a system of equations:

[0041]

[0042] Solving the system of equations will yield the three-dimensional point X. w .

[0043] The three-dimensional point X is solved based on this system of equations. w This requires knowing the correspondence between the coordinates of camera pixels and projector pixels. This correspondence is obtained through... Expression, in which and These represent the phases of the camera and the projector, respectively.

[0044] In this scheme, a high-precision phase-shifting method is used to solve for the phase value of the projector. The projection encoding should conform to the following:

[0045]

[0046]

[0047] The grayscale value of the projected image is represented by f, which represents the frequency / number of stripes, and x represents the coordinate of a column in the image. R Px Indicates image width / number of columns. This represents the phase shift amount, and N represents the number of phase shift steps.

[0048] like Figure 5 After the projected stripes are captured by the camera, the grayscale value of each pixel conforms to the following:

[0049]

[0050] I n (x, y) represents the grayscale values ​​of the image captured by the camera. A represents the phase value, B represents the offset, and A represents the amplitude.

[0051] The phase value can be obtained using the following standard phase-shifting formula:

[0052]

[0053] At the same time, A and B can also be obtained:

[0054]

[0055]

[0056] Due to the periodicity of the fringes, the solved phase will appear as follows: Figure 6 The “wrapping” phenomenon shown is due to the fact that the wrapping phase is calculated by the arctangent function. The phase will show a sawtooth wave-like fault at the boundary of each period and is constrained between [-π, π].

[0057] For the final calculated three-dimensional point X w Because the phase value has periodic ambiguity, there exists a one-to-many relationship between camera image points and projected image points. This scheme utilizes all possible 3D points calculated based on this one-to-many relationship. Specifically, let the camera pixel coordinates be (x, y), and the corresponding horizontal and vertical fringe periods be N. x N y The phase value is The ambiguous phase value corresponding to the camera image point is:

[0058]

[0059]

[0060] If the raw point cloud data obtained from ambiguous imaging is directly imaged, layering will occur. In subsequent processing, all raw point cloud data are used as candidate points and uniquely determined to obtain high-precision point cloud data.

[0061] Step S120: Filter the raw point cloud data collected by each camera to obtain the corresponding valid point cloud data.

[0062] Based on the characteristics of fringe projection structured light imaging, the point cloud quality can be evaluated by utilizing the correspondence between camera image points, phase values, and point clouds, and further, the effective point cloud data corresponding to each camera can be obtained.

[0063] In the specific implementation process, step S120 can be achieved through steps S121 and S122:

[0064] Step S121: For the raw point cloud data acquired by each camera, calculate the phase error, epipolar deviation, and target ray off-axis angle of the raw point cloud data based on the parameters corresponding to the camera.

[0065] Step S122: Obtain raw point cloud data where the phase error, epipolar deviation, and target ray off-axis angle are all within the corresponding threshold ranges, thus obtaining the effective point cloud data for each camera.

[0066] Phase accuracy directly determines the accuracy of the camera and projector coordinate matching. Phase quality can effectively evaluate the uncertainty of the reconstructed point cloud. In actual measurements, phase noise mainly originates from the sensor's white noise, which is generally considered to follow a Gaussian distribution σ ~ N(0, 1). Considering the error propagation of the phase-shifting method, the phase error is related to the number of phase-shifting encoding steps, the frequency, and the contrast of the acquired stripe image. The phase error is calculated as follows:

[0067]

[0068] Where Q1 represents the phase error, N represents the number of phase shift encoding steps, f represents the projection frequency, and B represents the phase shift amplitude. The phase error reflects the contrast of the stripes hitting the target surface; the higher the contrast, the smaller the phase error, and the higher the quality of the imaged point cloud.

[0069] According to the theory of epipolar geometry, such as Figure 6As shown, the optical center of the camera, the optical center of the projector, and the object point should all lie on the same plane. Projector rays and camera rays intersecting the object point should also fall on this plane. An accurate projector image point should fall on the straight line intersecting the light plane and the projected image plane; this relationship is commonly referred to as epipolar constraint. This scheme uses epipolar deviation to evaluate the distance of the projector rays from the light plane, such as... Figure 6 As shown, compared to the accurate three-dimensional coordinates X w , X′ w The greater the deviation of the epipolar line from the light plane, and the greater the deviation in 3D coordinates, the lower the reliability of the point cloud. In this scheme, the projector uses a reverse camera model to establish geometric relationships, thus forming a stereo geometric imaging model with the camera. Based on epipolar constraints, the epipolar line deviation is calculated as follows:

[0070] Q2 = q x -q y +q z

[0071]

[0072]

[0073]

[0074] Where Q2 represents the polar deviation, and the optical center of the camera. The direction vector of the camera at the image point The optical core of the projector The direction vector of the projector at the image point

[0075] In the imaging process of complex scenes, some surfaces exhibit both specular and diffuse reflection. These areas are susceptible to the influence of projected light and the camera's acquisition angle, leading to significant differences in the image quality captured by the camera. The closer the angle of the measured surface is to the central optical axis of the camera, the stronger the direct reflected light is, and the better the resistance to interference from indirect illumination such as ambient light. The angle between the target surface normal vector and the camera's optical axis is called the target ray off-axis angle; the smaller the angle, the higher the quality of that point. The target ray off-axis angle is calculated as follows:

[0076]

[0077] Where Q3 represents the target ray off-axis angle, Represents the normal vector of an image point. Indicates the central optical axis of the camera.

[0078] Different parameter indicators have different screening thresholds. Raw point cloud data within the threshold range are retained as valid point cloud data. That is, raw point cloud data in which phase error, epipolar deviation and target ray off-axis angle are all within the corresponding threshold range are obtained to obtain the valid point cloud data for each camera.

[0079] Step S130: Map the valid point cloud data corresponding to each camera to a reference coordinate system, wherein the space of the reference coordinate system is divided into pixel blocks of a preset size.

[0080] In the specific point cloud structuring process, point cloud data can be mapped to the same coordinate system, namely the reference coordinate system. The space of the reference coordinate system is divided into pixel blocks of a preset size, generally with the projected image coordinates of the projector as a reference. However, the resolution of the projector is limited, and one projection unit may correspond to multiple point clouds, that is, one pixel block corresponds to multiple point cloud data. After multiple point cloud data are fused, data loss may occur due to fusion, resulting in sparse point clouds. In this solution, the reference coordinate system is the projector image coordinate system of the three-dimensional structured light system;

[0081] The range of the pixel block is set in the following manner:

[0082] m′ p =INT[sm p ]

[0083] Where, m′ p Indicates the range of the set pixel block, m p =(u p v p ) represents the reference projection image coordinates of the projector, where INT[] represents the rounding operation, and s represents the quantization coefficient used to set the pixel block size.

[0084] like Figure 7 As shown, assuming the projector has 2x2 pixels, when S=2, upsampling yields 4x4 pixels. Therefore, the quantization coefficient determines the density of the fused point cloud.

[0085] In the specific mapping process, for the j-th 3D coordinate X generated by the i-th camera i,j When mapped onto the reference coordinate system determined by the projector coordinate system, the mapped coordinates are:

[0086]

[0087] Where K ci Let R be the intrinsic parameter matrix. i and t i Let m be the heterodyne matrix of the camera relative to the projector. p,i,j =(up，i，j v p，i，j ) is a two-dimensional coordinate.

[0088] Figure 8 Presented in The mapping relationship between the 3D points of a camera and the projector image, in a locally magnified single projection unit, may contain point clouds collected by cameras at different angles.

[0089] Step S140: Perform weighted fusion on the valid point cloud data in each pixel block to obtain the point cloud coordinates corresponding to each pixel block.

[0090] The effective point cloud data is specifically obtained by point cloud fusion using Gaussian quality weighted averaging to obtain accurate 3D coordinates. It is assumed that the real surfaces in each pixel block of the projector image are considered continuous and smooth. Due to the influence of random noise in actual measurements, the fusion weights follow a Gaussian distribution in the projected image coordinate system, i.e.:

[0091]

[0092] Point clouds already come with quality assessment metrics, and the larger the value, the greater the error in point cloud assessment. Therefore, the fusion weights should be inversely proportional to these metrics.

[0093] The point cloud coordinates are fused using the following formula:

[0094]

[0095]

[0096] Among them, X i，j Let Q represent the j-th 3D coordinate of the i-th camera. 1，i，j Q 2，i，j and Q 3，i，j Let these represent the phase error, epipolar deviation, and target ray off-axis angle at the j-th coordinate of the i-th camera, respectively. This indicates the mapping on the projector image coordinates (u p v p ) three-dimensional coordinates, (u p，i，j v p，i，j ) represents X i，j Two-dimensional coordinates mapped onto a reference coordinate system. The specific fusion process is as follows: Figure 9 As shown.

[0097] Ultimately Figure 10 As shown, the fused point cloud is smoother and removes redundant point cloud data while retaining effective point cloud data.

[0098] The above describes the acquisition of raw point cloud data from a 3D structured light imaging system, which includes multiple cameras and at least one projector. The raw point cloud data acquired by each camera is filtered to obtain corresponding valid point cloud data. The valid point cloud data corresponding to each camera is mapped to a reference coordinate system, the space of which is divided into pixel blocks of a preset size. The valid point cloud data in each pixel block is then weighted and fused to obtain the point cloud coordinates corresponding to each pixel block. By filtering the point cloud and dividing it into preset pixel blocks, the processing efficiency of the point cloud data can be improved while ensuring the integrity of the valid point cloud data, resulting in a smoother fused point cloud.

[0099] Figure 11 This is a schematic diagram of a multi-camera structured light three-dimensional imaging device provided in an embodiment of the present invention. (Reference) Figure 11 The multi-camera structured light 3D imaging device includes: a data acquisition unit 210, a data filtering unit 220, a pixel block division unit 230, and a data fusion unit 240.

[0100] The system includes a data acquisition unit 210 for acquiring raw point cloud data collected by a three-dimensional structured light imaging system, which includes multiple cameras and at least one projector; a data filtering unit 220 for filtering the raw point cloud data collected by each camera to obtain corresponding valid point cloud data; a pixel block division unit 230 for mapping the valid point cloud data corresponding to each camera to a reference coordinate system, wherein the space of the reference coordinate system is divided into pixel blocks of a preset size; and a data fusion unit 240 for weighted fusion of the valid point cloud data in each pixel block to obtain the point cloud coordinates corresponding to each pixel block.

[0101] Based on the above embodiments, the data filtering unit 220 includes:

[0102] The index calculation module is used to calculate the phase error, epipolar deviation and target ray off-axis angle of the raw point cloud data collected by each camera based on the parameters corresponding to that camera.

[0103] The data filtering module is used to acquire raw point cloud data where phase error, epipolar deviation, and target ray off-axis angle are all within their respective threshold ranges, thus obtaining the effective point cloud data for each camera.

[0104] Based on the above embodiments, the phase error is calculated in the following manner:

[0105]

[0106] Where Q1 represents the phase error, N represents the number of phase shift coding steps, f represents the projection frequency, and B represents the phase shift amplitude.

[0107] Based on the above embodiments, the polar deviation is calculated as follows:

[0108] Q2 = q x -q y +q z

[0109]

[0110]

[0111]

[0112] Where Q2 represents the polar deviation, and the optical center of the camera. The direction vector of the camera at the image point The optical core of the projector The direction vector of the projector at the image point

[0113] Based on the above embodiments, the target ray off-axis angle is calculated as follows:

[0114]

[0115] Where Q3 represents the target ray off-axis angle, Represents the normal vector of an image point. Indicates the central optical axis of the camera.

[0116] Based on the above embodiments, the reference coordinate system is the projector image coordinate system of the three-dimensional structured light system;

[0117] The range of the pixel block is set in the following manner:

[0118] m′ p =INT[sm p ]

[0119] Where, m′ p Indicates the range of the set pixel block, m p =(u p v p ) represents the reference projection image coordinates of the projector, where INT[] represents the rounding operation, and s represents the quantization coefficient used to set the pixel block size.

[0120] Based on the above embodiments, the point cloud coordinates are fused using the following formula:

[0121]

[0122]

[0123] Among them, X i，j Let Q represent the j-th 3D coordinate of the i-th camera. 1，i，j Q 2，i，j and Q 3，i，j Let these represent the phase error, epipolar deviation, and target ray off-axis angle at the j-th coordinate of the i-th camera, respectively. This indicates the mapping on the projector image coordinates (u p v p ) three-dimensional coordinates, (u p，i，j v p，i，j ) represents X i，j Two-dimensional coordinates mapped onto a reference coordinate system.

[0124] The multi-camera structured light three-dimensional imaging device provided in this embodiment of the invention is included in an electronic device and can be used to execute the multi-camera structured light three-dimensional imaging method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0125] It is worth noting that in the embodiments of the multi-camera structured light three-dimensional imaging device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0126] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 12 As shown, the electronic device includes a processor 510, a memory 520, an input device 530, an output device 540, and a communication device 550; the number of processors 510 in the electronic device can be one or more. Figure 12 Taking a processor 510 as an example; the processor 510, memory 520, input device 530, output device 540, and communication device 550 in the electronic device can be connected via a bus or other means. Figure 12 Taking the example of a connection between China and Israel via a bus.

[0127] The memory 520, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the multi-camera structured light 3D imaging method in this embodiment of the invention (e.g., the data acquisition unit 210, data filtering unit 220, pixel block division unit 230, and data fusion unit 240 in the multi-camera structured light 3D imaging device). Of course, the program instructions / modules that implement the functions of different electronic devices are located in the corresponding electronic devices. The processor 310 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 320, thereby realizing the above-described multi-camera structured light 3D imaging method.

[0128] The memory 320 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 320 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include memory remotely located relative to the processor 310, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0129] Input device 330 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 340 may include display devices such as a display screen.

[0130] The aforementioned electronic device includes a multi-camera structured light 3D imaging device, which can be used to perform any multi-camera structured light 3D imaging method and has corresponding functions and beneficial effects.

[0131] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform relevant operations in the multi-camera structured light three-dimensional imaging method provided in any embodiment of this application, and have corresponding functions and beneficial effects.

[0132] Those skilled in the art will understand that embodiments of this application may be provided as methods, systems, or computer program products.

[0133] Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce implementations of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0135] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0137] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A multi-camera structured light three-dimensional imaging method, characterized in that, include: Acquire raw point cloud data collected by a three-dimensional structured light imaging system, which includes multiple cameras and at least one projector; For the raw point cloud data acquired by each camera, the phase error, epipolar deviation, and target ray off-axis angle of the raw point cloud data are calculated based on the parameters corresponding to that camera. Obtain raw point cloud data where phase error, epipolar deviation, and target ray off-axis angle are all within the corresponding threshold ranges to obtain effective point cloud data for each camera; The effective point cloud data corresponding to each camera is mapped to a reference coordinate system, and the space where the reference coordinate system is located is divided into pixel blocks of a preset size. The effective point cloud data in each pixel block are weighted and fused according to the phase error, epipolar deviation and target ray off-axis angle to obtain the point cloud coordinates corresponding to each pixel block.

2. The method according to claim 1, characterized in that, The phase error is calculated as follows: in, Indicates phase error, N Indicates the number of phase shift coding steps. f B represents the projection frequency, and B represents the phase shift amplitude. This indicates a Gaussian distribution.

3. The method according to claim 1, characterized in that, The polar deviation is calculated as follows: in, Indicates the polar deviation, the position vector of the camera's optical center. ; The direction vector of the camera at the image point The position vector of the projector's optical center The direction vector of the projector at the image point .

4. The method according to claim 1, characterized in that, The off-axis angle of the target ray is calculated as follows: in, Indicates the off-axis angle of the target ray. Represents the normal vector of an image point. This represents the direction vector of the camera at the image point. The position vector of the optical center of the camera is represented.

5. The method according to claim 1, characterized in that, The reference coordinate system is the projector image coordinate system of the three-dimensional structured light imaging system; The range of the pixel block is set in the following manner: in, This indicates the pixel coordinates within a defined pixel block. Represents the image coordinates within the reference projection range of the projector, where INT[] This indicates a rounding operation, and s represents the quantization coefficient used to set the pixel block size.

6. The method according to claim 1, characterized in that, The point cloud coordinates are fused using the following formula: in, Indicates the first i The first camera j Three-dimensional coordinates, , and They represent the first i The first camera j Phase error, epipolar deviation, and target ray off-axis angle for each coordinate. This indicates the mapping on the projector image coordinates ( The three-dimensional coordinates of ) express Two-dimensional coordinates mapped onto a reference coordinate system. This indicates a Gaussian distribution.

7. A multi-camera structured light three-dimensional imaging device, characterized in that, include: The data acquisition unit is used to acquire raw point cloud data collected by the three-dimensional structured light imaging system, which includes multiple cameras and at least one projector. The data filtering unit is used to filter the raw point cloud data collected by each camera to obtain the corresponding valid point cloud data. The pixel block division unit is used to map the effective point cloud data corresponding to each camera to a reference coordinate system, wherein the space of the reference coordinate system is divided into pixel blocks of a preset size. The data fusion unit is used to perform weighted fusion of the effective point cloud data in each pixel block to obtain the point cloud coordinates corresponding to each pixel block; The data filtering unit includes: The index calculation module is used to calculate the phase error, epipolar deviation and target ray off-axis angle of the raw point cloud data collected by each camera based on the parameters corresponding to that camera. The data filtering module is used to obtain raw point cloud data in which phase error, epipolar deviation and target ray off-axis angle are all within the corresponding threshold range, so as to obtain the effective point cloud data for each camera. The data fusion unit is specifically used to perform weighted fusion of the effective point cloud data in each pixel block according to the phase error, epipolar deviation and target ray off-axis angle to obtain the point cloud coordinates corresponding to each pixel block.

8. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements the multi-camera structured light three-dimensional imaging method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the multi-camera structured light 3D imaging method as described in any one of claims 1-6.

Images