Method and device for three-dimensional reconstruction of high-reflective surface based on polarized structured light camera

Abstract

The application discloses a kind of high light reflection surface three-dimensional reconstruction method and device based on polarized structured light camera.The method comprises: polarizing mirror is set to the lens of projector, the linearly polarized light stripe of the object to be measured is projected to the field of view of camera by the projector through the polarizing mirror, wherein the camera is polarized camera;The camera vertically images multiple polarization state images for the object to be measured, the polarization angle of the projected linearly polarized light stripe is not perpendicular to the multiple polarization states, and then multiple projection images with different brightness are obtained, wherein the number of projection images is consistent with the number of polarization state images;For the multiple projection images, the corresponding multiple point clouds are obtained by point cloud reconstruction;The multiple point clouds are fused to obtain the three-dimensional reconstruction result of the object to be measured.The application improves the efficiency and effect of object three-dimensional reconstruction, and is especially suitable for high light reflection object surface.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and more particularly to a high-reflective surface three-dimensional reconstruction method and device based on a polarized structured light camera. BACKGROUND

[0002] With the rapid development of computer vision technology, vision sensors capable of three-dimensional measurement are increasingly widely used in industry. The structured light three-dimensional reconstruction technology based on a projector and a camera has the advantages of fast imaging, high speed, high precision, etc. However, for some objects with high-reflective surfaces, scanning with a projector can easily cause overexposure, which seriously affects the imaging of the camera, resulting in missing points in the three-dimensional reconstruction point cloud and the generation of noise, etc. Point cloud missing and noise are actually irreversible information loss in the measurement process, and the fitting information obtained through various technologies is unreliable, which reduces the precision of industrial measurement. The electromagnetic field vibration direction of ordinary light is random, and polarized elements can filter randomly propagating light and only allow polarized light with regular changes in electromagnetic field over time to pass through. Using polarized light technology can filter random scattered light and only obtain effective light of a specified polarization state.

[0003] In recent years, with the rapid development of three-dimensional reconstruction technology in computer vision, more and more manufacturers have begun to introduce related three-dimensional scanning devices. As one of the most important three-dimensional vision sensors, structured light 3D cameras based on projectors and cameras can achieve high-precision and fast scanning reconstruction. However, there are a large number of metal workpieces and other samples with high-reflective surfaces in industrial manufacturing. High-reflective phenomena can seriously affect the accuracy of three-dimensional reconstruction, resulting in irreversible loss of measurement information. Therefore, how to suppress and eliminate high-reflective phenomena has become a difficult problem that needs to be solved in structured light three-dimensional reconstruction. Existing methods usually focus on reducing light intensity and polarized light technology to remove high-reflective phenomena and achieve the integrity of structured light three-dimensional reconstruction.

[0004] For example, patent application CN113554575A provides a high-reflective object surface highlight removal method based on polarization principles. According to the optimal polarization angle principle, the method obtains imaging pictures of high-reflective objects under multiple polarization angles and synthesizes them to improve the signal-to-noise ratio of the images and weaken the highlights. At the same time, a normalization weighting algorithm is proposed to collect high-light images under multiple exposure times and synthesize them to restore the surface information in the high-light area.

[0005] The patent application CN115235377A provides a three-dimensional measurement method based on the best projection intensity of polarization. This method can directly estimate the required best projection intensity by establishing the camera response function under the polarization system, which is used to compensate for the image light intensity reduced by the additional polarizer. This method does not require rotating the polarizer and multiple exposure times. In addition, the best fringe image is obtained by using an image fusion algorithm.

[0006] The patent application CN113237435A provides a high-reflective surface three-dimensional vision measurement system and method, which includes: establishing a model between the projector light intensity and the camera imaging gray scale, projecting a uniform saturated gray scale image, collecting images under high and low exposure times, obtaining the imaging area of the high-reflective area on the object surface in the camera image through pixel marking, and calculating the low gray scale projection intensity that makes the imaging not saturated; judging the absolute phase of the camera pixel, obtaining one or more camera pixels corresponding to each pixel to be adjusted in the projection image, calculating the gray scale of the projection pixel and the average gray scale of the corresponding camera pixel, and forming a gray scale matching pair; taking the matched projection pixel gray scale and camera pixel gray scale as samples to fit the projection intensity model parameters; calculating the best projection intensity according to the projection intensity model to generate an adaptive sinusoidal image; the projector projects the adaptive sinusoidal image, the industrial camera collects the image, the multi-frequency heterodyne phase unwrapping method is used to unwrap the phase, and the three-dimensional morphology of the object is obtained according to the fringe projection profilometry vision measurement model.

[0007] The patent application CN108645354A provides a structured light three-dimensional imaging method and system for high-reflective object surfaces. The method includes: projecting a plurality of groups of binary phase shift coding patterns with the same frequency and different illumination intensities onto the surface of the measured object, generating a plurality of modulation picture groups under different illumination intensities according to the reflected images, and arranging them in descending order of illumination intensity; obtaining the illumination saturation intensity of each pixel point in the group with the maximum illumination intensity, obtaining the saturated pixel points, determining the saturated area, obtaining the replacement pixel points corresponding to each saturated pixel point in the corresponding area of each group and having the minimum illumination saturation intensity and the maximum illumination intensity, and calculating the phase, replacing the phase of the saturated pixel points with the phase of the replacement pixel points, and obtaining the repaired three-dimensional image of the measured object.

[0008] Through analysis, the existing technology mainly has the following technical defects:

[0009] 1) For the existing structured light 3D camera, due to overexposure, point cloud loss and noise phenomena occur, for some simple regular surfaces, a global or local fitting method is usually used to calculate the missing information. However, the information obtained by fitting has a lot of uncertainty, and if a complex surface is encountered, this method often produces fitting errors and is difficult to use for accurate measurement.

[0010] 2) Existing structured light 3D camera can also use multi-exposure method, imaging under different exposure time, reconstructing to get several pieces of point cloud, so that the highlight position can also obtain actual information, and finally point cloud fusion is carried out to obtain complete point cloud. This method has good reconstruction effect in actual application, but for some objects with strong reflection phenomenon, it is still impossible to completely reconstruct; the scanning time is increased, the introduced noise may be increased, and the efficiency is reduced.

[0011] 3) The existing structured light 3D three-dimensional reconstruction combined with the polarization light technology eliminates the high reflection method, mostly installs the polarizing mirror on the camera lens, adjusts the angle to obtain multiple polarization state reconstruction point clouds, and obtains complete point cloud through point cloud fusion. This method needs to manually rotate the polarizing mirror to adjust the angle, which is low in efficiency in actual application. SUMMARY

[0012] The purpose of the present application is to overcome the defects of the prior art, and provide a high-reflective surface three-dimensional reconstruction method and device based on a polarized structured light camera.

[0013] According to the first aspect of the present application, a high-reflective surface three-dimensional reconstruction method based on a polarized structured light camera is improved. The method comprises the following steps:

[0014] The polarizing mirror is arranged in front of the lens of the projector, and the projector projects linearly polarized light stripes of the object to be measured into the field of view of the camera through the polarizing mirror, wherein the camera is a polarized camera;

[0015] The camera images multiple polarization state images vertically for the object to be measured, the polarization angle of the projected linearly polarized light stripes is not perpendicular to the multiple polarization state images, and multiple projection images with different brightness are obtained, wherein the number of projection images is consistent with the number of polarization state images;

[0016] For the multiple projection images, corresponding multiple point clouds are obtained through point cloud reconstruction;

[0017] The multiple point clouds are fused to obtain the three-dimensional reconstruction result of the object to be measured.

[0018] According to the second aspect of the present application, a high-reflective surface three-dimensional reconstruction device based on a polarized structured light camera is provided. The device comprises a projector, a camera, a polarizing mirror, a point cloud reconstruction unit and a point cloud fusion unit, the polarizing mirror is arranged in front of the lens of the projector, and the camera is a polarized camera, wherein:

[0019] The projector projects linearly polarized light stripes of the object to be measured into the field of view of the camera through the polarizing mirror;

[0020] The camera images multiple polarization state images of the object to be measured perpendicularly, the polarization angle of the projected linearly polarized light stripe is not perpendicular to the multiple polarization states, and multiple projection images with different brightness are obtained, wherein the number of projection images is consistent with the number of polarization state images;

[0021] The point cloud reconstruction unit is used for obtaining corresponding multiple pieces of point cloud through point cloud reconstruction for the multiple projection images;

[0022] The point cloud fusion unit is used for performing point cloud fusion on the multiple pieces of point cloud to obtain a three-dimensional reconstruction result of the object to be measured.

[0023] Compared with the prior art, the application has the advantages of high reliability, short imaging time, and convenient operation, and the like.

[0024] Other features and advantages of the present application will become apparent from the following detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

[0026] Figure 1 is a flowchart of a high-reflective surface three-dimensional reconstruction method based on a polarization structured light camera according to an embodiment of the application;

[0027] Figure 2 is a schematic diagram of a polarization structured light 3D system device according to an embodiment of the application;

[0028] Figure 3 is a schematic diagram of a polarization mirror principle according to an embodiment of the application;

[0029] Figure 4 is a schematic diagram of a Bayer array according to an embodiment of the application;

[0030] Figure 5 is a schematic diagram of a polarization camera pixel array according to an embodiment of the application;

[0031] Figure 6 is a schematic diagram of a mathematical model of a structured light system according to an embodiment of the application;

[0032] Figure 7 is a schematic diagram of a polarization projection image according to an embodiment of the application;

[0033] Figure 8 is a schematic diagram of reconstructed point cloud and point cloud fusion according to an embodiment of the application;

[0034] Figure 9 is a different method of reconstructing point cloud contrast map according to one embodiment of the present application. DETAILED DESCRIPTION

[0035] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions, and numerical values set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated.

[0036] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the scope of the application its application or uses.

[0037] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification, where appropriate.

[0038] In all of the examples shown and discussed herein, any specific values should be interpreted as merely exemplary, and not as a limitation. Thus, other examples of the exemplary embodiments can have different values.

[0039] It should be noted that like reference numerals and letters refer to like items in the following drawings, and thus, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0040] The present application proposes a method of removing high reflection using polarized light technology to achieve complete reconstruction of structured light. In short, the method uses a polarization camera that can image four polarization states at a time, and adds a linear polarizer in front of the projector lens to make the projected light linearly polarized at a fixed angle. Four point clouds are reconstructed using four different polarization state images, and complete point cloud is obtained by fusing the point clouds. This method can simultaneously reconstruct four polarization state point clouds with one projection. Since the projected light is polarized light, the image intensity received by each polarization state will be different, which can well compensate for the point cloud information at high light and dim positions. It should be understood that although it is described based on four polarization state images, the present application does not limit the number of polarization state images imaged and the number of point clouds reconstructed.

[0041] Referring to Figure 1 As shown, the provided method for three-dimensional reconstruction of high-reflective surfaces based on a polarized structured light camera includes the following steps:

[0042] Step S110, the projector projects linearly polarized light stripes into the field of view of the polarization camera through the polarizer arranged in front of the lens.

[0043] In combination with Figure 2As shown, the polarization structured light 3D device adopted by the present application comprises a projector, a polarizer and a polarization camera, wherein the polarizer is arranged in front of the lens of the projector, and the projector projects linearly polarized light stripes of the object to be measured into the field of view of the camera through the polarizer. In actual application, various types of commercially available or special polarizers, projectors and polarization cameras can be used. For example, a linear polarizer is currently used, but there is a certain degree of light intensity attenuation in practice, so a circular polarizer can also be used to achieve the same function and maintain the original light intensity.

[0044] The principle of the polarizer can be seen from Figure 3 Specifically, light is an electromagnetic wave, and the directions of the electric field and the magnetic field are perpendicular to the propagation direction of the light. It can be assumed that the polarizer is composed of an array of regular fine metal wires parallel to each other, and when the fine metal wires are thin enough, it can be approximately considered that the electrons can only move in one direction. Therefore, only the electromagnetic field component with the same direction as the polarizer does work on the electrons, and in fact the energy of the light is also attenuated to a certain extent, finally making the light vibrate only in one plane. The polarizer is installed in front of the lens of the projector, and the randomly vibrating projector light becomes linearly polarized light with a fixed polarization angle after passing through the polarizer, and is projected into the field of view of the camera.

[0045] In step S120, the polarization camera vertically images four polarization states of the image, and the polarization angle of the projected stripe is not perpendicular to the four polarization states, and four images with different brightness are obtained.

[0046] A common color RGB camera saves the light intensity of different color channels through a Bayer array as shown in Figure 4 , so as to convert a gray-scale picture into a color picture. The polarization camera adopts a similar method, as shown in Figure 5 , each of the four adjacent pixels saves a polarization state, which are 0°, 45°, 90° and 135° respectively. After completing image acquisition, the pixels corresponding to the polarization states are taken out, and four pictures are segmented and composed, so that four images with different polarization states can be obtained at one time, and each image is one quarter of the original image resolution.

[0047] In step S130, four pieces of point cloud are reconstructed according to the four projected images obtained.

[0048] The structured light based three-dimensional reconstruction technology is an active three-dimensional reconstruction method, which projects a coded stripe pattern to the target surface in the field of view of the camera through the projector, and the camera captures the target and decodes the coded stripe pattern, and then the three-dimensional information of the object can be obtained through the principle of triangulation. In an embodiment, 18 stripe patterns made of Gray code plus line shift coding are used.

[0049] Based on the camera imaging model, the relationship between the planar coordinates of the camera and its spatial three-dimensional point can be quickly established. In the structured light reconstruction technology, the camera is regarded as an inverse light path camera, so the camera model can also be used to establish the relevant relationship. As shown in Figure 6 M is a point on the object to be measured, and M is the corresponding point of the projector image p =(u p ,v p ) T , the corresponding point of the camera image is m c =(u c ,v c ) T .

[0050] Assuming the focal length of the camera is f c , according to the projection relationship:

[0051]

[0052] The coordinates of point M in the projector space coordinate system are M p =(x p ,y p ,z p ) T , and the coordinates in the camera space coordinate system are M c =(x c ,y c ,z c ) T , and the relationship between the two types of coordinates is as follows:

[0053] M p =R·M c +T(2)

[0054] Where R and T are the rotation matrix and spatial translation vector between the projector coordinate system and the camera coordinate system.

[0055] According to the camera model, the image coordinate system and the space coordinate system have the following relationship:

[0056]

[0057]

[0058] Where K is the internal parameter of the camera model.

[0059] Using the principle of triangulation, the depth information z c can be obtained:

[0060]

[0061] Finally, the three-dimensional coordinates of the space point M in the camera coordinate system Mc For:

[0062] M c = u c z c f c v c z c f c z c ) T (6)

[0063] The projector projects linearly polarized fringes, and the polarized camera takes 18 fringe patterns, and four fringe patterns of polarization states are obtained, and 72 fringe patterns are actually obtained. The above structured light three-dimensional reconstruction method can be used to reconstruct the point cloud under four polarization states.

[0064] In step S140, based on the obtained four point clouds, the point cloud fusion is performed based on the image brightness to obtain the three-dimensional reconstruction result of the object to be measured.

[0065] For the point cloud obtained by reconstruction, the overexposure and overdarkness of the image pixel will cause the point cloud to be missing, so it is necessary to implement the corresponding fusion operation by judging whether the pixel value of the gray image and the point cloud are missing. At the same time, the gray image pixels of the four polarization states are traversed, firstly, it is judged whether the corresponding pixel is overexposed, if overexposed, the pixel gray value will exceed 255, then it is checked whether the four point clouds of the corresponding pixel have points.

[0066] Specifically, if there is a gray value exceeding 255 and the point position is missing, it is considered that the point cloud is missing due to overexposure, at this time the fusion strategy is to select the point cloud of the polarization state with the lowest gray value to fill in;

[0067] If the gray value does not exceed 255 and there is no point cloud missing, at this time the fusion strategy is to take the average value of the point position coordinates of the four point clouds as the point position coordinates of the pixel;

[0068] If the gray value does not exceed 255, but there is a point cloud missing, it is considered that the point cloud is missing due to overdarkness, at this time the fusion strategy is to select the point cloud of the polarization state with the highest gray value to fill in.

[0069] Accordingly, the present invention also provides a three-dimensional reconstruction device for highly reflective surfaces based on a polarized structured light camera, used to implement one or more aspects of the above-mentioned method. For example, the device includes a projector, a camera, a polarizing mirror, a point cloud reconstruction unit, and a point cloud fusion unit. The polarizing mirror is positioned in front of the projector lens, and the camera is a polarizing camera. The projector projects linearly polarized light fringes of the object under test through the polarizing mirror into the camera's field of view. The camera vertically images multiple polarization states of the object under test, with the polarization angle of the projected linearly polarized light fringes not perpendicular to any of the multiple polarization states, thus obtaining multiple projected images with different brightness levels, wherein the number of projected images is consistent with the number of polarization state images. The point cloud reconstruction unit is used to obtain multiple point clouds corresponding to the multiple projected images through point cloud reconstruction. The point cloud fusion unit is used to fuse the multiple point clouds to obtain the three-dimensional reconstruction result of the object under test. In this device, the point cloud reconstruction unit and the point cloud fusion unit can be implemented using a processor, FPGA, or dedicated hardware.

[0070] To further verify the effectiveness of this invention, experimental verification was conducted. The experimental sample was a freeform metal relief plate. Both an existing structured light 3D camera with the same field of view and the device of this invention were used for photographic reconstruction. The reconstruction effects of single exposure, multiple exposure, and the device of this invention were compared. See also Figures 7 to 9 As shown, where, Figure 7 It is a polarization projection image. Figure 8 This is a diagram illustrating point cloud reconstruction and point cloud fusion. Figure 9 This is a schematic diagram illustrating the reconstructed point cloud of this invention under single and multiple exposure conditions. The image in the lower right corner corresponds to this invention. Figure 9 It is known that a single exposure will result in missing point clouds due to overexposure, while multiple exposures can repair the missing parts of the point cloud to a certain extent. The point cloud reconstructed according to the present invention has a complete effect.

[0071] In summary, this invention employs a structured light 3D system built using a polarizing camera, a projector, and a polarizing mirror, with the polarizing mirror mounted in front of the projector lens. The structured light 3D system is used to project images of different polarization states, eliminating high reflectivity and achieving complete reconstruction of point clouds.

[0072] Compared with the prior art, the present invention has the following advantages:

[0073] 1) Compared with existing solutions that use fitting methods to obtain missing information and eliminate the influence of high reflectivity to obtain complete point clouds, this invention is more reliable and trustworthy, and has more practical significance for industrial inspection.

[0074] 2) Compared with multi-exposure point cloud fusion, the present invention can obtain four point clouds for fusion in one imaging, without the need for multiple shooting projections, which shortens the time required and reduces the probability of introducing noise during the shooting process.

[0075] 3) Compared with installing a polarizer in the camera lens and manually adjusting the polarization angle to obtain images of different light intensity, the application installs a polarizer in front of the projector lens to project linearly polarized light, adopts a polarization camera capable of obtaining four polarization states to obtain images of different light intensity, does not need manual adjustment and can image four images at a time, and is convenient to use.

[0076] The application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith to implement various aspects of the application.

[0077] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a magneto-optical or other optical medium, and / or any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0078] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0079] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0080] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0081] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0082] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0083] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0084] Embodiments of the application have been described above, and the description is intended to be illustrative of the embodiments of the application and not exhaustive. Numerous modifications and adaptations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The scope of the application is defined by the appended claims.

Claims

1. A method for 3D reconstruction of high-reflective surface based on polarized structured light camera, comprising the following steps: a polarizer is arranged in front of the lens of a projector, the projector projects linearly polarized light fringe to the object to be measured into the field of view of a camera through the polarizer, wherein the camera is a polarized camera; the camera vertically images multiple polarization state images of the object to be measured, the polarization angle of the projected linearly polarized light fringe is not perpendicular to the multiple polarization states, and then multiple projection images with different brightness are obtained, wherein the number of projection images is consistent with the number of polarization state images; for the multiple projection images, corresponding multiple pieces of point cloud are obtained by point cloud reconstruction; the multiple pieces of point cloud are fused to obtain the 3D reconstruction result of the object to be measured; wherein the number of polarization state images and projection images is four, and each projection image is obtained according to the following steps: for the imaging of the camera, each of the four adjacent pixels stores a pixel of a polarization state, which is 0°, 45°, 90° and 135° respectively; the camera acquires the pixels of the corresponding polarization state after completing the image acquisition, and divides and synthesizes four pictures, so that four images of different polarization states are obtained by one shooting, and each image is one quarter of the original image resolution; wherein the projector projects linearly polarized light fringe of the object to be measured into the field of view of the camera through the polarizer includes: the projector projects a coded fringe pattern onto the surface of the object to be measured in the field of view of the camera, the camera captures the target and decodes the coded fringe pattern, and then the three-dimensional information of the object is obtained by the principle of triangulation; wherein the coded fringe pattern is a 18 fringe pattern made of Gray code plus line shift coding; wherein the point cloud fusion of the multiple pieces of point cloud comprises the following steps: traverse the gray scale pixels of the four polarization states, first judge whether the corresponding pixel exists overexposure, if overexposure, the pixel gray value exceeds 255, then check whether the four pieces of point cloud of the corresponding pixel have points: if the gray value exceeds 255 and the point position is missing, it is considered that the point cloud is missing due to overexposure, at this time, the polarization state point cloud with the lowest gray value is selected to fill in; if the gray value does not exceed 255 and there is no point cloud missing, the point position coordinates of the four pieces of point cloud are averaged as the point position coordinates of the pixel; if the gray value does not exceed 255, but there is a point cloud missing, it is considered that the point cloud is missing due to overdarkness, at this time, the polarization state point cloud with the highest gray value is selected to fill in.

2. The method of claim 1, wherein, In the process of point cloud reconstruction, for a point on the object to be measured, the corresponding three-dimensional coordinates are obtained according to the following steps: For the case of a camera focal length of The projective relationship is expressed as: wherein, is a point on the object to be measured, is the corresponding point in the camera image, is the corresponding point in the camera image, , is the coordinate in the projector space coordinate system, is the coordinate in the camera space coordinate system, ; the relationship between the projector space coordinate system and the camera space coordinate system is represented as: wherein, and R and t are the rotation matrix and the spatial translation vector between the projector coordinate system and the camera coordinate system, respectively; according to the camera model, the image coordinate system and the space coordinate system have the following relationship: Depth information is obtained using the principle of triangulation is expressed as: obtaining points three-dimensional coordinates in the camera coordinate system are represented as: wherein, is an intrinsic parameter of the camera model.

3. The method of claim 1, wherein, the polarizer comprises a regular array of fine metal wires parallel to each other.

4. The method of claim 1, wherein, The polarizer is installed in front of the lens of the projector, the randomly vibrating projector light becomes linearly polarized light with a fixed polarization angle after passing through the polarizer, and is projected into the field of view of the camera.

5. A high-reflective surface three-dimensional reconstruction device based on polarized structured light camera, comprising a projector, a camera, a polarizer, a point cloud reconstruction unit and a point cloud fusion unit, the polarizer is arranged in front of the lens of the projector, and the camera is a polarized camera, wherein: the projector projects linearly polarized light stripes to the camera field of view through the polarizer; the camera vertically images multiple polarization state images for the object to be measured, the polarization angle of the projected linearly polarized light stripes is not perpendicular to the multiple polarization states, and then multiple projection images with different brightness are obtained, wherein the number of projection images is consistent with the number of polarization state images; the point cloud reconstruction unit is used for obtaining corresponding multiple point clouds by point cloud reconstruction for the multiple projection images; the point cloud fusion unit is used for performing point cloud fusion on the multiple point clouds to obtain a three-dimensional reconstruction result of the object to be measured; wherein the number of polarization state images and projection images is four, and each projection image is obtained according to the following steps: for the imaging of the camera, each of the four adjacent pixels stores a pixel of a polarization state, which are 0°, 45°, 90° and 135° respectively; the camera acquires pixels of corresponding polarization states after completing image acquisition, and divides and composes four pictures, so that four images of different polarization states are obtained by one-time shooting, and each image is one quarter of the original image resolution; wherein the projector projects linearly polarized light stripes of the object to be measured to the camera field of view through the polarizer, which includes: the projector projects an encoded stripe pattern to the surface of the object to be measured in the camera field of view, the camera decodes the target and the encoded stripe pattern, and then obtains three-dimensional information of the object by the principle of triangulation; wherein the encoded stripe pattern is 18 stripe patterns made by Gray code plus line shift encoding; wherein the point cloud fusion of the multiple point clouds includes the following steps: traversing the gray scale pixels of the four polarization states, first judging whether the corresponding pixel is overexposed, if overexposed, the pixel gray value exceeds 255, then checking whether the four point clouds of the corresponding pixel have points: if the gray value exceeds 255 and the point position is missing, it is considered that the point cloud is missing due to overexposure, at this time, the polarization state point cloud with the lowest gray value is selected to fill in; if the gray value does not exceed 255 and there is no point cloud missing, the point position coordinates of the four point clouds are averaged as the point position coordinates of the pixel; if the gray value does not exceed 255, but there is a point cloud missing, it is considered that the point cloud is missing due to underexposure, at this time, the polarization state point cloud with the highest gray value is selected to fill in. The computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 4. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 6. A computer readable storage medium having stored thereon a computer program, wherein, ​

Images