Light projectors based on photomasks, depth cameras, and item picking systems

Abstract

This invention provides a photomask-based light projector, depth camera, and item picking system, comprising: a light source for projecting a linear laser array; a microlens array disposed on the light-emitting side of the light source for receiving the linear laser array and homogenizing it into a surface light; and a photomask, including a pre-set pattern structure disposed on the light-emitting side of the microlens array for receiving the surface light and projecting a structured light pattern according to the pattern structure. In this invention, a rotating mirror drives the rotation of multiple light-reflecting surfaces to reflect the linear laser projected by the light source onto the microlens array to form a linear laser array. The microlens array then homogenizes the linear laser array into a surface light, which is then projected into a structured light pattern through the photomask. This not only significantly improves the light projection power compared to VCSEL-based structured light technology but also provides a significantly denser light spot compared to galvanometer scanning devices, thereby improving the resolution of the depth camera.

Description

Technical Field

[0001] This invention relates to depth sensing, and more specifically, to a photomask-based light projector, a depth camera, and an item picking system. Background Technology

[0002] In recent years, with the continuous development of the electronics industry, 3D cameras with depth sensing capabilities have received increasing attention from the consumer electronics sector. Currently, the most mature depth measurement method is the structured light scheme, which projects a specific structured light pattern onto an object and then calculates the depth of the object at different locations by measuring the deformation or displacement of the pattern.

[0003] Structured light-based 3D imaging requires actively projecting structured light onto the object being measured. The dimensional parameters of the object are determined by the deformation of the structured light; hence, it is called active 3D measurement. There are many types of structured light, including point structured light, line structured light, surface structured light, and coded optical patterns. After the structured light is projected onto the surface of the object, it is highly modulated by the object. The modulated structured light is then captured by a camera system and transmitted to a computer for analysis and calculation to obtain the 3D surface data of the object.

[0004] Widely used stripe projection technology is actually a generalized form of surface structured light. Its main principle is as follows: a line laser emits a linear laser beam; this linear laser beam is reflected by a scanning mirror and projected onto the surface of a target object. As the scanning mirror rotates, the laser beam deflects at an angle, thus scanning the object's surface and forming a pre-defined stripe pattern. A CCD camera is used to capture the degree of curvature of the stripes modulated by the object; the curvature of the stripes is demodulated to obtain the phase, and then the phase is converted into the height of the entire field.

[0005] Striped projection structured light is often used in industrial grasping applications. In these applications, rapid imaging of the object to be grasped is required. However, striped projection via scanning galvanometer reflection is inefficient and does not help reduce the time required for each grasp, resulting in low grasping efficiency. Furthermore, the stripe density of structured light projected via scanning galvanometer reflection is affected by the rotation speed of the scanning galvanometer, making it difficult to further increase the stripe density and thus failing to improve the resolution of the depth camera. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the purpose of this invention is to provide a light projector, a depth camera, and an item picking system based on a photomask.

[0007] The photomask-based light projector provided by the present invention includes:

[0008] Light source, used to project a linear laser array;

[0009] A microlens array, disposed on the light-emitting side of the light source, is used to receive the linear laser array and homogenize the linear laser array into surface light;

[0010] A photomask, including a pre-set pattern structure, is disposed on the light-emitting side of the microlens array for receiving surface light and projecting a structured light pattern according to the pattern structure.

[0011] Preferably, the light source includes a laser, a rotating mirror, and a drive motor arranged sequentially along the optical path;

[0012] Linear laser, used to project linear laser light;

[0013] A rotating mirror, disposed on the light-emitting side of the linear laser, includes multiple light-reflecting surfaces arranged sequentially along the circumference, for receiving the linear laser and reflecting the linear laser onto the target object through the light-reflecting surfaces;

[0014] A drive motor is used to drive the rotating mirror to rotate so that the light reflecting surface can reflect the linear laser onto the target object to form a linear laser array.

[0015] Preferably, the photomask includes a transparent substrate and an opaque layer disposed on the substrate;

[0016] The opaque layer is formed into multiple light-transmitting areas by an etching process, and the multiple light-transmitting areas form the pattern structure.

[0017] The surface light projected by the microlens array passes through the multiple light-transmitting areas to form the structured light pattern.

[0018] Preferably, the linear laser comprises a laser, a collimating lens, and a Powell prism arranged sequentially along the optical path;

[0019] The laser is used to project a laser beam;

[0020] The collimating lens is disposed on the light-emitting side of the laser and is used to collimate the laser beam.

[0021] The Powell prism is disposed on the light-emitting side of the collimating lens to receive the collimated laser beam and emit the linear laser.

[0022] Preferably, the rotating mirror comprises a prism structure;

[0023] Each side of the prism structure forms the light-reflecting surface;

[0024] The drive motor is used to drive the rotating mirror to rotate 360° infinitely along the central axis of the prism structure, so that each of the light reflecting surfaces reflects the linear laser onto the target object to form a set of linear laser arrays.

[0025] Preferably, the rotating mirror is detachably connected to the drive motor;

[0026] The number of rotating mirrors is multiple, and multiple rotating mirror groups are formed among the multiple rotating mirrors. The number of side edges of the rotating mirrors in different rotating mirror groups is different.

[0027] The field of view (FOV) of the linear laser array projected by the rotating mirror decreases as the number of side edges of the rotating mirror increases.

[0028] Preferably, the rotating mirror includes a double-sided mirror structure;

[0029] Each side of the double-sided mirror structure forms the light-reflecting surface;

[0030] The drive motor is used to drive the double-sided mirror structure to rotate 360° infinitely along the central axis in the height direction, so that each of the light-reflecting surfaces reflects the linear laser onto the target object to form a set of linear laser arrays.

[0031] The depth camera provided according to the present invention includes the light projector, and further includes a camera module and a processor module;

[0032] The light projector is used to project a structured light pattern onto the target object;

[0033] The camera module is used to receive the first dot matrix image generated by the linear laser array after reflection from the target object;

[0034] The processor module is used to generate a first depth image based on the first dot matrix image.

[0035] The depth camera provided according to the present invention includes the light projector and also includes a binocular sensing module;

[0036] The light projector is used to project structured light patterns onto the target object;

[0037] The binocular sensing module is used to receive the linear laser array reflected from the target object to generate a second dot matrix image and a third dot matrix image, and then generate a second depth image based on the parallax of each pixel in the second dot matrix image and the third dot matrix image.

[0038] The item picking system provided by the present invention includes the aforementioned depth camera, and further includes:

[0039] Unit 1 and Unit 2 are used for storing and / or transporting materials;

[0040] The depth camera has a visual scanning area that at least covers the first unit for storing or transporting the material, and is used to visually scan the material, acquire a target depth image of the material, and generate the pose information of the material based on the target depth image.

[0041] The robot unit is communicatively connected to the depth camera and is used to receive the position and pose information, and to grasp the material in a corresponding grasping posture according to the position and pose information and then transfer it to the second unit.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] In this invention, a rotating mirror drives the rotation of multiple light-reflecting surfaces to reflect the linear laser projected by the linear laser onto a microlens array to form a linear laser array. The microlens array then homogenizes the linear laser array into surface light, which is then projected onto a structured light pattern through a photomask. This not only significantly improves the light projection power compared to VCSEL-based structured light technology, but also provides a significantly denser light spot compared to galvanometer scanning devices. This improves the resolution of the depth camera and facilitates its application in industrial grasping scenarios to accurately identify the pose of the object to be grasped.

[0044] In this invention, a rotating mirror drives the rotation of multiple light-reflecting surfaces to reflect the linear laser projected by the linear laser onto a microlens array to form striped structured light. This is unlike a scanning galvanometer, which requires resetting and deflecting to a certain angle after each reflection of the linear laser before it can reflect the linear laser again. This saves the time of projecting the striped structured light, improves the imaging speed of the depth camera, and is convenient for use in industrial grasping applications to quickly obtain the pose of the object to be grasped, thereby speeding up the grasping process. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0046] Figure 1 This is a schematic diagram of the principle of a photomask-based light projector in an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the principle of the light projector based on the rotating mirror in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the microlens array structure in an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of a drive motor driving a rotating mirror to rotate in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the optical path of a linear laser in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the principle of the rotating mirror-based light projector in a modified embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of the circuit principle of the photomask-based light projector in an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram illustrating the working principle of a binocular depth camera in an embodiment of the present invention;

[0054] Figure 9 This is a schematic diagram illustrating the working principle of a structured light-based depth camera in an embodiment of the present invention.

[0055] Figure 10 This is a schematic diagram of the item picking system in an embodiment of the present invention.

[0056] In the picture:

[0057] 1 is a linear laser; 2 is a rotating mirror; 3 is a microlens array; 301 is a first-layer microlens array; 302 is a second-layer microlens array; 4 is a photomask; 5 is a projection optical element; 6 is a drive motor; 7 is a target object; 101 is a laser; 102 is a collimating lens; 103 is a Powell prism; 11 is a first camera; 12 is a light projector; 13 is a second camera; 14 is a processor module. Detailed Implementation

[0058] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0059] It should be noted that when a component is referred to as "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as "connected to" another component, it can be directly connected to or indirectly connected to that other component. Furthermore, a connection can be for both fixing and circuit connection purposes.

[0060] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0061] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0062] Figure 1 This is a schematic diagram of the principle of a photomask-based light projector in an embodiment of the present invention, as shown below. Figure 1 As shown, the photomask-based light projector provided by the present invention includes:

[0063] Light source, used to project a linear laser array;

[0064] Microlens array 3 is disposed on the light-emitting side of the light source and is used to receive the linear laser array and homogenize the linear laser array into surface light;

[0065] The photomask 4 includes a pre-set pattern structure and is disposed on the light-emitting side of the microlens array 3 for receiving the surface light and projecting a structured light pattern according to the pattern structure.

[0066] The projection optical element 5 is disposed on the light-emitting side of the photomask and is used to receive the structured light pattern and image the structured light pattern onto the target object.

[0067] The pattern structure can be configured as a stripe structure or a dot matrix structure. Therefore, the structured light pattern can correspondingly be stripe structured light or dot matrix structured light. The photomask 4 includes a transparent substrate and an opaque layer disposed on the substrate. The opaque layer is etched to form multiple transparent areas, which together form the pattern structure. The laser beam emitted by the laser emitter passes through these transparent areas to form the structured light pattern. The substrate is made of a transparent material, preferably high-purity, low-reflectivity, and low-thermal-expansion-coefficient quartz glass. The opaque layer can be configured into pattern structures of different shapes, and these pattern structures can be set, thereby controlling the shape of the structured light pattern.

[0068] Typically, the minimum linewidth of the pattern structure formed by etching is between 2-5 μm, and each light-transmitting area can be C-shaped, cross-shaped, triangular, circular, or linear.

[0069] In embodiments of the present invention, such as Figure 2 As shown, the linear laser 1 includes a laser 1, a rotating mirror 2, and a drive motor 6 arranged sequentially along the optical path;

[0070] Linear laser 1, used to project linear laser light;

[0071] A rotating mirror 2 is disposed on the light-emitting side of the linear laser 1 and includes multiple light-reflecting surfaces arranged sequentially along the circumference. These surfaces are used to receive the linear laser and reflect it onto the target object 7.

[0072] A drive motor 6 is used to drive the rotating mirror 2 to rotate so that the light reflecting surface can reflect the linear laser onto the target object 7 to form a linear laser array.

[0073] In this embodiment of the invention, the linear laser projected by the linear laser 1 is reflected onto the microlens array 3 by the rotation of multiple light-reflecting surfaces driven by the rotating mirror 2, forming a linear laser array. The microlens array 3 then homogenizes the linear laser array into surface light, which is then projected onto a structured light pattern through the photomask 4. This not only significantly improves the light projection power compared to VCSEL-based structured light technology, but also provides a significantly denser light spot compared to galvanometer scanning devices, improving the resolution of the depth camera. This facilitates its application in industrial grasping scenarios to accurately identify the pose of the object to be grasped. The rotation of multiple light-reflecting surfaces driven by the rotating mirror 2 reflects the linear laser projected by the linear laser 1 onto the microlens array 3 to form a linear laser array, unlike scanning galvanometers which require resetting and deflecting to a certain angle after each reflection before reflecting the linear laser again. This saves the projection time of the linear laser array, increases the imaging speed of the camera 1, and facilitates its application in industrial grasping scenarios to accurately identify the pose of the object to be grasped, thereby accelerating the grasping process.

[0074] Because VCSEL lasers are prone to overheating when the drive current is high, their output optical power decreases as the drive current increases, making it difficult for the total output optical power to exceed 1W, resulting in low power distributed across individual light spots. In contrast, the output optical power of edge-emitting lasers and semiconductor lasers increases with the drive current, with peak output optical power reaching tens of W. Therefore, it is easy to achieve output optical power of several W, such as 2W. Furthermore, by using a rotating mirror 2 to sequentially flood the entire surface of the target object 7 with linear laser beams from the edge-emitting laser and semiconductor laser, and ensuring that each reflection of the linear laser beam has an output optical power of 2W, this output power is significantly greater than that of a single light spot in a VCSEL laser.

[0075] Because the etching technology is mature and has low error, the pattern structure formed on the photomask 4 has high precision. The minimum linewidth during etching is distributed between 2-5 μm, thus allowing for the formation of denser light-transmitting holes on the photomask 4. Furthermore, it ensures that the size and shape of the pattern structure meet design requirements, enabling the camera module to more accurately identify the structured light pattern reflected from the target object 7, thereby improving the accuracy of acquiring the three-dimensional feature information of the target object 7 and enhancing depth imaging quality.

[0076] Figure 3 This is a schematic diagram of the microlens array structure in an embodiment of the present invention, as shown below. Figure 3 As shown, the microlens array 3 includes a substrate and a first layer microlens array 301 and a second layer microlens array 302 disposed on the substrate;

[0077] By setting up at least two layers of microlens arrays, the linear laser array projected by the linear laser 1 can form a more uniform surface light.

[0078] The centers of the microlenses in the microlens array 3 are randomly or regularly distributed, and the curved surfaces of the microlenses are randomly distributed or arranged with the same curved surface. The distance between any two adjacent microlenses is between 1 μm and 100 μm. The microlenses are composed of continuous curved surfaces, including convex surfaces, concave surfaces, and wavy surfaces.

[0079] The microlens are spherical or aspherical. The radius of curvature R of the microlens is 3μm-300μm. The substrate is hard glass or soft plastic. The thickness of the microlens array 3 is 0.01-0.1mm.

[0080] Figure 4 This is a schematic diagram of the drive motor driving the rotating mirror to rotate in an embodiment of the present invention, as shown below. Figure 2As shown, the rotating mirror 2 rotates under the drive of the high-speed drive motor 6. The rotating mirror 2 reflects the linear laser output by the linear laser 1 and then projects it onto the surface of the object being measured. As the incident position of the linear laser on the light reflection surface changes during the rotation of the rotating mirror 2, the linear laser scans across the entire surface of the object being measured, forming a linear laser array on the surface of the object being measured.

[0081] Figure 5 This is a schematic diagram of an optical path of a linear laser in an embodiment of the present invention, such as... Figure 5 As shown, the linear laser 1 includes a laser 101, a collimating lens 102, and a Powell prism 103 arranged sequentially along the optical path;

[0082] The laser 101 is used to project a laser beam;

[0083] The collimating lens 102 is disposed on the light-emitting side of the laser 101 and is used to collimate the laser beam.

[0084] The Powell prism 103 is disposed on the light-emitting side of the collimating lens 102, and is used to receive the collimated laser beam and emit the linear laser.

[0085] In one embodiment of the present invention, the laser 101 is a semiconductor laser or a side-emitting laser. In actual measurements, since common objects absorb less red light, a red laser with a wavelength of 650nm is chosen as the light source. While high laser power can make the projected Gray code image more distinct, excessively fast pattern projection can lead to insufficient light intensity accumulation, causing the lattice laser array to be overwhelmed by background light or resulting in unclear stripes due to a low signal-to-noise ratio. Considering both imaging clarity and measurement speed, a 50-80mW laser is ultimately selected.

[0086] In one embodiment of the present invention, the laser 101 is a semiconductor laser, and laser stroboscopic imaging is achieved by switching the laser 101 on and off using PWM modulation. With the cooperative operation of the rotating mirror 3, Gray code-encoded striped structured light is generated. The modulation frequency of the semiconductor laser is as high as 200MHz, which is sufficient to meet the stroboscopic imaging requirements of the measurement equipment.

[0087] The Powell prism 103 can shape an elliptical parallel laser beam into a linear laser with a certain linewidth. The front ridge of the Powell prism 103 is perpendicular to the long axis of the collimating lens 102. The geometric centers of the laser 101, the collimating lens 102, and the Powell prism 103 are located on the same optical axis.

[0088] The laser 101 is located at the front focal point of the collimating lens 102, thereby achieving collimation of the laser beam. The distance between the collimating lens 102 and the Powell prism 103 is 6 to 20 mm, preferably 10 to 12 mm. It should be noted that the distance between the collimating lens 102 and the Powell prism 103 can be designed according to actual needs.

[0089] In this embodiment of the invention, the rotating mirror 2 includes a prism structure;

[0090] Each side of the prism structure forms the light-reflecting surface;

[0091] The drive motor 6 is used to drive the rotating mirror 2 to rotate along the central axis of the prism structure, so that each of the light reflecting surfaces reflects the linear laser onto the target object 7 to form a set of linear laser arrays.

[0092] The prism structure can be any prism structure such as equilateral triangular prism, equilateral quadrangular prism, equilateral pentagonal prism, equilateral hexagonal prism, equilateral heptagonal prism, and equilateral octagonal prism, with the equilateral octagonal prism being preferred.

[0093] The rotating mirror 2 rotates at speeds ranging from 1000 to 10000 revolutions per minute. Assuming the rotating mirror 2 has eight surfaces, this rotational speed range translates to a frequency variation of 8K to 80K times per minute for the light-reflecting surfaces, which is equivalent to a frequency of 133.33 Hz to 1333.33 Hz. The scanning galvanometer, however, requires a frequency <100 Hz. Therefore, the frequency of the rotating mirror 2 is significantly higher than that of the scanning galvanometer. As the rotational speed of the rotating mirror 2 increases, the linear laser array approaches surface light infinitely.

[0094] In one embodiment of the present invention, the rotating mirror 2 is detachably connected to the drive motor 6;

[0095] The number of rotating mirrors 2 is multiple, and multiple rotating mirror groups are formed among the multiple rotating mirrors 2. The number of side edges of the rotating mirrors 2 in different rotating mirror groups is different.

[0096] Each of the aforementioned rotating mirror groups has at least one rotating mirror 2.

[0097] The field of view (FOV) of the linear laser array projected by the rotating mirror 2 decreases as the number of side edges of the rotating mirror 2 increases.

[0098] Therefore, in this embodiment, the FOV of the linear laser array can be easily adjusted, and different FOV application scenarios can be achieved by changing the rotating mirror 2.

[0099] Figure 6 This is a schematic diagram of the principle of a rotating mirror-based light projector in a variation of the present invention, as shown below. Figure 6 As shown, the rotating mirror 2 includes a double-sided mirror structure;

[0100] Each side of the double-sided mirror structure forms the light-reflecting surface;

[0101] The drive motor 6 is used to drive the rotation along the central axis of the double-sided mirror structure in the height direction, so that each of the light reflecting surfaces reflects the linear laser onto the target object 7 to form a set of linear laser arrays.

[0102] Each of the light-reflecting surfaces is a plane with the same shape and size.

[0103] That is, by rotating a double-sided mirror along the central axis in the height direction, two sets of linear laser arrays are projected by alternately reflecting linear laser light through two light-reflecting surfaces.

[0104] Figure 7 This is a schematic diagram of the circuit principle of a photomask-based light projector in an embodiment of the present invention, as shown below. Figure 7 As shown, the PC acts as the host computer, communicating with the FPGA via a serial port and sending commands to the FPGA. The FPGA then synchronously sends signals to the linear laser 1, the rotating mirror 2, and the camera. The linear laser 1 projects a radio-frequency strobe linear laser, thus forming alternating bright and dark stripes on the surface of the object under test as the rotating mirror 2 rotates. The microlens array 3 homogenizes the alternating bright and dark stripes into surface light. The photomask 4 receives the surface light and projects a structured light pattern according to the pattern structure. The camera takes a picture from the rising edge of the voltage level, setting an appropriate exposure time to cover one projection and reflection process of the linear laser, thus completing one picture. The FPGA sends a series of continuous signals to control the shooting of multiple pictures with different stripe structures to complete one imaging process.

[0105] Figure 8 This is a schematic diagram illustrating the working principle of a binocular depth camera in an embodiment of the present invention, as shown below. Figure 8 As shown, the depth camera provided by the present invention includes the light projector 12 and also includes a binocular sensing module.

[0106] The light projector 12 is used to project a structured light pattern onto the target object 7;

[0107] The binocular sensing module is used to receive the structured light pattern reflected by the target object 7 to generate a first dot matrix image and a second dot matrix image, and then generate a depth image based on the parallax of each pixel in the first dot matrix image and the second dot matrix image.

[0108] In this embodiment of the invention, the binocular sensing module 13 includes a first camera 11, a second camera 13, and a processor module 14, and the first camera 11 and the second camera 13 are fixedly connected. Specifically, the first camera 11 and the second camera 13 can be fixedly connected by a rigid structure, or the fixed connection can be achieved in other ways, and the specific method is not limited here. In this embodiment of the invention, the internal parameters and relative positions (external parameters) of the first camera 11 and the second camera 13 can be obtained through pre-calibration.

[0109] The first camera 11 receives the structured light pattern reflected from the target object 7 and generates a first dot matrix image; the second camera 13 receives the structured light pattern reflected from the target object 7 and generates a second dot matrix image; the processor module 14 generates a depth image based on the parallax of each pixel in the first and second dot matrix images. The processor module 14 is also used to control the light projector 12 and the first and second cameras 11 to be simultaneously turned on or off.

[0110] The processor module 14 can be an independent dedicated chip, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc., or it can contain a general-purpose processor. For example, when the depth camera is integrated into a smart terminal such as a mobile phone, computer, or controller of an item picking system, the processor in the terminal can be at least a part of the processor module 14.

[0111] Figure 9 This is a schematic diagram illustrating the working principle of a structured light-based depth camera in an embodiment of the present invention, as shown below. Figure 9 As shown, the depth camera provided by the present invention includes the light projector 12, and also includes a camera module 1 and a processor module 14;

[0112] The light projector 12 is used to project a structured light pattern onto the target object 7;

[0113] The camera module 1 is used to receive the structured light pattern reflected by the target object 7 and generate a first dot matrix image.

[0114] The processor module 14 is used to generate a first depth image based on the first dot matrix image.

[0115] In this embodiment of the invention, the camera module 1 includes a third camera, which employs a CMOS light sensor, a CCD light sensor, and a SPAD light sensor. A CMOS light sensor is preferred. However, the type of light sensor is not limited to those described above and may include other types of light sensors.

[0116] As shown above, in this embodiment, when the depth camera is working, the light projector 12 internally uses a rotating mirror 2 to drive the rotation of multiple light reflecting surfaces, thereby reflecting the linear laser projected by the linear laser 1 onto the microlens array 3 to form a linear laser array. Then, the microlens array 3 homogenizes the linear laser array into surface light and projects a structured light pattern through a photomask. This not only significantly improves the light projection power compared to VCSEL-based structured light technology, but also provides a significantly denser light spot compared to galvanometer scanning devices, thus improving the resolution of the depth camera. This makes it easier to apply in industrial grasping applications to accurately identify the pose of the object to be grasped. The rotation of multiple light reflecting surfaces by the rotating mirror 2 to reflect the linear laser projected by the linear laser 1 onto the microlens array 3 to form a linear laser array, instead of requiring a reset and deflection to a certain angle after each reflection of the linear laser as with scanning galvanometers, saves the projection time of the linear laser array, improves the imaging speed of the depth camera, and makes it easier to apply in industrial grasping applications to accurately identify the pose of the object to be grasped and accelerate the grasping process.

[0117] Figure 10 This is a schematic diagram of the item picking system in an embodiment of the present invention, as shown below. Figure 10 As shown, the item picking system provided by the present invention includes the depth camera 100, and further includes:

[0118] Unit 1 and Unit 2 are used for storing and / or transporting materials;

[0119] The depth camera 100 has a visual scanning area that at least covers the first unit for storing or transporting the material, and is used to visually scan the material, acquire a target depth image of the material, and generate the pose information of the material based on the target depth image.

[0120] The robot unit 400 is communicatively connected to the depth camera 100, and is used to receive the position and pose information, and to grasp the material in a corresponding grasping posture according to the position and pose information and then transfer it to the second unit.

[0121] In one embodiment of the present invention, the first unit may be configured as a storage unit 200 and a feeding unit 300;

[0122] Storage unit 200 is used for storing materials;

[0123] A feeding unit 300 is provided for placing and transporting the storage unit 200;

[0124] The depth camera 100 has a visual scanning area that covers at least part of the transport path of the transported material. It is used to visually scan the transported material after it enters the visual scanning area, acquire a target depth image of the transported material, and generate the pose information of the material based on the target depth image.

[0125] The robot unit 400 is communicatively connected to the depth camera 100, and is used to receive the position and pose information, and to grasp the material in a corresponding grasping posture according to the position and pose information and then transfer it to the second unit.

[0126] The second unit can be configured to perform barcode recognition and to transport or store the recognized materials.

[0127] As shown above, in operation, the item picking system of this embodiment utilizes a rotating mirror 2 to rotate multiple light-reflecting surfaces within the depth camera 100. This reflects the linear laser beam projected by the linear laser 1 onto the microlens array 3, forming a linear laser array. The microlens array 3 then homogenizes the linear laser array into surface light, which is then projected onto a structured light pattern through a photomask 4. This not only significantly improves the light projection power compared to VCSEL-based structured light technology but also provides a denser light spot compared to galvanometer scanning devices, thereby improving the depth camera's resolution and facilitating its application in industrial applications. In industrial grasping applications, the linear laser projected by the linear laser 1 is reflected onto the microlens array 3 by rotating multiple light-reflecting surfaces driven by the rotating mirror 2 to form a linear laser array, instead of requiring the scanning galvanometer to reset and deflect to a certain angle after each reflection before it can reflect the linear laser again, as is the case with the scanning galvanometer. This saves the projection time of the linear laser array, improves the imaging speed of the depth camera 100, and facilitates its application in industrial grasping applications to accurately identify the pose of the object to be grasped, thereby accelerating the grasping rhythm of the object picking system.

[0128] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0129] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A photomask-based light projector, characterized by, The application relates to a light projector. The light projector comprises: a light source for projecting a linear laser array; a microlens array arranged on the light exit side of the light source for receiving the linear laser array and homogenizing the linear laser array into a plane light; a light mask comprising a pre-set pattern structure arranged on the light exit side of the microlens array for receiving the plane light and projecting a structured light pattern according to the pattern structure. The light source comprises a linear laser, a rotating mirror and a driving motor arranged in sequence along the light path. The linear laser is used for projecting a linear laser. The rotating mirror is arranged on the light exit side of the linear laser and comprises a plurality of light reflection surfaces arranged in sequence along the circumference, and is used for receiving the linear laser and reflecting the linear laser onto the microlens array through the light reflection surfaces.

2. The photomask-based light projector of claim 1, wherein, The driving motor is used for driving the rotating mirror to rotate so that the light reflection surfaces can reflect the linear laser onto the microlens array to form a linear laser array. The light mask comprises a transparent substrate and an opaque layer arranged on the substrate. The opaque layer forms a plurality of light transmission zones through an etching process, and the plurality of light transmission zones form the pattern structure.

3. The light mask-based light projector according to claim 1, wherein, The plane light projected by the microlens array passes through the plurality of light transmission zones to form the structured light pattern. The linear laser comprises a laser, a collimating lens and a Powell prism arranged in sequence along the light path. The laser is used for projecting a laser beam. The collimating lens is arranged on the light exit side of the laser and is used for collimating the laser beam.

4. The light mask-based light projector according to claim 1, wherein, The Powell prism is arranged on the light exit side of the collimating lens and is used for receiving the collimated laser beam and emitting the linear laser. The rotating mirror comprises a prism structure. Each side surface of the prism structure forms the light reflection surface.

5. The photomask-based light projector of claim 3, wherein, The driving motor is used for driving the rotating mirror to rotate 360 degrees along the central axis of the prism structure so that each light reflection surface reflects the linear laser onto a target object to form a group of linear laser arrays. The rotating mirror is detachably connected with the driving motor. The number of the rotating mirrors is plural, and a plurality of rotating mirror groups are formed in the plurality of rotating mirrors, and the number of side edges of the rotating mirrors in different rotating mirror groups is different.

6. The light mask-based light projector according to claim 1, wherein, The FOV of the linear laser array projected by the rotating mirror decreases with the increase of the number of side edges of the rotating mirror. The rotating mirror comprises a two-sided mirror structure. Each side surface of the two-sided mirror structure forms the light reflection surface.

7. A depth camera, characterized by The driving motor is used for driving the two-sided mirror structure to rotate 360 degrees along the central axis in the height direction so that each light reflection surface reflects the linear laser onto a target object to form a group of linear laser arrays. The light projector comprises the camera module and the processor module. The light projector is used for projecting a structured light pattern to a target object. The camera module is used for receiving the structured light pattern reflected by the target object to generate a first dot array image.

8. A depth camera, characterized by The processor module is used for generating a first depth image according to the first dot array image. The light projector comprises the binocular sensing module. The light projector is used for projecting a structured light pattern to a target object. The binocular sensing module is configured to receive the second dot array image and the third dot array image generated by the linear laser array after the target object reflects the linear laser array, and further generate a second depth image according to the parallax of each pixel in the second dot array image and the third dot array image.

9. An article picking system, characterized by, The depth camera of claim 7 or 8, further comprising: The first unit and the second unit are used for storing or / and transporting materials; The depth camera has a visual scanning area covering at least the first unit storing or transporting the materials, and is used for visually scanning the materials, collecting a target depth image of the materials, and generating pose information of the materials according to the target depth image; The robot unit is in communication connection with the depth camera, is used for receiving the pose information, and moves the materials to the second unit after grabbing the materials in a corresponding grabbing pose according to the pose information.

Images