Multi-dimensional material sensing system and method

Abstract

Multi-dimensional light imaging, detection, and ranging (LIDAR) systems and methods optionally include a laser device configured to generate a plurality of light pulses emitted toward an object; a detector configured to receive a portion of the plurality of light pulses returned from the object; and a processor configured to generate a point cloud representing the object based on the plurality of light pulses received by the detector, the point cloud having a plurality of points, each point having three-dimensional position coordinates representing a position of the point on the object and having at least one additional value, the at least one additional value representing at least one of material information or optical information, the material information indicating a material of the object at the position of the point on the object, the optical information indicating at least one optical property of the plurality of light pulses returned from the surface of the object from the position of the point on the object.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 835,256, filed April 17, 2019. The entire disclosure of the above application is incorporated herein by reference. Technical Field

[0003] The present disclosure relates to material sensing systems and methods, and more particularly, to multi-dimensional material sensing systems and methods for making and using these systems. Background Art

[0004] This section provides background information related to the present disclosure which is not necessarily prior art.

[0005] LIDAR is a surveying method that measures the distance to an object by illuminating it with a pulsed laser and measuring the reflected pulses with a sensor. The difference in the laser's return time and wavelength can then be used to create a digital 3D representation of the detected object. LIDAR can be used to produce high-resolution maps and has applications in geodesy, surveying, archaeology, geography, geology, geomorphology, seismology, forestry, atmospheric physics, laser guidance, airborne laser swath mapping (ALSM), and laser altimetry. LIDAR technology can also be used for the control and navigation of autonomous vehicles.

[0006] Conventional LIDAR devices operate as follows. A laser source generates pulses of polarized or unpolarized light of a specific wavelength. A time-of-flight sensor records the initial time when the light is first emitted. The time-of-flight is used to determine the total distance the light travels from the source to the detector, using the speed of light.

[0007] The emitted light is then "steered" at a given angle. This "steering" can also include splitting the light pulse into multiple pulse components for different angles. The steering angle will vary over time to obtain a specific field of view for comprehensive mapping of the environment. After aiming, the light can pass through linear polarization optics before and after emission. These types of LIDAR are called polarization LIDARs, and polarization optics can be used at the registration step.

[0008] Conventional LIDAR devices typically use bulky and expensive optical lenses. In addition, due to the sensitivity of optical lenses used in conventional LIDAR devices to moisture, the optical lenses require a lot of protective packaging, which increases the weight, size and complexity of LIDAR devices using optical lenses. Implementing LIDAR systems with rotating optics (e.g., Velodyne-HDL64) in autonomous vehicles and robotsTM A well-known problem with LIDAR systems (models) is the large size and high cost of LIDAR systems. Rotating the entire device to steer the laser beam reduces reliability, limits miniaturization, and increases energy consumption. LIDAR systems based on solid-state beam steering address this problem, but their implementation is hampered by insufficient accuracy and range. Another issue is the performance of LIDAR and all other sensors in inclement weather. Currently used laser beams, with wavelengths of approximately 900nm to 940nm, are strongly scattered by rain, fog, and snow, making their readings highly uncertain in such conditions.

[0009] Furthermore, conventional LIDAR devices and their accompanying analysis systems have proven limited in their ability to accurately perform object recognition. For example, known LIDAR point clouds are based solely on distance readings from the laser source to the object. In this representation of the human world, a person sitting on a bench and a statue of the person sitting on it appear identical. This problem also applies to a sleeping baby and a similarly sized plastic doll lying next to it, or when trying to distinguish a black car from a sidewalk in the distance. The burden of distinguishing these objects and identifying the surrounding environment falls on the computational processing of these 3D maps.

[0010] Adequately classifying objects based on their geometry is a non-trivial problem, requiring complex algorithms and significant computing power, especially given the highly dynamic nature of various environments. Furthermore, conventional LIDAR hardware makes adequate object recognition and classification even more difficult because current beam steering methods lead to clustering and grouping of points in the LIDAR cloud, which results in ambiguous interpretations of the 3D image and its individual points. Consequently, geometry-based perception of the surrounding environment requires high computational costs, high energy consumption, and long processing times.

[0011]

[0006] Therefore, there is a need for improved LIDAR systems and methods, particularly LIDAR systems and methods that provide the ability to identify the material forming an object. Summary of the Invention

[0012] This section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.

[0013] In certain aspects, the present disclosure provides a system comprising a laser device configured to generate a plurality of light pulses emitted toward an object; a detector configured to receive a portion of the plurality of light pulses returned from the object; a processor configured to generate, based on the plurality of light pulses received by the detector, a point cloud representing the object, the point cloud having a plurality of points, each point having three-dimensional position coordinates representing a position of the point on the object and having at least one additional value, the at least one additional value representing at least one of material information or optical information, the material information being indicative of a material of the object at the position of the point on the object, the optical information being indicative of at least one optical property of the plurality of light pulses returned from a surface of the object from the position of the point on the object.

[0014] In an aspect, the optical information comprises at least one of polarization information, Raman scattering information, and circular dichroism information.

[0015] In an aspect, the at least one additional value represents material information indicative of a material on a surface of the object at the position of the point, and the processor is configured to generate the material information based on the optical information indicative of at least one optical property of a plurality of reflections returned from the surface of the object at the position of the point.

[0016] In another aspect, the processor is configured to generate the material information based on at least one of polarization information, Raman scattering information, and circular dichroism information of a reflection returned from a surface of the object at the position of the point.

[0017] In an aspect, the object is one of a plurality of objects, and the system further comprises a conveyor configured to transport the plurality of objects past the laser device; and a sorter configured to sort the plurality of objects into a plurality of containers; wherein the at least one additional value represents the material information indicative of a material on a surface of the object at the position of the point, and the processor is configured to control the sorter to sort the object into one of the plurality of containers based on the material information.

[0018] In one aspect, the system further comprises at least one additional laser device configured to generate a plurality of additional light pulses emitted toward the object, the at least one additional laser device being located at a different location than the laser device; and at least one additional detector configured to receive a portion of the plurality of additional light pulses returned from the object, the at least one additional detector being located at a different location than the detector; wherein the processor is configured to generate a point cloud representing the object additionally based on the plurality of additional light pulses received by the at least one additional detector.

[0019] In other variations, the present disclosure provides a method comprising initializing, using at least one processor, parameters of a neural network based on physical laws of light / matter interaction; emitting, using a laser device, a plurality of light pulses toward an object having a predetermined material composition; receiving, using a detector, a plurality of reflections of the plurality of light pulses returning from the object; inputting, using the processor, optical properties of the plurality of reflections into the neural network; receiving, using the processor, an expected material composition of the object from the neural network based on the input optical properties; comparing, using the processor, the expected material composition with the predetermined material composition; and adjusting, using the processor, the parameters of the neural network based on the comparison.

[0020] On one hand, tuning parameters is performed using machine learning algorithms.

[0021] In one aspect, the method further comprises repeatedly adjusting the parameters of the neural network based on comparing additional expected material compositions for additional objects with additional known material compositions for additional objects until a difference between one of the expected material compositions and a corresponding one of the additional known material compositions is less than a predetermined error margin, the additional expected material compositions being generated by inputting optical characteristics of a plurality of additional reflections of a plurality of additional light pulses reflected by the laser device off the additional object.

[0022] In other variations, the present disclosure provides another system comprising a polarization camera configured to receive ambient light reflected off a surface of an object and generate image data associated with the object and polarization data associated with the ambient light reflected off the surface of the object. The system also includes a processor configured to generate a point cloud representing the object based on the image data and the polarization data, the point cloud having a plurality of points, each point having three-dimensional position coordinates and material information, the three-dimensional position coordinates representing a position of the point on the object, the material information indicating a material of the object at the position of the point on the object, and the processor further configured to determine the material information based on the polarization of the ambient light reflected off the object at the position of the point on the object.

[0023] On the one hand, the processor is also configured to: access a material database that stores relevant information between the polarization of ambient light and the material of the object; and determine the material information indicating the material of the object at the position of each point on the object based on the material database and based on the polarization of the ambient light reflected away from the object from the position of each point on the object.

[0024] On the one hand, the processor is also configured to: determine the multiple materials constituting the object based on material information indicating the material of the object at each point on the object; and perform at least one of classifying or identifying the object based on the multiple materials constituting the object.

[0025] On the other hand, the processor is further configured to: access an object database that stores relevant information between multiple objects and the materials constituting each of the multiple objects; and perform at least one of classifying or identifying the objects based on the relevant information stored in the object database and the multiple materials constituting the objects.

[0026] In one aspect, the processor is further configured to determine an edge within the image data detected by the polarization camera between the object and an additional object based on a difference between the polarization data associated with the ambient light reflected off the surface of the object and the additional polarization data associated with the ambient light reflected off the surface of the additional object.

[0027] In one aspect, an additional light source can be used to image an object with a polarization camera. This light source can be a laser from a LIDAR used in the same perception system.

[0028] On one hand, because the polarization characteristics of a material vary with the reflection and scattering angle of light leaving an object, material identification can be achieved by using the time- and distance-progression of images acquired by a polarization camera. AI / ML analysis of the time- and distance-correlations of polarization images increases the accuracy of material and object identification.

[0029] In one aspect, the object is one of a plurality of objects, and the system further includes: a conveyor configured to transport the plurality of objects past the polarization camera; and a sorter configured to sort the plurality of objects into a plurality of containers. The processor is configured to control the sorter to sort the object into one of the plurality of containers based on the material information.

[0030] In one aspect, the system further comprises at least one additional polarization camera located at a different location from the polarization camera and configured to generate additional image data and additional polarization data associated with the object; and the processor is configured to generate a point cloud representing the object additionally based on the additional image data and the additional polarization data generated by the at least one additional polarization camera.

[0031] In other variations, the present disclosure provides another method, comprising: initializing, using at least one processor, parameters of a neural network based on physical laws of light / matter interaction; and receiving, using a polarization camera, a plurality of reflections of ambient light reflected off the object. The method further comprises: determining, using the processor, polarization information of the plurality of reflections of ambient light reflected off the object; inputting, using the processor, the polarization information of the plurality of reflections into the neural network; and receiving, using the processor, an expected material composition of the object from the neural network based on the input polarization information. The method further comprises: comparing, using the processor, the expected material composition with the predetermined material composition; and adjusting, using the processor, the parameters of the neural network based on the comparison.

[0032] In one aspect, adjusting parameters is performed using a machine learning algorithm and / or an artificial intelligence algorithm.

[0033] In one aspect, the method further comprises repeatedly adjusting the parameters of the neural network based on a comparison of additional expected material compositions of additional objects with additional known material compositions of additional objects until a difference between one of the expected material compositions and a corresponding one of the additional known material compositions is less than a predetermined error margin, the additional expected material compositions being generated by inputting additional polarization information of multiple additional reflections of ambient light reflected off the additional objects and received by the polarization camera.

[0034] Further areas of applicability will become apparent from the description provided herein.The description and specific examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure.

[0036] Figure 1 is a functional diagram illustrating an M-LIDAR system according to certain aspects of the present disclosure;

[0037] Figure 2a to Figure 2b is a scanning electron microscope (SEM) image of a nano-kirigami nanocomposite sheet configured for use in an M-LIDAR system according to certain aspects of the present disclosure;

[0038] Figures 3a to 3c is a graphene composite material based on nano-kirigami at various strain levels ( Figure 3a 0% of Figure 3b 50% and Figure 3c Image of the laser diffraction pattern under 100% of the image;

[0039] Figures 4a to 4d illustrates a representative simplified process for fabricating nano-kirigami-based optical elements according to certain aspects of the present disclosure;

[0040] Figures 5a to 5c Figure 1 illustrates a nano-kirigami nanocomposite optical element fabricated on a wafer according to certain aspects of the present disclosure. According to certain aspects of the present disclosure, Figure 5a Photographs showing nanokirigami nanocomposite optical elements, Figure 5b Shown Figure 5a SEM image of nano-kirigami nanocomposite optical element at 0% strain, and Figure 5c Pictured Figure 5a SEM image of nano-kirigami nanocomposite optical element under 100% strain;

[0041] Figure 6 illustrates a confusion matrix for MST using an artificial intelligence algorithm and polarization information according to certain aspects of the present disclosure;

[0042] Figure 7 illustrates a confusion matrix for detecting simulated black ice compared to other materials according to certain aspects of the present disclosure;

[0043] Figure 8 illustrates one embodiment of a black ice detection unit incorporating an M-LIDAR system according to certain aspects of the present disclosure;

[0044] Figure 9 is a flow chart illustrating a method for performing object classification using an M-LIDAR system according to certain aspects of the present disclosure;

[0045] Figure 10 is a schematic diagram of a planar composite material having a representative plurality of kirigami cuts formed therein in a linear pattern;

[0046] Figure 11 is a diagram illustrating an M-LIDAR system for mounting on a vehicle according to certain aspects of the present disclosure;

[0047] Figure 12 illustrates a multi-dimensional point cloud of an object according to certain aspects of the present disclosure;

[0048] Figure 13 is a flow chart illustrating a method for calibrating a neural network that outputs expected material information of an object based on input optical characteristics generated by a LIDAR system, according to certain aspects of the present disclosure;

[0049] Figure 14 is a diagram illustrating an embodiment recycling system according to certain aspects of the present disclosure;

[0050] Figure 15 illustrates a multi-dimensional point cloud of an object according to certain aspects of the present disclosure;

[0051] Figure 16 illustrates a multi-dimensional point cloud of a patient's skin according to certain aspects of the present disclosure;

[0052] 17A to 17C is the image from the polarization camera;

[0053] Figure 18 illustrates images of a set of objects using a non-polarized camera and a polarized camera; and

[0054] Figure 19 Illustrated is a confusion matrix for MST using an artificial intelligence algorithm and polarization information, in accordance with certain aspects of the present disclosure.

[0055] Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings. DETAILED DESCRIPTION

[0056] Example embodiments are provided so that this disclosure will be thorough and will fully convey the scope to those skilled in the art. Many specific details, such as specific components, parts, equipment and methods, are set forth to provide a thorough understanding of the embodiments of the present disclosure. It will be apparent to those skilled in the art that specific details need not be adopted and that example embodiments can be embodied in many different forms and should not be construed as limiting the scope of the present disclosure. In some example embodiments, well-known processes, well-known device structures and well-known technologies are not described in detail.

[0057] The technical terms used herein are only used for the purpose of describing specific example embodiments and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" may be intended to also include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of the features, elements, compositions, steps, integers, operations, and / or components, but do not exclude the presence or increase of one or more other features, integers, steps, operations, elements, components and / or their groups. Although the open term "comprising" should be understood as a non-limiting term for describing and claiming the various embodiments set forth herein, in some respects, the term may alternatively be understood as a more restrictive and constrained term, such as "consisting of" or "consisting essentially of". Thus, for any given embodiment that recites compositions, materials, components, elements, features, integers, operations, and / or process steps, the present disclosure also specifically includes any given embodiment that consists of, or consists essentially of, such recited compositions, materials, components, elements, features, integers, operations, and / or process steps. In the case of "consisting of," alternative embodiments exclude any additional compositions, materials, components, elements, features, integers, operations, and / or process steps, while in the case of "consisting essentially of," any additional compositions, materials, components, elements, features, integers, operations, and / or process steps that materially affect the basic and novel characteristics are excluded from such embodiments, but any compositions, materials, components, elements, features, integers, operations, and / or process steps that do not materially affect the basic and novel characteristics may be included in the embodiments.

[0058] Any method steps, processes, and operations described herein should not be construed as necessarily being performed in the particular order discussed or illustrated, unless expressly specified as an order of performance. It should also be understood that additional or alternative steps may be employed unless otherwise indicated.

[0059] When a component, element or layer is referred to as being "on another element or layer," "engaged to," "connected to," or "coupled to" another element or layer, the component, element or layer may be directly on, directly engaged with, connected to, or coupled to another component, element or layer, or there may be intermediate elements or layers. In contrast, when an element is referred to as being "directly on," "directly engaged to," "directly connected to," or "directly coupled to" another element or layer, there may be no intermediate elements or layers. Other words used to describe the relationship between elements (e.g., "between" versus "directly between," "adjacent" versus "directly adjacent," etc.) should be interpreted in a similar manner. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0060] Although the term first, second, third etc. can be used herein to describe various steps, elements, components, regions, layers and / or parts, unless otherwise indicated, these steps, elements, components, regions, layers and / or parts should not be limited by these terms.These terms can only be used to distinguish a step, element, component, region, layer and / or part from another step, element, component, region, layer and / or part.Such as " first ", " second " and other numerical terms when used in this article, unless context clearly indicates, otherwise do not imply sequence or order.Therefore, without departing from the teaching of example embodiments, the first step, the first element, the first assembly, the first area, the first layer or the first component discussed below can be referred to as the second step, the second element, the second assembly, the second area, the second layer or the second component.

[0061] Spatial or temporal relational terms (e.g., "before," "after," "inside," "outside," "under," "beneath," "beneath," "above," "over," etc.) may be used herein to simplify the description of one element or feature's relationship to another element or feature as illustrated in the drawings. Spatial or temporal relational terms may also be intended to encompass different orientations of the device or system in use or operation in addition to the orientation depicted in the drawings.

[0062] Throughout this disclosure, numerical values ​​represent approximate measurements or limits of ranges to encompass minor deviations from a given value and embodiments having approximately the stated value and embodiments having exactly the stated value. Except in the working examples provided at the end of the specific embodiments, all numerical values ​​for parameters (e.g., quantities or conditions) in this specification, including the appended claims, should be understood as being modified in all instances by the term "about," regardless of whether "about" actually appears before the numerical value. "About" indicates that the numerical value allows for some slight imprecision (using some method for the accuracy of the value; approximately or reasonably close to the value; close). If the imprecision provided by "about" is not understood in this ordinary sense in the art, "about" as used herein at least represents the variation that can be caused by ordinary methods of measuring and using these parameters. For example, "about" can include variations of less than or equal to 5%, optionally less than or equal to 4%, optionally less than or equal to 3%, optionally less than or equal to 2%, optionally less than or equal to 1%, optionally less than or equal to 0.5%, and in some aspects optionally less than or equal to 0.1%.

[0063] Additionally, disclosure of ranges includes disclosure of all values ​​within the entire range and further divided ranges, including endpoints and sub-ranges stated for the ranges.

[0064] Example embodiments will now be described more fully with reference to the accompanying drawings.

[0065] In certain aspects, the present disclosure provides LIDAR systems and methods configured to detect not only the distance of an object but also the material composition of the object. According to some embodiments, material composition classification can be achieved through polarization analysis processed using machine learning algorithms.

[0066] The optical elements of the systems described herein can be configured to cause all light emitted from a light source (e.g., a laser) to be in a known polarization state so that the change in polarization can later be accurately measured. The light then travels until it reaches an interface with an object (the object being composed of one or more materials), at which point a portion of the light will be diffusely reflected back.

[0067] This disclosure describes, among other things, a new method for perceiving the surrounding environment by creating a semantic map of 3D space by adding material and surface texture (MST) classification at each point in the LIDAR cloud. The MST classification inferred from the polarization characteristics of the returning photons can reduce the ambiguity of the 3D point cloud and help identify various objects (metal points, glass points, rough dielectric points, etc.). Polarization classification can precede surface tangent plane estimation, and thus objects can be pre-identified by grouping points with similar polarization characteristics. A LIDAR equipped with MST classification will be referred to herein as M-LIDAR.

[0068] According to one embodiment of the present disclosure, M-LIDAR technology can be configured to be lightweight and adaptable by using kirigami optics instead of conventional bulky optics, such as near-infrared optics, etc. For example, a LIDAR system can employ kirigami composite materials as modulators, such as those described in U.S. application Ser. No. 16 / 518,230, filed July 22, 2019, by Kotov et al., entitled “Kirigami Chiroptical Modulators for Circular Dichroism Measurements in Terahertz and Other Parts of Electromagnetic Spectrum.” Such a system may include utilizing periodically modified sheets of ultrastrong material capable of diffracting light, such as described in Xu et al., "Kirigami Nanocomposites as Wide-Angle Diffraction Gratings," ACS Nano, 10(6), 6156-6162 (2016); Lizhi Xu et al., "Origami and Kirigami Nanocomposites," ACS Nano, 11(8) 7587-7599 (2017). All patents, patent applications, articles, and references cited or referenced in this disclosure are hereby incorporated by reference in their relevant portions. Varying the strain applied to the kirigami sheet assembly varies the diffraction angle of the laser beam. Modulation can also be achieved through piezoelectric devices or by applying an electric field to a conductive kirigami composite (Jing Lyu et al., "Stretchable conductors by kirigami patterning of aramid-silver nanocomposites with zero conductivity gradient," Appl. Phys. Lett., III, 161901 (2017)). Incorporating such a kirigami composite modulator could provide a wide-angle, lightweight, and low-cost LIDAR.In addition, kirigami modulators can be applied to modulate a wide range of laser frequencies from optical to terahertz and higher (WJ Choi et al., "Chiroptical Kirigami Modulators for Terahertz Circular Dichroism Spectroscopy Biomaterials", Nature Materials, 18, 820-826 (2019)).

[0069] According to one embodiment, the M-LIDAR system and method described herein can be used for black ice detection for vehicles with different levels of automation.

[0070] Now refer to Figure 1 , a representative simplified M-LIDAR system is provided. The M-LIDAR system 100 may include a laser 102, a beam steering device 106, a first polarizer 114, a second polarizer 116, a first polarization detector 122, a second polarization detector 124, and a processor 126. Although Figure 1 A first polarizer 114 and a second polarizer 116 are illustrated, as well as a first polarization detector 122 and a second polarization detector 124, but according to some embodiments, only a single polarizer (e.g., first polarizer 114) and a single polarization detector (e.g., first polarization detector 122) may be included as part of system 100 without departing from the teachings of the present disclosure. Furthermore, according to certain embodiments, more than two polarizers and / or more than two polarization detectors may be included as part of system 100 without departing from the teachings herein.

[0071] For the purposes of simplicity and illustration, throughout the remainder of this disclosure, the first polarizer 114 will be referred to as an s-polarization linear polarizer 114. Similarly, for the purposes of simplicity and illustration, the second polarizer 116 will be referred to as a p-polarization linear polarizer 116. Furthermore, the first polarization detector 122 will be referred to as a p-polarization detector 122, and the second polarization detector will be referred to as an s-polarization detector 124.

[0072] However, as will be appreciated by one of ordinary skill in the art, the polarizers 114, 116 can be configured for a variety of different types of polarization without departing from the teachings herein. For example, a given polarizer can be configured to perform linear polarization (e.g., s-type or p-type linear polarization), right-handed circular polarization, left-handed circular polarization, elliptical polarization, or any other suitable type of polarization known in the art. Similarly, a given detector can be configured to detect linearly polarized light (e.g., s-type or p-type linear polarization), right-handed circular polarization, left-handed circular polarization, elliptical polarization, or any other type of polarized light known in the art. According to some embodiments, the polarization of a light beam (i.e., a combination of two or more light pulses) can be modulated on a pulse-by-pulse basis to obtain additional information about one or more objects under consideration.

[0073] As discussed in further detail below, system 100 can be configured to detect one or more materials comprising object 110 and classify object 110 based at least in part on the detected materials. According to some embodiments, object classification can be performed using one or more artificial intelligence algorithms, including but not limited to neural network-based artificial intelligence.

[0074] In operation, the system 100 may function as follows. The laser 102 may be configured to generate (i.e., emit) one or more polarized or unpolarized light pulses that collectively form a polarized / unpolarized light beam 104. Figure 1 In the embodiment shown in FIG, each pulse includes an s-polarization component (in Figure 1 ) and the transverse p-polarization component (represented by points along the beam 104 in Figure 1 104 in a vertical direction). Alternatively (and in conjunction with the previous discussion regarding different types of polarized light), the pulses may include, for example, a sequence of left-handed circularly polarized light and a sequence of right-handed circularly polarized light, a sequence of elliptically polarized light, any combination of the foregoing, or any other suitable sequence of polarized light.

[0075] According to some embodiments, the laser 102 can be configured to generate any pulse from one pulse to more than one million pulses per second. In addition, according to some embodiments, the laser 102 can generate light with a wavelength in the range of visible light, ultraviolet light, infrared light, etc. Visible light has a wavelength in the range of about 390nm to about 750nm and infrared radiation (IR) (including the near infrared (NIR) of about 0.75μm to about 1.4μm and the far infrared (FIR) of about 15μm to 1mm). The far infrared (FIR) part of the electromagnetic spectrum (electromagnetic spectrum) also known as the terahertz (THz) region has a photon wavelength of about 100μm to about 1mm and an energy of about 0.001eV to about 0.01eV. In some variations, without departing from the teachings of the present disclosure, the laser 102 can generate a 550 nanometer (nm), 808nm, 905nm or 1550nm pulsed laser, or a laser of any other suitable wavelength. For example, implementations of home robots, autonomous vehicles, and machine vision may use lasers with eye-safe frequencies above 800 nm. For outdoor applications, a light beam in the water transparency window, such as approximately 900 nm to 1550 nm, may be appropriately employed. According to some implementations, after a given pulse is generated by the laser 102, the processor 126, which can execute instructions, may record the initial time at which the pulse was generated. This "time of flight" information may then be used to calculate the distance to the object 110 using the speed of light.

[0076] The light beam 104 can be directed by the laser 102 to pass through the beam redirector 106. The beam redirector 106 can be configured to generate polarization-adjusted light pulses. In some aspects, the polarization of each polarized / unpolarized pulse of the polarized / unpolarized light beam 104 is adjusted by the beam redirector 106. As used herein, adjusting polarization can include imparting polarization or changing polarization. Thus, the beam redirector 106 can adjust the polarization of each polarized / unpolarized pulse of the polarized / unpolarized light beam 104 to produce one or more linearly polarized light pulses (the linearly polarized light pulses together form a linearly polarized light beam a). Although the aforementioned embodiments contemplate linear polarization, according to some embodiments, the beam redirector 106 can circularly polarize (e.g., left-handed or right-handed) or elliptically polarize the light beam 104. According to another embodiment, the beam redirector 106 may not impose any polarization on the light beam. For example, if the light beam 104 is already polarized when entering the beam redirector 106, the beam redirector 106 can further modify the characteristics of the generated polarization-adjusted light pulses (e.g., separate or modulate the pulses), but may not need to adjust the polarization of the previously polarized light pulses. Furthermore, according to some embodiments, the beam redirector 106 can polarize a first pulse of a beam according to a first type of polarization and polarize a second pulse of the same beam according to a second, different type of polarization. In addition to or in lieu of performing polarization of the beam 104, the beam redirector 106 can also control the direction of any beam emitted therefrom (e.g., beam 108). Still further, the beam redirector 106 can separate a beam (e.g., beam 104) into several different beams, thereby emitting one or more beams at defined angles to steer multiple beams at once. This concept is particularly useful in Figure 1 FIG. 1 shows a plurality of diverging arrows emanating from the beam redirector 106 .

[0077] Additionally or alternatively, in some embodiments, the beam redirector 106 can be configured to modulate the linearly polarized light beam 108. In one embodiment, the beam redirector 106 can include a kirigami nanocomposite beam redirector or the like. In accordance with this embodiment, and as discussed in additional detail below, the beam redirector 106 can be configured to polarize and / or modulate the linearly polarized light beam 108 by increasing or decreasing the amount of strain applied to the kirigami nanocomposite beam redirector.

[0078] Furthermore, according to one embodiment, the beam steering device 106 may be configured to linearly polarize each unpolarized pulse of the unpolarized light beam 104 into p-polarization. Figure 1 As shown in Figure 1As can be seen in FIG, after passing through the beam redirector 106, the light beam 104 no longer includes any s-polarization component (i.e., the "dot" component shown in the light beam 104 is absent from the linearly polarized light beam 108). In alternative aspects, the linearly polarized light beam 108 can instead be p-polarized. Furthermore, in certain aspects, the beam redirector 106 can modify, control, and steer the linearly polarized light beam 108 toward the object 110, as will be discussed further herein. The beam redirector 106 can implement dynamic, wavelength-dependent beam steering and amplitude modulation of electromagnetic waves.

[0079] Continue to refer to Figure 1 , linearly polarized light beam 108 can be diffusely reflected from object 110. One or more light pulses collectively form light beam 112, which constitutes a reflected version of linearly polarized light beam 108. According to some embodiments, reflected linearly polarized light beam 112 can have a different polarization than linearly polarized light beam 108 (i.e., the light beam before reflection from object 110). This difference in state is illustrated by light beam 112 including both p-polarized and s-polarized components (reflected by the dot and double-headed arrow along the path of light beam 112, respectively), while light beam 108 is shown as including only a p-polarized component. Furthermore, object 110 can include any suitable object (or target) composed of one or more different materials that it is desired to detect. Although discussed above and in subsequent sections as reflecting a "linearly" polarized light beam 112, according to certain embodiments, reflected light beam 112 can be polarized in a variety of different ways, including circularly polarized or elliptically polarized, without departing from the teachings herein.

[0080] The reflected linearly polarized light beam 112 diffusely reflected, scattered, or otherwise emitted by the object 112 can pass through the s-polarization linear polarizer 114 and / or the p-polarization linear polarizer 116 of the system 100. In some aspects, various portions of the reflected, scattered, or otherwise emitted linearly polarized light beam 112 pass through both the s-polarization linear polarizer 114 and / or the p-polarization linear polarizer 116 of the system 100. The s-polarization linear polarizer 114 is configured to linearly polarize one or more light pulses comprising the light beam 112 into s-polarization to produce one or more reflected s-polarized light pulses (the one or more reflected s-polarized light pulses collectively form a reflected s-polarized light beam 118). Similarly, the p-polarization linear polarizer 116 is configured to linearly polarize one or more light pulses comprising the light beam 112 into p-polarization to produce one or more reflected p-polarized light pulses (the one or more reflected p-polarized light pulses collectively form a reflected p-polarized light beam 120). According to some embodiments, the s-polarization linear polarizer 114 and / or the p-polarization linear polarizer 116 may comprise a kirigami nanocomposite material or the like, such as described above with respect to the beam redirector 106 and / or below with respect to the beam redirector 106. Figures 4a to 4d and Figures 5a to 5cHowever, one of ordinary skill in the art will recognize that, according to some embodiments, non-kirigami nanocomposites or other optical devices may be employed as part of system 100 without departing from the teachings herein.

[0081] According to some embodiments, similar arrangements of polarizers 114 , 116 may be used for polarization of left-handed circularly polarized light and right-handed circularly polarized light or elliptically polarized light reflected, scattered, or otherwise emitted from / by object 110 .

[0082] The s-polarization detector 122 can be configured to detect an intensity of each of the one or more reflected s-polarized light pulses that form the reflected s-polarized light beam 118. Furthermore, according to some implementations, the s-polarization detector 122 can be configured to detect an angle of incidence associated with the reflected s-polarized light beam 118. The detected intensities of the one or more reflected s-polarized light pulses that form the reflected s-polarized light beam 118 and / or the detected angle of incidence associated with the reflected s-polarized light beam 118 can be utilized by the processor 126 to perform material type detection (using, for example, MST classification), as discussed in additional detail below.

[0083] Similarly, the p-polarization detector 124 can be configured to detect an intensity of each of the one or more reflected p-polarized light pulses that form the reflected p-polarized light beam 120. Furthermore, according to some implementations, the p-polarization detector 124 can be configured to detect an angle of incidence associated with the reflected p-polarized light beam 120. The detected intensities of the one or more reflected p-polarized light pulses that form the reflected p-polarized light beam 120 and / or the detected angle of incidence associated with the reflected p-polarized light beam 120 can also be utilized by the processor 126 to perform material type detection, as discussed in additional detail below.

[0084] Processor 126 is configured to detect at least one material of object 110 based on: (i) the detected intensity of one or more light pulses forming light beams 118 and / or 120; and / or (ii) the detected angle of incidence associated with reflected s-polarized light beam 118 and / or reflected p-polarized light beam 120. More specifically, according to some embodiments, processor 126 is configured to apply a machine learning algorithm to detect one or more materials comprising object 110. As used herein, "applying a machine learning algorithm" may include, but is not limited to, executing executable instructions stored in a memory and accessible to the processor. Furthermore, according to one embodiment, the specific machine learning algorithm used for material detection may include an artificial neural network. However, other machine learning algorithms known in the art may be appropriately employed without departing from the teachings of the present disclosure.

[0085] Furthermore, according to some embodiments, processor 126 may be configured to classify object 110 based on the material detected by object 110 by applying a machine learning algorithm. Similarly, the machine learning algorithm used for object classification may include an artificial neural network. However, other machine learning algorithms known in the art may be appropriately employed without departing from the teachings of the present disclosure.

[0086] Before turning to FIG. 2 , the following reflects the use of an M-LIDAR system (e.g. Figure 1 An overview of a process for detecting material(s) of an object using the system 100 shown.

[0087] As mentioned above, one objective of the present disclosure is to enable the detection of object materials and reduce the data processing required by modern LIDAR devices by obtaining more data at each point in the point cloud. This additional polarization data, when combined with machine learning algorithms, enables material detection, thereby simplifying object identification for a variety of applications, including but not limited to autonomous vehicles, machine vision, medical applications (e.g., devices to assist the blind), and advanced robotics.

[0088] An M-LIDAR system according to an example implementation of the present disclosure may operate as follows. Return light (e.g., one or more light pulses constituting a reflected pattern of linearly polarized light beam 112) may be measured using a pair of detectors (e.g., detectors 122, 124) with vertically oriented linear polarizers (e.g., polarizers 114, 116). Some of the backscattered light (e.g., reflected light 112) may be directed at the detectors (e.g., detectors 122, 124) and pass through a narrowband interference filter (e.g., linear polarizer pair 114, 116) placed in front of each detector pair (e.g., detector pair 122 / 124). The narrowband interference filter may allow only a small range of wavelengths to pass (e.g., 1 nm to 2 nm), which may reduce unwanted noise from ambient lighting or external sources.

[0089] According to other embodiments of the foregoing system, the system may be configured to detect circularly polarized light and / or elliptically polarized light reflected, scattered, or otherwise emitted from an object and perform machine learning processing.

[0090] Due to this selectivity, an M-LIDAR system according to the present disclosure (e.g., system 100) can be configured to measure multiple wavelengths simultaneously, completely independently. The coherent light can then be polarized using, for example, co-polarizers and / or crossed polarizers. As the light travels through the polarizers, the intensity of the light may decrease by an amount depending on the change in polarization when reflected from an object (e.g., object 110). Beam focusing optics (e.g., polarizers 114, 116) can direct the coherent polarized light (e.g., beams 118, 120) toward a detection surface (e.g., the surface of detectors 122, 124), and the angle at which the return light travels (i.e., the angle of incidence) can be detected based on the location at which the light impinges on the detection screen.

[0091] Once the detector identifies the light, a time-of-flight sensor (e.g., one implemented in processor 126) can record the travel time of the light pulse. Each light pulse can have a light pulse intensity measured for both the co-polarization detector and the cross-polarization detector, and the combination of these two values ​​enables quantification of the polarization effects induced during reflection.

[0092] After this process, the following parameters can be measured: (i) the initial angle at which the beam is diverted; (ii) the angle at which the backscattered light returns; (iii) the time of flight from emission to detection; and (iv) the intensity at each detector.

[0093] Note that because the detectors are located at different locations, the amount of time it takes for a single light pulse to reach each detector may be slightly different. By understanding the geometry of the system and the relationship between intensity and distance, this difference can be compensated for and the intensity at one detector can be precisely adjusted. The time-of-flight data can be used to determine the distance between the source (e.g., laser 102) and the object (e.g., object 110), and the initial and return angles can be combined to determine the specific location of that point in space relative to the M-LIDAR system. These compensated intensity values ​​can contain information indicating the material that reflected the light pulse. Using these values, a machine learning algorithm can provide robust and comprehensive material recognition capabilities.

[0094] The process described above of emitting a light pulse, diffusely reflecting the light from an object, measuring the reflected light with a detector, and determining the object's position relative to the source can be repeated approximately one to millions of times per second. Each time, points are generated and mapped onto the same coordinate system to create a point cloud.

[0095] Once the point cloud is generated, one or more machine learning algorithms can be used to cluster the points into objects and ultimately characterize the corresponding material(s) of each object (e.g., where multiple objects are detected). The points can be clustered based on, for example, one or more values ​​of measured intensity and, in some embodiments, proximity of similar points.

[0096] Once a cluster is determined using intensity values, a machine learning or artificial intelligence algorithm can be used to correlate the measurements with a database of known materials to classify the material of that cluster point. This process can be repeated for all clusters in the system. Knowledge of the material of the surrounding environment enables the system (e.g., a system implemented in a car, robot, drone, etc.) to make faster and more informed decisions about what the object itself might be. From this, factors such as the risk involved can be assessed, and a decision can subsequently be made (e.g., if the system detects black ice in front of a vehicle). As this process continues over time, more information can be extracted from the perceived changes, and an even better understanding of the surrounding environment can be achieved.

[0097] The MST classification technique is also applicable to detecting objects whose surfaces have been modified to enhance detection, for example, where the surface of the object is coated or textured with a macroscale, microscale, nanoscale or molecular pattern to produce a reflected light beam with a specific optical response suitable for rapid MST classification by LIDAR. Embodiments of such surface treatments include coatings containing additives that produce reflected, scattered or otherwise emitted light with a specific linear polarization, circular polarization or elliptical polarization. In one case, metal nanowires / metal microwires / metal macrowires or axial carbon nanomaterials are added to the base coating. The alignment pattern can be random, linear, spiral, herringbone or any other pattern that produces a specific polarization feature, thereby enabling rapid identification of specific objects. By way of non-limiting example, this can be used to create markings on roads, road signs, obstacles, pylons, guardrails, vehicles, bicycles, clothing and other objects.

[0098] Another implementation of a surface treatment that facilitates MST classification can include adding chiral inorganic nanoparticles to the base paint used to coat such objects as road markings, vehicles, bicycles, clothing, etc. The chiral nanoparticles can exhibit a specific and very strong circular polarization response to the light beam used by LIDAR. The chiral nanoparticles can be mixed into the paint in specific proportions to create a polarization signature (e.g., a "barcode") for a specific object.

[0099] Another embodiment of polarization tagging of an object may include the use of surface textures that create a specific polarization response. One embodiment of such a texture may include creating a nanoscale pattern of metallic, semiconductor, insulator, or ceramic nanoparticles with specific geometric properties that results in a defined polarization response to laser light in LIDAR. Two examples of such patterns include (a) linear nanoscale or microscale surface features that result in a linear polarization of light reflected, scattered, or emitted from the object, and (b) an out-of-plane raised chiral pattern on a metal surface that results in a specific chirality and corresponding circular polarization of light reflected, scattered, or emitted from the object.

[0100] According to some embodiments, the aforementioned systems and methods may be employed to accurately identify materials for use in autonomous vehicles, machine learning, medical applications, and advanced robotics.

[0101] Existing conventional LIDAR systems primarily work by measuring the distance between an object and a laser source, typically using time-of-flight data or phase shift. In this case, objects are classified based on the geometry and pattern of the arrangement of points in the cloud. Some more advanced LIDAR point cloud classification methods use an additional parameter: overall intensity.

[0102] Based on how strong the signal intensity of the returning light pulse is, the system can effectively detect differences in color. This additional data segment makes it easier to identify object boundaries within the point cloud, reducing the amount of processing required to classify all points. However, applications like autonomous vehicles may require a higher degree of certainty that cannot be achieved with overall intensity. Furthermore, detection of distant objects can be achieved through single-point detection using MST classification, rather than the type of multi-point detection and processing employed in conventional LIDAR systems.

[0103] The method described in this paper therefore changes the current approach taken by machine vision to object recognition. Instead of relying solely on geometry, motion, and color to determine the nature of an object, the system described in the text takes into account another parameter: polarization. After reflecting off a material interface, light undergoes some polarization change. This polarization change is quantified by measuring the intensity of the light after it passes through both a co-polarizing filter and a cross-polarizing filter. This additional data can be paired with machine learning methods to significantly improve clustering and, in turn, object recognition capabilities. Traditional object recognition methods are very computationally expensive. The method described in this paper can significantly reduce the processing power required by LIDAR by using a material-based approach rather than the current geometry-based methods.

[0104] In addition to traditional distance measurements, the polarization data collected by the real-time system allows machine learning algorithms to determine the material(s) that make up an object. Current LIDAR systems lack knowledge or understanding of the materials in the surrounding environment. Such information, when implemented, provides context for better situational awareness and more informed decisions. In the context of autonomous vehicles, this enhanced understanding of the environment could lead to improved passenger safety, as accurate and timely detection of potential hazards leads to improved decision-making capabilities.

[0105] Now turn Figure 2a to Figure 2b , illustrates a scanning electron microscope (SEM) image of a nano-kirigami sheet that can be used to form a nano-kirigami nanocomposite optical component incorporated into an M-LIDAR system. According to some embodiments of the present disclosure, an optically active kirigami sheet (such as Figure 2a to those shown in FIG2 ) can be made of ultra-strong nanoscale composites having cut patterns with lengths of 0.5 μm to 5 μm. In some aspects, composites (including highly conductive composites) can be modified by using an idea from the ancient Japanese art of paper cutting called “kirigami”. See Lizhi Xu et al., “Kirigami Nanocomposites as Wide-Angle Diffraction Gratings”, ACS Nano, 10(6), 6156-6162 (2016). Lizhi Xu et al., “Origami and Kirigami Nanocomposites”, ACS Nano, 11(8), 7587-7599 (2017). Kirigami methods can be used to design elasticity by using multiple cuts or notches that create a network on a planar polymeric material (e.g., a composite or nanocomposite). Such cuts (e.g., where material in a polymer or composite material extends from one side to the other) can be made using top-down patterning techniques (e.g., photolithography) to uniformly distribute stress within the polymer or nanocomposite material and suppress uncontrolled high-stress singularities within the material. By way of non-limiting example, this approach can prevent unpredictable localized failures and increase the ultimate strain of a rigid sheet from 4% to 370%.

[0106] By using micro-scale paper-cut patterning, rigid nanocomposite sheets can be made to obtain high ductility. In addition, significantly relative to most stretchable conductive materials, the composite sheet of paper-cut cutting patterning maintains its conductivity within the entire strain state. Paper-cut structures can include composite materials, such as nanocomposites. In some aspects, paper-cut structures can be multilayer structures with at least two layers, wherein at least one layer is a polymer material. The polymer material can be a composite material or a nanocomposite material. The composite material includes a matrix material, such as a polymer, a polymer electrolyte or other matrix (such as cellulose paper), and at least one reinforcing material distributed therein. In some aspects, nanocomposites are particularly suitable for paper-cut structures, which are composite materials comprising enhanced nanomaterials (such as nanoparticles). In some variations, the composite material can be in the form of a sheet or film.

[0107] "Nanoparticles" are solid or semisolid materials that can have a variety of shapes or morphologies, however, they are generally understood by those skilled in the art to mean particles having at least one spatial dimension of less than or equal to about 10 μm (10,000 nm). In certain aspects, the nanoparticles have a relatively low aspect ratio (AR) (defined as the length of the longest axis divided by the diameter of that portion): less than or equal to about 100, optionally less than or equal to about 50, optionally less than or equal to about 25, optionally less than or equal to about 20, optionally less than or equal to about 15, optionally less than or equal to about 10, optionally less than or equal to about 5, and in certain variations, equal to about 1. In other aspects, nanoparticles having a tubular or fibrous shape have a relatively high aspect ratio (AR): greater than or equal to about 100, optionally greater than or equal to about 1,000, and in certain variations, optionally greater than or equal to about 10,000.

[0108] In some embodiments, the nanoparticle is selected to be included in the nanocomposite material and is the conductive nanoparticle that produces conductive nanocomposite material.Nanoparticle can be substantially circular nanoparticle, and it has low aspect ratio as defined above, and has the form or the shape that comprises sphere, spherical class, hemispherical, disc-like, spherical, annular, ring-shaped, cylindrical, flat circular, dome-shaped, egg-shaped, elliptical, circular (orbed), ovate etc.In some preferred variant, the form of nanoparticle has spherical shape.Alternatively, nanoparticle can have alternative shape, for example filament, fiber, rod, nanotube, nano star or nano shell.Nanocomposite material can also comprise the combination of any this type of nanoparticle.

[0109] In addition, in some aspects, particularly suitable nanoparticles for use in accordance with the present teachings have a particle size greater than or equal to about 10 nm to less than or equal to about 100 nm (for the average diameter of the multiple nanoparticles present). Conductive nanoparticles can be formed from various conductive materials including metal, semiconductor, ceramic and / or polymer nanoparticles having a variety of shapes. Nanoparticles can have magnetic or paramagnetic properties. Nanoparticles can include conductive materials such as carbon, graphene / graphite, graphene oxide, gold, silver, copper, aluminum, nickel, iron, platinum, silicon, cadmium, mercury, lead, molybdenum, iron, and alloys or compounds thereof. Thus, suitable nanoparticles can be, for example, but not limited to, graphene oxide, graphene, gold, silver, copper, nickel, iron, carbon, platinum, silicon, seed metals, CdTe, CdSe, CdS, HgTe, HgSe, HgS, PbTe, PbSe, PbS, MoS2, FeS2, FeS, FeSe, WO 3-X , and nanoparticles of other similar materials known to those skilled in the art. Graphene oxide is a conductive material that is particularly suitable for use as a reinforcing material in a composite material. For example, in some variations, the nanoparticles may include carbon nanotubes, such as single-walled nanotubes (SWNTs) or multi-walled nanotubes (MWNTs). SWNTs are formed from a single sheet of graphite or graphene, while MWNTs include a plurality of cylindrical shapes arranged concentrically. The typical diameter of SWNTs can be in the range of about 0.8 nm to about 2 nm, while MWNTs can have a diameter exceeding 100 nm.

[0110] In some variations, the nanocomposite material can include a total amount of a plurality of nanoparticles, the total amount of these plurality of nanoparticles being greater than or equal to about 1 % by weight to less than or equal to about 97 % by weight of the total amount of nanoparticles in the nanocomposite material, optionally greater than or equal to about 3 % by weight to less than or equal to about 95 % by weight, optionally greater than or equal to about 5 % by weight to less than or equal to about 75 % by weight, optionally greater than or equal to about 7 % by weight to less than or equal to about 60 % by weight, optionally greater than or equal to about 10 % by weight to less than or equal to about 50 % by weight. Certainly, the appropriate amount of nanoparticles in the composite material depends on the material properties, percolation threshold and other parameters of the nanoparticles of the material type in the specific matrix material.

[0111] In certain variations, the nanocomposite may comprise a total amount of polymeric matrix material that is from greater than or equal to about 1 wt % to less than or equal to about 97 wt %, optionally from greater than or equal to about 10 wt % to less than or equal to about 95 wt %, optionally from greater than or equal to about 15 wt % to less than or equal to about 90 wt %, optionally from greater than or equal to about 25 wt % to less than or equal to about 85 wt %, optionally from greater than or equal to about 35 wt % to less than or equal to about 75 wt %, optionally from greater than or equal to about 40 wt % to less than or equal to about 70 wt %, of the total amount of polymeric matrix material in the nanocomposite.

[0112] In certain variations, the nanocomposite material comprises a plurality of conductive nanoparticles and has a conductivity greater than or equal to about 1.5 x 10 3 In certain other aspects, the nanocomposite material may include a plurality of conductive nanoparticles as reinforcing nanomaterials and thus may have a conductivity of less than or equal to about 1x10 4 In certain other variations, the impedance (Z) of the conductive nanocomposite material comprising the plurality of nanoparticles may be less than or equal to about 1×10 4 Ohm (for example, measured using an AC sinusoidal signal with an amplitude of 25 mV, where the impedance value is measured at a frequency of 1 kHz).

[0113] The polymer or nanocomposite material may be in a planar form (e.g. a sheet) in the initial state (before cutting), but may be folded or formed into a three-dimensional structure after the cutting process and thus be used as a structural component. By way of example, Figure 10 , a structure 220 is illustrated that includes a portion of an exemplary nanocomposite sheet 230 having a surface with a checkerboard pattern of cuts. The sheet 230 includes a first row 232 of first discontinuous cuts 242 (extending through the sheet 230 to create openings), the pattern of the first discontinuous cuts 242 defining first uncut areas 252 between the discontinuous cuts 242. Discontinuous cuts are localized or discrete cuts formed in the sheet so that the entire sheet retains its original dimensions, rather than being divided into separate smaller pieces or portions. If there are multiple discontinuous cuts 242, at least some of the cuts are non-adjacent and unconnected to each other, so that at least one uncut area remains on the sheet as a bridge between discontinuous pieces. Although many cutting patterns are possible, they will be described herein as follows. Figure 10 A simple paper cut pattern of straight lines arranged in a centered rectangle is shown as an exemplary pattern. The first uncut area 252 has a length "x." Each discontinuous cut 242 has a length "L."

[0114] In certain aspects, the length of each discontinuous cut (e.g., discontinuous cut 242) can be on a microscale, a mesoscale, a nanoscale, and / or a macroscale. Macroscale is generally considered to have a size greater than or equal to about 500 μm (0.5 mm), while mesoscale is greater than or equal to about 1 μm (1,000 nm) to less than or equal to about 500 μm (0.5 mm). Microscale is generally considered to be less than or equal to about 100 μm (0.5 mm), while nanoscale is generally less than or equal to about 1 μm (1,000 nm). Therefore, conventional mesoscale, microscale, and nanoscale dimensions can be considered to overlap. In some aspects, the length of each discontinuous cut can be on a microscopic scale, for example, less than about 100 μm (i.e., 100,000 nm), optionally less than about 50 μm (i.e., 50,000 nm), optionally less than about 10 μm (i.e., 10,000 nm), optionally less than or equal to about 5 μm (i.e., 5,000 nm), and in some aspects less than or equal to about 1 μm (i.e., 1,000 nm). In some aspects, discontinuous cut 42 can have a length of less than about 50 μm (i.e., 50,000 nm), optionally less than about 10 μm (i.e., 10,000 nm), and optionally less than about 1 μm (i.e., less than about 1,000 nm).

[0115] In certain other variations, these dimensions can be reduced at least 100-fold to the nanoscale, for example, the cut has a length less than or equal to about 1 μm (1,000 nm), optionally less than or equal to about 500 nm, and in certain variations, optionally less than or equal to about 100 nm.

[0116] It should be noted that "x" and "L" may vary within a row depending on the pattern being formed, although in a preferred aspect these dimensions remain constant.

[0117] A second row 234 of second discontinuous cuts 244 is also patterned on sheet 230. The second discontinuous cuts 244 define a second uncut region 254 between the second discontinuous cuts. A third row 236 of third discontinuous cuts 246 is also patterned on sheet 230. The third discontinuous cuts 246 define a third uncut region 256 between the third discontinuous cuts. It should be noted that the first row 232, the second row 234, and the third row 236 are used for illustrative and nomenclature purposes, but it can be seen that the checkerboard pattern on the surface of sheet 230 has more than three distinct rows. The first row 232 is spaced apart from the second row 234, as indicated by the label "y." The second row 234 is similarly spaced apart from the third row 236. It should be noted that "y" can vary between rows, though in some aspects, "y" remains constant between rows. As described above, this spacing between rows can also be at a microscale, a mesoscale, a nanoscale, and / or a macroscale.

[0118] Notably, the first discontinuous cuts 242 in the first row 232 are offset in the transverse direction (along the dimension / axis shown as "x") from the second discontinuous cuts 244 in the second row 234, thereby forming a checkerboard pattern. Similarly, the second discontinuous cuts 244 in the second row 234 are offset in the transverse direction from the third discontinuous cuts 246 in the third row 236. Thus, the first uncut areas 252, the second uncut areas 254, and the third uncut areas 256 in each respective row cooperate to form a structural bridge 260 that extends from the first row 232, through the second row 234, and to the third row 236.

[0119] In this regard, sheet 230 having a patterned checkerboard surface with a plurality of discontinuous cuts (e.g., 242, 244, and 246) can be stretched in at least one direction (e.g., along the dimension / axis shown as "y" or "x"). As a result, sheet 230 formed from the nanocomposite material exhibits certain advantageous properties including enhanced strain.

[0120] In various aspects, an optical device is contemplated that includes a stretchable multilayer polymeric material or composite material formed by a kirigami process. "Stretchable" refers to the ability of materials, structures, components, and devices to withstand strain without cracking or other mechanical failure. Stretchable materials are extensible and, therefore, capable of being stretched and / or compressed, at least to some extent, without damage, mechanical failure, or significant degradation of performance.

[0121] Young's modulus is a mechanical property that refers to the ratio of stress to strain for a given material. Young's modulus is given by the following expression:

[0122]

[0123] where engineering stress is σ, tensile strain is ε, E is Young's modulus, L0 is the equilibrium length, ΔL is the change in length under applied stress, F is the applied force, and A is the area over which the force is applied.

[0124] In certain aspects, stretchable composites, structures, components and devices can withstand at least about 50% of a maximum tensile strain without breaking; optionally greater than or equal to about 75% without breaking, optionally greater than or equal to about 100% without breaking, optionally greater than or equal to about 150% without breaking, optionally greater than or equal to about 200% without breaking, optionally greater than or equal to about 250% without breaking, optionally greater than or equal to about 300% without breaking, optionally greater than or equal to about 350% without breaking, and in certain embodiments, greater than or equal to about 370% without breaking.

[0125] In addition to being stretchable, a stretchable material can also be flexible and thus be able to significantly elongate, flex, bend, or otherwise deform along one or more axes. The term "flexible" can refer to the ability of a material, structure, or component to deform (e.g., into a curved shape) without undergoing permanent deformation that introduces significant strain, such as strain that indicates a failure point for the material, structure, or component.

[0126] Therefore, the present disclosure provides a stretchable polymer material in certain aspects. In a further aspect, the present disclosure provides a stretchable composite material comprising a polymer and a plurality of nanoparticles or other reinforcing materials. The polymer can be an elastomer or a thermoplastic polymer. By way of non-limiting example, a suitable polymer includes polyvinyl alcohol (PVA).

[0127] For example, for certain materials, creating a surface with patterned kirigami cuts according to certain aspects of the present disclosure can increase the ultimate strain of an initially rigid sheet to greater than or equal to about 100%, optionally greater than or equal to about 500%, optionally greater than or equal to about 1,000%, and in certain variations optionally greater than or equal to about 9,000%, from the initial ultimate strain prior to any cutting.

[0128] It is worth noting that a wide range of maximum achievable strain or extension levels can be achieved based on the geometry of the cutting pattern used. Therefore, the ultimate strain is determined by the geometry. The ultimate strain (% strain) is the ratio between the final achievable length when stretched to the point before the structure breaks and the original or initial length (Li):

[0129]

[0130] Among them L cis the length of the cut, x is the spacing between the discontinuous cuts, and y is the distance between discrete rows of discontinuous cuts. Thus, in certain variations, according to certain aspects of the present disclosure, a polymeric material (e.g., a nanocomposite material) having a surface with patterned cuts can increase the ultimate strain to greater than or equal to about 100%, optionally greater than or equal to about 150%, optionally greater than or equal to about 200%, optionally greater than or equal to about 250%, optionally greater than or equal to about 300%, optionally greater than or equal to about 350%, and in certain variations, optionally greater than or equal to about 370%. Additional discussion of kirigami composites and methods for their manufacture is described in U.S. Publication No. 2016 / 0299270, filed on April 7, 2016, by Kotov et al., entitled “Kirigami Patterned Polymeric Materials and Tunable Optic Devices Made Therefrom,” filed as U.S. Application Serial No. 15 / 092,885, the relevant portions of which are incorporated herein by reference.

[0131] In certain aspects, the kirigami nanocomposites can form tunable grating structures that maintain stable periodicity on macroscopic length scales even under 100% stretching. The lateral spacing of the diffraction pattern is negatively correlated with the amount of stretching, which is consistent with the inverse relationship between the size in the diffraction pattern and the spacing of the corresponding grating. Due to the relatively small change in longitudinal periodicity with lateral stretching, the longitudinal spacing in the diffraction pattern exhibits low dependence on the amount of stretching. The diffraction pattern also shows a significant dependence on the wavelength of the incident laser. The polymer stretchable tunable grating structure exhibits elastic behavior with stretching and spontaneously recovers to the relaxed (i.e., unstretched) geometry when the stretch is removed under cyclic mechanical actuation. The diffracted beam forms a clear pattern that changes consistently with the deformation of the polymer stretchable tunable grating structure. This behavior indicates excellent capabilities for dynamic, wavelength-dependent beam steering.

[0132] Therefore, due to the out-of-plane surface features, three-dimensional (3D) kirigami nanocomposites offer a new dimension to conventional reflective and refractive optical devices, such as Figure 2a to Figure 2b For example, Figure 2a to Figure 2b The reconfigurable fins and slits formed by the cuts in the nano-kirigami sheet shown allow for efficient modulation of light through the reversible expansion (or strain level) of the kirigami sheet. Figure 2a to Figure 2bThe nano-kirigami sheets shown in the figure are incorporated into one or more optical components of the M-LIDAR system described herein. More specifically, these light, thin, and inexpensive optical components can be used, for example, in the red and infrared parts of the spectrum to achieve beam steering and / or polarization modulation. According to some implementations, Figure 2a to Figure 2b Kirigami nanocomposites of the type shown can be used to form Figure 1 The system shown includes a beam redirector 106 , an s-polarization linear polarizer 114 , and / or a p-polarization linear polarizer 116 .

[0133] In some variations, the kirigami nanocomposites can form nanocomposites assembled from ultra-strong layer-by-layer (LbL) structures (Kotov, N.A. et al., "Ultrathin graphite oxide-polyelectrolyte composites prepared by self-assembly: Transition between conductive and non-conductive states" Adv. Mater., 8, 637-641 (1996) and Kotov, N.A. et al., "Layer-by-Layer Self-assembly of Polyelectrolyte-Semiconductor Nanoparticle Composite Films" Films)," J. Phys. Chem., 99, 13065-13069 (1995). These nanocomposites have high strength, e.g., about 650 MPa, and elastic modulus (E), e.g., about 350 GPa, thereby providing excellent mechanical properties, environmental robustness, and a wide operating temperature range (e.g., from -40°C to +40°C), and proven scalability. The high elasticity of the LbL composite makes it reconfigurable, and its high-temperature resilience enables integration with different types of actuators and CMOS compatibility. In certain aspects, the nanocomposite can be coated with a plasma film (e.g., titanium nitride, gold, etc.) to enhance interaction with photons of a target wavelength (e.g., 1550 nm photons where the laser source has a wavelength of 1550 nm).

[0134] In certain other variations, the kirigami nanocomposite sheet can include a magnetic material distributed therein or coated thereon. For example, a nickel layer can be deposited on the ultra-strong composite. The nickel layer can serve as both a magnetic and a reflective layer, thereby providing a magnetically active kirigami element. Thus, the kirigami unit can be directly integrated with a LIDAR assembly and used as a beam steering device (e.g., using first- and second-order diffracted beams) or as a polarizer (e.g., using first-order diffracted beams).

[0135] Now refer to Figures 3a to 3c , depicting an image depicting a laser diffraction pattern from a nano-kirigami-based graphene composite. As a reference, Figures 3a to 3c The scale bar shown in the upper right corner represents 25 mm. Figure 3a Depicted are the laser diffraction patterns of the nanokirigami-based graphene composite for 0% strain (relaxed state). Figure 3b The laser diffraction pattern from the nanokirigami-based graphene composite material subjected to 50% strain is depicted. Figure 3c Depicted are laser diffraction patterns from the nanokirigami-based graphene composite for 100% strain.

[0136] Polarization modulation of the LIDAR beam and polarization analysis of the returning photons will enable the acquisition of information about the material of an object, which is currently lacking in, for example, automotive safety and robotic vision equipment. Machine learning (ML) algorithms can be trained to recognize different materials, and MST classification can be achieved based on the material's unique polarization signature. One of the advantages of MST classification of objects by material is that it can speed up object recognition and improve the accuracy of a machine's perception of its surroundings.

[0137] In the turn Figures 4a to 4d Before going into detail, it is worth noting that according to some embodiments of the present disclosure, non-nano-kirigami optical elements can be used to perform Figure 1 The system 100 and corresponding material detection and object classification methods are shown. In fact, for certain applications (e.g., where size and weight are not primary issues), such M-LIDAR systems based on non-nano-kirigami optical elements may be preferred. Therefore, without departing from the teachings herein, implementations of the presently disclosed M-LIDAR system that do not use nano-kirigami optical elements may equally use other conventional optical components, such as (i) IR polarizers; (ii) beam splitters; (iii) lenses made of CdS, ZnS, silicon; and / or (iv) similar suitable optical elements. However, nano-kirigami optical elements are generally advantageous for M-LIDAR systems that benefit from being light and small.

[0138] With the above as background, Figures 4a to 4dA step-by-step photolithographic type process for manufacturing a nano-cut paper optical element (e.g., a beam splitter or linear polarizer) is illustrated. According to one embodiment, Figures 4a to 4d The process for manufacturing a nano-cut paper optical element set forth in U.S. Publication No. 2016 / 0299270 can include the use of vacuum assisted filtration (VAF), whereby a nano-composite material can be deposited as a layer on a rigid (e.g., plastic) substrate suitable for photolithographic patterning. As noted above, U.S. Publication No. 2016 / 0299270 describes a method of manufacturing such a nano-composite material, which includes a vacuum assisted filtration (VAF) and layer-by-layer (LBL) deposition process technique. Nano-composite materials manufactured according to this process are known to have high toughness and strong light absorption.

[0139] Figure 4a is a simplified illustration of the first step of the process 400, in which a layer of nano-composite material 404a is deposited on a substrate 402 via VAF, layer-by-layer deposition (LBL), or any other suitable deposition method known in the art. Figure 4b is an illustration of the second step in the process 400, in which the nano-composite material 404a of FIG. 4 is patterned to produce a patterned cut paper nano-material 404b through select areas of the nano-composite material layer 404a, e.g., via a photolithographic cutting process on top of the substrate 402. Figure 4c is an illustration of the third step in the process 400, whereby the cut or patterned cut paper nano-composite material 404b is released (e.g., lifted) from the substrate 402. Finally, Figure 4d is an illustration of the final step of the process 400, whereby at least a portion of the patterned cut paper nano-composite material 408 has been incorporated into a subassembly configured for beam steering and / or modulation, etc.

[0140] Figure 4d The illustrated subassembly includes the patterned cut paper nano-composite material portion 408, a microfabricated silicon layer 406 housing, and one or more flexural beam actuators 410. The double-sided arrows 412 illustrate possible directions of actuator 410 motion. As discussed in additional detail below, the flexural beam actuators 410 can be configured to exert a reversible strain on the cut paper technology nano-composite material portion 408 to, e.g., adjust the size and / or orientation of the various slits and / or fins that make up the pattern of the cut paper nano-composite material portion 408. Thus, the cut paper nano-composite material portion 408 can be reversibly stretched with a strain level ranging from 0% to 100%.

[0141] Next is Figures 4a to 4d is a more detailed discussion of the patterning aspect of the process 400 illustrated. The manufacturing of a cut paper transmissive optical module can follow Figures 4a to 4dThe step-by-step diagram is shown. Modulation of the LIDAR laser beam in the visible and IR ranges may require feature sizes of, for example, 3 μm created over a width of 0.1 cm to 1 cm. The feasibility of such patterns has been demonstrated. The 2D geometry of the pattern can be selected based on computer simulations of the 2D to 3D reconfiguration of the pattern when stretched or strained. The 3D geometry can be modeled to obtain optical properties, such as polarization modulation in the desired wavelength range. Photolithography can be the primary patterning tool, which is achieved through the chemical reaction of the VAF composite material described above. The patterning protocol can be essentially similar to that currently used for large-scale microfabrication. For example, after patterning the image using a commercially available conventional mask aligner, the VAF composite material on a glass substrate can be coated with a standard SU8 photoresist. Figures 5a to 5c An example of a prepared paper-cut pattern is shown in FIG, which will be discussed in more detail below.

[0142] The kirigami nanocomposite sheets can be prepared by combining them with e.g. Figure 4d Commercial microelectromechanical actuators are shown integrated to create kirigami optical elements. Microelectromechanical system (MEMS) kirigami units can be directly integrated with LIDAR components and used as beam steering devices (using, for example, first- and second-order diffracted beams) and / or polarizers (using, for example, first-order diffracted beams). Considering the almost infinite number of kirigami patterns and a wide variety of 2D to 3D reconfigurations, kirigami optical elements with both beam steering and polarization functions, as well as other optical functions, are contemplated within the teachings herein.

[0143] Brief Reference Figures 5a to 5c , illustrates various images of example kirigami optical elements. For example, Figure 5a Based on the above Figures 4a to 4d An image of a kirigami optical element, such as the ones described herein, fabricated on a wafer by process 400 is depicted. Figure 5b yes Figure 5a SEM image of the kirigami optical element at 0% strain. Figure 5c yes Figure 5a SEM image of the kirigami optical element under 100% strain. Figures 5a to 5c The scale bar depicted in the upper right corner is 50 μm.

[0144] Photolithography can be used to fabricate kirigami transmissive or reflective optical modules / elements. By way of example, the modulation of a LIDAR laser beam having a wavelength of about 1550 nm can have a feature size ranging from greater than or equal to about 1 μm to less than or equal to about 2 μm, and a width ranging from greater than or equal to about 0.1 cm to less than or equal to about 1 μm. Figures 5a to 5cThe pattern is created on about 1 cm as shown in the current illustration. The 2D geometry of the pattern can be selected based on computer simulations of the two-dimensional (2D) to three-dimensional (3D) reconfiguration of the pattern upon stretching. The 3D geometry can be modeled to extract optical properties, such as polarization modulation over a desired wavelength range.

[0145] Photolithography is the primary patterning technique that can be used in conjunction with LbL composites to form kirigami optical elements. In one embodiment, the patterning protocol can include providing an LbL composite on a glass substrate, which is coated with a standard SU-8 photoresist and then photopatterned using a commercial mask aligner (UM Lurie Nanofabrication Facility, LNF). Such a process can form kirigami elements, such as Figures 5a to 5c Those paper-cut elements are shown in .

[0146] Figure 11 Another representative simplified compact M-LIDAR system 300 for use in a vehicle (e.g., an autonomous vehicle) is provided. To some extent, the components in the M-LIDAR system 300 are similar to those in the Figure 1 The components of the M-LIDAR system 100 are similar, and for the sake of brevity, the functions of the components will not be repeated herein. The M-LIDAR system 300 may include a laser 310, a beam steering device 312, one or more polarizers (not shown, but similar to Figure 1 1 and 126). In the M-LIDAR system 300, a pulse generator 312 is connected to the laser 310 and generates a first light pulse 314 that is polarized or unpolarized and a second light pulse 316 that is polarized or unpolarized. The pulse generator 312 is connected to an oscilloscope 324. The first light pulse 314 and the second light pulse 316 generated by the laser 310 are directed to a beam steering device 318, which in some aspects can be a kirigami-based beam steering device as discussed above. The beam steering device 318 is connected to and controlled by a servo motor / Arduino 352. As a non-limiting example, the servo motor / Arduino 352 is connected to a MATLAB® controller that can be a MATLAB® controller. TM Controller 350 of the control. As described above, beam redirector 318 can polarize, modify, split, and / or modulate one or both of first light pulse 314 and second light pulse 316 by the non-limiting embodiments described above. First light pulse 314 and second light pulse 316 are then directed toward object 340 to be inspected.

[0147] First light pulse 314 and second light pulse 316 may be diffusely reflected from object 110. One or more light pulses together form first reflected light beam 342 and second reflected light beam 344 that constitute reflected versions of first light pulse 314 and second light pulse 316. According to some embodiments, first reflected light beam 342 and second reflected light beam 344 may have a different polarization than first light pulse 314 and second light pulse 316 (i.e., before reflection from object 340). After reflection from object 340, first reflected light beam 342 and second reflected light beam 344 may be directed toward off-axis parabolic reflector / mirror 330, which redirects first reflected light beam 342 and second reflected light beam 344 toward beam splitter 360.

[0148] Thus, first reflected light beam 342 is split and directed to both first detector 362 and second detector 364. First detector 362 and second detector 364 can be connected to oscilloscope 324. First detector 362 can be an s-polarization detector configured to detect the intensity of one or more reflected s-polarized light pulses forming first reflected light beam 342. Similarly, second detector 364 can be a p-polarization detector configured to detect the intensity of one or more reflected p-polarized light pulses forming first reflected light beam 342. After passing through beam splitter 360, second reflected light beam 344 is directed to both first detector 362 and second detector 364, where the intensity of the s-polarized light pulses and / or the p-polarized light pulses can be detected from second reflected light beam 344. First detector 362 and second detector 364 can be connected to a processor (not shown) that further analyzes the information received from the processor as described above. By way of non-limiting example, M-LIDAR system 300 is compact and can have dimensions of approximately 7 inches x 12 inches, making it particularly suitable for installation in a vehicle.

[0149] As described above, MST classification can be achieved using light source-based MST classification and a light polarization classifier added to the point cloud according to embodiments of the present disclosure. In one embodiment, for each 3D distance measurement of the point cloud, the linear polarization / circular polarization of the return photon can be obtained. Furthermore, although the relationship between surface properties and polarization state may be chaotic in some cases due to surface roughness, local curvature and local scattering conditions can be directly determined based on the polarization state of the return photon.

[0150] Now refer to Figure 6 , performs MST polarization analysis on the reflected laser light using AI data processing via a neural network algorithm to produce Figure 6 More specifically, based on the confusion matrix of the s-polarized beam and the p-polarized beam (e.g. Figure 1) is analyzed to generate a confusion matrix. Along the x-axis, the predicted material type for the test object subjected to the M-LIDAR system and processing method described herein is identified. Along the y-axis, the actual material type for the test object is identified. The accuracy of the various predictions of the AI ​​algorithm for various material types is reflected at the intersection of the predicted material type and the actual material type. As shown, these polarized beams can be used to implement the material detection function of the M-LIDAR system with a high accuracy (including 99% or more in some cases).

[0151] Now refer to Figure 7 , shows a confusion matrix for the detection of simulated black ice compared to other materials. Likewise, the material detection functionality of the M-LIDAR system can be achieved with a high degree of accuracy (including 100% in some cases).

[0152] Figure 8 An embodiment of an M-LIDAR device 800 for use, for example, in black ice detection (e.g., when mounted in a vehicle, etc.) is illustrated. While the present embodiment focuses on black ice detection applications, one of ordinary skill in the art will recognize that the device 800 is not limited to black ice detection and may be suitably used in a wide range of material detection and object classification applications, including autonomous vehicles. The device 800 includes a housing 802, an emitter 804 (i.e., an emitter for emitting light pulses that constitute a laser beam), a first detector 806a, and a second detector 806b. According to one embodiment, one or more of the detectors 806a, 806b include orthogonal polarization analyzers. Furthermore, according to one embodiment, one or more of the emitter 804, detector 806a, and / or detector 806b may be made from kirigami optical elements. While the primary embodiment of the device is for use within an automobile, the device may also be used, for example, within an aircraft such as a drone.

[0153] Now refer to Figure 9 , a flow chart illustrating a method 900 for performing object classification using an M-LIDAR system is provided. Method 900 begins at 902, where an unpolarized light pulse is generated. At 904, the unpolarized light pulse is linearly polarized to produce a linearly polarized light pulse. The linearly polarized light pulse can be emitted toward an object and reflected from the object to produce a reflected linearly polarized light pulse. At 906, the reflected linearly polarized light pulse can be linearly polarized to s-polarization to produce a reflected s-polarized light pulse.

[0154] At 908, the reflected linearly polarized light pulse may be linearly polarized to p-polarization to generate a reflected p-polarized light pulse. At 910, the intensity of the reflected s-polarized light pulse may be detected. At 912, the intensity of the reflected p-polarized light pulse may be detected. At 914, at least one material of the object may be detected based on the intensity of the reflected s-polarized light pulse and the intensity of the reflected p-polarized light pulse. Finally, at 916, the object may be classified based on the detected at least one material. After 916, method 900 ends.

[0155] Finally, according to some embodiments, kirigami patterns can be used as MST tags for polarization-based object detection. Mass-produced kirigami technology compositions can also be added to coatings to impart a specific polarization response on road signs, clothing, markings, vehicles, household items, or any other suitable objects.

[0156] In certain variations, the disclosed LIDAR system can provide modulation of both the transmitted and reflected beams. Kirigami-based optical elements can be added to the transmitter side of the LIDAR to act as beam steering devices, replacing conventional body-rotating or liquid crystal phase array beam steering devices. A magnetic actuation module can be integrated with a 1550 nm laser source. To reduce the size of the beam steering device, optical fibers can be directly coupled to the module.

[0157] In certain variations, the LIDAR systems provided by the present disclosure can provide enhanced detection in precipitation and / or humid atmospheric conditions. For example, by way of non-limiting example, by employing a laser having a wavelength of approximately 1550 nm, the LIDAR systems contemplated by the present disclosure can be particularly well-suited for use in low-visibility conditions, providing enhanced detection and performance during adverse weather conditions, including those associated with fog, rain, and snow. Such LIDAR systems can enable long-range warnings of up to 200 meters, for example, which is particularly useful for highway driving conditions. Conventional LIDARs use lasers having a wavelength of approximately 900 nm, which is convenient for silicon-based detectors. However, these conventional laser beams experience relatively strong scattering in humid atmospheric conditions. LIDARs operating at 1550 nm can take advantage of the high transparency of humid air, which is advantageous for various levels of autonomy, from proximity warning to assisted driving and fully autonomous driving. However, due to the high weight and cost of near-infrared optics, such LIDARs can be bulky and expensive. According to certain aspects of the present disclosure, kirigami-based optical elements can address this problem by exploiting the space charge and subwavelength effects feasible with patterned kirigami sheets, see e.g. Figure 2a to Figure 2b .like Figure 3a to Figure 3bAs shown, this kirigami sheet can be used to efficiently modulate and beam-steer a near-infrared laser using its reconfigurable out-of-plane pattern. A 1550nm beam-steering device incorporating this kirigami-based optical element can be used as a thin, lightweight, and inexpensive solid-state LIDAR. Furthermore, the versatility of the kirigami technique allows for tailoring the pattern to specific applications, for example, customizing the LIDAR system for a specific vehicle and / or adapting it to the varying curvatures of automotive parts.

[0158] In certain aspects, the present disclosure can provide relatively fast detection for LIDAR systems by using a two-stage object proposal and detection approach, without sacrificing accuracy for latency. For example, improved model accuracy and generalizability for classification models can include enhancing static object classifiers by adding material dimensions to the data. Objects with distinctive material fingerprints, such as plastic, wood, and brick, are unlikely to be moving, while objects with distinctive metal or fabric features are more likely to be pedestrians and vehicles. Furthermore, because material dimensions are more robust to scene changes, these models generalize better to rare and complex cases, such as construction sites and streets with intricate holiday decorations. Consequently, distinctive features significantly improve the model accuracy of point cloud-associated models, impacting tracking and autonomous vehicle mapping. For example, the material dimension of a point cloud can make detection and classification more reliable, such as picking up a cyclist as a point cloud with metallic material on the underside and some fabric or skin features from the pedestrian. This makes it easier for autonomous driving systems to distinguish cyclists from pure pedestrians. Likewise, the material distinctive features of the objects make it easier for the system to associate the point cloud with the correct object classification, thus helping to maintain correct and consistent classification of composite objects.

[0159] The present disclosure thus provides an inexpensive and compact LIDAR system with enhanced object recognition capabilities, including the ability to distinguish between material types, provide an early detection and warning system, including the ability to identify objects within milliseconds, and high performance in low visibility conditions, among other benefits.

[0160] In certain variations, the present disclosure provides a system comprising a laser configured to generate light pulses; a beam diverter configured to generate polarized adjusted light pulses emitted toward an object; at least one polarizer configured to polarize reflected light, scattered light, or emitted light returned from the object; and a processor configured to detect at least one material of the object based on intensity and polarization of the polarized reflected light, the polarized scattered light, or the polarized emitted light from the object. In an aspect, the beam diverter comprises a paper-cutting nanocomposite. In an aspect, the at least one polarizer comprises a paper-cutting nanocomposite. In an aspect, the processor is further configured to classify the object based on the detected at least one material of the object.

[0161] In another aspect, the processor is configured to classify the object based on the detected at least one material of the object by applying a machine learning algorithm. In another aspect, the machine learning algorithm comprises an artificial neural network algorithm.

[0162] In an aspect, the beam diverter is configured to adjust polarization of the light pulses to produce the polarized adjusted light pulses. In an aspect, the beam diverter is configured to adjust the polarization of the light pulses by at least one of imparting polarization to unpolarized light pulses and changing polarization of polarized light pulses. In another aspect, the beam diverter is configured to adjust the polarization of the light pulses by applying at least one of the following types of polarization: linear polarization, circular polarization, and elliptical polarization. In another aspect, applying linear polarization comprises applying at least one of s-type linear polarization and p-type linear polarization.

[0163] In an aspect, the at least one polarizer is configured to polarize the reflected light, the scattered light, or the emitted light returned from the object by applying at least one of the following types of polarization: linear polarization, circular polarization, and elliptical polarization. In another aspect, applying is applying linear polarization comprising applying at least one of s-type linear polarization and p-type linear polarization. In an aspect, the at least one polarizer comprises a plurality of polarizers.

[0164] In an aspect, the system further comprises at least one polarization detector connected to the at least one polarizer and the processor, wherein the at least one polarization detector is configured to detect intensity of the polarized reflected light, the polarized scattered light, or the polarized emitted light from the object. In another aspect, the at least one polarization detector comprises a plurality of polarization detectors. In another aspect, the at least one polarization detector is configured to detect an angle of incidence associated with the polarized reflected light, the polarized scattered light, or the polarized emitted light from the object.

[0165] In another aspect, the processor is further configured to detect the at least one material of the object based on the angle of incidence associated with the polarized reflected light, the polarized scattered light, or the polarized emitted light from the object.

[0166] In another variation, the present disclosure provides a method comprising generating a light pulse; adjusting the polarization of the light pulse to generate a polarization-adjusted light pulse emitted toward an object; polarizing reflected light, scattered light, or emitted light returned from the object; and detecting at least one material of the object based on the intensity and polarization of the polarized reflected light, polarized scattered light, or polarized emitted light from the object.

[0167] In one aspect, adjusting the polarization of a light pulse is performed by a beam steering device comprising a kirigami nanocomposite. In one aspect, the kirigami nanocomposite is fabricated using a vacuum-assisted filtration (VAF) process. In one aspect, the kirigami nanocomposite is fabricated using a layer-by-layer (LBL) deposition process. In one aspect, the method further includes classifying an object based on at least one material detected in the object.

[0168] In one aspect, classifying the object comprises classifying the object by applying a machine learning algorithm. In one aspect, the machine learning algorithm comprises an artificial neural network algorithm.

[0169] In other aspects, the present disclosure provides LIDAR systems and methods configured to detect the relative distance or position and shape of an object, as well as additional information about the object (e.g., the object's material composition) and / or additional information about a returning laser pulse (such as the optical properties of the reflected light beam). The additional information about the light beam returning from the object may include, for example, polarization information, Raman scattering information, circular dichroism information, and the like. Thus, the additional information about the object and / or about the returning / reflected light beam adds an additional dimension of information that is stored and associated with the distance or position and shape information about the object. In other words, the LIDAR systems and methods of the present disclosure are multidimensional because they are configured to detect information in addition to the three-dimensional information representing the relative distance or position and shape of the object. For example, the material composition of the object may add a fourth dimension associated with the information about the object. Additionally or alternatively, the additional information about the light beam returning from the object may include polarization information, Raman scattering information, circular dichroism information, and the like, which may add one or more additional dimensions to the information associated with the object.

[0170] For example, an object detected by the LIDAR systems and methods of the present disclosure can be represented by a point cloud in a three-dimensional XYZ coordinate system. The point cloud can include multiple points, each representing a location on the surface of the object and having associated XYZ coordinates. As discussed in further detail above, generating the point cloud can include a repeated process of emitting light pulses, which diffusely reflect off the object, the reflected light being measured by a detector, and the position of the point on the object's surface relative to the light source being determined. This process can then be repeated many times per second, for example, on the order of millions of times per second, generating a point each time. Each point is then mapped onto the same XYZ coordinate system to create a point cloud.

[0171] For each point in the point cloud representing the object, the LIDAR systems and methods of the present disclosure can also determine a fourth dimension representing the material composition at the specific point on the surface of the object. Additionally or alternatively, for each point in the point cloud representing the object, the LIDAR systems and methods of the present disclosure can also determine an additional dimension corresponding to additional information about a return (e.g., reflected) beam of the laser pulse beam associated with the specific point on the surface of the object (such as optical properties of the reflected beam). As described above, the additional information can include, for example, polarization information, Raman scattering information, circular dichroism information, etc., about the return (e.g., reflected) beam associated with the specific point on the surface of the object. In this manner, the LIDAR systems and methods of the present disclosure can also determine a multi-dimensional point cloud representing the detected object, the multi-dimensional point cloud including XYZ coordinates of points representing the surface of the object, each point having additional associated information in addition to the XYZ coordinates, such as information representing the material composition of the surface of the object at the specific point and / or additional information about the reflected beam from the specific point (such as polarization information, Raman scattering information, circular dichroism information, etc.). As described above, the material classification method of the present disclosure can determine material composition information based on optical information about the reflected light beam (such as polarization information, Raman scattering information, and circular dichroism information). In this way, the LIDAR system and method of the present disclosure can generate and store any combination of material composition information and / or additional optical information associated with each point in a point cloud representing a detected object.

[0172] Reference Figure 12 , a four-dimensional point cloud 1000 generated by the LIDAR system and method of the present disclosure is shown using a person on a bicycle as an example detected object. Figure 12 While only the X-axis and Y-axis of the XYZ coordinate system are shown, it can be understood that the point cloud 1000 omits the three-dimensional Z-axis for ease of explanation. The point cloud 1000 includes a plurality of points, each of which represents a position on a detected object, for example, a position on the surface of a detected object, in this case, a person on a bicycle. Figure 12 As shown, each XYZ point in point cloud 1000 also includes associated material composition information. As shown in key 1002, point 1004 in point cloud 1000 corresponding to a person's head is indicated as having material composition information corresponding to skin. Points 1006 in point cloud 1000 corresponding to the person's torso, arms, and legs are indicated as having material composition information corresponding to cotton. Point 1008 in point cloud 1000 corresponding to a bicycle's frame is indicated as having material composition information corresponding to aluminum. Point 1010 in point cloud 1000 corresponding to a bicycle's tire is indicated as having material composition information corresponding to rubber. Figure 12The point cloud 1000 shown may be used, for example, by an autonomous vehicle system. Figure 12 The point cloud 1000 shown determines that the detected object is a person riding a bicycle. The autonomous vehicle system can also use the point cloud 1000 to distinguish between an object that is an actual person riding a bicycle and, for example, a statute of a person riding a bicycle.

[0173] Although the embodiment point cloud 1000 includes material composition information as a fourth dimension associated with each point in the point cloud, as described above, other additional information about the reflected light beam may be stored with each point in addition to or in lieu of the material composition information. The additional information may include, for example, polarization information, Raman scattering information, circular dichroism information, etc.

[0174] The LIDAR systems and methods of the present disclosure can generate and utilize a material database that stores material composition information and corresponding optical information, such as polarization information, Raman scattering information, circular dichroism information, and the like. Thus, the LIDAR systems and methods of the present disclosure can compare the optical information associated with a reflected light beam from an object with the stored optical information to locate and determine the corresponding material composition of the object. The LIDAR systems and methods of the present disclosure can use neural networks and machine learning techniques to generate and construct the material database and calibrate relevant parameters for determining the material composition of the object based on the optical information associated with the reflected light beam from the object.

[0175] For example, a neural network layer can be initialized using a neural network layer that includes parameter values ​​based on the physical laws of light / matter interaction. In other words, the neural network layer can be initialized using input optical information associated with a reflected light beam from an object to provide the object's expected material composition based on the physical laws of light / matter interaction. The system can then be trained using a large number of objects (e.g., thousands, millions, or billions of objects), each with a known composition. For example, a LIDAR system can scan each object, compare the determined material composition to the object's known composition, and use machine learning techniques to adjust the neural network's parameter values ​​based on the comparison. This process can be repeated multiple times for each of the large number of objects, with each iteration resulting in a higher accuracy in the determined material composition as the system calibrates and adjusts the neural network's parameter values ​​over time. Because the optical information associated with the reflected light beam can depend on the object's orientation, position, and shape, the object can be rescanned multiple times using different orientations and positions to improve the system's accuracy. Furthermore, as discussed in further detail below, the system can include multiple LIDAR lasers and detectors positioned at different orientations relative to the object, and information from each of the multiple LIDAR lasers and detectors can be used in parallel to determine the object's material composition and improve the system's accuracy.

[0176] Reference Figure 13 , a flow chart illustrating a method 1100 for training a neural network for identifying the material composition of an object (including individual points on a surface of the object) according to the present disclosure is provided. The method 1100 begins at 1102. At 1104, the neural network is initialized based on the physical laws of light / matter interaction. For example, initialization parameters of the neural network are initialized and set based on the physical laws of light / matter interaction. At 1106, an object having a predetermined or known material composition is detected using a LIDAR system, and the material composition of each point in a point cloud representing the object is determined based on optical information of a returning (e.g., reflected) light beam and based on the neural network.

[0177] At 1108, the material composition determined by the system is compared to the known material composition of the object. At 1110, if necessary, the parameters of the neural network are adjusted based on the comparison using machine learning techniques.

[0178] At 1112, a determination is made as to whether the accuracy of the system is still acceptable. At 1112, when the accuracy of the system is not yet acceptable, the method loops back to 1106 and repeats the process by detecting another object and making further adjustments to the parameters. This process is repeated until it is determined at 1112 that the accuracy of the system is acceptable. When the accuracy of the system is determined to be acceptable, the process ends at 1114. For example, the accuracy of the system may be determined to be acceptable when the system accurately determines the material composition of a predetermined number of objects within a predetermined acceptable error range.

[0179] In this way, by scanning a large number of objects (e.g., thousands, millions, or billions of objects), the systems and methods of the present disclosure can build a large material database and associated neural network that can be used to accurately determine the material composition of objects detected using the LIDAR system of the present disclosure.

[0180] The multi-dimensional LIDAR system and method of the present disclosure may be used in a recycling system to identify the material composition of objects to be recycled and to sort the objects appropriately based on the determined material composition. Figure 14 , an embodiment recycling system 1200 is shown and includes a conveyor belt 1202. The conveyor belt 1202 moves objects from a recycling object bin 1204 that holds objects to be sorted and recycled to a sorter 1206. Figure 14 In the embodiment of FIG. 1 , a metal can 1220 is shown on a conveyor belt 1202 as it moves from a recycle bin 1204 toward a sorter. Figure 14 As shown in FIG, as objects 1220 are transported on a conveyor belt toward a sorter 1206, the objects 1220 are scanned by one or more LIDAR systems 1214a, 1214b, 1214c according to the present disclosure. Figure 14 Three LIDAR systems 1214a, 1214b, 1214c are shown, but any number of LIDAR systems may be used. For example, only one LIDAR system may be used. However, as described above, the use of additional LIDAR systems that can scan the object 1220 from multiple different angles and perspectives may increase the accuracy of material classification determinations.

[0181] The recycling system 1200 includes a control module 1218 in communication with LIDAR systems 1214a, 1214b, and 1214c, a sorter 1206, and a material database 1216. As an object 1220 passes by the LIDAR systems 1214a, 1214b, and 1214c, the object 1220 is scanned by the LIDAR systems 1214a, 1214b, and 1214c, which then transmit resulting information regarding the reflected light beams from the object 1220 to the control module 1218. Based on the aforementioned classification techniques, the control module 1218 is configured to access the material database 1216 and determine a multi-dimensional point cloud for the object 1220 based on the optical information of the reflected light beams received by each of the LIDAR systems 1214a, 1214b, and 1214c. For example, the multi-dimensional point cloud may include material classification information for each point of the point cloud representing the object 1220. Based on the material classification information, the control module 1218 may then control the sorter 1206 to sort the objects 1220 into appropriate classification bins for recycling.

[0182] For example, Figure 14 As shown in FIG, recycling system 1200 includes a glass bin 1208 for receiving glass objects, a metal bin 1210 for receiving metal objects, and a plastic bin 1212 for receiving plastic objects. Sorter 1206 may include baffles, doors, slots, and / or levers that are movable and configurable to direct objects 1220 to appropriate bins via, for example, chutes 1209, 1211, and 1213. For example, chute 1209 leads to glass bin 1208, chute 1211 leads to metal bin 1210, and chute 1213 leads to plastic bin 1212. Although the sorter 1206, recycling bins 1208, 1210, 1212, and chutes 1209, 1211, 1213 are illustrated as embodiments, any other sorting mechanism may be used and controlled by the control module 1218 to sort objects from the conveyor belt 1202 into appropriate recycling bins based on the material composition of the objects (as determined based on the systems and methods of the present disclosure).

[0183] In this way, the LIDAR system and method of the present disclosure can accurately determine the material composition of objects to be recycled and can be used in conjunction with a sorting system to appropriately and automatically sort the objects to be recycled into groups of recycled materials.

[0184] Reference Figure 15 , shows four-dimensional point clouds 1300, 1302, 1304 generated by the LIDAR system and method of the present disclosure for recycling objects. Figure 15Only the X and Y axes of the XYZ coordinate system are shown, but it can be understood that the point clouds 1300, 1302, and 1304 omit the three-dimensional Z axis for ease of explanation. Each of the point clouds 1300, 1302, and 1304 includes a plurality of points, each of which represents a position on the surface of the object being detected. Figure 15 As shown, each XYZ point in point clouds 1300, 1302, and 1304 also includes associated material composition information. As shown by key 1306, the points of point cloud 1300 are indicated as having material composition information corresponding to plastic. The points of point cloud 1302 are indicated as having material composition information corresponding to aluminum. The points of point cloud 1304 are indicated as having material composition information corresponding to paper. Point clouds 1300, 1302, and 1304 can be represented by, for example, Figure 14 The control module 1218 of the recirculation system shown in FIG.

[0185] The LIDAR system and other methods of the present disclosure can also be used to detect the health status of a living object, such as an organism such as an animal. In some aspects, the animal can be a human.

[0186] The LIDAR systems and other methods of the present disclosure can also be used in medical / cosmetic applications to identify, for example, skin areas exhibiting certain conditions and for diagnosis of skin diseases by mapping the skin surface and analyzing polarimetry data from points corresponding to the scanned patient skin surface.

[0187] Reference Figure 16 , shows four-dimensional point clouds 1400, 1402, 1404, 1406 generated by the LIDAR system and method of the present disclosure for scanning a patient's skin surface. Figure 16 Only the X and Y axes of the XYZ coordinate system are shown, but it is understood that the point clouds 1400, 1402, 1404, and 1406 omit the three-dimensional Z axis for ease of explanation. Each of the point clouds 1400, 1402, 1404, and 1406 includes a plurality of points, each of which represents a location on the patient's skin surface. Figure 16As shown, each XYZ point in point clouds 1400, 1402, 1404, and 1406 also includes associated material composition information. As shown by key 1408, points in point cloud 1400 are indicated as having material composition information corresponding to normal skin. Points in point cloud 1402 are indicated as having material composition information corresponding to acne. Points in point cloud 1404 are indicated as having material composition information corresponding to a lump. Points in point cloud 1406 are indicated as having material composition information corresponding to dry skin. Point clouds 1300, 1302, and 1304 can be generated, for example, by a LIDAR system utilizing a material database that stores information on material compositions corresponding to various skin conditions and diseases. Generally, four-dimensional point clouds can be used to distinguish between biological materials, cells, tissues, and the like, for example, the location of malignant cells versus benign cells, or the characteristics of selected biological materials. Thus, the LIDAR systems and other methods of the present disclosure can be used to detect the health status of living subjects, such as organisms such as animals. In some respects, animals can be people.

[0188] Similarly, the LIDAR systems and methods of the present disclosure can also be used for facial recognition.

[0189] Furthermore, the LIDAR systems and methods of the present disclosure may also be incorporated into other devices, such as personal computers, laptop computers, portable devices, smartphones, tablet devices, etc. For example, the lasers used for the laser pulses may be implemented using diode lasers, which may be included in other devices (e.g., personal computers, portable devices, smartphones, tablet devices, etc.). When incorporated into a smartphone, laptop computer, or tablet device, for example, the LIDAR systems and methods of the present disclosure may be used for facial recognition to allow access to the smartphone, laptop computer, or tablet device.

[0190] In addition, the LIDAR system of the present disclosure can be used to communicate with other LIDAR systems. For example, the LIDAR system can use the polarization of the transmitted laser pulses to convey data. As an embodiment, the LIDAR system can be incorporated into a vehicle, and when the vehicle turns left, the LIDAR system can transmit left-polarized pulses to the LIDAR systems of the surrounding vehicles. Similarly, when the vehicle turns right, the LIDAR system can transmit right-polarized pulses to the LIDAR systems of the surrounding vehicles. In addition, the two LIDAR systems can transmit pulses with a specific polarization sequence to convey information (such as verification information) to confirm and verify each other's identity and prevent the LIDAR systems from being counterfeited or hacked by third parties. For example, a first LIDAR system can request a second LIDAR system to send verification information using a specific polarization sequence known to both LIDAR systems.

[0191] The disclosed LIDAR systems and methods can be used to generate information about scanned materials by analyzing the polarization of reflected and scattered LIDAR beams. The disclosed material-sensing LIDAR systems and methods perform computational analysis of the polarization ratio between the s- and p-polarizations of a modulated laser beam impinging on the scanned item. Raman scattering and other information can also be obtained to supplement polarization processing.

[0192] The polarization of the impinging LIDAR beam can be circularly polarized. The polarization of the reflected and scattered LIDAR beams can be analyzed with respect to circular polarization. The chirality of the material, its composition, or its shape will be a fundamental material property that affects the polarization characteristics of the detected item. Each XYZ data point obtained by LIDAR is supplemented by another characteristic (such as a material detection characteristic, i.e., the polarization ratio or data obtained by comparing the polarization ratios through computational processing based on a database containing different materials). Each XYZ point is provided with additional data, making the point more than just a three-dimensional point. When material properties are included, the point becomes a four-dimensional point. When different Raman scattering or other property information is given to the data point at the same time, the point becomes multidimensional, having more than four dimensions.

[0193] As described above, the material information of each point can be specified by a deep learning neural network algorithm based on a material database. The material database can be obtained through a polarimetric LIDAR setup. The LIDAR beam is modulated to be linearly polarized or circularly polarized. Polarimetric LIDAR measures the polarization difference during reflection / scattering by analyzing the ratio of s-polarized light to p-polarized light of the reflected / scattered light. Based on both the measurement results and the electric field calculation, the angular dependence of the reflection and scattering physics of each material will be included in the database. According to the CIE color vector space, the color effect of each material will be included in the database based on both the measurement and the electric field calculation. Polarimetric Raman scattering and FTIR information can be obtained and included in the material database. The optical activity of the material including linear dichroism and circular dichroism will be included in the material database. The material database package will provide a comprehensive understanding of the material regardless of texture, incident angle and coating color.

[0194] Multidimensional material identification LIDAR can also be achieved by processing the intensity distribution of reflected and scattered light. In this case, the signal is processed by a machine learning algorithm that includes reflection physics and an evaluation of the reflected light distribution based on expected material parameters as one of the neural network layers. Not only the intensity maximum but also the decay of intensity and noise frequencies are included as parameters for training such a network.

[0195] In certain variations, the present disclosure provides a system that includes a polarization-enabled camera, also referred to as a polarization camera, that is configured to sense the polarization of light it detects. In these variations, because the system that utilizes the polarization camera detects the polarization of ambient light that reflects off of an object, the system does not need the laser and beam steerer described in the above-described systems and methods. Rather, in these variations, the system utilizes non-polarized ambient light and can detect the polarization of ambient light that reflects off of an object. Ambient electromagnetic radiation can include visible light, ultraviolet light, infrared light, etc. Particularly suitable visible and infrared electromagnetic radiation includes visible light having wavelengths in the range of about 390 nm to about 750 nm and infrared radiation (IR) including near infrared (NIR) of about 0.75 pm to about 1.4 pm and far infrared (FIR) of about 15 pm to 1 mm. The far infrared (FIR) portion of the electromagnetic spectrum, also referred to as the terahertz (THz) region, has photon wavelengths of about 100 pm to about 1 mm and energies of about 0.001 eV to about 0.01 eV. In addition to providing information for many areas of THz research from astronomy and solid-state physics to telecommunications, as non-limiting examples, THz circular dichroism (TCD) can also be used to understand and detect astronomy, solid-state physics, telecommunications, biological materials, biological molecules, and pharmaceuticals.

[0196] Referring to Figure 1 For example, a system that utilizes a polarization camera in accordance with these variations does not need the laser 102 and the beam steerer 106, and does not need to generate the light beam 104 or the light beam 108. In other words, in a system that utilizes a polarization camera, ambient light is used in place of the light beam 104 and the light beam 108 that are generated by the laser 102.

[0197] In an aspect, similar to the above-described systems and methods, the processor uses the sensed polarization information about ambient light that reflects off of a detected object to determine and sense at least one material of the detected object.

[0198] In other aspects, the systems and methods of the present disclosure can include a LIDAR and a polarization camera in the same perception system. In another aspect, the polarization camera is configured to sense polarization information about ambient light that reflects off of a detected object, and polarization information about laser light from the LIDAR that returns from the detected object.

[0199] In other aspects, the processor is configured to classify the object based on the at least one material of the detected object by applying a machine learning algorithm. In another aspect, the machine learning algorithm includes an artificial neural network algorithm.

[0200] In other aspects, the present disclosure provides polarization camera systems and methods configured to detect the relative distance, position, and shape of an object, as well as additional information about the object, such as polarization information of ambient light reflected off the object and detected by the polarization camera system. This polarization information can be used to determine the material composition of the object, as described above with respect to systems and methods for sensing polarization information using a laser, a beam redirector, and at least one polarizer. Thus, the additional information about the object and / or about the returning / reflected light beam adds an additional dimension of information that is stored and associated with the distance, position, and shape information about the object. In other words, the polarization camera systems and methods of the present disclosure are multidimensional because they are configured to detect information beyond the three-dimensional information representing the relative distance, position, and shape of the object. For example, the material composition of the object can add a fourth dimension to the information associated with the object. Additionally or alternatively, the additional information about the light beam returning from the object can include polarization information and material composition information, among others, which can add one or more additional dimensions to the information associated with the object.

[0201] For example, an object detected by the polarization camera system and method of the present disclosure can be represented by a point cloud in a three-dimensional XYZ coordinate system. The point cloud can include multiple points, where each point represents a location on the surface of the object and has associated XYZ coordinates. Similar to the LIDAR system described above, generating a point cloud using the polarization camera system and method can include a repeated process of detecting ambient light reflected off the object, measuring the reflected light with a polarization camera, and determining the location of the point on the object's surface. This process can then be repeated many times per second, for example, on the order of millions of times per second, generating a single point each time. Each point is then mapped to the same XYZ coordinate system to create a point cloud.

[0202] For each point in a point cloud representing an object, the polarization camera system and method of the present disclosure can also determine a fourth dimension representing the composition of the material located at the specific point on the surface of the object. Additionally or alternatively, for each point in the point cloud representing the object, the polarization camera system and method of the present disclosure can also determine an additional dimension corresponding to additional information about the returned (e.g., reflected) ambient light associated with the specific point on the surface of the object, such as optical properties of the reflected light. As described above, the additional information can include, for example, polarization information associated with the specific point on the surface of the object. In this manner, the polarization camera system and method of the present disclosure can also determine a multi-dimensional point cloud representing the detected object, the multi-dimensional point cloud comprising XYZ coordinates of points representing the surface of the object, wherein each point has additional associated information in addition to the XYZ coordinates, such as information representing the material composition of the object's surface at the specific point and / or additional information about the reflected light from the specific point (such as polarization information). As described above, the material classification method of the present disclosure can determine material composition information based on optical information about the reflected light (e.g., polarization information). In this manner, the polarization camera systems and methods of the present disclosure may generate and store any combination of material composition information and / or additional optical information, such as polarization information, associated with each point in a point cloud representing a detected object.

[0203] Reference 17A to 17C , shows a polarization camera 2000. Specifically, Figure 17A A perspective view of a polarization camera 2000 is shown. Figure 17B shows a front view of the polarization camera 2000, and Figure 17C A rear view of a polarization camera 2000 is shown. For example, the polarization camera 2000 can be a Triton polarization model polarization camera available from Lucid Vision Labs, which utilizes Polarsense technology from Sony. For example, the polarization camera 2000 can record RGB polarization video at 24 frames per second (FPS) and 5.0 megapixel (MP) resolution. Although the present disclosure describes systems and methods utilizing a Triton polarization model polarization camera, any other polarization camera that captures polarization information of light reflected off an object can be used in accordance with the present disclosure.

[0204] An image captured by a standard still camera contains RGB color data for each pixel in the image, but that's all. With the polarization camera 2000, each pixel also stores data about the polarization of the detected light. When light reflects off an object's surface, it undergoes a polarization shift based on the properties of the surface's material before being sent to the lens of the polarization camera 2000. Due to these unique polarization properties, if two objects are made of different materials, the polarization of the reflected light from each object will be different, and the two objects can be distinguished based on the different polarizations of the reflected light. Standard cameras and image recognition algorithms without polarization information would have difficulty distinguishing between the two objects. For example, a road covered in black ice would have a significantly different polarization image than a road without black ice. However, a standard camera would generate RGB images of the two roads that, if not identical, would appear very similar. Similarly, a black rubber tire and a black bag have very different polarization distributions due to the different textures and other properties of their constituent materials. Therefore, the polarization camera system and method of the present disclosure can advantageously distinguish a black rubber tire from a black plastic bag, as well as a road covered in black ice from one without.

[0205] The polarization camera 2000 is configured to sense the polarization of ambient light that reflects off a detected object. Polarization is a property of all electromagnetic radiation, including the light that is measured by the camera, in order to sense or detect objects in the surrounding environment. All light, even if initially completely unpolarized, will enter a new polarization state after reflecting off the surface of an object. This change in polarization depends on and corresponds to the specific composition of the material of the object the light is reflecting off, as well as the angle at which the light is reflected off the material of the object. The closer the light is to being perpendicular to the reflecting surface, the more significant the change in polarization value that the light will experience. Therefore, the further away the object is, the stronger the material-based effect of the object's surface on the light will be. Although increased distance makes object detection more difficult due to reduced resolution, the relative enhancement of light-surface interaction for distant objects provides some relief from this difficulty. Polarization information can be fed into the perception layer of the systems and methods of the present disclosure and used to determine the material composition of the sensed object, even when the sensed object is farther away and the resulting image of the object has a lower resolution than that of a closer object. For example, the systems and methods of the present disclosure can access a materials database (e.g. Figure 14 1216 ) to determine the material of an object and / or a point on the object based on the polarization of ambient light reflected off the object and / or a point on the object.

[0206] A particular polarization state can be any angle between 0 and 360 degrees. In a similar manner, chromaticity (a property of color) is often represented in a circular fashion, where each degree of chromaticity corresponds to one degree of rotation. Exploiting this fact, it is possible to create false color photographs that represent the kind of visual contrast provided by the polarization data generated by the polarization camera 2000. For example, Figure 18 Illustrated is an image 2050 of a group of objects shown without polarization data. The image 2050 was generated with a non-polarized camera. Figure 18 Also illustrated is image 2052 of the same set of objects generated by polarization camera 2000 and shown using false-color polarization to illustrate the additional contrast and feature visibility obtainable using polarization data captured by polarization camera 1000. In accordance with the present disclosure, by training a machine learning system to consider this polarization information in the same context as color, entirely new correlations can be formed that are stronger and more general than anything a standard camera vision system can achieve.

[0207] Reference Figure 19 , and with the above discussion Figure 6 Similarly, MST polarization analysis of reflected light is performed using AI data processing with a neural network algorithm to produce Figure 19 A confusion matrix is ​​shown. Along the x-axis, the predicted material type for a test object subjected to the processing methods described herein is identified. Along the y-axis, the true material type for the test object is identified. The accuracy of the AI ​​algorithm's various predictions for various material types is shown at the intersection of the predicted material type and the true material type. As shown, the material detection functionality of the systems and methods of the present disclosure can be achieved with a high degree of accuracy (including, in one case, equal to or greater than 96%).

[0208] Furthermore, there are many possible edge cases where an object may be substantially camouflaged due to its color, yet fully visible to the polarization camera system and method of the present disclosure that uses the polarization information of the object. For example, in an image of a child wearing a yellow raincoat standing in front of a yellow school bus, a conventional system may not be able to distinguish between the child and the school bus. However, a system using polarization information according to the present disclosure can recognize that the polarization of light reflected off the material of the yellow raincoat is distinguishable from the polarization of light reflected off the material of the yellow school bus. Therefore, a system using polarization camera 2000 according to the present disclosure can distinguish between two objects, classify different materials and objects, and identify boundaries and edges between two objects. Therefore, for detecting distant or visually obscured objects, the system and method of the present disclosure significantly improves the success rate of identifying materials and objects, while also reducing the amount of time required for classification, identification, etc.

[0209] Object detection and computer vision systems can divide an image into tens or hundreds of objects represented therein. The object detection data can then be fed into object classification algorithms that characterize each identified object. According to the present disclosure, adding polarization information to the perception layer of object detection and computer vision systems improves the systems' ability to discern and classify objects, while reducing processing time, compared to systems that do not utilize such polarization.

[0210] According to the present disclosure, the polarization camera 2000 may be used in place of the LIDAR, beam steering device, and at least one polarizer described in the above systems and methods.

[0211] For example, the polarization camera 2000 can be used to generate Figure 12 、 Figure 15 and Figure 16 Furthermore, by replacing the LIDAR system with a polarization camera system having a polarization camera 2000 and referring to, for example, Figure 13 Reference numeral 1106, using the detected polarization information as the reflected light information described above, the polarization camera 2000 can be used with the above reference Figure 13 The method 1100 for training a neural network for identifying the material composition of an object is used in conjunction with the method 1100 described above.

[0212] Additionally, one or more polarization cameras 2000 may be used with the polarization cameras 2000 described above. Figure 14 The embodiments described herein can be used in conjunction with the recycling system 1200. In each case, the LIDAR, beam steering device, and at least one polarizer described above with respect to each of those systems and methods can be replaced by a polarization camera 2000 configured to capture polarization information of ambient light reflected off an object. Figure 1 , laser 102 , beam steering device 106 , s-polarization linear polarizer 114 , p-polarization linear polarizer 116 , s-polarization detector 122 , and p-polarization detector 124 may be replaced with a polarization camera 2000 that detects the polarization of ambient light reflected off object 110 .

[0213] Reference Figure 14For example, one or more of the LIDAR systems 1214a, 1214b, and 1214c can be replaced with one or more polarization cameras 2000. In this case, the control module 1218 is configured to access the material database 1216 and determine a multidimensional point cloud for the object 1220 based on the polarization information of the ambient light reflected off the object 1220 and sensed by the one or more polarization cameras 2000, in accordance with the classification techniques described above. For example, the multidimensional point cloud can include material classification information for each point in the point cloud representing the object 1220. As described above, based on the material classification information, the control module 1218 can then control the sorter 1206 to sort the object 1220 into the appropriate bin for recycling. In this way, the polarization camera system and method of the present disclosure can accurately determine the material composition of objects to be recycled and can be used in conjunction with a sorting system to appropriately and automatically sort the objects to be recycled into groups of recyclable materials.

[0214] In addition, the polarization camera system and method of the present disclosure can be used in an autonomous driving system to sense objects near an autonomous vehicle and generate a four-dimensional point cloud for the objects to sense, classify, and identify the objects. For example, the polarization camera system and method of the present disclosure can be used to generate a four-dimensional point cloud of a cyclist, such as Figure 12 shown.

[0215] Furthermore, the polarization camera system and method of the present disclosure can be used in medical / cosmetic applications to identify, for example, skin areas exhibiting certain conditions and for diagnosis of skin diseases by mapping the skin surface and analyzing polarization information from points corresponding to the scanned patient skin surface. For example, the polarization camera system and method of the present disclosure can be used to generate Figure 16 As described above, a material database storing information corresponding to material components of different skin conditions and diseases can be used to generate a 4D point cloud. This 4D point cloud can then be used to differentiate between biological materials, cells, tissues, and the like. For example, the location of malignant cells relative to benign cells or the characteristics of selected biological materials can be determined.

[0216] Similarly, the polarization camera system and method of the present disclosure can be used to detect the health status or biological indicators of a living object (eg, an organism). The organism can be an animal, such as a human.

[0217] The following table (Table 1) provides a comparison between different camera / sensor types (such as standard cameras, hyperspectral cameras, polarization cameras, LIDAR systems, and radar systems) in terms of (i) the unique wavelengths of light measured, (ii) whether additional material-related data is provided, (iii) the number of polarization angles, (iv) the current estimated price range, (v) cycles per second, and (vi) associated data processing requirements. (The associated data processing requirements for LIDAR and radar systems are not shown.)

[0218]

[0219] In other aspects, the systems and methods of the present disclosure can be used to classify and / or identify objects based on the different types of materials that make up the objects. For example, machine learning techniques and algorithms can be used to build an object database that includes relevant information that associates a list of specific objects with the specific materials that make up each object. Figure 12 , for example, the object database may include a database entry for a cyclist that indicates that the object riding the bicycle is composed of skin, cotton, aluminum, and rubber. Thus, when the system and method of the present disclosure senses an object composed of skin, cotton, aluminum, and rubber, the object database may be queried based on the sensed materials. Based on the query, the system and method of the present disclosure may determine that the object is likely to be a cyclist based on the material composition of the object. Furthermore, the object database may include a percentage of each type of material, or a percentage range of each type of material, indicating the approximate percentage or percentage range of each type of material that can be found in a particular object. Thus, the system and method of the present disclosure may utilize material information to more accurately classify and identify objects. Object recognition and classification may be used in conjunction with a computer vision system that recognizes and classifies objects based on their size, shape, and / or other characteristics.

[0220] In this application, including the definitions below, the term "module" or the term "controller" may be replaced with the term "circuit". The term "module" may refer to, be part of, or include: an Application Specific Integrated Circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinatorial logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip.

[0221] The module may include one or more interface circuits. In some examples, the interface circuit may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module of the present disclosure may be distributed across multiple modules connected via the interface circuits. For example, multiple modules may allow for load balancing. In another example, a server (also referred to as a remote or cloud) module may perform certain functions on behalf of a client module.

[0222] As used above, the term "code" may include software, firmware and / or microcode, and may refer to programs, routines, functions, classes, data structures and / or objects. The term "shared processor circuit" includes a single processor circuit that executes some or all of the code from multiple modules. The term "group processor circuit" includes a processor circuit that, in combination with additional processor circuits, executes some or all of the code from one or more modules. References to multiple processor circuits include multiple processor circuits on discrete dies, multiple processor circuits on a single die, multiple cores of a single microprocessor circuit, multiple threads of a single processor circuit, or a combination of the foregoing. The term "shared memory circuit" includes a single memory circuit that stores some or all of the code from multiple modules. The term "group memory circuit" includes a memory circuit that, in combination with other memory circuits, stores some or all of the code from one or more modules.

[0223] The term "memory circuit" is a subset of the term "computer-readable medium." As used herein, the term "computer-readable medium" does not include transient electrical or electromagnetic signals propagated through a medium (e.g., on a carrier wave); thus, the term "computer-readable medium" may be considered to be tangible and non-transitory. Non-limiting examples of non-transitory, tangible computer-readable media are non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital magnetic tape or hard drives), and optical storage media (e.g., CDs, DVDs, or Blu-ray discs).

[0224] The apparatus and methods described in this application can be implemented partially or completely by a special-purpose computer, which is created by configuring a general-purpose computer to perform one or more specific functions embodied in a computer program. The above-mentioned functional blocks, process components, and other elements serve as software specifications, which can be converted into computer programs through routine work by technicians or programmers.

[0225] The computer program includes processor-executable instructions stored on at least one non-transitory, tangible computer-readable medium. The computer program may also include or rely on stored data. The computer program may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0226] A computer program may include: (i) descriptive text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation), (ii) assembly code, (iii) object code generated by a compiler from source code, (iv) source code executed by an interpreter; (v) source code compiled and executed by a real-time compiler, etc. By way of example only, the source code may be written using the syntax of a language including: C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Fortran, Perl, Pascal, Curl, OCaml, HTML5 (Hypertext Markup Language Version 5), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Lua, MATLAB, SIMULINK, and

[0227] No element recited in a claim is a means-plus-function element within the meaning of 35 U.S.C. §112(f) unless the element is expressly recited using the phrase "means for" or, in the case of a method claim, the element is expressly recited using the phrases "operate" or "step."

[0228] The foregoing description of the embodiments has been provided for the purpose of illustration and description. It is not intended to be exhaustive or to limit the present disclosure. The individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but are interchangeable where applicable and may be used in selected embodiments even if not specifically shown or described. The same may also be varied in many ways. Such variants should not be considered as departing from the present disclosure, and all such modifications are intended to be included within the scope of the present disclosure.

Claims

1. A method for detecting a material, the method comprising: Initializing parameters of the neural network using at least one processor based on physical laws of light / matter interactions; emitting a plurality of light pulses using a laser device toward an object having a predetermined material composition; receiving, using a detector, a plurality of reflections of the plurality of light pulses returning from the object; inputting, using the processor, the optical characteristics of the plurality of reflections into the neural network; receiving, using the processor, from the neural network an expected material composition of the object based on the input optical properties; comparing, using the processor, the expected material composition to the predetermined material composition; and Based on the comparison, the parameters of the neural network are adjusted using a processor.

2. The method according to claim 1, wherein Adjusting the parameters is performed using a machine learning algorithm.

3. The method according to claim 1, further comprising: repeatedly adjusting the parameters of the neural network based on comparing additional expected material compositions of additional objects with additional known material compositions of the additional objects until a difference between one of the expected material compositions and a corresponding one of the additional known material compositions is less than a predetermined error margin, the additional expected material compositions being generated by inputting optical characteristics of multiple additional reflections of multiple additional light pulses reflected by the laser device off the additional objects.

4. A method for detecting a material, the method comprising: Initializing parameters of the neural network using at least one processor based on physical laws of light / matter interactions; receiving, using a polarization camera, a plurality of reflections of ambient light reflected off the object; determining, using the processor, polarization information for the plurality of reflections of ambient light reflected off the object; inputting the polarization information of the plurality of reflections into the neural network using the processor; receiving, using the processor, from the neural network an expected material composition of the object based on the input polarization information; comparing, using the processor, the expected material composition to a predetermined material composition; and Based on the comparison, the parameters of the neural network are adjusted using a processor.

5. The method according to claim 4, wherein Adjusting the parameters is performed using a machine learning algorithm or an artificial intelligence algorithm.

6. The method according to claim 4, further comprising: The parameters of the neural network are repeatedly adjusted based on a comparison of additional expected material compositions for additional objects with additional known material compositions for the additional objects until a difference between one of the expected material compositions and a corresponding one of the additional known material compositions is less than a predetermined error margin, the additional expected material compositions being generated by inputting additional polarization information for multiple additional reflections of ambient light reflected off the additional objects and received by the polarization camera.

Images