Spatial profiling system and method

Abstract

A spatial profiling system for profiling an environment is disclosed. The spatial profiling system includes an optical transmitter for providing light, a beam director for directing the light in one or more directions toward the environment, and an optical receiver for receiving return light reflected by a surface or object in the environment. The return light carries information for determining a distance to the surface or object. The optical receiver is configured to detect (a) the specularity of the return light and (b) the polarization state of the return light. The spatial profiling system further includes a processing system configured to determine a material associated with the surface or object based on the detected specularity and the detected polarization state.

Description

[Technical Field]

[0001] The present disclosure generally relates to systems and methods for light-based estimation of terrestrial or extraterrestrial environments, such as for LiDAR systems and methods performed by LiDAR systems. [Background technology]

[0002] Spatial profiling refers to a 2D or 3D mapping of an environment over a two-dimensional (2D) or three-dimensional (3D) field of view of the environment. Each point or pixel in the field of view is associated with a distance to form a 2D or 3D representation of the environment. Spatial profiles may be useful in facilitating task automation by identifying objects and / or obstacles in the environment.

[0003] One technique for spatial profiling involves sending light in a particular direction into an environment and detecting any light that is reflected back from that direction, for example, by reflective surfaces in the environment. This technique is sometimes called light detection and ranging, or LiDAR. The reflected light carries relevant information for determining the distance to the reflective surface. The combination of this particular direction and distance forms a point or pixel in a three-dimensional representation of the environment. The above steps may be repeated for multiple different directions to form other points or pixels in the three-dimensional representation, thereby estimating the spatial profile of the environment within a desired field of view. Summary of the Invention [Means for solving the problem]

[0004] A spatial profiling system and components for a spatial profiling system, as well as related methods, are described, wherein the spatial estimates produced by the spatial profiling system may be of a terrestrial or extraterrestrial environment.

[0005] According to aspects of the present disclosure, a spatial profiling system for profiling an environment is provided, the spatial profiling system including: an optical transmitter for providing light; a beam director for directing the light in one or more directions toward the environment; an optical receiver for receiving returned light reflected by a surface or object in the environment, the returned light carrying information for determining a distance to the surface or object, the optical receiver configured to detect (a) a specularity of the returned light and (b) a polarization state of the returned light; and a processing system configured to determine a material associated with the surface or object based on the detected specularity and the detected polarization state.

[0006] The processing system may be configured to determine a material associated with the surface or object by classifying the material into one of a plurality of material categories. Classifying the material into one of a plurality of material categories may include, wherein the classification includes applying one or more machine learning algorithms.

[0007] The optical receiver may be further configured to detect specularity based on a speckle-related image or interference pattern, which in one embodiment represents a surface or objected spatial sample that reflects light.

[0008] The optical receiver may be further configured to detect specularity based on the plurality of despeckled signals. The optical receiver may be further configured to retrieve or provide an amplitude measure and a phase measure of one or more of the plurality of despeckled signals.

[0009] The processing system may be further configured to determine any one of a speckle contrast, a speckle granularity, and a speckle anisotropy based on the detected speckle contrast, a speckle granularity, and a speckle anisotropy. The processing system may be further configured to determine a material associated with the surface or object based on any one or more of the determined speckle contrast, a speckle granularity, and a speckle anisotropy.

[0010] The optical receiver may be further configured to detect the polarization state based on the degree of polarization state preservation. The degree of polarization state preservation may represent a degree of polarization of the returned light relative to a degree of polarization of the emitted light or the local oscillator. The processing system may be further configured to determine a material associated with the surface or object based on the degree of polarization state preservation.

[0011] As used herein, terms such as "first" and "second" are used to distinguish one entity from another, and are not used to indicate or require any particular order in time, location, or otherwise. For example, "a first port and a second port" has the same meaning as "a port and another port."

[0012] As used herein, the terms "optical port" and "port" refer to an area of ​​an optical component through which light passes, and do not necessarily require the presence of a physical structure or component. For example, one port may be formed by the end of a waveguide or optical fiber, in which case the perimeter of the port coincides with the inner surface of the waveguide or optical fiber, while another port may be within a larger area of ​​an input or output slab of a wavelength router, in which case the perimeter of the port does not coincide with any structure of the waveguide.

[0013] As used herein, "light" refers to electromagnetic radiation having optical frequencies, which includes far infrared, infrared, visible radiation, and ultraviolet radiation.

[0014] Any view or orientation designations used herein, such as top view, side view, horizontal, or vertical, are arbitrary for illustrative purposes and do not imply any required orientation.

[0015] Further aspects of the invention and further embodiments of the aspects described in the preceding paragraphs will become apparent from the following description, given by way of example with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 illustrates the layout of a spatial profiling system. [Figure 2] FIG. 2 illustrates the placement of light sources relative to the spatial profiling system of FIG. 1. [Figure 3] 2A and 2B are diagrams illustrating examples of sensor head placement for the spatial profiling system of FIG. 1. [Figure 4] FIG. 4 is a diagram showing sensor head components of the sensor head of FIG. 3. [Figure 5] FIG. 2 is a partial diagram of a photodetector for the spatial profiling system of FIG. 1. [Figure 6] FIG. 1 illustrates an example of a processing system for classifying surfaces of different materials. [Figure 7] FIG. 1 shows an example of an experimental setup for obtaining experimental observations. [Figure 8A] FIG. 7 shows a performance matrix of the processing system of FIG. 6 trained according to the disclosed training method. [Figure 8B] FIG. 7 shows a performance matrix of the processing system of FIG. 6 trained according to the disclosed training method. [Figure 9] FIG. 7 illustrates the accuracy of the processing system of FIG. 6 under different trained machine learning models. [Figure 10] FIG. 2 illustrates a portion of a photodetector for the spatial profiling system of FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0017] Light-based spatial profiling systems are sometimes called light detection and ranging (LiDAR) systems. LiDAR involves transmitting light into an environment and detecting light returned from the environment. By detecting the returned light, the system can determine information about the distance of reflective surfaces within its field of view (FOV), such as the surfaces of objects or obstacles, the contours of the ground, and / or the location of the horizon, and a spatial estimate of the environment may be formed.

[0018] There are various methods for determining distance in or by a LiDAR system. In some embodiments of a LiDAR system, the distance of a reflective surface may be determined based on the round-trip time of light. In a simple example, the round-trip time of a pulse of light may be determined, and from that round-trip time, the range to the reflective surface in the direction from which the pulse of light was transmitted may be determined. Alternatively or additionally, distance may be determined using frequency-modulated continuous wave (FMCW) technology. Examples of LiDAR range detection, including examples using FMCW technology, are discussed in International Patent Application PCT / AU2016 / 050899 (published as International Publication No. WO2017 / 054036(A1)), the entire contents of which are incorporated herein by reference. In some embodiments, pulses of light having a time-varying profile are emitted, and the time-varying profile is used for distance determination. In other embodiments, instead of detecting the round-trip time of a series of modulated pulses, the emitted light includes a linear frequency chirp or phase variation to detect the round-trip time.

[0019] In three-dimensional mapping, one dimension relates to the range of a point from the origin of the emitted light, while the other two dimensions relate to the two-dimensional space (e.g., a space definable by a Cartesian (x, y) or polar (theta, phi) coordinate system) across which the light is directed. The area or angular range over which the light is directed for detection of the return light is the field of view of the spatial profiling system. The field of view of a LiDAR system may be fixed or a controlled variable.

[0020] In some LiDAR systems, one or more beams of light are directed into the environment, and the one or more beams of light travel in two dimensions (i.e., the first and second dimensions of a two-dimensional field of view), and the combination of this travel and knowledge of the determined range provides information for spatial profiling.

[0021] In some other LiDAR systems, light is emitted over a wider area, up to the entire field of view of the LiDAR system. For example, different colors of light may be emitted in different directions within the field of view to enable both direction and range determination. The remainder of this specification is provided primarily with reference to LiDAR systems having output light in the form of one or more beams of light, rather than systems that emit light simultaneously across the entire field of view.

[0022] In some embodiments, the LiDAR system or a processing system in communication with the LiDAR system may determine speed or velocity information of an entity, such as a vehicle, and / or reflective surfaces in the environment in which the LiDAR system is located. This speed or velocity determination may be based on detected light returned from the environment, which may be based directly, for example, on a Doppler-shifted signal contained in the returned light, or based on changes in distance determination over time. For example, in an FMCW system, a coherent beat tone of a chirp waveform would indicate a Doppler shift. Additionally or alternatively, speed information may be obtained or determined from external information not originating from the LiDAR system.

[0023] Figure 1 shows an example layout of a spatial profiling system 100. As indicated in the drawing legend, electrical connections (e.g., analog or digital data or control signals) are represented by solid lines, and optical connections (e.g., guided or free-space optical transmission) are represented by dashed lines in Figure 1. Optical input and output ports of components are represented by solid-filled circles.

[0024] Spatial profiling system 100 includes an optical transmitter 101, a sensor head 103, an optical receiver 104, and a processing and control system 105. Spatial profiling system 100 forms an outgoing light path P1 for outgoing light L1 that is provided to the environment for spatial profiling, and an incoming light path P2 for incoming light L2 that is provided to optical receiver 104 for detection. Incident light L2 includes outgoing light L1 that is reflected from the environment.

[0025] The optical transmitter 101 includes a light source 102 for generating the output light L1. The light source 102 may include one or more light generators, such as one or more laser diodes. In some embodiments, the light source 102 is tunable to selectively provide light in one or more selectable wavelength ranges. For example, the light source may include one or more tunable laser diodes. In some embodiments, the light source 102 provides light having a single polarization orientation. In some embodiments, the optical transmitter 101 includes one or more optical amplifiers to provide gain to the output light L1 and / or one or more optical modulators to impart time variation to at least one characteristic of the output light L1.

[0026] Output light L1 from optical transmitter 101 is provided to sensor head 103. Output light L1 may be provided directly from optical transmitter 101 to sensor head 103 or may be provided indirectly via one or more other optical components in output light path P1, such as, for example, a collimator.

[0027] The sensor head 103 directs the emitted light L1 towards the environment. In embodiments where the emitted light L1 is in the form of one or more beams of light, the sensor head 103 includes a beam director for controlling the direction of the emitted light L1.

[0028] When the light source 102 is tunable, the sensor head 103 may include one or more wavelength-based beam directors that direct one wavelength of the light source 102 in one direction and another wavelength in another direction. Thus, a range of wavelengths may be directed in a range of directions. Depending on the beam director implementation, there may be a one-to-one correspondence between selectable wavelengths and directions, or one set of multiple selectable wavelengths may be directed in a single direction and another set of selectable wavelengths may be directed in another direction.

[0029] Additionally or alternatively, sensor head 103 may include one or more beam directors including one or more mechanically movable components for controlling the direction of the emitted light, such as, for example, one or more scanning mirrors and / or rotating or tilting dispersive or diffractive components. Thus, the emitted light L1 is directed in one direction at one time when the mechanically movable component is in one position or orientation, and in another direction at another time when the mechanically movable component is in another position or orientation, and so on, providing a range of directions.

[0030] The sensor head 103 may include both a wavelength-based beam director and a mechanical beam director. For example, the sensor head 103 may include one or more diffractive and / or dispersive components that direct light based on wavelength, and the directed light may be provided to a scanning mirror for mechanical beam steering. In another example, at least one diffractive or dispersive component for wavelength-based beam steering is mounted on a rotating platform, and rotation of the diffractive or dispersive component results in mechanical beam steering. A spatial profiling system having both wavelength and mechanical beam steering components may be considered to have a wavelength dimension and a mechanical dimension. The wavelength dimension and the mechanical dimension may be orthogonal or substantially orthogonal.

[0031] Additionally, sensor head 103 receives incident light L2 along incident light path P2. In the illustrated embodiment, sensor head 103 includes a bidirectional port through which both exit light L1 and incident light L2 traverse. In other words, exit light path P1 and incident light path P2 coincide or overlap at least at the bidirectional port of sensor head 103. Exit light path P1 and incident light path P2 may share a common optical axis or have parallel optical axes at the bidirectional port. This sharing of a common optical axis or the existence of parallel optical axes may continue through at least one of the one or more beam directors of sensor head 103.

[0032] In some embodiments, sensor head 103 separates incident light L2 from output light L1. This separation may be achieved by directing incident light L2 to a different port than the port at which sensor head 103 receives incident light L1 (as represented by the separated ports in FIG. 1 ) and / or by providing incident light L2 from sensor head 103 such that light path P2 is not parallel to light path P2. In other embodiments, this separation occurs at another location along light paths P1, P2, such as near or within optical receiver 104.

[0033] In still other embodiments, the outgoing light path P1 and the incoming light path P2 do not coincide or overlap on or within the sensor head 103. In these embodiments, the sensor head may optionally be separated into two physical components, one for providing the outgoing light path P1 and one for providing the incoming light path P2.

[0034] Incoming light L2 traversing incident light path P2 is received by optical receiver 104. Light may be provided directly from sensor head 103 to optical receiver 104, or may be provided indirectly via one or more other optical components in incident light path P2, such as, for example, an optical filter.

[0035] The optical receiver 104 includes a photodetector 106. The photodetector generates a signal S1 based on the incident light L2. The signal S1 represents information conveyed by the detected incident light L2 for determining the distance to a reflective surface. As shown in FIG. 1, the signal S1 may be an analog data signal. The photodetector 106 may include one or more photodetectors. An example of a photodetector is an avalanche photodiode (APD). The optical receiver 104 may include two photodiodes for balanced detection. When the processing and control system 105 is a digital system, an analog-to-digital converter 107 converts the analog data signal to a digital signal S2.

[0036] In some embodiments, light from the light source 102 is also provided to the detector 106 to provide a reference or local oscillator optical signal L3. The local oscillator optical signal L3 is provided to the optical receiver 104. The detector circuitry may then be configured to reject non-reflected light based on wavelength or modulation differences between the emitted and non-reflected light. For example, the photodetector 106 may include one or more balanced detectors for coherently detecting reflected light in the incident light L2 mixed with the reference light. Thus, the spatial profiling system 100 may achieve coherent (homodyne or heterodyne) detection of the incident light L2. Through coherent detection, the photodetector 106 is configured to recover or provide measures of both the amplitude (E) and phase (φ) of the incident light L2, such as functions of both time (E(t)) and phase (φ(t)). In one example, the photodetector 106 includes an in-phase and quadrature (IQ) optical demodulator. The IQ demodulator is configured to combine a first portion of the incident light L2 with a first (in-phase) portion of the reference light L3, for example, via an optical coupler, to provide a first combination. The IQ demodulator is further configured to combine a second portion of the incident light L2 with a second (quadrature) portion of the reference light L3, for example, via another optical coupler, to provide a second combination. The first (in-phase) portion and the second (quadrature) portion of the reference light L3 are phase-separated by 90 degrees of pi / 2 radians. The IQ demodulator may include an optical path length, such as an optical delay line, to facilitate phase separation. Alternatively, the IQ demodulator may include one or more multi-mode interference (MMI) couplers to facilitate phase separation. The IQ demodulator is configured to generate an electrical in-phase signal (of magnitude I) based on the first combination and an electrical quadrature signal (of magnitude Q) based on the second combination. The in-phase signal (of magnitude I) and the quadrature signal (of magnitude Q) can be further combined, such as during digitization by ADC 107 discussed below, to recover the amplitude (E) and phase (φ) of the incident light L2. For example, a complex-valued representation of the incident light L2 may be characterized as I+jQ=Eexp(jφ).In general, the values ​​of I, Q, E, and φ are all functions of time. Other detection methods may be used, such as direct direction detection. In direct detection, the local oscillator optical signal L3 is not required.

[0037] The digital signal S2 is received and processed by the processing and control system 105. The processing and control system 105 may determine the distance to a reflective surface (or object) in the environment based on the digital signal S2.

[0038] Optical transmitter 101 may be controlled by processing and control system 105 via control signals over control line C1. In some embodiments, processing and control system 105 also controls aspects of the operation of other components in the system, such as controlling one or more components of sensor head 103 over control line C2 and / or one or more components of optical receiver 104 over control line C3. Two or more of control lines C1-C3 may optionally be combined with other control lines into a control bus, where the controlled components are individually addressable.

[0039] The processing and control system 105 may determine distances to reflective surfaces in the environment based on its knowledge of controlling the components of the spatial profiling system 100. The processing and control system 105 may determine a spatial profile of the environment based on the collection of distance determinations. Alternatively, the processing and control system 105 may include a communications interface with another data processing system and may communicate signals with the other data processing system to enable it to make spatial profiling determinations based on the distance determinations by the processing and control system 105, or to enable it to make distance and / or spatial profiling determinations.

[0040] The processing and control system 105 may include one or more application-specific devices configured to perform the operations described herein, such as one or more manufactured or configured programmable logic devices, such as application-specific integrated circuits or field programmable gate arrays, or one or more general-purpose computing devices, such as microcontrollers or microprocessors, along with computer-readable memory that stores instructions for causing the computing device(s) to perform the operations.

[0041] In the case of an application-specific device, the instructions and / or data for controlling the operation of the processing unit may be implemented in whole or in part by firmware or hardware elements, including configured logic gates, which may be integrated on a common substrate, for example as a system on a chip integrated circuit, or distributed among devices on separate substrates.

[0042] In the case of a general-purpose computing device, the processing and control system 105 may include, for example, a single computing device (e.g., a central processing unit, a graphics processing unit, or other computing device) or may include multiple computing devices. The processing and control system 105 may also include a communication bus in data communication with one or more machine-readable storage (memory) devices that store instructions and / or data for controlling aspects of the operation of the processing units. The memory devices may include system memory (e.g., BIOS), volatile memory (e.g., random access memory), and non-volatile memory (e.g., one or more hard disks or solid-state drives for providing non-transitory storage). Operations for spatial profiling are generally controlled by instructions in non-volatile memory and / or volatile memory.

[0043] Additionally, the processing and control system 105 includes one or more interfaces, such as an interface to control lines C1-C3 or a control bus, and an interface for receiving signal S2. The external interface may provide an option for updating firmware and / or software of the processing and control system 105. The external interface may provide an option for multiple LiDAR systems to communicate, such as to share information and / or share spatial profiles for spatial profiling, to enable decisions and actions based on the spatial profiling activities of the two or more LiDAR systems.

[0044] In some embodiments, the control operations and the data processing operations are performed by separate physical devices. In other embodiments, one or more physical devices may perform both the control and data processing operations.

[0045] In some embodiments, the spatial profiling system 100 separates functional components into two or more physical units. For example, the sensor head 103 may be included in one physical unit, and the optical transmitter 101, the optical receiver 104, and the processing and control system 105 may be included in one other physical unit, or one or more of them may be in additional physical units. In some embodiments, the sensor head 103 is remote from one or more of the other components. This remote sensor head 103 may be coupled to the other units via one or more guided optical connections, such as, for example, waveguides or optical fibers. The spatial profiling system may include multiple sensor heads 103. Each of the multiple sensor heads 103 may be optically coupled to the optical receiver 104 by a respective guided optical connection. The multiple sensor heads 103 may be positioned at different locations and / or oriented to have different fields of view. In some embodiments, the optical transmitter 101 and the optical receiver 104 are implemented in the same optical subassembly. In another embodiment, the optical transmitter 101 and the optical receiver 104 are implemented in different optical subassemblies. In either embodiment, the ADC 107 and the processing and control system 106 may be implemented in the same or a different printed circuit board assembly, separate from any optical subassembly or subassemblies. The printed circuit board assembly or subassemblies may include or correspond to a system-on-a-chip (SoC) or system-on-a-module (SoM).

[0046] 2 shows an example arrangement of an optical transmitter 201, which may form, for example, the optical transmitter 101 of the spatial profiling system 100 described with reference to FIG. 1. In this example, the optical transmitter 201 includes a tunable laser 202, for example a tunable laser diode, as a source of a beam of light. The tuned wavelength of the tunable laser 202 may be based on one or more currents applied to the laser diode, for example, an injection current to one of more wavelength tuning elements in the laser cavity. In the spatial profiling system 100, the current is controlled in response to a control signal over control line C1.

[0047] Thus, optical transmitter 201 transmits a plurality of selectable wavelength channels (each with its own respective center wavelength λ1, λ2, . . . λ) N and (b) are configured to provide a beam of output light at a selected one or more of the wavelengths (represented by ). In some embodiments, the wavelength range of the tunable light source is at least 20 nm, or at least 25 nm, or at least 30 nm, or at least 35 nm. The resolution (i.e., minimum wavelength step) of the tunable light source may be at most 0.2 nm, preferably at most 0.1 nm, more preferably at most 0.05 nm, and even more preferably at most 0.01 nm. In some embodiments, the wavelength channel is about 1550 nm. Other wavelengths, such as, for example, about 905 nm, may also be used. The optical transmitter 201 may select one wavelength channel at a time or may provide two or more different selected wavelength channels (i.e., channels having different center wavelengths) simultaneously.

[0048] Light from the light source may pass through polarizer 203 so that the light emitted to the environment is polarized light. In some embodiments, polarizer 203 is a single polarizer. In other embodiments, polarizer 203 is a cross-polarizer, which may include two polarizers oriented perpendicular to each other, or may provide orthogonal polarizations when the source light has a single polarization. In some embodiments, the polarizer produces linearly polarized light.

[0049] Polarized light from the light source may pass through optical splitter 204, where a majority of the light continues along the output light path and the remainder of the light is provided as a local oscillator signal. For example, optical splitter 204 may be a 90 / 10 optical fiber coupler that provides 90% of the light as output light and 10% of the light as a local oscillator signal for coherent detection.

[0050] The optical transmitter 101 may also include an optical amplifier 205 for amplifying (providing gain to) the output light. In some embodiments, the optical amplifier 205 is one or more stages of an erbium-doped fiber amplifier (EDFA). In other embodiments, one or more stages of a semiconductor optical amplifier (SOA), a booster optical amplifier (BOA), or a solid-state amplifier (e.g., an Nd:YAG amplifier) ​​may be used. In the spatial profiling system 100, the gain may be controlled in response to a control signal over control line C1. In some embodiments, the optical amplifier 205 is omitted.

[0051] In some embodiments, the optical transmitter 201 includes a modulator 206 for imparting a time-varying profile to the output light. This modulation may be in addition to any wavelength tuning described previously herein. In other words, this modulation results in modulation of light at a tuned wavelength. It will be appreciated that the tuned wavelength may indicate the center frequency or other measure of the generated wavelength channel. The time-varying profile may be, for example, one or more of variations in intensity, frequency, phase, or code imparted to the output light. The operation of the modulator 206 (e.g., modulating the waveform) may be controlled by the processing and control system 105 by a control signal over control line C1.

[0052] In one example, modulator 206 is an external modulator to the laser diode (e.g., a Mach-Zehnder modulator, an electro-optic modulator, or an external SOA modulator). In another example, modulator 206 is a phase modulator. While FIG. 2 shows an example in which modulator 206 is located after optical amplifier 204, it will be appreciated that the modulator may be located either before or after optical amplifier 205 in the output light path. In one example, the optical transmitter's modulator is a semiconductor optical amplifier (SOA) or Mach-Zehnder modulator integrated on the laser diode of the light source. Varying the current applied to the SOA over time may vary the amplification of the CW light generated by the laser over time, which in turn provides output light with a time-varying intensity profile. In yet another example, instead of including an integrated or external modulator, the light source includes a laser with a gain medium into which a pump current is controllably injected to impart a time-varying intensity profile to the output light. In some embodiments, the light source, optical amplifier, and modulator are provided by a sampled-grating distributed Bragg reflector (SG-DBR) laser.

[0053] In another example, the optical transmitter 101 may include a broadband light source and one or more tunable spectral filters to provide substantially continuous-wave (CW) optical intensity at a selected wavelength or wavelengths. In another example, the optical transmitter 101 may include multiple laser diodes, each tunable over a respective range, whose respective outputs are combined to form a single output. The respective outputs may be combined using a wavelength combiner, such as an optical splitter or an arrayed waveguide grating (AWG).

[0054] Optical transmitter 101 or optical transmitter 201 may be controllable to provide 10 Gbps modulation, operate over a 35 nm wavelength range, and change from one wavelength channel to another in less than 500 ns, 200 ns, or 100 ns. The wavelength channels may have center frequencies spaced about 1 GHz or more apart.

[0055] 3 illustrates an example arrangement of a sensor head 301, which may form, for example, the sensor head 103 of the spatial profiling system 100 described with reference to FIG. 1. The sensor head 301 includes an optical circulator 302 and a beam director 303, which includes a fast-axis beam director (e.g., wavelength-based beam director 304) and a slow-axis beam director (e.g., mechanical beam director 305). Generally, the fast-axis beam director is configured to direct the beam along a first axis (the "fast axis") at a higher speed than the slow-axis beam director, and the slow-axis beam director is configured to direct the beam along a second axis (the "slow axis") that is orthogonal or substantially orthogonal to the first axis. As shown in FIG. 3, the beam director 303 may be downstream of the optical circulator 302 in the output light direction. Alternatively, the optical circulator 302 may be downstream of the beam director 303 in the output light direction. Furthermore, in the beam director 303, the fast axis beam director may be downstream of the slow axis beam director in the output light direction. Alternatively, in the beam director 303, the fast axis beam director may be downstream of the slow axis beam director in the output light direction.

[0056] In some embodiments, optical circulator 302 may be omitted. Separation of output optical path P1 and input optical path P2 (see FIG. 1) may be performed by a 2x1 optical coupler. Additionally, as previously mentioned, in some embodiments, only wavelength beam steering or only mechanical beam steering may be performed by a beam director. Additionally, combined wavelength and mechanical beam steering may be performed through mechanical movement of components for wavelength-based beam steering.

[0057] The blocks in Figures 1-3 represent functional components of spatial profiling system 100. The functions may be provided by different physical components or integrated physical components. For example, the photodetector may be separate or integrated with the analog-to-digital converter (ADC). In another example, the optical circulator 302 (or optical coupler) and the wavelength-based beam director 304 may be separate physical components or a single integrated component.

[0058] Figure 4 shows a sensor head component 400, such as, for example, components of sensor head 301 of Figure 3, which may form part of the spatial profiling system of Figure 1. The components of Figure 4 are described in this context below. This component includes a wavelength router 401, which may form all or part of wavelength-based beam director 304 of Figure 3, and an optical circulator 401, which may be optical circulator 302 of Figure 3. As previously mentioned, a 2x1 optical coupler may be used in place of the optical circulator.

[0059] Wavelength router 400 may include or be an arrayed waveguide grating (AWG) or an Echelle grating or a photonic lantern. An AWG may be fabricated as an integrated circuit chip, for example, in Si, SiO2, or SiN. Those skilled in the art will understand that descriptions herein referring to an AWG are applicable to an Echelle grating or a photonic lantern without minor modification, all of which may, for example, distinguish higher-order modes in the returned light from lower-order or fundamental modes.

[0060] Wavelength router 400 includes an input slab 402, an output slab 403, and a waveguide array 404. Because this wavelength router is a bidirectional component, the terms input and output are used herein with reference to output light path P1. For input light path P2, output slab 403 effectively acts as the input slab of the AWG. Waveguide array 404 contains waveguides of different lengths, thereby creating the interference pattern of the AWG. Wavelength router 400 also includes an array 405 of single-mode optical fibers distributed across input slab 402.

[0061] Optical fiber 407 of array of single mode optical fibers 405 forms part of output light path P1 and receives output light from optical circulator 401. Optical circulator 401 receives output light L1 through optical path 408, which may be free space or guided optical components. Optical fiber 407 is connected to a central location of input slab 402. The other optical fibers of array of single mode optical figures 405 are positioned symmetrically across input slab 402 with respect to optical fiber 407.

[0062] The emitted light L1 is from a light source 102, which emits selectable wavelengths λ1 to λ N For example, the output light L1 can be tunable from λ1 to λ2 to cover the wavelength dimension. NThe light source may cycle through each of the wavelength channels. The light source may vary the wavelength continuously between the wavelength channels or may include a step change in wavelength between the wavelength channels. Interference within the output slab 403 caused by the waveguide array 404 causes different wavelengths to exit the output slab 403 at different angles. This angular difference may be used for beam steering. In some embodiments, at least one lens or other suitable optical component is provided downstream of the output slab 403 in the output light path P1, such as a collimating lens to collimate the output light and / or a lens to increase the field of view by increasing the angular difference and / or a polarization wave-plate.

[0063] In an ideal scenario without the effects of speckle, the propagation of reflected light in incident light L2 through wavelength router 400 would result in the reflected light being imaged back to where the light originated, i.e., at optical fiber 407. In a practical scenario, the reflected light in incident light L2 is speckled (diffused), and the propagation of reflected light in the return light through wavelength router 400 results in the reflected light being imaged as a diffused field at input slab 402. Some of the reflected light is received by optical fiber 407, and some of the reflected light is received by other optical fibers in array of single-mode optical fibers 405. Thus, an image is formed across array of single-mode optical fibers 405, and the image is formed by interfering signals in input slab 402. This image may therefore be described as an interference pattern. Such an image or interference pattern represents a spatial sample of the surface or object that reflected the light. In this manner, the target (i.e., the part of the environment that reflects the outgoing light) is spatially sampled and can be used to detect or mitigate speckle effects.

[0064] Thus, each of the array of single-mode optical fibers 405 provides a return signal R-1 through R-7. The return signals R-1 through R-7, which resolve or deconstruct the speckle effect, are referred to herein as "despeckled" signals. In one example, the despeckled signal includes a fundamental mode signal and a set of one or more higher-order mode signals. A plurality of these despeckled signals may be provided to the optical receiver 104 for detection, such as the coherent detection discussed above. The optical detector 106 may be configured to detect specularity of the return signals, such as detecting an image or interference pattern related to speckle. In the case of coherent detection, the optical detector 106 may further be configured to retrieve or provide a measure of the amplitude and phase of each despeckled signal. In other words, the optical detector 106 may be configured to detect specularity based on the amplitude and phase of the incident light in a spatially resolved manner. While this example shows seven fibers, there may be more or fewer fibers, with corresponding changes in receiver channels, ADCs, and processing resources. In some embodiments, the photodetector 106 is further configured to determine the state of polarization of the returned light, e.g., any one or more of the despeckle signals R-1 through R-7. For example, the photodetector 106 may include one or more polarizers for each of the despeckle signals R-1 through R-7. Determining the state of polarization provides an indication of the degree of polarization of the returned light, e.g., how well the degree of polarization of the returned light was preserved upon reflection from a surface or object. Furthermore, based on the state of polarization of each of the despeckle signals R-1 through R-7, the degree of polarization of the returned light may be determined in a spatially resolved manner, facilitating the indication of material characteristics of the surface or object. In particular, the degree of polarization of the returned light may be determined relative to the degree of polarization of the output light or the local oscillator.

[0065] In some embodiments, each of the despeckle signals R-1 through R-7 may be converted by optical receiver 104 to an electrical signal, which may then be converted to a digital signal, such as by ADC 107. Thus, signal S2 in FIG. 1 may be considered a composite signal including signal components corresponding to each of the despeckle signals R-1 through R-7. Signal processing, such as by processing and control system 105, may then combine the digital signals representing the despeckle signals R-1 through R-7 to reduce speckle effects. Additionally, the characteristics of the despeckle signals R-1 through R-7, such as one or more of spatially resolved amplitude, phase, and polarization, may be used to characterize or classify the incident light L2 and thus provide input for characterizing or classifying the surface producing at least the reflected light component of the incident light L2.

[0066] 5 partially illustrates a photodetector 501, such as, for example, photodetector 106 of FIG. 1, for use with an optical transmitter 201 including a polarizer 203 and a sensor head component 400. The photodetector 501 may be used in the spatial profiling system 100 of FIG. 1. The photodetector 501 may include a polarization-diverse photodetector configured for coherent detection, such as, for example, in-phase and quadrature (IQ) demodulation.

[0067] Optical detector 501 receives an LO signal, such as from optical splitter 204. The LO signal may be further split by optical splitter 502 to provide N LO signals, where N is the number of return signals for detection. In the example of Figures 4 and 5, N is 7. In other embodiments, N may be less or greater.

[0068] The despeckle signal R-1 and the LO signal are polarized by first and second polarizers 503 and 504, respectively. In one example, the first and second polarizers 503 and 504 are configured to polarize light in orthogonal polarization orientations, e.g., corresponding to the polarized light from polarizer 203, respectively. In other words, the first polarizer 503 has a polarization orientation corresponding to one polarization of the polarized light, and the second polarizer 505 has a polarization orientation corresponding to the other polarization of the polarized light. In another example, the first and second polarizers 503 and 504 have aligned polarization orientations, e.g., both corresponding to the polarized light from polarizer 203, respectively, and both corresponding to the polarized light from polarizer 203, respectively.

[0069] The polarized light from each of the first and second polarizers 503, 504 is provided to both a first mixer 505 and a second mixer 506. Each of the first and second mixers 505, 506 generates a mixed signal. This mixed signal is provided to a first and second photodetector 509, 510, respectively, which generate electrical signals S1-1 and S2-2, respectively. The electrical signals S1-1 and S2-2 convey information for determining the state of polarization of the returned light, such as the degree of polarization, which includes the degree to which the polarization state of the returned light is preserved.

[0070] The despeckle signal R-2 is provided to a third mixer 507 along with the LO signal. The third mixer generates a mixed signal for detection by a third photodetector 510, which generates an electrical signal S1-3. The other despeckle signals R-3 and onward are similarly mixed with the LO signal, including despeckle signals R-7, which are mixed by an eighth mixer 508 to generate electrical signals S1-8. Each of signals S1-1 through S1-8 includes a beat frequency resulting from the mixing. Signals S1-1 through S1-8 form components of signal S1 in FIG. 1. These signals are provided from the ADC 107 to the processing and control system 105.

[0071] In the embodiment shown in Figure 5, the despeckle signals R-2 through R-7 are not polarized. In other words, their corresponding electrical signals S1-3 through S1-8 are therefore not polarization-resolved. However, in other embodiments, some or all of these despeckle signals are similarly polarized in the same manner as the return signal R-1 and mixed in a pair of mixers, thereby doubling the signal content of S1 due to the presence of the despeckle signals. In these other embodiments, the electrical signals S1-3 through S1-8 are polarization-resolved.

[0072] 5 is configured for use with polarized output light. When the output light has a single polarization, the second polarizer 504, the second mixer 506, and the second photodetector 510 may be omitted. In these embodiments, the first mixer 1 receives the LO signal from the optical splitter 502.

[0073] The despeckle signals R-1 to R-7 and the subsequent signals S1 and S2, particularly the component portions of S1, provide information usable for spatial profiling decisions, such as by the processing and control system 105. Two different reflective surfaces may result in different responses of the return signals to the outgoing polarized light. For example, the different responses may include different detected polarization states. The difference in the detected polarization states may be based on a different magnitude ratio between signals S1-1 and S1-2, which conveys information for determining the polarization state of any one or more of the despeckle signals (e.g., R-1). Similarly, two different reflective surfaces may result in different speckle responses of the return signals to the outgoing light. For example, the different responses may include different detected specularities. The difference in detected specularities may be based on a magnitude ratio between signal S1-1 and any one of signals S1-3 to S1-8, or between signal S1-2 and any one of signals S1-3 to S1-8. These differences are detectable in the component portions of S1.

[0074] In some embodiments, the absolute and / or relative intensities of or between the detected polarized light are used to distinguish surfaces. For example, metal may have a high degree of polarization preservation, while brick, wood, and leaves may have a low degree of polarization preservation, and textiles may have an intermediate degree of polarization preservation. In another example where coherent detection is used, the phase delay between two (e.g., orthogonal) polarizations detected by the photodetector may be used to distinguish surfaces. The use of the detected polarization state may apply only to the despeckle signal R-1 or may be extended to one or more of the despeckle signals R-2 through R-7. In another example, the relative magnitudes of two or more of the despeckle signals R-1 through R-7 may be used to distinguish surfaces.

[0075] In some embodiments, detected specularity is used to distinguish surfaces of different materials. The processing and control system 105 may be configured to make a decision based on the detected specularity to classify surfaces into different categories. The detected specularity may include any one of speckle contrast, speckle granularity, and speckle anisotropy. For example, speckle contrast may be determined based on the standard deviation of intensity normalized by the mean intensity or the severity of the speckles. For example, speckle granularity may be determined based on the distribution of speckles at different particle sizes. For example, speckle anisotropy may be determined based on directional inhomogeneity or whether the particles are elongated in a particular direction.

[0076] In some embodiments, one or more features derived from detected polarization are used in combination with one or more features derived from detected specularity to distinguish between surfaces of different materials. Furthermore, in some embodiments, polarization and / or specularity are used in conjunction with additional information, such as information about the position within the field of view and / or determinations made on areas adjacent to the surface within the field of view.

[0077] In some embodiments, an associated processing system, such as processing and control system 105, uses a lookup process to distinguish between surfaces of different materials. For example, the measured polarization state(s) and / or specularity may be matched to a lookup table, with each row of the table having a unique combination of polarization state(s) and / or specularity and surface category. The surface category may be specific, such as "wood" or "highly reflective," or non-specific, such as "Category 1." It will be appreciated that the surface category may be used for decisions and / or actions, such as by a control system of an autonomous or semi-autonomous vehicle.

[0078] In some embodiments, particularly but not exclusively, in embodiments where there are two or more inputs to a process for distinguishing surfaces of different materials, an associated processing system, such as processing and control system 105, determines the material category based on prior machine learning of the relationship between the inputs and the surface category. The machine learning may be supervised or unsupervised. The machine learning algorithm may include the use of artificial neural networks or other machine learning algorithms.

[0079] In some embodiments, determining the material category includes classifying the surface into one of a plurality of material categories based on the detected polarization state and the detected specularity, for example, an associated processing system configured to apply a machine learning algorithm, such as a support vector machine (SVM), a K-nearest neighbors algorithm (k-NN), or a decision tree.

[0080] 6 shows an example of a processing system 600 for classifying surfaces of different materials based on a machine learning framework. The processing system 600 includes a machine learning model 602, inputs for receiving one or more specularity parameters 604 and one or more polarization parameters 606, and an output for material classification 608. The specularity parameter(s) 604 may include relative weights or ratios associated with one or more of the despeckle signals R-1 through R-7. The relative weights or ratios may be based on the magnitude (e.g., power, intensity, or amplitude) of one or more despeckle signals. Alternatively or additionally, the relative weights or ratios may be based on the phase of one or more despeckle signals. For example, in a spatial profiling system configured to generate three despeckle signals R-1, R-2, and R-3, where R-1 is associated with the fundamental mode and R-2 and R-3 are associated with higher-order modes, if the reflective surface is a smooth material (e.g., metal), the received power for R-1 is expected to be substantially higher than the received power for R-2 and R-3. In this case, the specularity parameter(s) 604 may be characterized by a 90:5:5 ratio, indicating that 90% of the total received power is due to the sole contribution of despeckle signal R-1 and 10% of the total received power is contributed equally by despeckle signals R-2 and R-3 (or a 5% contribution each). The 90:5:5 ratio may alternatively be represented by the relative weights of 90, 5, and 5 for each despeckle signal. Conversely, if the reflective surface is a rough material (e.g., brick), the received power for R-1 is expected to be equivalent to the received power for R-2 and R-3. In this case, the specularity parameter(s) 604 may be characterized by a ratio of 40:30:30, indicating that 40% of the total received power is due to the sole contribution of despeckled signal R-1, and 60% of the total received power is contributed equally by despeckled signals R-2 and R-3 (or a 30% contribution each). The 40:30:30 ratio may alternatively be represented by the relative weights of 40, 30, and 30 for the respective despeckled signals R-1, R-2, and R-3.The polarization parameter(s) 606 may include one or more Stokes parameters S0, S1, S2, and S3, collectively commonly known as a Stokes vector, each generally ranging from -1 to +1, although each may be scaled by any factor.

[0081] In some embodiments, the processing system 600 is trained by a training method based on training and validation data sets. The training method may include obtaining the training and validation data sets through experimental observations. FIG. 7 shows an example of an experimental setup 700 for obtaining experimental observations. The experimental setup 700 includes a spatial profiling system 100, a known material under test (test material) 702, and data storage 704. The experimental setup 700 is configured to measure one or more sets of specularity parameter(s) and polarization parameter(s) associated with several different test materials via the spatial profiling system 100 one or more times. The experimental setup 700 is further configured to store each set of measured specularity parameter(s) and polarization parameter(s) in the data storage 704. In one example, the training method includes obtaining a total of 30,208 sets of measurements for four different test materials and separating the sets of measurements into a training dataset of 24,224 measurement sets and a validation dataset of 6,056 measurement sets. The training method further includes adapting the obtained datasets based on one or more machine learning models, for example, according to a standard machine learning framework (e.g., sklearn). Adapting the obtained datasets may include iteratively determining a value of an optimization function (e.g., based on classification accuracy). At each iteration of determining the value of the optimization function, the machine learning model may be adjusted, for example, by increasing or decreasing the values ​​of one or more parameters of the machine learning model to reach a different value of the optimization function. The iterative determination of the value of the optimization function may be stopped in response to a predetermined maximum or minimum value being achieved (e.g., a set classification accuracy being achieved).

[0082] 8A and 8B show performance matrices of a processing system 600 trained according to the disclosed training method under each of two machine learning models. The different test materials include a black painted panel material, a woven material, a 90% diffuse material, and a wood material. The machine learning models include a linear classifier, such as logistic regression (FIG. 8A). Alternatively, the machine learning models include a nonlinear classifier, such as a decision tree (FIG. 8B). Both machine learning models result in an accuracy of 99.9% or greater in classifying the four different test materials. FIG. 9 shows the accuracy of a processing system trained under different additional machine learning models. It can be seen that each of the machine learning models, logistic regression, linear discriminant analysis, K-nearest neighbors, decision tree, and Gaussian Native-Bayes, results in an accuracy of 99% or greater.

[0083] In one embodiment, the processing system 600 is configured to classify materials based on a single specularity parameter 604 and a single polarization parameter 606. The single specularity parameter 604 may be related to the relative magnitude of one despeckle signal. The single polarization parameter 606 may be related to one Stokes parameter. In this example, the spatial profiling system 100 is configured to measure the received power weight (R1-weight) of the despeckle signal R1 relative to all other despeckle signals. This relative weight is a single number between 0 and 1. Additionally, the spatial profiling system 100 is configured to measure the Stokes parameter S1 (R1-S1) of the despeckle signal R1. The Stokes parameter S1 is a single real value (negative or positive). The R1-weight and R1-S1 values ​​are measured multiple times for four different materials (wood material, white diffuse material, black panel material, and textile material). FIG. 10 shows how different materials are associated with this single specularity parameter and this single polarization parameter. Figure 10 shows a clear separation of clusters, each representing one of the test materials, based on a single specularity parameter and a single polarization parameter. This clear separation of clusters corresponds to a high degree of accuracy of machine learning-based classification based on a single specularity parameter and a single polarization parameter. In addition, Figure 10 suggests that when either a single specularity parameter or a single polarization parameter is used to classify the test materials, the separation of the clusters will be lost, and that machine learning-based classification based on either single parameter will not be as accurate.

[0084] It will be understood that the invention disclosed and defined in this specification extends to all alternative combinations of two or more of the individual features mentioned or made apparent by the text or drawings, all of which different combinations constitute various alternative aspects of the invention.

Claims

1. A spatial profiling system for profiling an environment, wherein the spatial profiling system is A light transmitter for providing light, A beam director for directing the light toward one or more directions toward the environment, A light receiver for receiving reflected light from a surface or object in the environment, wherein the reflected light transmits information for determining the distance to the surface or object, and the light receiver is configured to detect (a) the specularity of the reflected light and (b) the polarization state of the reflected light. A spatial profiling system comprising: a processing system configured to determine the material associated with the surface or the object based on the detected specularity and the detected polarization state.

2. The spatial profiling system according to claim 1, wherein the processing system is configured to determine the material associated with the surface or object by classifying the material into one of a plurality of material categories.

3. The spatial profiling system according to claim 2, wherein classifying the material into one of a plurality of material categories includes applying one or more machine learning algorithms to the classification.

4. The spatial profiling system according to any one of claims 1 to 3, wherein the optical receiver is further configured to detect specularity based on an image or interference pattern related to speckle.

5. The spatial profiling system according to claim 4, wherein the image or interference represents a spatial sample of the surface or the opposite surface that reflected light.

6. The spatial profiling system according to any one of claims 1 to 3, wherein the optical receiver is further configured to detect specularity based on a plurality of speckle-removed signals.

7. The spatial profiling system according to claim 6, wherein the optical receiver is further configured to retrieve or provide one or more amplitude and / or phase measures of the plurality of speckle-removed signals.

8. The spatial profiling system according to claim 7, wherein the specularity is characterized by one or more specularity parameters, and each specularity parameter is associated with the amplitude of one or more of the plurality of speckle-removed signals.

9. The spatial profiling system according to claim 8, wherein the polarization state is characterized by one or more Stokes parameters associated with one of the one or more speckle-removing signals.

10. The spatial profiling system according to any one of claims 1 to 3, wherein the processing system is further configured to determine one of speckle contrast, speckle grain size, and speckle anisotropy based on the detected specularity.

11. The spatial profiling system according to claim 10, wherein the processing system is further configured to determine the material associated with the surface or the object based on one or more of the determined speckle contrast, speckle grain size, and speckle anisotropy.

12. The spatial profiling system according to any one of claims 1 to 3, wherein the optical receiver is further configured to detect the polarization state based on the degree to which the polarization state is maintained.

13. The spatial profiling system according to claim 12, wherein the degree of retention of the polarization state represents the polarization degree of the reflected light with respect to the polarization degree of the emitted light or the local oscillator.

14. The spatial profiling system according to claim 12, wherein the processing system is further configured to determine the material associated with the surface or the object based on the degree of retention of the polarization state.

Images