Controlling the automated driving vehicle based on the determined yaw parameter of the additional vehicle

Abstract

The present invention relates to controlling an autonomous vehicle based on determined yaw parameters of an additional vehicle. A yaw parameter (e.g., at least one yaw rate) of an additional vehicle that is additional to a vehicle being autonomously controlled is determined, and autonomous control of the vehicle is adapted based on the determined yaw parameter of the additional vehicle. For example, autonomous steering, acceleration, and / or deceleration of the vehicle may be adapted based on the determined yaw rate of the additional vehicle. In many embodiments, the yaw parameter of the additional vehicle is determined based on data from a phase coherent light detection and ranging (LIDAR) component of the vehicle, such as a phase coherent LIDAR monopulse component and / or a frequency modulated continuous wave (FMCW) LIDAR component.

Description

[0001] This application is a divisional application of an application which entered the Chinese national phase on September 30, 2020, with PCT application number PCT / US2019 / 024672, international application date March 28, 2019, Chinese application number 201980024410.0, and invention name “Controlling an autonomous driving vehicle based on the determined yaw parameters of an additional vehicle”. Background Art

[0002] As computing and vehicle technology continue to evolve, autonomous-related features have become more powerful and widely available, and are able to control vehicles in a wide variety of situations. For automobiles, for example, the Society of Automotive Engineers (SAE) has established a standard (J3016) that identifies six driving autonomous driving levels from "no autonomous driving" to "full autonomous driving". The SAE standard defines Level 0 as "no autonomous driving" where the human driver fully performs all aspects of the dynamic driving task even when augmented by a warning or intervention system. Level 1 is defined as "driver assistance" where the vehicle controls steering or acceleration / deceleration (but not both) in at least some driving modes, thereby letting the operator perform all remaining aspects of the dynamic driving task. Level 2 is defined as "partial autonomous driving" where the vehicle controls steering and acceleration / deceleration in at least some driving modes, thereby letting the operator perform all remaining aspects of the dynamic driving task. Level 3 is defined as "conditional autonomous driving" where the autonomous driving system performs all aspects of the dynamic driving task for at least some driving modes while expecting the human driver to respond appropriately to the intervention request. Level 4 is defined as "highly autonomous driving" where the autonomous driving system performs all aspects of the dynamic driving task for only certain conditions even if the human driver does not respond appropriately to the intervention request. Certain conditions at Level 4 may be, for example, certain types of roads (e.g., highways) and / or certain geographic areas (e.g., a metropolitan area that has been appropriately mapped). Finally, Level 5 is defined as "full self-driving" where the vehicle is able to operate without operator input in all conditions.

[0003] The fundamental challenges of any autonomy-related technology involve collecting and interpreting information about the vehicle's surroundings, and planning and executing commands to appropriately control vehicle motion to navigate the vehicle safely through its current environment. As a result, there has been a constant effort to improve each of these aspects, and by doing so, autonomous vehicles are increasingly able to operate reliably in increasingly complex environments and adapt to both expected and unexpected interactions within the environment. Summary of the invention

[0004] In some embodiments, a method for adapting autonomous control of an autonomous vehicle based on a determined yaw parameter of an additional vehicle is provided. The method includes receiving a group of phase-coherent light detection and ranging (LIDAR) data points of a sensing period of a phase-coherent LIDAR component from a phase-coherent light detection and ranging (LIDAR) component of the vehicle. Each of the phase-coherent LIDAR data points of the group indicates a corresponding range and a corresponding speed of a corresponding point in an environment, and each of the phase-coherent LIDAR data points of the group is generated based on a corresponding sensing event of the phase-coherent LIDAR component during the sensing period. The method further includes determining that a subgroup of the phase-coherent LIDAR data points of the group corresponds to an additional vehicle other than the vehicle. The method further includes determining a yaw parameter of the additional vehicle based on a plurality of phase-coherent LIDAR data points in the subgroup based on determining that the subgroup corresponds to the additional vehicle. The method further includes adapting autonomous control of the vehicle based on the determined yaw parameter of the additional vehicle.

[0005] These and other implementations of the techniques described herein can include one or more of the following features.

[0006] In some embodiments, determining that a subset of the group of phase-coherent LIDAR data points corresponds to an additional vehicle includes: processing the group of phase-coherent LIDAR data points using a trained machine learning model; generating an output indicating that the subset has a vehicle classification based on the processing of the group of phase-coherent LIDAR data points using the trained machine learning model; and determining that the subset corresponds to an additional vehicle based on the output indicating that the subset has a vehicle classification.

[0007] In some embodiments, determining a yaw parameter of an additional vehicle based on a plurality of phase-coherent LIDAR data points in a subset includes: determining a first set of one or more of the plurality of phase-coherent LIDAR data points of the subset; determining a second set of one or more of the plurality of phase-coherent LIDAR data points of the subset based on the second set being spatially offset from the first set; and determining the yaw parameter based on a comparison of the first set with the second set. In some versions of those embodiments, determining that the first set and the second set are spatially offset relative to each other is based at least in part on corresponding ranges indicated by the first set and the second set. In some additional or alternative versions of those embodiments, determining a yaw parameter of an additional vehicle based on a comparison of the first set with the second set includes: determining the yaw parameter based on a comparison of a first set speed magnitude of the first set with a second set speed magnitude of the second set. The first set speed magnitude is based on a corresponding speed of the first set, and the second set speed magnitude is based on a corresponding speed of the second set. In various embodiments, the yaw parameter of the additional vehicle may include a yaw rate of the additional vehicle, and determining the yaw parameter of the additional vehicle based on the comparison of the first set with the second set may further include: determining a distance based on a spatial offset between the first set and the second set; and determining the yaw rate based on a comparison of the first set speed magnitude with the second set speed magnitude and the distance. Determining the yaw rate based on a comparison of the first set speed magnitude with the second set speed magnitude and the distance may include: determining a speed difference between the first set speed magnitude and the second set speed magnitude based on a comparison of the first set speed magnitude and the second set speed magnitude; and converting the speed difference into a yaw rate based on the distance. In various embodiments, the yaw parameter of the additional vehicle may be a lower limit yaw rate of the additional vehicle, and the method may further include determining an additional yaw parameter of the additional vehicle, the additional yaw parameter being an upper limit yaw rate and determined based on the comparison between the first set and the second set. For example, the method may further include determining a speed difference between the first set speed magnitude and the second set speed magnitude based on a comparison of the first set speed magnitude and the second set speed magnitude, and determining the lower limit yaw rate may include dividing the speed difference by the distance, and determining the upper limit yaw rate may include dividing the speed difference by a value that is based on the distance, but the magnitude of the value is reduced relative to the distance. In various embodiments of determining the first set and the second set of phase-coherent LIDAR data points, the first set includes a plurality of the plurality of phase-coherent LIDAR data points, and the first set speed magnitude is determined based on corresponding speeds of a plurality of the plurality of phase-coherent LIDAR data points. For example, the first set speed magnitude may be based on an average of corresponding speeds of a plurality of the plurality of phase-coherent LIDAR data points.In various embodiments in which the yaw parameter includes a yaw rate and in which determining the yaw parameter of the additional vehicle is based on a comparison of the first set with the second set, determining the yaw parameter includes: determining a speed difference based on a comparison of a first set speed magnitude with a second set speed magnitude; identifying a stored model of the additional vehicle; and converting the speed difference to a yaw rate based on the stored model of the additional vehicle. The first set speed magnitude may be based on a corresponding speed of the first set, and the second set speed magnitude may be based on a corresponding speed of the second set. In some of those various embodiments, identifying the stored model of the additional vehicle includes selecting a stored model from a plurality of candidate stored models based on the subset of phase coherent LIDAR data points.

[0008] In various embodiments, the yaw parameter includes a yaw rate and a yaw direction. In some of those embodiments, determining the yaw parameter of the additional vehicle based on the subset of the plurality of phase-coherent LIDAR data points includes: processing the subset of the plurality of phase-coherent LIDAR data points using a trained machine learning model; generating an output indicative of a yaw rate based on processing the subset of phase-coherent LIDAR data points using the trained machine learning model; and determining the yaw rate and the yaw direction based on the output. In some versions of those embodiments, processing the subset of the plurality of phase-coherent LIDAR data points using the trained machine learning model includes processing all of the subset of phase-coherent LIDAR data points using the trained machine learning model.

[0009] In some embodiments, the yaw parameter of the additional vehicle is a speed difference indicative of a yaw rate. In some of those embodiments, adapting autonomous control of the vehicle based on the determined yaw parameter of the additional vehicle comprises adapting autonomous control based on the speed difference exceeding a threshold.

[0010] In some embodiments, adapting autonomous control of the vehicle based on the determined yaw parameter of the additional vehicle includes changing a speed of the vehicle and / or a direction of the vehicle. In some of those embodiments, the yaw parameter is a yaw rate, and adapting autonomous control of the vehicle based on the determined yaw rate of the additional vehicle is responsive to determining that the determined yaw rate satisfies a threshold.

[0011] In some embodiments, adapting autonomous control of the vehicle based on the determined yaw parameter of the additional vehicle includes: determining at least one candidate trajectory for the additional vehicle based on the determined yaw parameter; and adapting autonomous control of the vehicle based on the at least one candidate trajectory.

[0012] In some embodiments, the set of phase-coherent LIDAR data points includes a three-dimensional point cloud.

[0013] In some embodiments, the phase-coherent LIDAR assembly is a phase-coherent LIDAR monopulse assembly. In some of those embodiments, the corresponding sensed events of the phase-coherent LIDAR assembly each include a first receiver sensed event at a first receiver of the phase-coherent LIDAR monopulse assembly and a second receiver sensed event at a second receiver of the phase-coherent LIDAR monopulse assembly. In some versions of those embodiments, the corresponding sensed events of the phase-coherent LIDAR assembly further include a third receiver sensed event at a third receiver of the phase-coherent LIDAR monopulse assembly.

[0014] In some embodiments, the phase coherent LIDAR assembly is a frequency modulated continuous wave (FMCW) LIDAR assembly.

[0015] In some embodiments, a method for adapting autonomous control of an autonomous vehicle based on a determined yaw rate of an additional vehicle is provided, and the method includes receiving phase-coherent LIDAR data capturing an environment of the vehicle from a phase-coherent LIDAR component of the vehicle. For each of a plurality of points in the environment of the vehicle, the phase-coherent LIDAR data indicates at least one corresponding range and at least one corresponding speed based on a corresponding sensed event of the phase-coherent LIDAR component. The method further includes determining that a subset of the phase-coherent LIDAR data corresponds to an additional vehicle other than the vehicle in the environment. The method further includes determining a yaw rate of the additional vehicle and adapting autonomous control of the vehicle based on the determined yaw rate of the additional vehicle. Determining the yaw rate of the additional vehicle is based on a plurality of corresponding speeds indicated by the subset of phase-coherent LIDAR data.

[0016] These and other implementations of the techniques described herein can include one or more of the following features.

[0017] In some embodiments, determining that a subgroup of the phase-coherent LIDAR data corresponds to an additional vehicle includes: processing the phase-coherent LIDAR data using a trained machine learning model; generating an output indicating that the subgroup has a vehicle classification based on processing the phase-coherent LIDAR data using the trained machine learning model; and selecting the subgroup based on the output indicating that the subgroup has a vehicle classification.

[0018] In some embodiments, determining the yaw rate of the additional vehicle includes: determining a velocity gradient based on a plurality of corresponding velocities indicated by the subset of phase-coherent LIDAR data; and determining the yaw rate based on the velocity gradient.

[0019] In some embodiments, adapting autonomous control of the vehicle based on the determined yaw rate of the additional vehicle comprises changing one or both of the speed of the vehicle and the direction of the vehicle.

[0020] In some embodiments, the phase coherent LIDAR data includes: a range Doppler image, a three-dimensional point cloud, and / or an intermediate frequency waveform. The intermediate frequency waveform can be generated based on mixing of a local optical oscillator with time-delayed reflections during a corresponding sensing event.

[0021] In some embodiments, the phase-coherent LIDAR assembly is a phase-coherent LIDAR monopulse assembly comprising at least a first coherent receiver and a second coherent receiver. In some of those embodiments, the corresponding sensing events of the phase-coherent LIDAR assembly each comprise a first receiver sensing event at the first coherent receiver of the phase-coherent LIDAR monopulse assembly and a second receiver sensing event at the second coherent receiver of the phase-coherent LIDAR monopulse assembly (and optionally further receiver sensing events at further receivers).

[0022] In some embodiments, the phase coherent LIDAR component is a FMCW LIDAR component.

[0023] In some embodiments, a method for adapting autonomous control of an autonomous vehicle based on determined yaw parameters of an additional vehicle is provided, and the method includes receiving phase-coherent LIDAR single-pulse data that captures an environment of the vehicle from a phase-coherent LIDAR single-pulse assembly of the vehicle. The phase-coherent LIDAR single-pulse assembly includes a laser source and at least a first coherent receiver and a second coherent receiver. For each of a plurality of points in the environment of the vehicle, the phase-coherent LIDAR single-pulse data indicates at least one corresponding range and at least one corresponding speed based on a corresponding sensing event of the phase-coherent LIDAR single-pulse assembly. The corresponding sensing events of the phase-coherent LIDAR assembly each include a first receiver sensing event performed by a first receiver and a second receiver sensing event performed by a second receiver. The method further includes determining a yaw parameter of an additional vehicle in the environment of the vehicle based on a plurality of corresponding speeds indicated by the phase-coherent LIDAR single-pulse data. The method further includes adapting autonomous control of the vehicle based on the determined yaw parameters of the additional vehicle.

[0024] These and other implementations of the technology disclosed herein may optionally include one or more of the following features.

[0025] In some embodiments, determining a yaw parameter of an additional vehicle based on phase-coherent LIDAR monopulse data includes: determining a first set of phase-coherent LIDAR monopulse data; determining a second set of phase-coherent LIDAR monopulse data based on its spatial offset from the first set; and determining the yaw parameter based on a first set speed and a second set speed. The first set speed is based on at least one corresponding speed of the first set, and the second set speed is based on at least one corresponding speed of the second set. In some of those embodiments, the yaw parameter of the additional vehicle includes a yaw rate of the additional vehicle. In some versions of those embodiments, determining the yaw parameter of the additional vehicle based on a comparison of the first set speed and the second set speed further includes: determining a distance based on a spatial offset between the first set and the second set; and determining the yaw rate based on the first set speed, the second set speed, and the distance. In some of those versions, determining the yaw rate based on the first set speed, the second set speed, and the distance includes: determining a speed difference between the first set speed and the second set speed; and converting the speed difference into a yaw rate based on the distance.

[0026] In some embodiments, the yaw parameter includes a yaw rate, and determining the yaw parameter based on a plurality of corresponding speeds indicated by the phase coherent LIDAR monopulse data includes: identifying a stored model of an additional vehicle; and determining the yaw rate based on the plurality of corresponding speeds and based on the stored model of the additional vehicle. In some of those embodiments, determining the yaw rate based on the plurality of corresponding speeds and based on the stored model of the additional vehicle includes: determining a speed difference based on the plurality of corresponding speeds; and converting the speed difference to a yaw rate based on the stored model of the additional vehicle.

[0027] In some embodiments, the yaw parameter includes a yaw rate, and determining the yaw parameter of the additional vehicle based on the phase-coherent LIDAR single-pulse data includes: processing at least a portion of the phase-coherent LIDAR single-pulse data using a trained machine learning model; generating an output indicating the yaw rate based on processing at least a portion of the phase-coherent LIDAR single-pulse data using the trained machine learning model; and determining the yaw rate based on the output.

[0028] In some embodiments, the phase coherent LIDAR component is a FMCW LIDAR component.

[0029] In some embodiments, a method is provided that includes receiving phase coherent LIDAR data from a phase coherent LIDAR component of a vehicle that captures an environment of the vehicle. For each of a plurality of points in the environment of the vehicle, the phase coherent LIDAR data indicates at least one corresponding range and at least one corresponding speed based on a corresponding sensed event of the phase coherent LIDAR component. The method further includes determining that a subset of the phase coherent LIDAR data corresponds to an additional vehicle in the environment and attached to the vehicle. The method further includes determining a yaw parameter of the additional vehicle based on a plurality of corresponding speeds indicated by the subset of phase coherent LIDAR data. The method further includes providing the determined yaw parameter to at least one subsystem of the vehicle, such as a planning subsystem and / or a control subsystem of the vehicle.

[0030] In some embodiments, a method is provided that includes receiving phase-coherent LIDAR data from a phase-coherent component of a vehicle, the phase-coherent LIDAR data capturing an environment of the vehicle and indicating, for each of a plurality of points in the environment of the vehicle, at least one corresponding range and at least one corresponding speed based on a corresponding sensed event of the phase-coherent LIDAR component. The method further includes determining a yaw parameter of an additional vehicle in the environment of the vehicle based on a plurality of corresponding speeds indicated by the phase-coherent LIDAR data. The method further includes providing the determined yaw parameter to at least one subsystem of the vehicle, such as a planning subsystem and / or a control subsystem of the vehicle.

[0031] The two embodiments described above, as well as other embodiments of the technology disclosed herein, may include one or more of the following features.

[0032] In some embodiments, the yaw parameter includes a yaw rate, and determining the yaw parameter of the additional vehicle based on the phase-coherent LIDAR data includes processing at least a portion of the phase-coherent LIDAR data using a trained machine learning model; generating an output indicative of the yaw rate based on processing at least a portion of the phase-coherent LIDAR data using the trained machine learning model; and determining the yaw rate based on the output.

[0033] In some embodiments, the phase coherent LIDAR component is a phase coherent monopulse component and the phase coherent LIDAR data is phase coherent LIDAR monopulse data. In some additional or alternative embodiments, the phase coherent LIDAR component is a FMCW LIDAR component.

[0034] In some embodiments, a method for training a machine learning model is provided. The machine learning model is to be used for autonomous control of at least one autonomous vehicle, and the method includes generating a plurality of training instances. Generating each training instance includes: generating a training instance input for the training instance based on a corresponding instance of visual data from a visual component of the corresponding autonomous vehicle; and generating a supervised training instance output for the training instance based on a corresponding instance of additional vehicle data, the corresponding instance of additional vehicle data being based on one or more sensors of the corresponding additional vehicle captured by the corresponding instance of visual data. The corresponding instance of the additional vehicle data indicates a corresponding current state of at least one dynamic characteristic of the corresponding additional vehicle. The corresponding instance of the additional vehicle data is used to generate a supervised training instance output based on determining that the corresponding instance of the additional vehicle data corresponds in time to the corresponding instance of the visual data. The method further includes training the machine learning model based on the plurality of training instances, and providing the trained machine learning model for controlling at least one autonomous vehicle.

[0035] These and other implementations of the technology disclosed herein may optionally include one or more of the following features.

[0036] In some embodiments, the corresponding example of visual data is light detection and ranging (LIDAR) data, and the visual component is a LIDAR component. In some of those embodiments, the LIDAR component is a phase coherent LIDAR component, and the LIDAR data includes a set of LIDAR data points for a sensing period of the phase coherent LIDAR component. Each of the set of LIDAR data points indicates a corresponding range and a corresponding speed of a corresponding point in the environment, and each is generated based on a corresponding sensing event of the sensing period.

[0037] In some implementations, the supervised training instance output for the training instance further includes an indication of a restricted region that restricts a portion of the visual data captured corresponding to the additional vehicle.

[0038] In some embodiments, training the machine learning model includes: processing a training instance input of a training instance using the machine learning model to generate a predicted output; generating a loss based on the predicted output and a supervised training instance output of the training instance; and updating the machine learning model based on the generated loss.

[0039] In some embodiments, examples of additional vehicle data include one or more yaw parameters, and the one or more sensors corresponding to the additional vehicle include at least a yaw rate sensor. In some of those embodiments, the additional vehicle data is determined based on monitoring a controller area network corresponding to the additional vehicle.

[0040] In some embodiments, examples of additional vehicle data include one or both of speed and acceleration corresponding to the additional vehicle.

[0041] In some embodiments, examples of additional vehicle data include yaw parameters and one or both of velocity and acceleration.

[0042] In some embodiments, the method further includes: processing, by at least one processor of a given autonomous vehicle of at least one autonomous vehicle, given visual data (captured by a given visual component of the given autonomous vehicle) using a trained machine learning model; generating, based on the processing, a predicted state of a given additional vehicle captured by the given visual data; and controlling the given autonomous vehicle based on the predicted state. In some of those embodiments, the given visual data is a subset of visual data captured during a sensing cycle of the visual component, and the method further includes generating a subset of visual data based on determining that the subset corresponds to the given additional vehicle. In some versions of those embodiments, determining that the subset corresponds to the given additional vehicle includes: processing the visual data using an additional object detection and classification model; and determining that the subset corresponds to the given additional vehicle based on output generated from processing the visual data using the additional object detection and classification model.

[0043] In some embodiments, a method for controlling an autonomous vehicle using a trained machine learning model is provided. The method is implemented by one or more processors of the autonomous vehicle, and the method includes processing given visual data using the trained machine learning model, wherein the given visual data is captured by a visual component of the autonomous vehicle. The method further includes generating a predicted state of at least one dynamic characteristic of a given additional vehicle captured by the given visual data based on the processing. The method further includes controlling the autonomous vehicle based on the predicted state. The trained machine learning model is trained based on multiple training instances, each of which includes: a training instance input based on a corresponding instance of visual data from a visual component of the corresponding autonomous vehicle; and a supervised training instance output based on a corresponding instance of additional vehicle data, the corresponding instance of the additional vehicle data being based on one or more sensors of the corresponding additional vehicle captured by the corresponding instance of the visual data.

[0044] These and other implementations of the technology disclosed herein may include one or more of the following features.

[0045] In some embodiments, the given visual data is a subset of visual data captured during a sensing cycle of the vision component, and the method further includes generating the subset of visual data based on determining that the subset corresponds to a given additional vehicle. In some of those embodiments, determining that the subset corresponds to the given additional vehicle includes: processing the visual data using an additional object detection and classification model; and determining that the subset corresponds to the given additional vehicle based on an output generated from processing the visual data using the additional object detection and classification model.

[0046] In some embodiments, controlling the autonomous vehicle includes controlling the speed and / or direction of the autonomous vehicle.

[0047] In some embodiments, the at least one characteristic comprises a yaw rate and / or a yaw direction of the additional vehicle.

[0048] In some embodiments, the at least one characteristic includes a speed and / or acceleration of the additional vehicle.

[0049] In some embodiments, the at least one characteristic comprises a yaw parameter, a velocity and / or an acceleration of the additional vehicle.

[0050] In addition, some embodiments include an autonomous vehicle having one or more processors operable to execute instructions stored in an associated memory, and wherein the instructions are configured to cause any of the methods described herein to be performed. Some embodiments additionally or alternatively include one or more non-transitory computer-readable storage media storing computer instructions executable by one or more processors to perform any of the methods described herein.

[0051] It should be appreciated that all combinations of the foregoing concepts and additional concepts described in greater detail herein are contemplated as being part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are contemplated as being part of the subject matter disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 An example environment is illustrated in which implementations disclosed herein can be implemented.

[0053] Figure 2A Schematically illustrated Figure 1 Implementation of the FMCW LIDAR component.

[0054] Figure 2B Schematically illustrated Figure 1 Another implementation of an FMCW LIDAR assembly.

[0055] Figure 3An example additional vehicle is illustrated along with the additional vehicle's yaw axis, pitch axis, and roll axis.

[0056] Figure 4 is a block diagram illustrating an embodiment of adapting autonomous control of a vehicle based on a determined yaw parameter of an additional vehicle.

[0057] Figure 5 , Figure 6 , Figure 7 as well as Figure 8 Each illustrates an example environment of a vehicle in which the environment includes an additional vehicle, and each also illustrates an example of some FMCW LIDAR data points that may correspond to the additional vehicle in the environment.

[0058] Fig. 9 An example of how candidate trajectories for additional vehicles can be determined based on the determined yaw parameter and how autonomous control of the vehicle can be adapted based on the candidate trajectories for the additional vehicle is illustrated.

[0059] Fig.10 is a flow chart illustrating an exemplary method of determining a yaw parameter and controlling an autonomous vehicle based on the determined yaw parameter.

[0060] Fig.11 is a flow chart illustrating another exemplary method of determining a yaw parameter and controlling an autonomous vehicle based on the determined yaw parameter.

[0061] Fig.12 is a flow chart illustrating an example method for training a machine learning model that may be used to determine yaw rate based on FMCW LIDAR data. DETAILED DESCRIPTION

[0062] In various levels of autonomous control of a vehicle (e.g., any of the above-mentioned SAE Levels 1-5), it is necessary to determine various parameters of the so-called ego vehicle (i.e., the vehicle being autonomously controlled) - and adapt the autonomous control of the ego vehicle based on the various determined parameters of the autonomous vehicle itself. One such parameter is the yaw rate of the ego vehicle. The yaw rate of a vehicle is the angular velocity of the vehicle's rotation about the vehicle's yaw axis, typically measured in radians / second or degrees / second. For example, the yaw rate of the ego vehicle can be determined based on the output from a yaw rate sensor of the ego vehicle, such as a piezoelectric or micromechanical sensor that measures the Coriolis force of the ego vehicle. The yaw rate of the ego vehicle has been used in various vehicle control technologies, such as electronic stability control (ESC) of the ego vehicle. In ESC, the output from the yaw rate sensor and other sensors can be used to adapt, for example, the control of an anti-lock braking system (ABS), thereby mitigating traction loss / slip of the ego vehicle.

[0063] In various levels of autonomous control of an ego vehicle (also referred to hereinafter, for simplicity, as a "vehicle"), it may also be desirable and / or necessary to determine various parameters of additional vehicles that are in the environment of the ego vehicle and that are attached to the ego vehicle. For example, it may be necessary to determine the pose (position and optionally orientation) of the additional vehicle relative to the vehicle, and to determine how the additional vehicle is moving relative to the vehicle. For example, autonomous evasive braking and / or autonomous evasive steering of the vehicle may be necessary when the determination of the pose and movement of the additional vehicle may result in the additional vehicle being too close to the vehicle if the vehicle continues on its current intended trajectory.

[0064] Embodiments disclosed herein are generally directed to determining a yaw parameter of an additional vehicle, and adapting autonomous control of the vehicle based on the determined yaw parameter of the additional vehicle. For example, autonomous steering, acceleration and / or deceleration of the vehicle may be adapted based on the determined yaw parameter of the additional vehicle. The yaw parameter of the additional vehicle may include, for example, at least one yaw rate of the additional vehicle. The yaw rate may indicate the angular velocity of the additional vehicle's rotation around its yaw axis, and optionally indicate the direction of the rotation. The yaw parameter of the additional vehicle may additionally or alternatively include at least one speed difference indicating the magnitude of the speed difference between two parts of the additional vehicle, optionally together with an indication of the direction of rotation of the additional vehicle around its yaw axis. The yaw parameter may additionally or alternatively include an indication of whether the additional vehicle is experiencing a yaw movement (e.g., whether the additional vehicle is experiencing at least a threshold degree of movement around its yaw axis).

[0065] In many embodiments disclosed herein, the yaw parameter of the additional vehicle is determined based on data from a phase coherent light detection and ranging (LIDAR) component of the vehicle. The data from the coherent LIDAR component of the vehicle is referred to herein as "phase coherent LIDAR data". As described herein, the phase coherent LIDAR data can indicate at least the range of the point and the speed of the point for each of a plurality of points in the environment. The range of the point indicates the distance from the receiver of the phase coherent LIDAR component to the point. The speed of the point indicates the amplitude (and optionally the direction) of the speed of the point. The range and speed of the indicated point are based on the corresponding sensing events of the phase coherent LIDAR component. Therefore, by sensing the sensing events of the sensing cycle, the phase coherent LIDAR data indicating the instantaneous range and instantaneous speed of a large number of points in the environment can be determined. The embodiments disclosed herein can determine the instantaneous yaw parameter of the additional vehicle based on the instantaneous range and instantaneous speed indicated by at least some of the FMCW LIDAR data. Thus, the yaw parameters determined based on the phase-coherent LIDAR data of a sensing period provide yaw information for the additional vehicle based on the yaw motion of the additional vehicle during the sensing period without reference to phase-coherent LIDAR data from future sensing periods. This enables rapid resolution of the current yaw parameters of the additional vehicle, and as a result enables rapid determination of whether control of the vehicle needs to be adapted in view of the current yaw parameters (and enables rapid adaptation when it is determined that adaptation is required).

[0066] Typically, LIDAR is a technology that uses the reflection of electromagnetic waves in a spectrum to determine the range of a point in an environment. For example, some LIDAR implementations determine the range of a point based on measuring the round-trip delay of a light pulse to the point. The LIDAR component can include a light source (e.g., a laser) for generating light pulses in the spectrum, an optical device (e.g., a reflector) for guiding the light source (e.g., to scan azimuth and elevation), a sensor for measuring the reflected pulse, and a local processor and / or other electronic components for processing the reflected pulse. The reflected pulse can be processed to determine the range of the target to the point in the environment based on the time delay between the transmitted pulse and the reflected pulse (and using the speed of light). In some embodiments, the LIDAR component generates an output based on the processing, which is a three-dimensional (3D) point cloud that may have an intensity value (based on the intensity of the reflected pulse).

[0067] In general, phase coherent LIDAR is a technology that uses the reflection of electromagnetic waves in the spectrum to determine data indicating the range and radial velocity of a point in the environment. The phase coherent LIDAR component includes at least one tunable laser (or other light source) that generates a waveform, and includes at least one receiver, such as a coherent light receiver. The phase coherent LIDAR component can also include one or more components for guiding the tunable laser, such as an optical device that can be used to guide the waveform from the tunable laser (in azimuth and / or elevation) to multiple points in the environment. In addition, the phase coherent LIDAR component can include a local processor and / or other electronic components for processing the reflection of the waveform. The tunable laser of the phase coherent LIDAR component is tunable because it can be controlled so that the waveform it generates is encoded by adapting one or more properties of the waveform across the waveform. The tunable laser can be controlled to generate one or more of various types of coded waveforms, such as chirped waveforms, step waveforms, and / or pulse waveforms.

[0068] A frequency modulated continuous wave (FMCW) LIDAR component is a specific type of phase coherent LIDAR component. The FMCW LIDAR component at least selectively (e.g., always when in use) controls its tunable laser so that it generates a waveform that is both encoded by frequency modulation and continuous. For example, the waveform can be a continuous step waveform encoded by frequency modulation. For example, the tunable laser can be controlled to generate a step waveform encoded by adapting the step frequency. The data from the FMCW LIDAR component of the vehicle is referred to herein as "FMCW LIDAR data". Therefore, when "phase coherent FMCW LIDAR data" is generated by the FMCW LIDAR component based on a waveform that is both encoded by frequency modulation and continuous, the "phase coherent FMCW LIDAR data" can be "FMCW LIDAR data".

[0069] During a sensing cycle of the phase coherent LIDAR assembly, the transmitted coded waveform is sequentially directed to each of a plurality of points in the environment and sequentially reflected from each of a plurality of points in the environment. The reflected portions of the coded waveform are each detected by at least one receiver of the phase coherent LIDAR assembly in a corresponding sensing event of the sensing cycle. For example, the reflected portion of the sensing event can be detected by at least one coherent optical receiver using a coherent detection scheme in which the reflected portion is mixed with a local optical signal corresponding to the transmitted coded waveform. The coherent detection scheme can be, for example, homodyne mixing, heterodyne mixing, self-heterodyne mixing, or other coherent detection schemes. The resulting signal generated from mixing the reflected portion with the local optical signal indicates both: (1) the range to the corresponding point and (2) the velocity of the corresponding point. For example, the resulting signal can be processed by applying a two-dimensional Fourier transform to the signal to generate a 2D spectrum having frequency coordinates in a direct dimension and frequency coordinates in an indirect dimension. The frequency coordinates in the direct dimension can be used to determine the range, and the frequency coordinates in the indirect dimension can be used to determine the velocity. As a specific example, a homodyne detection scheme can be used, where the local optical signal is a separate portion of the transmitted coded waveform and is used as an optical local oscillator signal for the optical receiver. The local oscillator is mixed with the delayed reflected portion during a sensing event to produce a time-varying intermediate frequency (IF) waveform that is directly related to the range of the corresponding point. In addition, when the point is moving, a Doppler shift will be superimposed on the IF waveform, which can be used to determine the radial velocity of the point. Phase-coherent LIDAR data is generated for each of the multiple points in the vehicle's environment by detecting at multiple sensing events of a sensing cycle, and indicates the range, velocity, and optionally intensity of each point in the environment within the sensing cycle.

[0070] As described above, in various embodiments disclosed herein, phase coherent LIDAR data generated by a phase coherent LIDAR assembly is utilized when determining the yaw parameter of an additional vehicle. In those embodiments, the phase coherent LIDAR data of the sensing period directly or indirectly indicates a corresponding range and a corresponding speed for each of a plurality of points in the vehicle environment coupled to the phase coherent LIDAR assembly. In some of those embodiments, the phase coherent LIDAR data comprises a three-dimensional (3D) point cloud, wherein each point in the 3D point cloud defines a corresponding range (via its 3D coordinates in the 3D point cloud), a corresponding speed, and optionally a corresponding intensity. In some other embodiments, the phase coherent LIDAR data comprises a range Doppler image, such as a range Doppler image in which each pixel defines a corresponding speed (and optionally an intensity) for a corresponding range and a lateral range. In other words, each pixel in the range Doppler image corresponds to a corresponding range and a lateral range (which corresponds to a point in the environment), and a speed is defined for the range and the lateral range. In yet another embodiment, the phase-coherent LIDAR data includes a time-varying intermediate frequency (IF) waveform generated by an optical receiver of the FMCW LIDAR component based on mixing a local oscillator with delayed reflections. The time-varying IF waveform can be used directly to determine the yaw parameter and / or can be processed using Fourier transforms and / or other techniques, and the processed data is used to determine the yaw parameter. For example, each slice of the time-varying IF waveform can be processed to determine the range and velocity of the corresponding point in the environment, and the ranges and velocities of multiple points are utilized when determining the yaw parameter. Therefore, according to the embodiments disclosed herein, one or more of the aforementioned and / or other types of phase-coherent LIDAR data can be generated by the phase-coherent LIDAR component and utilized when determining the yaw parameter of the additional vehicle. The type of phase-coherent LIDAR data generated and utilized when determining the yaw parameter can depend on, for example, the specific phase-coherent LIDAR component being utilized. Additionally, note that in some embodiments, the speed indicated in the received phase-coherent LIDAR data can have the speed of the ego vehicle subtracted (e.g., based on the speed of the ego vehicle determined from the ego vehicle's sensors, or based on inferring the speed of the ego vehicle from the speeds of static objects in the phase-coherent LIDAR data). In some other embodiments, the speed of the ego vehicle may not be subtracted from the received phase-coherent LIDAR data. In those embodiments, the received phase-coherent LIDAR data may optionally be processed to subtract the speed of the ego vehicle.

[0071] In some embodiments, the phase coherent LIDAR data comes from a phase coherent LIDAR single pulse component including at least a first receiver and a second receiver such as a first coherent optical receiver and a second coherent optical receiver. In those embodiments, each sensing event includes a first receiver sensing event by the first receiver and a second receiver sensing event by the second receiver. For example, the first receiver sensing event can include a coherent mixing of a local oscillator with a first detection of a reflected portion of a waveform, and the second receiver sensing event can include a coherent mixing of a local oscillator with a second detection of a reflected portion of the waveform. The first receiver sensing event and the second receiver sensing event are represented as separate events in this article because they pass through separate receivers. However, it is noted that the sensing events can each have the same reflection and can occur simultaneously or substantially simultaneously (e.g., within one millisecond of each other, within one microsecond of each other, or within one nanosecond of each other).

[0072] The first receiver and the second receiver are offset in position relative to each other, so that the first detection and the second detection vary (e.g., vary in time and / or in characteristics (e.g., in phase)). This enables at least one additional channel of data to be determined for the sensed event based on a comparison of the first receiver sensed event and the second receiver sensed event. The additional channel can be used to improve the angular accuracy of the point determined based on the sensed event. For example, the additional channel can be based on the amplitude and / or phase difference between the first receiver sensed event and the second receiver sensed event (e.g., the difference in the time-varying IF waveform generated by each of the first receiver and the second receiver when mixed with the local oscillator signal). For example, for each of the one or more points determined based on the sensed event, the amplitude and / or phase difference can be compared to determine an angle (a specific angle or a range of angles), wherein each angle is resolved so that the point is less than the beamwidth of the reflection that produced the sensed event. Therefore, the angle of the corresponding range (e.g., azimuth and elevation relative to the receiver) can be determined to the sub-beamwidth level, thereby providing greater accuracy of the determined angle. In addition, because the position of the point is defined by the angle and the range, the point can be located to the sub-beamwidth level.

[0073] Note that in some embodiments, multiple range / velocity pairs can be determined based on a single sensing event (e.g., based on differences in range / or velocity among multiple ranges / velocities indicated by range Doppler images and / or IF waveforms). In those embodiments, a corresponding angle can be determined for each of the multiple range / velocity pairs, and the corresponding angles can each be resolved to a sub-beamwidth level. In other words, a single sensing event of a sensing cycle can be used to determine the corresponding range and corresponding velocity of each of two or more points. In some versions of those embodiments, the beamwidth of the electromagnetic output output by the laser of the phase-coherent LIDAR assembly can be intentionally thickened / widened to enable the beamwidth to cover more space compared to a non-thickened beamwidth. For example, the beamwidth can be fixed so that at a given distance from the phase-coherent LIDAR assembly (e.g., 10 meters, 15 meters, 20 meters, 50 meters, or other distance), there is no coverage gap between the beams of the sensing cycle. The improved angular accuracy techniques described herein can be utilized to determine the corresponding range and corresponding velocity for each of two or more points within the thickened beamwidth of each beam in a sensing cycle. This enables the range and velocity to be determined with a certain accuracy, which would otherwise require a larger number of beams and sensing events in a sensing cycle in the absence of the improved angular velocity techniques described herein. Therefore, the improved angular velocity techniques described herein enable the range and velocity to be resolved with a given accuracy in a sensing cycle while reducing the amount of sensing events in the sensing cycle required to achieve that accuracy. This can reduce the amount of sensing events that occur in each sensing cycle, thereby conserving hardware resources (e.g., laser resources and / or power resources). This can additionally or alternatively enable a given amount of range and velocity to be resolved for an area in a reduced amount of time, thereby providing excess time for resolving the range and velocity of additional areas (e.g., enabling a larger total area to be covered in each sensing cycle without increasing the duration of the sensing cycle) and / or enabling the updated range and velocity of the area to be quickly re-resolved (e.g., enabling the sensing cycle to be shortened without reducing angular accuracy).

[0074] Thus, with a phase coherent LIDAR monopulse assembly, super-pixel resolution can be achieved while reducing the signal-to-noise ratio (SNR) (i.e., increasing noise) by separating the reflections across two receivers. This can be beneficial in determining the range and velocity of points that are distant (far from the phase coherent LIDAR monopulse assembly) and / or small additional vehicles. For example, it can improve the angular accuracy with which the range and velocity can be determined for a given return / sensing event (a more accurate position in an angular sense for a given point) - thereby improving resolution and enabling determination (or at least more accurate determination) of yaw parameters of distant and / or small additional vehicles, whereas yaw parameters would not be determinable (or would at least be less accurate) for distant and / or small additional vehicles.

[0075] In various embodiments, a phase-coherent LIDAR single pulse assembly is utilized to reduce the signal-to-noise ratio (SNR) (i.e., increase noise) by separating reflections across two or more receivers, while being able to additionally and / or alternatively achieve super-resolution. Two or more receivers are offset in position relative to each other, so that the corresponding detections of the receivers vary (e.g., vary in time and / or in characteristics (e.g., in phase)). This enables non-redundant LIDAR data from sensing events at each receiver to be combined to generate LIDAR data (e.g., range Doppler images and / or IF waveforms) having a higher resolution (e.g., higher resolution range and / or velocity) than the LIDAR data from any one of the sensing events at the corresponding receivers. In other words, each sensing event can include a receiver sensing event at each receiver (e.g., a first receiver sensing event at a first receiver, a second receiver sensing event at a second receiver, etc.) and generate corresponding LIDAR data for the corresponding receiver sensing event. The non-redundant LIDAR data of the receiver sensing events can be combined to construct super-resolution LIDAR data of the sensing event. Non-redundant LIDAR data can be based on, for example, amplitude and / or phase differences between corresponding receiver sensing events (e.g., differences in a time-varying IF waveform generated by each of the first receiver and the second receiver when mixed with a local oscillator signal). Various techniques can be used to generate super-resolution LIDAR data from multiple receiver sensing events, such as frequency domain techniques (e.g., based on the shift and aliasing properties of continuous and discrete Fourier transforms) and / or spatial domain techniques (e.g., non-iterative and / or iterative techniques). Spatial domain techniques can be used to apply one or more flexible estimators when constructing super-resolution LIDAR data from lower resolution LIDAR data from receiver sensing events. Such flexible estimators include, for example, maximum likelihood, maximum a posteriori, convex set projection, Bayesian processing, etc. Additional descriptions of some phase-coherent LIDAR techniques including single-pulse components and techniques and some phase-coherent LIDAR components are provided herein.

[0076] As described above, during a sensing cycle of the phase coherent LIDAR assembly, the transmitted coded waveform is sequentially directed to each of a plurality of points in the environment and sequentially reflected from each of a plurality of points in the environment, and the reflected portion of the coded waveform is each detected by at least one receiver of the phase coherent LIDAR assembly in a corresponding sensing event of the sensing cycle. During the sensing cycle, the waveform is directed to a plurality of points in an area of ​​the environment and the corresponding reflections are detected without redirecting the waveform to those points during the sensing cycle. Therefore, the range and speed of the point indicated by the phase coherent LIDAR data of the sensing cycle can be instantaneous because it is based on a single sensing event without reference to a previous or subsequent sensing event. In some embodiments, each of a plurality of (e.g., all) sensing cycles can have the same duration, the same field of view, and / or the same waveform distribution pattern (by directing the waveform during the sensing cycle). For example, each of a plurality of sensing cycles, each of which is a scan, can have the same duration, the same field of view, and the same waveform distribution pattern. However, in many other embodiments, the duration, field of view, and / or waveform distribution pattern can vary between one or more sensing cycles. For example, a first sensing cycle can have a first duration, a first field of view, and a first waveform distribution pattern; while a second sensing cycle can have a second duration shorter than the first duration, a second field of view that is a subset of the first field of view, and a second waveform distribution pattern that is denser than the first waveform distribution pattern.

[0077] Now turning to the graph, Figure 1 An example autonomous vehicle 100 is illustrated within which the various techniques disclosed herein may be implemented. For example, the vehicle 100 may include a powertrain 102 including a prime mover 104 powered by an energy source 106 and capable of providing power to a drivetrain 108, and a control system 110 including a direction control 112, a powertrain control 114, and a brake control 116. The vehicle 100 may be implemented as any number of different types of vehicles, including vehicles capable of transporting people and / or cargo and capable of traveling by land, by sea, in the air, underground, under the sea, and / or in space, it being appreciated that the aforementioned components 102-116 may vary widely based on the type of vehicle utilizing the components.

[0078] For example, the embodiments discussed below will focus on wheeled land vehicles, such as cars, vans, trucks, buses, etc. In such embodiments, the prime mover 104 may include one or more electric motors and / or internal combustion engines (etc.). The energy source 106 may include, for example, a fuel system (e.g., providing gasoline, diesel, hydrogen, etc.), a battery system, a solar panel or other renewable energy source, and / or a fuel cell system. The transmission system 108 may include wheels and / or tires and a transmission and / or any other mechanical drive assembly suitable for converting the output of the prime mover 104 into vehicle motion, as well as one or more brakes configured to controllably stop or slow down the vehicle 100 and a direction or steering assembly suitable for controlling the trajectory of the vehicle 100 (e.g., a rack and pinion steering linkage that enables one or more wheels of the vehicle 100 to pivot around a generally vertical axis to change the angle of the plane of rotation of the wheel relative to the longitudinal axis of the vehicle). In some embodiments, a combination of a powertrain and an energy source may be used (e.g., in the case of an electric / gas hybrid vehicle), and in some cases, multiple electric motors (e.g., dedicated to individual wheels or axles) may be used as prime movers.

[0079] The direction control 112 may include one or more actuators and / or sensors for controlling and receiving feedback from the direction or steering components to enable the vehicle 100 to follow a desired trajectory. The powertrain control 114 may be configured to control the output of the powertrain 102, for example, controlling the output power of the prime mover 104, controlling the gears of the transmission in the transmission system 108, etc., thereby controlling the speed and / or direction of the vehicle 100. The brake control 116 may be configured to control one or more brakes that slow down or stop the vehicle 100, for example, disc or drum brakes coupled to the wheels of the vehicle.

[0080] As will be appreciated by those of ordinary skill having the benefit of this disclosure, other vehicle types, including but not limited to aircraft, space vehicles, helicopters, drones, military vehicles, all-terrain or tracked vehicles, ships, submarines, construction equipment, etc., will necessarily utilize different powertrains, drivetrains, energy sources, directional controls, powertrain controls, and brake controls. In addition, some of these components may be combined in some embodiments, for example, where directional control of the vehicle is handled primarily by changing the output of one or more prime movers. Thus, the embodiments disclosed herein are not limited to the specific application of the technology described herein in autonomous wheeled land vehicles.

[0081] In the illustrated embodiment, full or semi-autonomous control of the vehicle 100 is implemented in a vehicle control system 120, which may include one or more processors 122 and one or more memories 124, wherein each processor 122 is configured to execute program code instructions 126 stored in the memory 124. The processor 122 can include, for example, a graphics processing unit (GPU) and / or a central processing unit (CPU).

[0082] The sensors 130 may include various sensors suitable for collecting information from the vehicle's surroundings for use in controlling the operation of the vehicle. For example, the sensors 130 can include a frequency modulated continuous wave (FMCW) light detection and ranging (LIDAR) component 132 (FMCW) of the vehicle. As described herein, the FMCW LIDAR component 132 can generate FMCW LIDAR data that can be used to determine a corresponding yaw parameter for each of one or more additional vehicles in the environment of the vehicle 100. For each of a plurality of points in the vehicle's environment, the FMCW LIDAR data generated by the FMCW LIDAR component 132 indicates a corresponding range and a corresponding speed based on a corresponding sensed event of the FMCW LIDAR component. Although in Figure 1 Although an FMCW LIDAR component is illustrated in (and other figures) and described in association with many of the figures herein, in many embodiments, a phase coherent LIDAR component that is not an FMCW LIDAR component can be used instead. In addition, in those embodiments, phase coherent LIDAR data that is not FMCW LIDAR data can be used instead of FMCW LIDAR data.

[0083] The sensor 130 can also optionally include a radio detection and ranging (RADAR) component 134 and / or a LIDAR component 136 (non-phase coherent, which generates data that can be used to determine range rather than speed). In addition, the sensor can also optionally include a satellite navigation (SATNAV) component 138 compatible with any of various satellite navigation systems such as GPS, GLONASS, Galileo, Compass, etc. The SATNAV component 138 can be used to determine the location of the vehicle on the earth using satellite signals. The sensor 130 can also optionally include a camera 140, an inertial measurement unit (IMU) 142 and / or one or more wheel encoders 144. The camera 140 can be a monoscopic camera or a stereo camera and can record still and / or video imagers. The IMU 142 can include multiple gyroscopes and accelerometers that can detect linear and rotational motion of the vehicle 100 in three directions. The wheel encoder 142 can be used to monitor the rotation of one or more wheels of the vehicle 100.

[0084] The output of the sensor 130 may be provided to a set of control subsystems 150, including a positioning subsystem 152, a planning subsystem 156, a perception subsystem 154, and a control subsystem 158. The positioning subsystem 152 is primarily responsible for accurately determining the position and orientation (sometimes also referred to as "pose") of the vehicle 100 within its surroundings and typically within some reference frame. The perception subsystem 154 is primarily responsible for detecting, tracking, and / or identifying elements within the environment surrounding the vehicle 100. For example, and as described herein, the perception subsystem 154 may determine, among other things, the yaw rate of one or more additional vehicles in the environment surrounding the vehicle 100. The planning subsystem 156 is primarily responsible for planning the trajectory of the vehicle 100 during a certain time frame in view of a desired destination and stationary and moving objects within the environment. For example, and as described herein, the planning subsystem 156 is capable of planning the trajectory of the vehicle 100 based, at least in part, on the determined yaw parameters (determined by the perception subsystem 154) of the additional vehicles in the environment surrounding the vehicle 100. The control subsystem 158 is primarily responsible for generating appropriate control signals for controlling the various controls in the control system 110 in order to achieve the planned trajectory of the vehicle 100 .

[0085] It should be understood that Figure 1 The collection of components illustrated for the vehicle control system 120 in FIG. 1 is merely exemplary in nature. Individual sensors may be omitted in some embodiments. Additionally or alternatively, in some embodiments, Figure 1 Multiple sensors of the type illustrated in the figure can be used for redundancy and / or to cover different areas around the vehicle, and other types of sensors can be used. Similarly, different types and / or combinations of control subsystems can be used in other embodiments. In addition, although the subsystems 152-160 are illustrated as being separated from the processor 122 and the memory 124, it should be appreciated that in some embodiments, some or all of the functionality of the subsystems 152-160 can be implemented with program code instructions 126 that reside in one or more memories 124 and are executed by one or more processors 122, and these subsystems 152-160 can be implemented using the same processor and / or memory in some cases. In some embodiments, the subsystem can be implemented at least in part using various dedicated circuit logic, various processors, various field programmable gate arrays ("FPGA"), various application-specific integrated circuits ("ASIC"), various real-time controllers and / or subsystems, etc., and as noted above, multiple subsystems can utilize circuits, processors, sensors and / or other components. In addition, the various components in the vehicle control system 120 can be networked in various ways.

[0086] In some embodiments, the vehicle 100 may also include an auxiliary vehicle control system (not shown) that may be used as a redundant or backup control system for the vehicle 100. In some embodiments, the auxiliary vehicle control system may be able to fully operate the autonomous vehicle 100 in the event of an adverse event in the vehicle control system 120, while in other embodiments, the auxiliary vehicle control system may have only limited functionality, for example, to perform a controlled stop of the vehicle 100 in response to an adverse event detected in the primary vehicle control system 120. In still other embodiments, the auxiliary vehicle control system may be omitted.

[0087] Typically, numerous different architectures, including various combinations of software, hardware, circuit logic, sensors, networks, etc., can be used to implement Figure 1 100 . Each memory may be implemented, for example, as a microprocessor, and each memory may represent a random access memory (RAM) device including primary storage, as well as any supplementary levels of memory, such as cache memory, non-volatile or backup memory (e.g., programmable or flash memory), read-only memory, etc. In addition, each memory may be considered to include memory storage physically located elsewhere in the vehicle 100, such as any cache memory in the processor, and any storage capacity used as virtual memory, such as stored on a mass storage device or on another computer or controller. Figure 1 One or more of the processors illustrated in the figure or an entirely separate processor may be used to implement additional functionality in the vehicle 100 beyond the purpose of autonomous control, for example, to control the entertainment system, to operate doors, lights, convenience features, etc.

[0088] In addition, for additional storage, the vehicle 100 may also include one or more mass storage devices, such as a removable disk drive, a hard drive, a direct access storage device (DASD), an optical drive (e.g., a CD drive, a DVD drive, etc.), a solid-state storage drive (SSD), a network attached storage, a storage area network, and / or a tape drive, etc. In addition, the vehicle 100 may include a user interface 164 that enables the vehicle 100 to receive a number of inputs from a user or operator and generate outputs for the user or operator, such as one or more displays, touch screens, voice and / or gesture interfaces, buttons and other tactile controls, etc. Otherwise, the user input may be received via another computer or electronic device, such as via an application on a mobile device or via a web interface.

[0089] In addition, the vehicle 100 may include one or more network interfaces, such as network interface 162, suitable for communicating with one or more networks (e.g., a LAN, WAN, wireless network, and / or the Internet, etc.) to allow information to be transferred with other computers and electronic devices, including, for example, a central service, such as a cloud service, from which the vehicle 100 receives environmental and other data for use in its autonomous control.

[0090] As will be described in more detail below, Figure 1 Each processor illustrated in the vehicle 100 , as well as the various additional controllers and subsystems disclosed herein, typically operates under the control of an operating system and executes or otherwise relies on various computer software applications, components, programs, objects, modules, data structures, etc. In addition, the various applications, components, programs, objects, modules, etc. may also be executed on one or more processors on another computer coupled to the vehicle 100 via a network, such as in a distributed, cloud-based, or client-server computing environment, whereby the processing required to implement the functionality of the computer program may be distributed to multiple computers and / or services over the network.

[0091] In general, routines executed to implement the various embodiments described herein, whether implemented as part of an operating system or as a specific application, component, program, object, module or sequence of instructions or even a subset thereof, will be referred to herein as "program code". Program code typically includes one or more instructions that reside in various memories and storage devices at various times, and when read and executed by one or more processors, perform the steps necessary to implement the steps or elements that embody various aspects of the present invention. In addition, although the embodiments have been and will be described below in the context of fully functional computers and systems, it should be appreciated that the various embodiments described herein can be distributed as program products in various forms, and can be implemented regardless of the specific type of computer-readable medium used to actually perform the distribution. Examples of computer-readable media include tangible non-transitory media, such as volatile and non-volatile storage devices, floppy disks and other removable disks, solid-state drives, hard disk drives, tapes, and optical disks (e.g., CD-ROMs, DVDs, etc.), and the like.

[0092] In addition, various program codes described below can be identified based on the application implemented in a specific embodiment. However, it should be appreciated that any specific program terms that follow are used only for convenience, and therefore the present invention should not be limited to use in any specific application that is only identified and / or implied by such terms. In addition, given the usual countless ways in which computer programs can be organized into routines, programs, methods, modules, objects, etc., and the various ways in which program functionality can be distributed between various software layers (e.g., operating systems, libraries, APIs, applications, applets, etc.) residing in typical computers, it should be appreciated that the present invention is not limited to the specific organization and distribution of program functionality described herein.

[0093] Those skilled in the art who have the benefit of this disclosure will recognize that Figure 1 The exemplary environment illustrated in is not intended to limit the embodiments disclosed herein. In fact, those skilled in the art will recognize that other alternative hardware and / or software environments may be used without departing from the scope of the embodiments disclosed herein.

[0094] Now turn to Figure 2A and Figure 2B , schematically illustrates Figure 1 Two embodiments of the FMCW LIDAR assembly 132 are described. Figure 2A The FMCW LIDAR assembly 132A is shown to include a single receiver 132A3, however Figure 2B An FMCW LIDAR monopulse assembly 132B is shown including two receivers 132B3A and 132B3B.

[0095] exist Figure 2A In the embodiment, the FMCW LIDAR component 132A includes a tunable laser 132A1, whose optical output is controlled by a signal generator 132A2. The signal generator 132A2 drives the laser 132A1 so that the laser 132A1 generates an electromagnetic output that is encoded in the spectrum and by adapting the frequency of the electromagnetic output. The signal generator 132A2 generates a signal that causes the waveform to be encoded. Various types of frequency-encoded waveforms can be generated via the signal generator 132A2 and the laser 132A1.

[0096] Figure 2A Illustrated is a transmit beam 192A from laser 132A1, which is encoded as described in the previous paragraph and directed to a point on target object 191. Transmit beam 192A is reflected off the point on target object 191 and reflection 193A is received at receiver 132A3 during a sensing event. Transmit beam 192A includes slices of the encoded waveform and reflection 193A also includes slices of the encoded waveform. The slices of the encoded waveform are detected by receiver 132A3 during a sensing event.

[0097] Receiver 132A3 can be a coherent receiver and reflection 193A can be detected by receiver 132A3 using a coherent detection scheme in which reflection 193A is mixed with a local optical signal, such as a local optical signal that is a separated portion of transmit beam 192A. Note that detection of reflection 193A by receiver 132A3 will be delayed relative to detection of the local optical signal (e.g., the separated portion of transmit beam 192A). The coherent detection scheme can be implemented by processor 132A4 and can be, for example, homodyne mixing, heterodyne mixing, self-heterodyne mixing, or other coherent detection schemes. The resulting signal generated from mixing the reflected portion with the local optical signal indicates both: (1) the range to the point of target object 191, and (2) the velocity of the point of target object 191. In some embodiments, processor 132A4 provides the resulting signal as FMCW LIDAR data from the point of FMCW LIDAR assembly 132A. In some other embodiments, processor 132A4 performs further processing to generate FMCW LIDAR data for a point that directly indicates the range and velocity of the point. For example, the range and velocity of a point can be directly indicated in the FMCW LIDAR data, which is a 3D point cloud or range Doppler image indicating range and velocity values. By detecting multiple sensing events, in the event of a change in the direction of the light output from laser 132A1 (e.g., a change in azimuth and / or elevation), FMCW LIDAR data is generated by the FMCW LIDAR component for each of a plurality of points in the environment and indicates the range, velocity, and optionally intensity of each point in the environment.

[0098] Despite the above Figure 2A The description of describes an FMCW LIDAR assembly, but note that a phase coherent LIDAR assembly that is not FMCW can have the same and / or similar components. However, with a non-FMCW LIDAR assembly, the signal generator 132A2 can drive the laser 132A1 to generate a waveform that is non-continuous and / or encoded using a technique other than frequency modulation.

[0099] exist Figure 2B In the FMCW LIDAR single pulse component 132B, a tunable laser 132B1 is included, whose optical output is controlled by a signal generator 132B2. The signal generator 132B2 drives the laser 132B1 so that the laser 132B1 generates an electromagnetic output that is encoded in the spectrum and by adapting the frequency of the electromagnetic output. The signal generator 132B2 generates a signal that causes the waveform to be encoded. Various types of encoded waveforms can be generated via the signal generator 132B2 and the laser 132B1.

[0100] Figure 2BA transmit beam 192B from a laser 132B1 is illustrated, which is encoded and directed to a point on a target object 191 as described in the previous paragraph. The transmit beam 192B is reflected off the point on the target object 191, and during a sensing event, a first reflection 193B1 is received at a first receiver 132B3A and a second reflection 193B2 is received at a second receiver 132B3B. The transmit beam 192B includes slices of the encoded waveform and the reflections 193B1 and 193B2 also include slices of the encoded waveform. The first receiver 132B3A and the second receiver 132B3B can be coherent receivers, and the sensing event can include a first receiver sensing event and a second receiver sensing event. The first receiver sensing event can include a coherent mixing of a local oscillator (e.g., a separate portion of the transmit beam 192B) with a first detection of the first reflection 193B1, and the second receiver sensing event can include a coherent mixing of a local oscillator with a second detection of the second reflection 193B2. The first receiver 132B3A and the second receiver 132B3B are offset in position relative to each other, so that the first detection and the second detection are changed. This enables the processor 132B4 to determine at least one additional channel of data of the sensing event based on the comparison of the first receiver sensing event and the second receiver sensing event. For example, the processor 132B4 can determine the additional channel based on the amplitude and / or phase difference between the first receiver sensing event and the second receiver sensing event (for example, the difference in the time-varying IF generated by each of the first receiver and the second receiver when mixed with the local oscillator signal). For example, for each of the one or more points determined based on the sensing event, the amplitude and / or phase difference can be compared to determine the angle (a specific angle or an angle range), where each angle is resolved so that the point is smaller than the beam width of the reflection that produced the sensing event. Therefore, the angle of the corresponding range (for example, the azimuth and elevation relative to the receiver) can be determined to the sub-beam width level. In addition, because the position of the point is defined by the angle and range, the point can be located to the sub-beam width level. Thus, while reducing the signal-to-noise ratio (SNR) (i.e., increasing noise) by separating the reflections across two receivers, improved angular accuracy can be achieved. This may be beneficial when determining the range and speed of multiple points of distant (far away from the FMCW LIDAR monopulse component) and / or small additional vehicles. Additionally or alternatively, and as described herein, processor 132B4 can use non-redundant data from the first detection and the second detection to determine LIDAR data with super-resolution. For example, processor 132B4 can construct super-resolution LIDAR data from lower resolution LIDAR data from the first detection and the second detection. Thus, while reducing the SNR by separating the reflections across two receivers, super-resolution range and speed can be determined.

[0101] In some embodiments, processor 132B4 provides signals generated from the first receiver sensing event and the second receiver sensing event as FMCW LIDAR single pulse data for a point from FMCW LIDAR single pulse component 132B. In some other embodiments, processor 132B4 performs further processing to generate FMCW LIDAR single pulse data for a point that directly indicates the range and velocity of the point. For example, the range and velocity of a point can be directly indicated in the FMCW LIDAR single pulse data as a 3D point cloud or a range Doppler image. By detecting multiple sensing events, in the event of a change in the direction of the light output from laser 132B1 (e.g., a change in azimuth and / or elevation), FMCW LIDAR single pulse data is generated by FMCW LIDAR single pulse component 132B for each of multiple points in the environment and indicates the range, velocity, and optionally intensity of each point in the environment.

[0102] Despite the above Figure 2B The description of describes an FMCW LIDAR monopulse assembly, but note that a non-FMCW phase coherent LIDAR monopulse assembly can have the same and / or similar components. However, with a non-FMCW LIDAR assembly, the signal generator 132B2 can drive the laser 132B1 to generate a waveform that is non-continuous and / or encoded using a technique other than frequency modulation.

[0103] Reference now Figure 3 , in which an example additional vehicle 300 is shown, along with a yaw axis 371, a pitch axis 372, and a roll axis 373 of the additional vehicle 300. The yaw axis 371 of the additional vehicle 300 is perpendicular to the axles of the additional vehicle 300. In some cases, the yaw axis 371 may intersect the rear axle of the additional vehicle 300, although this is not required - and depending on the cause and / or extent of the yaw movement of the additional vehicle 300, the yaw axis 371 may even be shifted in some cases. The rotation of the additional vehicle 300 about the yaw axis 371 may be caused by various factors, such as turning of the additional vehicle 300, loss of traction of the additional vehicle 300 (e.g., hydroplaning), the additional vehicle 300 striking another object, etc. The yaw movement of the additional vehicle 300 is the rotation of the additional vehicle 300 about the yaw axis 371, and the yaw rate of the additional vehicle 300 is the angular velocity of its rotation about the yaw axis 371. The yaw rate is typically measured in radians per second or degrees per second.

[0104] The roll axis 373 of the additional vehicle 300 is perpendicular to the yaw axis 371 and extends from front to back along the additional vehicle 300. When the additional vehicle 300 is in motion, the additional vehicle will have an absolute object velocity along the roll axis 373. This absolute object velocity can also be considered in the control of the autonomous driving vehicle adapted in the embodiments described herein. However, it should be noted that the absolute object velocity is separate from the yaw rate of the additional vehicle 300. For example, the additional vehicle 300 can have a relatively high absolute object velocity during movement while having zero or minimal yaw rate (for example, if it is moving along a relatively straight path without any traction loss). The pitch axis of the additional vehicle 300 is perpendicular to the yaw axis 371 and the roll axis 373 and extends from side to side along the additional vehicle 300.

[0105] Reference now Figure 4 , a block diagram illustrating adaptation of autonomous control of a vehicle based on a determined yaw parameter of an additional vehicle is provided. Figure 4 Shown are examples of subsystems 152 , 154 , 156 , and 158 of the vehicle control system 120 and how they may interact to determine a yaw parameter of an additional vehicle and / or adapt autonomous control based on the determined yaw parameter.

[0106] The positioning subsystem 152 is generally responsible for locating the vehicle within its environment. For example, the positioning subsystem 152 can determine the pose (typically location, and in some cases orientation) of the autonomous vehicle 100 within its surroundings. The positioning subsystem 152 can determine the pose of the autonomous vehicle 100 using FMCW LIDAR data 492 from the FMCW LIDAR component 132 and / or data from other sensors 491, such as Figure 1 The camera 140, RADAR component 134, LIDAR component 136, and / or other sensors 130 (e.g., IMU 142, encoder 144) of the positioning subsystem 152. For example, the pose determined by the positioning subsystem 152 can be relative to a map utilized by the vehicle control system 120. The pose can be updated by the positioning subsystem 152 as the vehicle 100 moves around its environment, or the pose is provided to (or made accessible to) one or more of the other subsystems 154, 156, and / or 158 for use by the subsystem.

[0107] The perception subsystem 154 is primarily responsible for sensing dynamic objects in the environment, such as pedestrians and additional vehicles, and may utilize FMCW LIDAR data 492 and optionally data from other sensors 491 (e.g., camera 140, RADAR component 134) to identify and track dynamic objects. When tracking additional vehicles, the perception subsystem 154 may determine various characteristics of the additional vehicles, such as their pose (position and / or orientation) and their absolute object velocity and / or yaw parameters (e.g., yaw rate) in each of a plurality of iterations.

[0108] The perception subsystem 154 may include an object detection and classification module 154A that utilizes the FMCW LIDAR data 492 and / or data from other sensors 491 to detect various objects (e.g., determine the pose of the object) and optionally classify the detected objects. For example, the object detection and classification module 154A may process the FMCW LIDAR data 492 and / or data from other sensors 491 to detect and classify additional vehicles, pedestrians, and / or other dynamic objects in the environment of the vehicle 100. In various embodiments, the object detection and classification module 154A may utilize one or more trained machine learning models to detect and / or classify various objects. For example, the trained machine learning model may include a neural network model that is trained to process FMCW LIDAR data (and / or other data) to generate as output an indication of the pose of the object indicated in the data and optionally an indication of the corresponding classification of the object. For example, the output may indicate a bounding box, a bounding sphere, or other bounding area for each of one or more objects detected in the data, optionally together with a corresponding classification for each bounding area.

[0109] As a specific example, the FMCW LIDAR data 492 may include a 3D point cloud that defines a corresponding range (via its 3D coordinates in the 3D point cloud), a corresponding speed, and optionally a corresponding intensity. The object detection and classification module 154A may process the FMCW LIDAR data 492 using a trained machine learning model to generate an output indicating one or more bounding boxes, each of which corresponds to a portion of the 3D point cloud that includes a detected object. In some embodiments, the output may also indicate, for each bounding box, the classification to which the detected object belongs. For example, for each bounding box, the output may indicate a corresponding probability that the detected object is an additional vehicle, a pedestrian, an animal, etc. In some other embodiments, the trained machine learning model may be trained to detect only additional vehicles and may be used to generate an output indicating a bounding box and / or other bounding shapes without explicitly indicating the classification of the bounding shapes.

[0110] In some embodiments, the object detection and classification module 154A may additionally or alternatively process the FMCW LIDAR data 492 and / or data from other sensors 491 without utilizing a machine learning model when detecting and / or classifying various objects. For example, nearest neighbor and / or other clustering techniques may be utilized to determine portions of the data that may correspond to additional vehicles and / or other objects.

[0111] Regardless of the technology utilized, the object detection and classification module 154A can provide an indication of the pose 493 of the additional vehicle to the yaw module 154B of the perception subsystem 154. The pose can be relative to a map utilized by the vehicle control system 120 and / or relative to the FMCW LIDAR data 492. The yaw module 154B uses the provided pose 493 of the additional vehicle to determine a subset of the FMCW LIDAR data 492 that corresponds to the pose. In addition, the yaw module 154B determines one or more yaw parameters of the additional vehicle 494 based on the subset of the FMCW LIDAR data 492.

[0112] The yaw module 154B may determine a yaw parameter of the additional vehicle 494 based on the magnitude and optionally the direction indicated by all or a subset of the FMCW LIDAR data 492. For example, the yaw module 154B may determine a yaw parameter indicating whether the vehicle is experiencing yaw movement (e.g., at least a threshold degree of yaw movement) based on determining that at least a threshold gradient of velocity is indicated by at least some subset of the FMCW LIDAR data 492. For example, if a subset of the FMCW LIDAR data 492 has a velocity gradient of 4 m / s (e.g., a velocity range from +2 m / s to -2 m / s or a velocity range from +9 m / s to +5 m / s), the yaw module 154B may determine that the vehicle is experiencing yaw movement based on the velocity gradient of 4 m / s. On the other hand, if a subset of the FMCW LIDAR data 492 has a velocity gradient of less than 0.1 m / s, the yaw module 154B may determine that the vehicle is experiencing little or no yaw movement based on the low velocity gradient.

[0113] Likewise, for example, the yaw module 154B may determine a yaw parameter of the additional vehicle based on the speeds indicated by the plurality of sets of the subset of the FMCW LIDAR data 492. For example, the yaw module 154B may determine a speed difference based on a comparison of the speed of the first set of FMCW LIDAR data 492 with the speed of the second set of FMCW LIDAR data 492 that is spatially offset. The yaw module 154B may utilize the speed difference as a yaw parameter, or may determine a yaw rate based on the speed difference. In some embodiments, the yaw module 154B may determine a yaw rate based on the speed difference and based on a determined distance between the first set and the second set. In some of those embodiments, the yaw module 154B may determine the yaw rate based on dividing the speed difference by the distance (optionally multiplied by a constant) to determine a yaw rate in radians per second. In some versions of those embodiments, the yaw module 154B may determine a range of yaw rates. For example, a lower yaw rate may be based on a velocity difference divided by the distance, while an upper yaw rate may be based on a velocity difference divided by: the distance multiplied by a constant between 0 and 1, such as a value between 0.4 and 0.9, a value between 0.4 and 0.75, or a value between 0.4 and 0.7. For example, if the velocity difference is 1 m / s and the distance is 1.5 m, the lower yaw rate may be 0.67 radians / second ((1 m / s) / (1.5 m)), while if the velocity difference is 1 m / s and the distance is greater than 4.0 m, the lower yaw rate may be 0.25 radians / second. The value of the constant may be based, for example, on the cosine of the angle of the additional vehicle relative to the ego vehicle. For example, the angle of the additional vehicle relative to the FMCW LIDAR receiver, which may be based on one or more angles corresponding to the additional vehicle indicated by the FMCW LIDAR data.

[0114] As a specific example, the first set may be a single data point, the speed of the first set may be the speed of the single data point, the second set may be a separate single data point, and the speed of the second set may be the speed for the separate single data point. The yaw module 154B may determine the yaw rate based on a difference in the magnitude of the speeds of the first and second sets (speed difference) and based on a determined distance between the first and second sets. The yaw module 154B may also determine the direction of the yaw movement (when the speed difference is not zero) based on a comparison of the magnitudes of the speeds of the first and second sets (and optionally the direction in the case of extreme yaw movement (e.g., loss of control of an attached vehicle)).

[0115] As another specific example, yaw module 154B can determine a yaw rate of the additional vehicle based on determining first and second sets of FMCW LIDAR data, where the first set and the second set each include a plurality of corresponding data points. Additionally, yaw module 154B can determine an average speed or other statistical measure (e.g., an average) of the speed of the data points of the first set, a statistical measure of the speed of the data points of the second set, and determine the yaw rate based on a comparison of the statistical measure of the speed of the first set and the statistical measure of the speed of the second set. Additional description of these and other techniques for determining yaw rate is provided herein. Additionally, the following description of the yaw rate of the additional vehicle can be found in the accompanying drawings. Figures 5 to 8 The description of the associated provides additional description of various techniques.

[0116] In some embodiments, the yaw module 154B may additionally or alternatively determine the yaw rate of the additional vehicle based on the speed indicated by at least some subset of the FMCW LIDAR data 492 and based on a stored model of the additional vehicle. The stored model of the additional vehicle describes the geometric features of the additional vehicle. The stored model may be, for example, a three-dimensional (3D) model of the additional vehicle. The yaw module 154B may map the speed indicated by at least some subset of the FMCW LIDAR data 492 to a corresponding position on the stored model, and determine the yaw rate of the additional vehicle using the speed and its position on the stored model. For example, the yaw module 154B may apply a plurality of speeds to the stored model to determine the yaw rate of the stored model about the yaw axis at the applied speeds. The yaw module 154B may determine the yaw rate, for example, using a simulator that simulates the movement of the stored model at the applied speeds.

[0117] In some embodiments where stored models of additional vehicles are utilized, the yaw module 154B may optionally utilize a general stored model for all additional vehicles. In some other embodiments, the yaw module 154B may select a stored model for a given additional vehicle from a plurality of candidate stored models based on determining that the selected stored model most closely matches the sensor data capturing the additional vehicle. For example, the yaw module 154B may select a stored model based on determining that the FMCW LIDAR data of a subset (determined to correspond to the additional vehicle) more closely matches the selected model than any other stored model.

[0118] In some embodiments, the yaw module 154B may additionally or alternatively utilize a trained yaw machine learning (ML) model 154B1 to determine the yaw rate of the additional vehicle. For example, the yaw module 154B may process all or a portion of the FMCW LIDAR data 492 using the trained yaw ML model 154B1 to generate an output indicative of the yaw rate. For example, at least a subset of the FMCW LIDAR data 492 determined to correspond to the additional vehicle may be processed using the trained yaw ML model 154B1 to generate an output indicative of the yaw rate. The trained yaw ML model 154B1 may, for example, be used as described below with respect to Fig.12 Training is performed as described in method 1200.

[0119] The perception subsystem 154 provides the yaw parameters for the additional vehicle 494 to the planning subsystem 156 (or otherwise makes them available to the planning subsystem 156). The planning subsystem 156 is primarily responsible for planning the trajectory of the vehicle within a certain time frame (e.g., a few seconds), and may receive input from both the position subsystem 152 and the perception subsystem 154. The planning subsystem 156 may adapt the trajectory of the vehicle based on the provided yaw parameters of the additional vehicle 494. For example, if the yaw parameters 494 indicate that the speed difference and / or yaw rate of the additional vehicle exceeds a threshold indicating that the additional vehicle has lost control, the planning subsystem 154 may adapt the trajectory of the vehicle so as to avoid the additional vehicle. Also, for example, if the yaw parameters 494 indicate a yaw rate and a yaw direction, the planning subsystem 154 may determine one or more candidate trajectories for the additional vehicle using the yaw rate and yaw direction of the additional vehicle 494, and may adapt the trajectory of the vehicle so that if the additional vehicle travels through one of the candidate trajectories, the vehicle does not contact and / or enter within a threshold distance of the additional vehicle. Additional descriptions of these and other techniques for adapting control of a vehicle based on a determined yaw parameter of an additional vehicle are provided herein. Fig. 9 The instructions provide additional explanation of some techniques.

[0120] The trajectory and / or other data generated by the planning subsystem 156 in view of the determined yaw parameters is then communicated to the control subsystem 158 to convert the desired trajectory into a trajectory command 195 suitable for controlling the various vehicle controls 112, 114 in the control system 110, and also to provide position data to the control subsystem 158 so that the control subsystem 158 can issue appropriate commands to achieve the desired trajectory as the vehicle positioning changes over a time frame.

[0121] Reference now Figures 5 to 8 , each of these figures shows an example environment of a vehicle, where the environment includes an additional vehicle 300. In addition, Figures 5 to 8Each also shows examples of some FMCW LIDAR data points that may correspond to additional vehicles 300 in the environment. For example, Figure 5 Example FMCW LIDAR data points 582A-B, 583A-B, 584A-C, and 585A-C are shown. The data points may, for example, be instances from FMCW LIDAR data as a 3D point cloud, where each point of the 3D point cloud indicates a corresponding range (by its location in the 3D point cloud), and includes a corresponding rate / speed, and optionally a corresponding intensity value. It should be understood that additional FMCW LIDAR data points will be provided with the FMCW LIDAR data, including additional points corresponding to the vehicle 300 - as well as additional points corresponding to additional environmental objects. However, for simplicity, the data points are not shown in FIG. Figures 5 to 8 Only some FMCW LIDAR data points are shown in each of FIG. Figures 5 to 8 will not be presented in FMCW LIDAR data (although it can be visually determined via the corresponding FMCW LIDAR data points). Figures 5 to 8 In each of the FIG. 300, a vehicle 300 is shown for reference. In addition, it should be understood that although the FMCW LIDAR data points are described Figures 5 to 8 , but additional or alternative types of FMCW LIDAR data may be used to determine yaw parameters of additional vehicles, such as those also described herein.

[0122] exist Figure 5 , a bounding box 581 is depicted around the additional vehicle 300. The bounding box 581 indicates a subgroup of FMCW LIDAR data that may correspond to the additional vehicle 300. The bounding box 581 may be, for example, Figure 4 The object detection and classification module 154A may be based on processing the FMCW LIDAR data and / or other data to detect the vehicle and its pose to determine. For example, the object detection and classification module 154A may process the FMCW LIDAR data using a machine learning model to generate an output indicating that the FMCW LIDAR data points enclosed by the bounding box 581 correspond to the vehicle. Example FMCW LIDAR data points are shown within the bounding box 581. That is, a first set 582 of data points 582A-B, a second set 583 of data points 583A-B, a third set 584 of data points 584A-C, and a fourth set of data points 585A-C.

[0123] Figure 4The yaw module 154B can determine the yaw parameter of the additional vehicle 300 based on the comparison between the first set 582, the second set 583, the third set 584, and / or the fourth set 585. For example, the yaw module 154B can determine the statistical measurement of the speed of the first set 582 based on the speed indicated by the data points 582A and 582B. For example, if the data point 582A indicates a speed of +3.1 m / s and the data point 582B indicates a speed of +3.3 m / s, the yaw module 154B can determine that the average speed of the first set 582 is +3.2 m / s. In addition, the yaw module can also determine the statistical measurement of the speed of the second set 583 based on the speed indicated by the data points 583A and 583B. For example, if data point 583A indicates a velocity of +3.8 m / s and data point 583B indicates a velocity of +4.0 m / s, the yaw module 154B may determine that the average velocity of the second set 583 is +3.9 m / s. The yaw module 154B may then determine a velocity difference between the first set 582 and the second set 583. For example, where the first set 582 and the second set 583 have respective average velocities of +3.1 m / s and +3.9 m / s, the velocity difference may be 0.8 m / s. In some implementations, the velocity difference of 0.8 m / s may be used as a yaw parameter. In addition, the yaw module 154B may also determine a direction of the yaw movement based on a comparison of the velocity magnitudes of the first set 582 and the second set 583. For example, where the first set 582 and the second set 583 have respective average velocities of +3.1 m / s and +3.9 m / s, the direction of the yaw movement (the yaw direction) may be along Figure 5 The direction indicated by arrow 371A.

[0124] In some embodiments, the yaw parameter may additionally or alternatively include a yaw rate determined based on a speed difference of 0.8 m / s. For example, at least an upper yaw rate and a lower yaw rate may be determined based on a speed difference of 0.8 m / s and based on a distance 588 based on a spatial offset between the first set 582 and the second set 583. The distance 588 may be, for example, a 3D distance between an average distance between points 582A and 582B and an average distance between points 583A and 583B. As an example, if the distance 588 is 4.5 meters and the speed difference is 0.8 m / s, the lower yaw rate may be 0.18 radians / second ((0.8 m / s) / (4.5 m)) and the upper yaw rate may be 0.36 radians / second ((0.8 m / s) / (4.5 m)(0.5)).

[0125] The yaw module 154B may additionally or alternatively determine a statistical measurement of the speed of the third set 584, a statistical measurement of the speed of the fourth set 585, determine a speed difference between the third set 584 and the fourth set 585, and determine a yaw parameter based on the speed difference between the third set 584 and the fourth set 585. For example, the yaw parameter may be based only on the speed difference between the third set 583 and the fourth set 584, or may be based on a combination (e.g., an average) of the speed difference and the speed difference between the first set 582 and the second set 583. Furthermore, additional speed differences between additional sets (sets not shown and / or different combinations of the sets shown) may be determined and used to determine the yaw parameter.

[0126] exist Figure 6 , a bounding box 681 is depicted around the additional vehicle 300. The bounding box 681 indicates a subgroup of FMCW LIDAR data that may correspond to the additional vehicle 300. The bounding box 681 may be, for example, Figure 4 The object detection and classification module 154A determines based on processing of FMCW LIDAR data and / or other data to detect the vehicle and its pose. Example FMCW LIDAR data points 682A-H are shown within the bounding box 681.

[0127] Figure 4 The yaw module 154B may determine a yaw parameter of the additional vehicle 300 based on a comparison between two or more FMCW LIDAR data points 682A-H. For example, the yaw module 154B may: determine a first yaw rate (or a first lower and upper yaw rate) based on a first speed difference between the speeds indicated by data points 682E and 682H and a determined distance between data points 682E and 682H; determine a second yaw rate (or a second lower and upper yaw rate) based on a second speed difference between the speeds indicated by data points 682F and 682H and a determined distance between data points 682F and 682H; determine a third yaw rate (or a third lower and upper yaw rate) based on a third speed difference between the speeds indicated by data points 682A and 682D and a determined distance between data points 682A and 682D; and so on. Continuing with this example, the yaw module 154B can then determine a total yaw rate (or a total lower and upper bound yaw rates) based on a combination of the plurality of yaw rates (or a combination of the plurality of lower bound yaw rates and a combination of the plurality of upper bound yaw rates). In addition, the yaw module 154B can also determine a direction of the yaw movement based on a comparison of the magnitudes of the one or more data point pairs.

[0128] exist Figure 7 , the additional vehicle 300 is shown in Figure 5 and Figure 6The different postures shown (relative to the FMCWLIDAR component). That is, in Figure 7 In FIG. 3 , the additional vehicle 300 is perpendicular to the FMCW LIDAR assembly. Figure 7 A bounding box 781 is depicted around the additional vehicle 300 and indicates a subset of the FMCW LIDAR data that may correspond to the additional vehicle 300. Example FMCW LIDAR data points 782A and 782B are shown within the bounding box 781.

[0129] Since the additional vehicle 300 is Figure 7 300 is vertical, so if the additional vehicle 300 is not performing a yaw motion, the data points 782A and 782B will indicate a zero (or near zero) velocity. On the other hand, if the additional vehicle 300 is performing a yaw motion, the data points 782A and 782B will indicate a non-zero velocity. Figure 4 The yaw module 154B can determine a yaw parameter of the additional vehicle 300 based on a comparison between the data points 782A and 782B. For example, the yaw module 154B can determine whether the additional vehicle 300 is performing a yaw motion based on determining whether a first speed difference between the speeds indicated by the data points 782A and 782B satisfies a threshold.

[0130] exist Figure 8 In FIG. 8 , a bounding box 881 is depicted around the additional vehicle 300 and indicates that only two FMCW LIDAR data points 882A and 882B correspond to the additional vehicle 300. Figure 8 , the additional vehicle 300 is away from the FMCW LIDAR assembly. Figure 8 The FMCW LIDAR data points 882A and 882B in may be FMCW LIDAR single pulse data points generated by a FMCW LIDAR single pulse component. Figure 8 The example of is one example where a FMCW LIDAR single pulse data point may be particularly beneficial. That is, Figure 8 The following situation is shown where the greater angular accuracy of the FMCW LIDAR monopulse component enables the determination of two FMCW LIDAR data points 882A and 882B, whereas one (or none) of them might have been determined by the FMCW LIDAR (non-monopulse component). Figure 8 is an example, but it should be noted that Figures 5 to 7The FMCW LIDAR data points may also be FMCW LIDAR single pulse data points and may also be beneficial because, for example, they increase the angular accuracy of the FMCW LIDAR data of the additional vehicle 300 that may be used to determine the yaw rate according to embodiments disclosed herein. This greater angular accuracy of the FMCW LIDAR data may produce a more accurate yaw rate determination in some embodiments.

[0131] Figure 4 The yaw module 154B may determine a yaw parameter of the additional vehicle 300 based on a comparison of the FMCW LIDAR data points 882A and 882B. For example, the yaw module 154B may determine a yaw rate based on a speed difference between the speeds indicated by the data points 882A and 882B and a determined distance between the data points 882A and 882B. In addition, the yaw module 154B may also determine a direction of the yaw movement based on a comparison of the speed magnitudes of the data points 882A and 882B.

[0132] The yaw parameters determined according to the techniques disclosed herein can be used to adapt the control of the ego vehicle. Fig. 9 An example of such an adaptation is described, Fig. 9 An example of how a candidate trajectory for the additional vehicle 300 may be determined based on the determined yaw parameter and how the autonomous control of the vehicle 100 may be adapted based on the candidate trajectory for the additional vehicle 300 is shown.

[0133] exist Fig. 9 , the vehicle 100 may determine the yaw rate of the additional vehicle 300 in a given instance using FMCW LIDAR data from the FMCW LIDAR component of the vehicle 100. Candidate trajectory 991 illustrates a candidate trajectory that may be determined if the determined yaw rate is zero or close to zero. The candidate trajectory 991 may be determined based on the determined absolute object velocity direction and magnitude of the additional vehicle 300 without any adaptation based on yaw rate (because the determined yaw rate is zero or close to zero).

[0134] On the other hand, candidate trajectory 992 shows what would happen if the determined yaw rate indicated that the vehicle was moving relative to Fig. 9 The candidate trajectory 992 can be determined based on the determined absolute velocity direction and magnitude and the determined magnitude and direction of the determined yaw rate. Various techniques can be used to determine the absolute object velocity, such as appending vehicle tracking on multiple instances of sensor data (e.g., MCW LIDAR data and / or other data). Although in Fig. 9Only a single candidate trajectory 992 is shown in FIG. 1 , but in some embodiments, multiple candidate trajectories may be determined. For example, where lower and upper yaw rates are determined as described herein, a first candidate trajectory may be determined based on the lower yaw rate, a second candidate trajectory may be determined based on the upper yaw rate, and multiple trajectories between the first and second candidate trajectories (and optionally the first and second candidate trajectories therebetween) may be considered candidate trajectories (e.g., candidate trajectory regions) for additional vehicles 300.

[0135] By observation Fig. 9 It will be appreciated that if candidate trajectory 991 is determined for vehicle 300, then it may be necessary to adapt control of vehicle 100 to avoid additional vehicle 300 (because if additional vehicle 300 were to travel through candidate trajectory 991, it would enter the lane of vehicle 100). On the other hand, if candidate trajectory 992 is determined instead, then avoidance adaptation of control of the vehicle may not be necessary.

[0136] Fig. 9 Candidate trajectories that may be determined for a given instance based on a determined yaw rate for the given instance are shown. The yaw rate and candidate trajectory may continue to be generated at each of a plurality of additional instances (e.g., updated yaw rates and candidate trajectories may be generated at each control cycle (e.g., at 60 Hz and / or other rates)), thereby enabling continuous updating of the candidate trajectory to account for changing yaw rates. Additionally, when generating an instance trajectory at a given instance, one or more candidate trajectories and / or one or more yaw rates from one or more previous instances may be considered to smooth the candidate trajectory and prevent unnecessary avoidance adaptations at the given instance based on, for example, an erroneously determined yaw rate (e.g., based on noisy data) and / or brief and intermittent yaw rate changes.

[0137] Reference now Fig.10 , which provides a flowchart illustrating an example method 1000 of determining a yaw parameter and controlling an autonomous vehicle based on the determined yaw parameter. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. The system may include various components, such as Figure 1 At the same time, although the operations of method 1000 are shown in a particular order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0138] At block 1002, the system receives FMCW LIDAR data from a FMCW LIDAR component of a vehicle. The FMCW LIDAR data received at the iteration of block 1002 may be a most recently unprocessed (unprocessed using method 1000) instance of data generated by the FMCW LIDAR component. The instance of data may be generated by the FMCW LIDAR component during a sensing cycle and may be received by the system, for example, as part of a data stream or as a data packet, or received via access to a shared memory.

[0139] At block 1004, the system determines that a subgroup of the FMCW LIDAR data corresponds to an additional vehicle. Block 1004 may optionally include subblock 1004A, where the system determines that the subgroup has a vehicle classification based on processing the FMCW LIDAR data and / or other sensor data using a trained machine learning model. In some embodiments, subblock 1004A may be omitted, and the system may determine that the subgroup corresponds to an additional vehicle without utilizing a machine learning model. For example, the system may utilize nearest neighbor and / or other clustering techniques to determine that the subgroup of the FMCW LIDAR data may correspond to an additional vehicle.

[0140] At block 1006, the system determines one or more yaw parameters for the additional vehicle based on the subset of FMCW LIDAR data. Based on the subset of FMCW LIDAR data corresponding to the additional vehicle, the system determines the yaw parameters based on the subset of FMCW LIDAR data.

[0141] Block 1006 may optionally include sub-block 1006A and / or sub-block 1006B. At sub-block 1006A, the system determines a yaw parameter based on a comparison of the first set of FMCW LIDAR data of the sub-group with the second set of FMCW LIDAR data of the sub-group. For example, the system may determine the yaw rate based on a speed difference between the first set and the second set, and based on a 3D distance between the first set and the second set. The system may determine the yaw parameter based on the comparison of the first set and the second set alone, or also based on additional comparisons of additional sets.

[0142] At subblock 1006B, the system may determine a yaw parameter based on processing the FMCW LIDAR data of the subset using the trained yaw machine learning model. For example, the system may process all or a portion of the FMCW LIDAR data of the subset using the trained yaw machine learning model to generate an output indicative of yaw rate. For example, the system may determine a yaw parameter based on processing the FMCW LIDAR data of the subset using the trained yaw machine learning model. Fig.12 The trained yaw machine learning model is trained as described in method 1200.

[0143] In some embodiments, the system determines the yaw parameter using only one of sub-block 1006A and sub-block 1006B. In other embodiments, the system determines the yaw parameter using both sub-block 1006A and sub-block 1006B. For example, the system may determine a first yaw rate based on sub-block 1006A, determine a second yaw rate based on sub-block 1006B, and determine a final yaw rate based on a combination of the first yaw rate and the second yaw rate. Fig.10 Sub-blocks 1006A and 1006B are shown in FIG. 1 , but it should be understood that additional and / or alternative sub-blocks may be included in block 1006 , such as a sub-block for determining yaw parameters using a stored 3D model of an additional vehicle.

[0144] At block 1008, the system determines whether to adapt autonomous control of the vehicle based on the determined yaw parameter. For example, the system may selectively adapt autonomous steering and / or acceleration / deceleration based at least in part on the determined yaw parameter. Block 1008 may optionally include subblock 1008A, wherein the system determines a candidate trajectory for the additional vehicle based on the determined yaw parameter, and determines whether to adapt autonomous control of the vehicle based on the candidate trajectory. For example, if the candidate trajectory for the additional vehicle and the current intended trajectory of the vehicle indicate an additional vehicle and / or the additional vehicle indicates a likelihood of a collision between the vehicle and the additional vehicle, the system may adapt autonomous control of the vehicle.

[0145] At block 1008, the system returns to block 1002 where it receives new FMCW LIDAR data (e.g., recently generated data that has not yet been processed using method 1000) and then proceeds to perform another iteration of blocks 1002, 1004, 1006, and 1008. Fig.10 1002, 1004, and / or 1006 may be performed in parallel (e.g., by multiple processors and / or threads) prior to the previous iteration of one or more subsequent blocks. Fig.10 With continued iteration of the blocks while the vehicle is traveling, yaw parameters of additional vehicles may be continuously determined and updated (when additional vehicles are present), and autonomous control of the vehicle selectively adapted based on the yaw parameters.

[0146] Reference now Fig.11 , a flowchart illustrating another example method 1100 of determining a yaw parameter and controlling an autonomous vehicle based on the determined yaw parameter is provided. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. The system may include various components, such as Figure 1At the same time, although the operations of method 1000 are shown in a particular order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0147] At block 1102, the system receives FMCW LIDAR data from a FMCW LIDAR component of a vehicle. As a non-limiting working example, the FMCW LIDAR data may be a range Doppler image.

[0148] At block 1104, the system determines a yaw parameter for the additional vehicle based on the received FMCW LIDAR data. Block 1104 may optionally include sub-block 1004A, in which the system determines the yaw parameter based on processing the FMCW LIDAR data using a trained yaw machine learning model. Continuing with the working example, the system may process all or a portion of the range Doppler image using the trained yaw machine learning model to generate an output indicating at least the yaw rate of the additional vehicle captured in the range Doppler image. For example, the system may process the entire range Doppler image to generate an output indicating the yaw rate of the additional vehicle and an indication of the positioning of the additional vehicle in the range Doppler image. Similarly, for example, the system may process a portion of the range Doppler image corresponding to a single additional vehicle and generate an output indicating the yaw rate of the single additional vehicle. The trained yaw machine learning model may, for example, be used as described below with respect to Fig.12 Trained as described in method 1200.

[0149] At block 1106, the system determines whether to adapt autonomous control of the vehicle based on the determined yaw parameter. Block 1106 may optionally include sub-block 1106A, at which the system determines candidate trajectories for the additional vehicle based on the determined yaw parameter and determines whether to adapt autonomous control of the vehicle based on the candidate trajectories.

[0150] At block 1106, the system returns to block 1102 where it receives new FMCW LIDAR data (e.g., recently generated data that has not yet been processed using method 1100) and then proceeds to perform another iteration of blocks 1104 and 1106. Fig.11 1102, 1104, and / or 1106 may be performed in parallel (e.g., by multiple processors and / or threads) prior to the previous iteration of one or more subsequent blocks. Fig.11 With continued iteration of the blocks while the vehicle is traveling, yaw parameters of additional vehicles may be continuously determined and updated (when additional vehicles are present), and autonomous control of the vehicle selectively adapted based on the yaw parameters.

[0151] Reference now Fig.12 , a flowchart illustrating an example method 1200 of training a machine learning model that can be used to determine yaw rate based on FMCW LIDAR data is provided. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. The system may include various components, such as one or more processors of one or more computing devices, such as one or more graphics processing units of one or more computing devices. At the same time, although the operations of method 1200 are shown in a particular order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0152] At block 1202, the system generates training instances that each include a training instance input and a training instance output. The training instance input includes FMCW LIDAR data. The training instance output includes an indication of a yaw rate of an additional vehicle corresponding to the FMCW LIDAR data of the training instance input. The additional vehicles may include additional autonomous vehicles and / or non-autonomous vehicles.

[0153] As a specific example, the training instance input of each training instance can include the entire FMCW LIDAR data of the instance. For example, each training instance input can be a corresponding range Doppler image, a corresponding 3D point cloud (with velocity values ​​and intensity values), or a corresponding time-varying intermediate frequency (IF) waveform. In such a specific example, the training instance output of each training instance can include, for example: a bounding box (or other restriction indication) (if present), each bounding box indicates a portion of the FMCW LIDAR data including an additional vehicle; and the yaw rate and yaw direction of each bounding box (if present). For example, for a first training instance having a training instance input including FMCW LIDAR data that does not capture any additional vehicles, the training instance output can indicate that there is no restriction indication. Similarly, for example, for a second training instance having a training instance input including FMCW LIDAR data that captures a single additional vehicle, the training instance output can indicate a single restriction indication, which indicates the portion of the FMCW LIDAR data corresponding to the single additional vehicle, and the yaw rate and yaw direction of the restriction indication. Similarly, for example, for a third training instance having a training instance input including FMCW LIDAR data capturing a plurality of additional vehicles, the training instance output may indicate a plurality of constraint indications, each of the constraint indications indicating a portion of the FMCW LIDAR data corresponding to a corresponding one of the plurality of additional vehicles, and a corresponding yaw rate and a corresponding yaw direction for each constraint indication.

[0154] As another specific example, the training instance input for each training instance can include a subset of FMCW LIDAR data for the instance, wherein the subset of FMCW LIDAR data is selected based on determining a subset of FMCW LIDAR data corresponding to an additional vehicle (e.g., based on processing the FMCW LIDAR data of the instance using a separate trained object detection and classification machine learning model). Because the subset of FMCW LIDAR data corresponding to the additional vehicle may vary from instance to instance (e.g., based on the corresponding pose of the additional vehicle at the instance), one or more techniques can be used to generate training instance inputs that are each the same size. For example, padding data can be added to a subset that is smaller than the size. Similarly, for example, the FMCW LIDAR data can be upsampled (when smaller than the size) or downsampled (when smaller than the size). In such a specific example, the training instance output for each training instance can include, for example, a yaw rate and a yaw direction.

[0155] The yaw rate of the additional vehicle for the training instance output for the training instance can be based on the determination of the yaw rate at a time corresponding to the capture time of the FMCW LIDAR data of the training instance. In some embodiments and for at least some training instances, the yaw rate can be determined based on the output of the onboard yaw rate sensor from the additional vehicle. For example, the output from the onboard yaw rate sensor can be synchronized with the FMCW LIDAR data time and used to assign a yaw rate and yaw direction to each of the multiple FMCW LIDAR data instances. In these embodiments, it should be noted that the onboard yaw rate sensor of the additional vehicle can be used to generate the training instance output for the training instance for training the yaw machine learning model. However, at inference time, the trained yaw machine learning model can be used to generate the yaw rate prediction of the additional vehicle without utilizing the data from the yaw rate sensor of the additional vehicle. For example, at inference time, the yaw rate prediction of the given instance can be generated based on processing only the FMCW LIDAR data of the given instance using the trained yaw machine learning model.

[0156] In various embodiments, the output of the onboard yaw rate sensor of the additional vehicle used to generate the training instance output can be determined based on monitoring data from the controller area network (CAN bus) of the additional vehicle or other communication network of the additional vehicle including data from the yaw sensor. In some of those various embodiments, a laptop or other computing device can be incorporated in or on the additional vehicle during movement of the additional vehicle and can be coupled to the communication network of the additional vehicle to record yaw rate sensor data and / or other data transmitted on the communication network. In additional or alternative embodiments, the onboard yaw rate sensor of the additional vehicle can be an isolated sensor of the additional vehicle that does not communicate with the CAN bus and / or other integrated communication network of the additional vehicle. For example, during movement of the additional vehicle, a device kit including a yaw rate sensor and a computing device can be incorporated in or on the additional vehicle, and the computing device can be communicatively coupled to the yaw rate sensor to record and time-stamp sensor data from the sensor.

[0157] Various techniques may be utilized to time synchronize the output of an onboard yaw rate sensor (of an additional vehicle) with the FMCW LIDAR data (of the autonomous vehicle) to enable assignment of a given yaw rate and a given yaw direction to a corresponding FMCW LIDAR data instance. For example, a computing device of an additional vehicle that records yaw rate data from an additional vehicle may time stamp the yaw rate data using a local clock of the computing device. Likewise, the local clock of the autonomous vehicle may time stamp the FMCW LIDAR data. The timestamp of the yaw rate data and the timestamp of the FMCW LIDAR data may be used to pair the FMCW LIDAR data instance with the corresponding yaw rate data. For example, the local clock may have been recently synchronized prior to collecting the data, and the timestamps are considered synchronized. Likewise, for example, a delta between local clocks may be determined (either before or after collecting the data), which delta is used to determine which timestamps of the FMCW LIDAR data correspond to which timestamps of the yaw rate data.

[0158] In some additional or alternative embodiments, and for at least some training instances, the yaw rate may be determined based on an automated yaw rate determination technique that determines the yaw rate for a given instance based, at least in part, on an analysis of one or more previous / subsequent instances of FMCW LIDAR data (and / or other sensor data) that captured the additional vehicle. In these embodiments, it should be noted that previous and / or subsequent instances of FMCW LIDAR data may be utilized when generating a training instance output for training a training instance for a yaw machine learning model. However, at inference time, a yaw rate prediction for an additional vehicle for a given instance may be generated using the trained yaw machine learning model and FMCW LIDAR data for the given instance without employing previous and / or subsequent FMCW LIDAR data. For example, at inference time, a yaw rate prediction for a given instance may be generated based on processing the FMCW LIDAR data for the given instance using the trained yaw machine learning model without processing any subsequent FMCW LIDAR data (and optionally before subsequent FMCW LIDAR data has been generated and / or received).

[0159] In block 1204 , the system selects a training instance from among the training instances generated in block 1202 .

[0160] At block 1206, the system generates a prediction based on the training instance input of the instance processed using the machine learning model. In some implementations, the machine learning model can be a neural network model, such as a feed-forward neural network model. For example, the machine learning model can be a feed-forward convolutional neural network (CNN) including one or more convolutional layers.

[0161] At block 1208, the system generates a loss based on a comparison of the prediction to the training instance output of the training instance. For example, an error can be determined based on a difference between the prediction and the training instance output, and the loss can be determined based on the magnitude of the error.

[0162] At block 1210, the system updates the machine learning model based on the loss. For example, the system can update the perceptron weights of the machine learning model by performing back propagation of the machine learning model based on the loss.

[0163] At block 1212, the system determines whether to perform further training. If the system determines to perform further training, the system performs another iteration of blocks 1204, 1206, 1208, and 1210. In some embodiments, if there are still any unprocessed training instances, the system determines to perform further training. In some embodiments, the system determines whether to perform further training based on one or more additional or alternative criteria. For example, the system can determine whether to perform further training based on whether a threshold number of training epochs have been performed, based on determining that the most recent prediction generated using the machine learning model meets a criterion, and / or based on other criteria / criteria.

[0164] If, at block 1214, the system determines that no further training is to be performed, the system proceeds to block 1214. At block 1214, the system provides the trained machine learning model for use in determining the yaw rate of the additional vehicle based on the FMCW LIDAR data. For example, the system may provide the trained machine learning model for use in determining the yaw rate of the additional vehicle based on the FMCW LIDAR data. Fig.10 Method 1000 or Fig.11 Method 1100.

[0165] Although method 1200 is described with respect to generating a loss based on predictions and training instance outputs for a single training instance, in various embodiments, a batch processing technique may be utilized in which the loss is determined based on a batch of predictions and training instance outputs for a batch of training instances. For example, multiple iterations of blocks 1204 and 1206 may be performed prior to each iteration of block 1208, and at each iteration of block 1208, the system may generate a loss based on the most recent iteration of blocks 1204 and 1206.

[0166] As described in this article, it is possible to use Fig.12The trained machine learning model trained in an embodiment of method 1200 of the present invention determines the yaw parameter of the additional vehicle. As further described, some embodiments of the trained machine learning model can be used to process LIDAR data and generate an output based on the processing of the LIDAR data, the output directly or indirectly indicating the yaw parameter of the additional vehicle captured by the LIDAR data. In some of those embodiments, the processed LIDAR data can be a subset of the LIDAR data that has been determined to correspond to the additional vehicle. For example, the additional trained machine learning model can be used to detect a portion of the LIDAR data corresponding to the additional vehicle, and the trained machine learning model can be used to process the portion of the LIDAR data. As further described, various embodiments of the trained machine learning model can be used to process the LIDAR data and generate an output based on the processing of the LIDAR data, the output indicating the limit indication, each limit indication indicating a portion of the FMCM LIDAR data corresponding to the corresponding vehicle, and the corresponding yaw rate and corresponding yaw direction of each limit indication. Thus, in those various embodiments, the trained machine learning model can be trained to perform the detection of the additional vehicle, and determine the yaw parameter of the detected additional vehicle.

[0167] A description of training a machine learning model that can be used to determine yaw rate based on FMCM LIDAR data is given. Fig.12 However, in many embodiments, method 1200 can be adapted to train a machine learning model that can be used to determine additional and / or alternative parameters for additional vehicles based on FMCM LIDAR data and / or based on other sensor data (e.g., non-FMCMLIDAR data, camera-based images, and / or other visual sensor data).

[0168] As an example of such adaptation, in a variation of step 1202, the system can generate training instances, each of which includes a training instance input having phase coherent LIDAR data (FMCM LIDAR data or other phase coherent LIDAR data) and includes a training instance output that includes an indication of the speed, acceleration, and / or heading of an additional vehicle corresponding to the phase coherent LIDAR data of the training instance input. The additional vehicles can include additional autonomous vehicles and / or non-autonomous vehicles. In the training instance output, the indication of the speed, acceleration, and / or heading for a given additional vehicle can be based on the output of a corresponding sensor from the additional vehicle. For example, the indication can be determined based on monitoring of corresponding data from a CAN bus or other communication network of the additional vehicle and / or based on monitoring of an isolated sensor of the additional vehicle that does not communicate with the CAN bus and / or other integrated communication network of the additional vehicle. Various techniques may be utilized to time synchronize the velocity, acceleration, and / or heading (of the additional vehicle) with the phase-coherent LIDAR data (of the autonomous vehicle), thereby enabling a given velocity, a given acceleration, and / or a given heading to be assigned to a corresponding phase-coherent LIDAR data instance.

[0169] As a specific example of adaptation, the training instance input for each training instance can include the entire phase-coherent LIDAR data for the instance. For example, each training instance input can be a corresponding range Doppler image, a corresponding 3D point cloud (with velocity values ​​and intensity values), or a corresponding time-varying intermediate frequency (IF) waveform. The training instance output for each training instance in this specific example can include, for example: each indicating a portion of the phase-coherent LIDAR data that includes an additional vehicle (or other limiting indication) (if any); and an indication of velocity, acceleration, and / or heading for each bounding box (if any).

[0170] As another specific example of adaptation, the training instance input for each training example can include a subset of phase coherent LIDAR data for the instance, wherein the subset of phase coherent LIDAR data is selected based on determining that the subset of phase coherent LIDAR data corresponds to an additional vehicle (e.g., based on processing the phase coherent LIDAR data for the instance using a separate trained object detection and classification machine learning model). The training instance output for each training instance in this specific example can include, for example, an indication of speed, acceleration, and / or heading for the additional vehicle.

[0171] Thus, various embodiments described herein can generate training instances that include training instance inputs based on sensor data of an autonomous vehicle and training instance outputs based on sensor data of an additional vehicle, wherein the additional vehicle is captured at least in part by the sensor data of the autonomous vehicle. Training a machine learning model based on these training instances can enable the use of a machine learning model to process sensor data from an autonomous vehicle, thereby predicting one or more characteristics of the additional vehicle captured at least in part based on the sensor data. It should be understood from the description herein that the generation of these training instances can be performed automatically and without (or with little) human intervention. For example, supervisory labels for training instances can be generated based on sensor data of additional vehicles, eliminating the need for manual reviewers to provide such supervisory labels (and avoiding the consumption of various computing resources associated with such manual labeling), and / or enabling accurate automatic supervisory labeling in the event that a human is unable to generate labels based on a review of the training instance input.

Claims

1. A method comprising: receiving, from a phase coherent light detection and ranging (LIDAR) component of the vehicle, a set of LIDAR data points for a sensing period of a phase coherent LIDAR component, each of the set of LIDAR data points indicates a corresponding instantaneous range and a corresponding instantaneous velocity of a corresponding point in an environment of the vehicle, and each of the set of LIDAR data points is generated based on a corresponding sensing event of the phase coherent LIDAR component during the sensing period; determining that a subset of the LIDAR data points of the group corresponds to a dynamic object in the environment; Based on determining that the subgroup corresponds to the dynamic object: determining at least one instantaneous vehicle characteristic of the dynamic object based on a plurality of corresponding instantaneous velocities of the LIDAR data points in the subset; and Autonomous control of the vehicle is adapted based on the determined at least one instantaneous vehicle characteristic of the dynamic object.

2. The method according to claim 1, wherein: Determining at least one instantaneous vehicle characteristic of the dynamic object based on a plurality of corresponding instantaneous velocities of a plurality of the LIDAR data points in the subset includes: The at least one instantaneous vehicle characteristic is determined based on a plurality of corresponding instantaneous velocities of the LIDAR data points in the subset and based on a stored model for the dynamic object, the stored model describing geometric features of the dynamic object.

3. The method according to claim 2, wherein: The stored model is a three-dimensional model of the dynamic object.

4. The method according to claim 2, further comprising: The stored model is selected from a plurality of candidate stored models based on a determination that sensor data from the vehicle most closely corresponds to the stored model.

5. The method according to claim 4, wherein: The sensor data includes the set of LIDAR data points.

6. The method according to claim 2, wherein: Determining the at least one instantaneous vehicle characteristic based on a plurality of corresponding instantaneous velocities of the LIDAR data points in the subset and based on a stored model for the dynamic object comprises: determining a speed difference based on a comparison of a first set speed magnitude and a second set speed magnitude, wherein the first set speed magnitude is based on a first set of corresponding instantaneous speeds of the LIDAR data points of the subset, and wherein the second set speed magnitude is based on a second set of corresponding instantaneous speeds of the LIDAR data points of the subset; and The speed difference is converted into the at least one instantaneous vehicle characteristic based on a stored model of the dynamic object.

7. The method according to claim 2, wherein: Determining the at least one instantaneous vehicle characteristic based on a plurality of corresponding instantaneous velocities of the LIDAR data points in the subset and based on a stored model for the dynamic object comprises: A simulator is used to simulate movement of the stored model if corresponding instantaneous velocities of the LIDAR data points in the subset are applied to the stored model.

8. The method according to claim 1, wherein: Determining at least one instantaneous vehicle characteristic of the dynamic object based on corresponding instantaneous velocities of a plurality of the LIDAR data points in the subset includes: determining a first set of one or more of said plurality of said LIDAR data points of said subset; determining a second set based on a second set of one or more of the plurality of the LIDAR data points of the subset being spatially offset from the first set; and The at least one instantaneous vehicle characteristic is determined based on a comparison of one or more of the corresponding instantaneous speeds of the first set with one or more of the corresponding instantaneous speeds of the second set.

9. The method according to claim 8, wherein: The at least one instantaneous vehicle characteristic of the dynamic object is a lower limit yaw rate of the dynamic object, and further includes determining an additional instantaneous vehicle characteristic of the dynamic object based on a comparison between the first set and the second set, the additional instantaneous vehicle characteristic being an upper limit yaw rate.

10. The method according to claim 1, wherein: The at least one instantaneous vehicle characteristic comprises a yaw rate and a yaw direction, and wherein determining the at least one instantaneous vehicle characteristic of the dynamic object based on corresponding instantaneous velocities of the subset of the plurality of the LIDAR data points comprises: processing corresponding instantaneous velocities of a plurality of said LIDAR data points of said subset using the trained neural network model; generating an output indicative of the yaw rate and the yaw direction based on processing the set of the LIDAR data points using the trained neural network model; and The yaw rate and the yaw direction are determined based on the output.

11. The method according to claim 1, wherein: The at least one instantaneous vehicle characteristic of the dynamic object is a speed difference indicative of a yaw rate, and wherein adapting autonomous control of the vehicle based on the determined at least one instantaneous vehicle characteristic of the dynamic object comprises adapting a speed of the vehicle and / or a direction of the vehicle based on the speed difference exceeding a threshold value.

12. The method according to claim 1, wherein: Adapting autonomous control of the vehicle based on the determined at least one instantaneous vehicle characteristic of the dynamic object comprises: determining at least one candidate trajectory for the dynamic object based on the determined at least one instantaneous vehicle characteristic; and Autonomous control of the vehicle is adapted based on the at least one candidate trajectory.

13. The method according to claim 12, wherein: The dynamic object is an additional vehicle, and wherein adapting autonomous control of the vehicle based on the at least one candidate trajectory includes performing autonomous avoidance steering.

14. The method according to claim 1, wherein: The LIDAR component is a LIDAR monopulse component, and wherein the corresponding sensed events of the LIDAR component each include a first receiver sensed event at a first coherent receiver of the LIDAR monopulse component and a second receiver sensed event at a second coherent receiver of the LIDAR monopulse component, the first coherent receiver being positionally offset from the second coherent receiver.

15. The method according to claim 14, wherein: The set of LIDAR data points are super-resolution LIDAR data points generated based on combining the first receiver sensed events and the second receiver sensed events.

16. A method comprising: receiving LIDAR data capturing the vehicle's environment from a phase-coherent light detection and ranging (LIDAR) monopulse component of the vehicle, the LIDAR data indicating, for each of a plurality of points in an environment of the vehicle, at least one corresponding instantaneous range and at least one corresponding instantaneous velocity based on corresponding sensed events of a LIDAR monopulse assembly, the corresponding sensed events of the LIDAR monopulse assembly each comprising a first receiver sensed event at a first coherent receiver of the LIDAR monopulse assembly and a second receiver sensed event at a second coherent receiver of the LIDAR monopulse assembly, the first coherent receiver being positionally offset from the second coherent receiver; determining that a subset of the LIDAR data corresponds to a dynamic object in the environment; determining an instantaneous vehicle characteristic of the dynamic object, the determining being based on a plurality of corresponding instantaneous velocities indicated by the LIDAR data of the subset: The instantaneous vehicle characteristics are provided to one or both of: a planning subsystem of the vehicle and a control subsystem of the vehicle.

17. The method according to claim 16, wherein: The LIDAR data is super-resolution LIDAR data generated based on combining the first receiver sensing events and the second receiver sensing events.

18. The method according to claim 17, wherein: Combining the first receiver-sensed event and the second receiver-sensed event includes combining using one or both of: a frequency domain technique and a spatial domain technique.

19. The method according to claim 16, wherein: in, The dynamic object is an additional vehicle, and wherein providing the instantaneous vehicle characteristic to one or both of the planning subsystem and the control subsystem comprises providing an instantaneous yaw rate to the planning subsystem.

20. An autonomous driving vehicle, comprising: Phase coherent light detection and ranging (LIDAR) components; one or more processors that execute stored computer instructions to: receiving LIDAR data of the environment of the captured vehicle from the phase coherent LIDAR component, the LIDAR data indicating, for each of a plurality of points in an environment of the vehicle, at least one corresponding instantaneous range and at least one corresponding instantaneous velocity based on a corresponding sensed event of a LIDAR monopulse component; determining at least one instantaneous vehicle characteristic of a dynamic object in the environment of the vehicle based on a plurality of corresponding instantaneous velocities indicated by the LIDAR data: Autonomous control of the vehicle is adapted based on the determined at least one instantaneous vehicle characteristic of the dynamic object.

Images