Attitude generation via lidar sensor measurements

Abstract

The present disclosure provides attitude generation via lidar sensor measurements. A computer includes a processor and a memory, and the memory stores instructions executable by the processor to: generate a set of points from a measurement scan obtained by a lidar sensor; and generating an expected termination distance for the set of points based on the neuroimplicit representation of the set of points. The instructions may additionally be used to calculate a loss function based on the expected termination distance of a learning point of the set of points, the loss function including a relatively low margin related to a variance or standard deviation of a training distribution centered on the learning point, the learning point being learned by the neuroimplicit representation. The instructions may additionally be used to generate key frames from the set of points and to generate a pose of the lidar sensor based on the key frames.

Description

Technical Field

[0001] This disclosure describes techniques for computing the pose of a sensor (e.g., a lidar sensor) using a representation of a scene obtained via output signals from the lidar sensor. Background Art

[0002] Modern vehicles typically include a variety of sensors. Some sensors detect static or moving objects outside the vehicle, such as other vehicles, lane markings of the road, traffic lights and / or signs, animals, natural objects, etc. Types of vehicle sensors include radar sensors, ultrasonic sensors, scanning laser rangefinders, and light detection and ranging (lidar) devices. Summary of the invention

[0003] In an example, a scene can be represented using a multilayer perceptron that implements a neural implicit representation of the scene. In an example, the neural implicit representation includes characteristics of a neural radiance field, except that the neural radiance field operates using images captured by a camera, which can then be viewed from perspectives not included in the training dataset. In contrast, the neural implicit representation as described herein relies, at least in some instances, on measurements obtained using a lidar sensor (i.e., without using output data from a camera), and operates without radiance estimates.

[0004] As described herein, a neural implicit representation operates to model the geometry of a scene via training of a neural network, which may be, for example, a multilayer perceptron trained to implicitly represent a scene using an output signal representing lidar measurements obtained during a lidar measurement scan. In this context, a multilayer perceptron refers to a fully connected feedforward artificial neural network having at least three layers (an input layer, an output layer, and at least one hidden layer). In an example, the input layer operates to receive a query (e.g., And the output layer operates to provide a decision or prediction in response to the received query. The combination of the input layer, hidden layer, and output layer of the multilayer perceptron operates as a computing resource and is capable of approximating many continuous or discontinuous functions. In this context, "function" means a three-dimensional function (e.g., F(x, y, z)) that expresses a three-dimensional position (e.g., x coordinate, y coordinate, and z coordinate). In an example, the volume density of each point of the scene can be rendered by the multilayer perceptron relative to the observation position. In an example, the function can be about a novel posture (e.g., x coordinate, y coordinate, z coordinate, and rotation around the x, y, and z coordinate axes) of a lidar sensor (e.g., lidar sensor 108A). As further described below, using a multilayer perceptron, a scene can be represented via a set of discrete points obtained by a measurement scan performed by a lidar sensor. In an example, at each point of the set of points, a density value can indicate whether the point is occupied (i.e., filled with an object) or unoccupied (i.e., free space, sky, or another position where a reflected signal transmitted by a lidar sensor is not detected).

[0005] In an example, a computer may be programmed to train a multilayer perceptron implementing a neural implicit representation using continuous sampling iterations. In response to continuous sampling iterations, a relatively low margin (ε) may be assigned to "learned" points, which in this context means lidar measurement points in the scene that have previously been determined to be occupied or unoccupied by the neural implicit representation. Also in this context, a higher margin (ε) may be assigned to points in the scene that have not previously been determined to be occupied or unoccupied by the neural implicit representation (e.g., unlearned points). Also in this context, the margin (ε) may be related to the variance or standard deviation of a training distribution centered at the point (e.g., learned point or unlearned point). Thus, in response to continuous sampling iterations, lidar measurements of points representing the geometry of the scene may converge with a reduced "loss," which in this context means the difference between the actual three-dimensional scene geometry and the geometry predicted via the neural network implicit representation. Similarly, a "loss function" is meant to express the difference (e.g., error) between the density value of a point measured via a lidar sensor (e.g., 108A) and the predicted or expected density calculated by the multilayer perceptron. As described herein, by utilizing data about points learned by the multilayer perceptron during previous lidar measurement sweeps, the loss of the neural implicit representation of points of a scene can be reduced by informing the neural implicit representation of such previous measurements.

[0006] In an example, in response to successive sampling iterations, the pose of a lidar sensor may be generated or calculated based on a neural implicit representation of a scene using lidar measurements. The pose of the lidar sensor may be refined in response to additional keyframes, which may operate to further inform the neural implicit representation, thereby further reducing the loss between the actual three-dimensional scene geometry and the geometry predicted by the neural implicit representation. In an example, after calculating and / or refining the pose of the lidar sensor, the generated and / or refined pose may be transmitted to a vehicle autonomous driving application of a vehicle (e.g., vehicle 100) on which a lidar sensor (e.g., lidar sensor 108A) is mounted. The vehicle may then perform motion planning as part of an autonomous / semi-autonomous vehicle driving application. In an example, vehicle pose calculation and / or vehicle motion planning may be performed without an input signal from a camera (such as a camera mounted on vehicle 100).

[0007] An example system may include a computer having a processor and a memory including instructions executable by the processor to generate a set of points from a measurement scan obtained by a lidar sensor. The instructions may also be used to generate an expected termination distance for the set of points based on a neural implicit representation of the set of points, and to calculate a loss function based on the expected termination distance of a learning point of the set of points, the loss function including a relatively low margin related to a variance or standard deviation of a training distribution centered on the learning point, the learning point being learned by the neural implicit representation. The instructions may also be used to generate a keyframe from the set of points and generate a pose of the lidar sensor based on the keyframe.

[0008] In an example, the instructions for generating a pose of the lidar sensor may additionally include instructions for modifying the pose of the lidar sensor to align with an implicit representation of the scene.

[0009] In an example, the instructions for calculating the loss function may include instructions for performing the following operations: based on the unlearned points in the set of points not being learned by the neural implicit representation, assigning a relatively higher margin associated with the variance or standard deviation of the training distribution centered on the unlearned points.

[0010] In an example, the instructions may additionally be used to transmit the generated pose of the lidar sensor to an autonomous vehicle driving application.

[0011] In an example, the instructions may additionally be for performing, by the autonomous vehicle driving application, motion planning based on the generated pose.

[0012] In an example, the calculated loss may be based on a combination of the primary loss and the opacity loss of the learning point.

[0013] In an example, the instructions for calculating the loss function include instructions for calculating a depth loss for a learning point, wherein the depth loss represents a difference between an expected distance to the learning point based on the neural implicit representation and a distance extracted from the measurement scan.

[0014] In an example, the instructions for calculating the loss function may include instructions for calculating the gradient of the loss function, and include instructions for performing the following operations: utilizing the calculated gradient to update the generated pose estimate and weights of the neural implicit representation via gradient descent to reduce the magnitude of the loss function.

[0015] In an example, the instructions may additionally be for calculating a margin of the learning point based on the neural implicit representation of the learning point and a weight of the learning point derived from the measurement scan.

[0016] In an example, the instructions may additionally be for calculating margins for both the learned and unlearned points based on neural implicit representations of the learned and unlearned points and weights of the learned or unlearned points derived from a measurement scan.

[0017] In an example, the instructions may further be for allocating a minimum margin to the learning point in response to the allocated margin being less than a first threshold.

[0018] In an example, the instructions may additionally be for allocating a maximum margin to the learning point in response to the allocated margin being greater than a second threshold.

[0019] In an example, the instructions may additionally be for assigning a zero weight to any point in the set of points in response to an absence of a return signal transmitted during the measurement scan.

[0020] In an example, the instructions may additionally be for generating a mesh representation of the measurement scan based on the neural implicit representation of the set of points.

[0021] In an example, the neural implicit representation may include a continuous function of an observation position of the lidar sensor measurement scan.

[0022] In an example, the neural implicit representation may include a continuous function representing the geometry of the three-dimensional scene.

[0023] In an example, a method may include: generating a set of points from a measurement scan obtained by a lidar sensor; and generating an expected termination distance for the set of points based on a neural implicit representation of the set of points. The method may further include calculating a loss function based on the expected termination distance of a learning point of the set of points, the loss function including a relatively low margin assigned to the learning point, the learning point being learned by the neural implicit representation. The method may further include generating a keyframe from the set of points, and generating a pose of the lidar sensor based on the keyframe.

[0024] In an example, the method may further include assigning a relatively higher margin to an unlearned point in the set of points based on the unlearned point not being learned by the neural implicit representation.

[0025] In an example, the method may further include transmitting the updated pose of the lidar sensor to an autonomous vehicle driving application.

[0026] In an example, the method may additionally include performing, by the autonomous vehicle driving application, motion planning based on the generated pose.

[0027] In an example, the method may additionally include assigning a zero weight to any point in the set of points based on an absence of a return signal received in response to a signal transmitted during the measurement scan. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a block diagram of an example vehicle.

[0029] Figure 2 is a diagram showing processing of lidar sensor measurements and rendering of objects of a scene external to the lidar sensor.

[0030] Figure 3A is a graph showing the sample weights resulting from queries to the neural implicit representation and the distances to unlearned measurement points.

[0031] Figure 3B is a graph showing the sample weights and distances to the learned measurement points resulting from queries to the neural implicit representation.

[0032] Figure 4A Depicts the scene outside the lidar measurement system.

[0033] Figure 4B yes Figure 4A Representation of an image obtained via LiDAR measurement points.

[0034] Figure 5 is a flow chart of a method for generating pose via lidar measurements. DETAILED DESCRIPTION

[0035] refer to Figure 1 , the vehicle 100 may be any passenger or commercial vehicle, such as a car, truck, sport utility vehicle, crossover, van, minivan, taxi, bus, etc. The vehicle 100 may include a computer 104, a communication network 106, vehicle components 110, a human machine interface (HMI) 112, a communication interface 114 (e.g., to provide Wi-Fi communication, communication with a satellite or terrestrial network, communication with other vehicles, etc.), and a sensor group 108. The sensor group 108 may include a lidar scanner or other lidar-based measurement system, as indicated by the lidar sensor 108A, which may be operated to determine the distance between the lidar sensor 108A and any static or moving objects in the driving environment of the vehicle 100 as the vehicle proceeds along the path 50. The sensor group 108 may include other sensors, such as radar sensors, ultrasonic sensors, vehicle navigation sensors (e.g., sensors of a satellite positioning system), etc.

[0036] The vehicle component 110 includes a propulsion system to convert stored energy (e.g., gasoline, diesel fuel, electric charge) into motion to propel the vehicle 100. The vehicle component 110 may include a conventional vehicle propulsion subsystem, such as a conventional powertrain system, which includes an internal combustion engine coupled to a transmission that transmits torque generated by the engine to the wheels of the vehicle 100. The vehicle component 110 may also include a hybrid powertrain system that utilizes elements of a conventional powertrain system and an electric powertrain system; or may include any other type of powertrain system. The vehicle component 110 may include an electronic control unit (ECU) or the like that communicates with and receives input from the computer 104 and / or a human operator. The human operator may control the propulsion system via, for example, a pedal and / or a gear shift lever.

[0037] The vehicle component 110 may include a conventional vehicle steering subsystem to control the rotation of the wheels of the vehicle 100. The steering subsystem may include a rack-and-pinion steering member with electric power steering, a steer-by-wire system, or another suitable system. The steering subsystem may include an electronic control unit (ECU) or the like that communicates with and receives input from the computer 104 and / or a human operator. The human operator may control the steering subsystem via, for example, a steering wheel.

[0038] The HMI 112 presents information to and receives information from an operator of the vehicle 100. The HMI 112 may include controls and displays located, for example, on an instrument panel in a passenger compartment of the vehicle 100, or may be located at another location accessible to an operator of the vehicle 100. The HMI 112 may include dials, digital readouts, screens, speakers, etc., for providing information to an operator of the vehicle 100. The HMI 112 may include buttons, knobs, keypads, microphones, etc., for receiving information from an operator.

[0039] The computer 104 of the vehicle 100 may include a microprocessor-based computing device, such as a general-purpose computing device (including a processor and memory, an electronic controller, etc.), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a combination of the foregoing, and the like. In an example, a hardware description language such as VHDL (VHSIC (Very High Speed ​​Integrated Circuit) Hardware Description Language) may be used in electronic design automation to describe digital and mixed signal systems such as FPGAs and ASICs. For example, an ASIC is manufactured based on VHDL programming provided before manufacturing, and the logic components inside the FPGA may be configured based on, for example, VHDL programming stored in a memory electrically connected to the FPGA circuit. Therefore, the computer 104 may include a processor, a memory, and the like. The memory of the computer 104 may include a tangible medium for storing instructions executable by a processor and for electronically storing data and / or a database. Alternatively or in addition, the computer 104 may include a structure such as the aforementioned structure that provides programming. In an example, the computer 104 may be a plurality of computers connected together to operate as a single computing resource.

[0040] The computer 104 can transmit and receive data via the communication network 106. The communication network 106 can include, for example, a controller area network (CAN) bus, Ethernet, WiFi, a local interconnect network (LIN), an on-board diagnostic connector (OBD-II), and / or any other wired or wireless communication network. The computer 104 can be communicatively coupled to the lidar sensor 108A, the vehicle component 110, the HMI 112, the communication interface 114, and other vehicle systems and / or subsystems via the communication network 106.

[0041] The lidar sensor 108A may include a scanning lidar transmitter and receiver that can detect the distance of an object by emitting laser pulses of a specific wavelength and measuring the flight time of the pulses to an object (such as a static or moving object in the driving environment of the vehicle 100) and returning to the lidar sensor 108A. The lidar sensor 108A may also include a transmitter that transmits a continuous beam and measures the phase shift in the received signal. Thus, the lidar sensor 108A may include any suitable type of scanning lidar signal transmitter and receiver that can cooperate to provide lidar sensor measurements to the computer 104. Thus, in an example, the lidar sensor 108A may be arranged to include a stack of separate lidar signal transmitters and receivers oriented in an upward direction relative to the vehicle structure 103, to which the lidar sensor 108A is mounted. In such an arrangement, the lidar sensor 108A may be scanned to transmit lidar signals and receive lidar signals reflected from objects located in any direction relative to the vehicle body 102. In an example, the lidar sensor 108A may perform a sweeping lidar scan to detect an object outside the vehicle body 102 by first scanning the lidar sensor in a direction to the right of the vehicle body 102, then scanning in an area in a vehicle forward direction relative to the vehicle body 102, and terminating by scanning an area to the left of the vehicle body 102. In other examples, the lidar sensor 108A may include a spindle-type lidar, a solid-state lidar, a flash lidar, etc.

[0042] Exemplary System Operation

[0043] Figure 2 1 is a diagram showing the processing of lidar sensor measurements and the rendering of objects external to the lidar sensor 108A. In the example, the pose tracking component 210 and the scene mapping component 240 represent a set of programming instructions that operate to perform simultaneous localization and mapping (SLAM) by dividing the position and mapping into two separate processing threads. Figure 2In the example of , the pose tracking component 210 operates to obtain an initial estimate of the pose of the lidar sensor 108A. The scene mapping component 240 can then use the initial pose estimate in loss calculations and optimization operations to reconstruct the scene outside the lidar sensor 108A (mounted on the vehicle structure 103). The pose tracking component 210 executes programming to implement iterative closest point (ICP) using continuous measurement scans of the lidar sensor 108A to obtain initial pose estimates of key frames for potential selection later. To reduce motion distortion that may occur during the left and right scans of the lidar sensor 108A, which occurs for a non-zero or non-negligible duration when the vehicle 100 is in motion, a motion compensation module can be used to remove distortion in the measurement points generated by scanning the lidar measurement points selected to form the key frames. Measurement points pointing toward the sky can be sampled for later use. In this context, a "keyframe" means a set of measurement points generated by a lidar scan that is periodically or intermittently selected by the scene mapping component 240 for use in the loss calculation and optimization steps, instead of an implicit representation of a set of measurement points. In an example, the pose tracking component 210 uses a set of measurement points generated by a lidar scan that is not selected for use in a keyframe to obtain an initial pose estimate for a keyframe lidar scan. Therefore, in an example, a keyframe can be constructed from measurement points generated by a lidar scan from a lidar sensor 108A. The scene mapping component 240 operates to reconstruct the scene 205 based on the keyframe. The scene mapping component 240 performs the optimization process using a window that includes the current keyframe from the pose tracking component 210 and a keyframe previously obtained from the pose tracking component 210. In this context, a "window" means a set of keyframes gathered or collected within an interval before the current keyframe. For example, the window may include a current keyframe and keyframes that are clustered or collected at a predetermined number of time intervals (e.g., 0.2 second intervals, 0.5 second intervals, 1.0 second intervals, etc.) prior to the current keyframe (e.g., 4, 5, 7, 9, etc.). In an example, the scene mapping component 240 randomly selects previously obtained keyframes from the pose tracking component 210 for use during the optimization process. In an embodiment, the randomly selected keyframes are used in conjunction with the time-based keyframe selection operation to improve efficiency in the SLAM process, which can be confirmed by ablation studies. In this context, "ablation" means removing information about Figure 2 A portion of one or more procedural steps described to determine the contribution of the removed procedural steps to enhancing scene reconstruction and refining pose estimation.

[0044] like Figure 2 As shown in , lidar sensor 108A can output signal measurements representing a scene (such as scene 205) for parallel processing by gesture tracking component 210 and scene mapping component 240. Figure 2 In the example of FIG. 2 , the pose tracking component 210 operates to process a set of points from a measurement scan obtained in response to the lidar sensor 108A scanning the scene 205. The pose tracking component 210 can be operated to estimate the range using an iterative closest point (ICP) alignment algorithm (via the ICP alignment component 212) to estimate the motion of the lidar sensor 108A via the motion compensation component 214. Figure 2 In the example of , the ICP alignment component 212 operates to minimize the difference between the two sets of measurement points obtained during the measurement scan of the lidar sensor 108A. The ICP alignment component 212 can operate by first determining the correspondence between the measurement points in the first set of measurement points and the measurement points in the second set of measurement points. During the alignment process, the ICP alignment component 212 performs continuous iterations until the minimum distance between the corresponding points of the first set of measurement points and the second set of measurement points is reached. Alternatively or in addition, the ICP alignment component 212 can operate to minimize the difference between the first set of measurement points and the plane including the second set of measurement points.

[0045] In an example, use of the ICP alignment component 212 allows programming of the computer 104 to compensate for movement of the lidar sensor 108A as the vehicle 100 moves along the path 50. In an example, the ICP alignment component 212 can utilize an identity transform as an initial alignment estimate of the vehicle 100 relative to features present in the scene 205. In response to the alignment estimate, the pose of the lidar sensor 108A can be modified to align with a keyframe representing the scene 205 during a measurement interval of the lidar sensor 108A, or with features within a keyframe. The pose tracking component 210 additionally includes a sky sampling component 216 that operates to determine whether any area of ​​the lidar scan in which the sensor did not detect a point corresponds to, for example, a sky area oriented at a positive angle relative to the lidar sensor 108A. The sky sampling component 216 can additionally operate to reduce or eliminate random noise in the estimated sky area.

[0046] The scene mapping component 240, operating in parallel with the pose tracking component 210, can utilize the output data representing the measurement scans from the lidar sensor 108A during the current (ongoing) scan, and utilize selected previous scans as key frames via the key frame selection component 242. Figure 2In an example of , the neural implicit representation 250 can be updated using keyframes, thereby operating via the programming of the computer 104. In the example, the neural implicit representation 250 is a fully connected neural network combined with a multi-level hierarchical feature grid encoding. Updating the neural implicit representation 250 can include updating the weights of the constituent nodes of the layers (e.g., all layers) of the fully connected neural network and the parameters of the multi-level hierarchical feature grid encoding. The point sampling component 244 cooperates with the loss function calculation component 246 to sample points obtained during the lidar measurement scan and calculate the loss between the predicted values ​​or expected values ​​output from the neural implicit representation 250.

[0047] exist Figure 2 In the example of , the signal measurements representing points obtained from the measurement scan via the lidar sensor 108A can be extracted to a fixed frequency, such as 2 Hz, 5 Hz, 10 Hz, etc. In the example, the relative transformation P is estimated using point-to-plane iterative closest point estimation using the ICP alignment component 212 i-1 , i∈SE(3), where SE(3) indicates both translational and rotational motion in three-dimensional space from the previous scan to the current scan. This estimate can be refined later via mapping optimization and sky sampling via sky sampling component 216. Lidar sensor 108A(x i ∈SE(3)) can then be estimated as In response to obtaining the previous pose and the current pose, the lidar sensor 108A may be motion compensated using the motion compensation component 214, such that, for example, a constant velocity is assumed between motions between scans obtained from the lidar sensor 108A.

[0048] Scene 205 can be represented as a multilayer perceptron in a hierarchical multilevel feature grid. In an example, a three-dimensional feature grid can be constructed starting from random initialization values ​​uniformly distributed in a three-dimensional space. The input X, Y, Z points in a Cartesian coordinate system can be encoded by first determining in which cell the input X, Y, Z points fall in the three-dimensional feature grid. Secondly, the features on the eight corners can be linearly interpolated to obtain output features. The process can be repeated at different resolution levels. The results can be stacked or assembled into a feature vector, which represents the features at different resolution levels. The output vector can then be passed through a multilayer perceptron. Therefore, the features of the feature grid can be learned and updated via back propagation of a multilayer perceptron through a neural implicit representation 250. As previously described, back propagation through a multilayer perceptron can include updating the weights of the constituent nodes of a layer (e.g., all layers) of a fully connected neural network. Such a process can operate to provide a learned multi-level three-dimensional feature grid encoding.

[0049] In this example, during online training, the parameters of the multilayer perceptron and the feature grid are updated (e.g., query θ) to predict the volume density (σ) for each measurement point measured during the scan obtained via the lidar sensor 108A. In a training exercise, such as training a multilayer perceptron and estimating the distance to a measurement point (e.g., point 206), a volume rendering procedure may be followed. For example, for Represents a lidar signal (e.g., ray) with an origin and direction Example distance t i ∈[t 近 , t 远 ], which generates Ns samples, In the example, t 近 and t 远 may correspond to the measurement range of the lidar sensor 108A (e.g., t 近 = 0.5 m, 1.0 m, etc.), and t 远 (e.g., 100 meters, 200 meters, etc.) may depend on the scale of the scene 205. In an example, a feature grid and a multi-layer perceptron (which may be collectively referred to as To predict the occupancy state σ i In an example, the transmittance T of a measurement point (eg, point 206 ) may be calculated according to the following expressions (1) and (2): i and weight w i :

[0050]

[0051] w i =T i σ i (2)

[0052] where δ j =t j+1 -t j , and σ i is the sample s predicted by the multilayer perceptron i The expected volume density at . The weight w i The loss function (e.g., calculated by the loss function calculation component 246) is used to represent the probability that the measurement point (e.g., 206) of the ray (e.g., transmitted via the lidar sensor 108A) is predicted or expected (e.g., by the multilayer perceptron) to terminate at the predicted or expected measurement point. Therefore, the ray can be estimated according to the following expression (3): Expected end distance of:

[0053]

[0054] The weight w i corresponds to the weights used by the loss function, and where t i ∈[t 近 , t 远 ]. The scene mapping component 240 operates to receive scanning lidar measurements (performed by the lidar sensor 108A) from the pose tracking component 210. The scene mapping component 240 operates to determine whether a keyframe is formed. In an example, in response to selecting (e.g., randomly selecting) an optimization window of keyframes, the scene mapping component 240 operates on the optimization window to jointly optimize the neural implicit representation and estimated pose of each keyframe in the optimization window. In an example, the keyframe selection component 242 selects keyframes on a temporal basis, meaning that in response to a predetermined time having passed since the previous keyframe was accepted, the keyframe selection component 242 adds a new keyframe. In response to accepting a new keyframe, the scene mapping component 240 can update the optimizer programmed to operate via the keyframe selection component 242 based on the accepted new keyframe and the previously accepted keyframe. In the example, a total of N is used in the update. w keyframes, including the newly accepted keyframe and N randomly selected keyframes w -1 previous keyframe.

[0055] In the example, in response to selecting (e.g., randomly selecting) a window of keyframes, the scene mapping component 240 uses the keyframes and past keyframes to jointly optimize the pose and neural implicit representation of the lidar sensor. In this context, and "optimization window" means a total of N w In this example, for a keyframe with an estimated pose in the global reference frame Distorted Vector Key frame KF i , where the symbol Indicates three values ​​expressing angular rotational velocity (e.g., relative to the pitch axis, the roll axis, the yaw axis, or in another coordinate system) and a vector of values ​​expressing translational velocity (e.g., relative to the X axis, the Y axis, and the Z axis, or in another coordinate system). In the example, for Represents the estimated pose The axis-angle representation of the rotational components and Both, and represent the translation component. In the forward pass scene mapping component 240, the twist vector Can be converted back to stance and is used to calculate the starting point of the signal transmitted via the lidar sensor 108A. In an example, a set of points (N) from a measurement scan obtained by the lidar sensor 108A may be R) is randomly sampled to provide N representations of each ray using the occupancy grid. S distance samples.

[0056] The loss function calculation section 246 utilizes the dynamic headroom loss, which is combined with the depth loss and the sky loss according to the following expression (4):

[0057]

[0058] in represents the dynamic headroom loss calculated using Jensen-Shannon divergence, and where and represents the depth loss and sky loss of a lidar measurement point in response to a transmission signal from a lidar sensor (e.g., 108A). The constant λ 1 and λ 2 represents a positive weight value. In this example, λ 1 and λ 2 An integer value (e.g., 1, 2, 3, 5, etc.) may be assigned, or a floating point value (e.g., 0.1, 0.2, 1.1, 1.2, etc.) may be assigned. In an example, the loss calculated via the Jenson-Shannon divergence of expression (4) operates to calculate the dynamic headroom of the lidar measurement point in response to the transmission signal from the lidar sensor 108A. In an example, the use of the dynamic headroom loss enhances training convergence and reconstruction accuracy.

[0059] In a typical formulation of the loss function, a single margin is used for each lidar measurement point in response to a transmission signal from a lidar sensor (e.g., 108A). In contrast, expression (4) allows for the assignment of a unique (e.g., dynamic) margin related to the variance or standard deviation of a training distribution centered at each point. Thus, for each ray terminating at a measurement point in response to a transmission signal from a lidar sensor (e.g., 108A), The samples obtained along the ray can be expressed as and z * , which operates to represent each ray As discussed previously, Indicates that the end is at a point with a direction The starting point of the ray of the measurement point. i Represents individual training samples along the ray In this context, the truncated Gaussian distribution has a bounded domain parameterized by a relatively low margin related to the variance or standard deviation of the training distribution centered at the learning point. In the example, the margin (ε) is taken as the training distribution, where (i.e., a standard deviation of ε / 3), where the margin ε / 3 represents an example of a relatively low margin. In other examples, other margins such as ε / 2, ε / 4, ε / 5, etc. may be utilized. Thus, the target weight may be expressed as In this example, the Jensen-Shannon loss function can be expressed according to the following expression (5):

[0060]

[0061] The amount represents the main loss, and the amount ||1-∑ i ω 1 || 1 Denotes the opacity loss. In the example, the opacity loss provides a means to weight each ray so as to maintain a sum of 1.0 or less. Thus, the Jensen-Shannon loss function of expression (5) is calculated based on a combination of the primary loss and the opacity loss of a set of lidar measurement points of the scene.

[0062] In applications performing simultaneous localization and mapping (SLAM), continuous optimization, sparse sampling, and incremental input from a lidar sensor (e.g., lidar sensor 108A), different regions of a scene can be learned (e.g., by neural implicit representation 250) at different rates during online training. For example, in an image reconstructed from lidar measurements (e.g., image 260), a scan of region 262 of scene 205 can be learned at a different rate than region 264. Mesh representation 270 can be generated via a lidar sensor 108A transmission signal and then sampling measurement points along the transmission signal to form a query to neural implicit representation 250. In response to the query, neural implicit representation 250 can estimate a weight (w i ), whose value is between 0 and 1. Then, the threshold can be set to i ) classifies free space and occupied space to generate a mesh representation 270.

[0063] As reference Figure 3A and Figure 3BAs described (below), use of the Jensen-Shannon loss function allows rays that terminate at lidar measurement points for scenes with unknown geometry to be assigned a relatively high margin (ε), while relatively low margin (ε) can be assigned to rays that terminate at measurement points for scenes with known or learned geometry. This assignment of high margin / low margin allows the neural implicit representation 250 to learn new regions while maintaining and refining the learned geometry. In an example, in response to multiple rays terminating at measurement points with relatively small margins, the multiple rays can be utilized in a keyframe generated for use by the scene mapping component 240. The scene mapping component 240 can then generate a pose of the lidar sensor based on the updated (learned) weights of the neural implicit representation 250 based on the generated keyframe.

[0064] In the backward pass, the weights of the neural implicit representation 250 can be used to calculate the loss function through the scene mapping component 240. The neural implicit representation 250, the feature grid parameters θ and the warp vector The gradient is calculated (e.g., gradient descent) to reduce the magnitude of the loss function. In response to the optimization being completed, the optimized twist vector Convert to SE(3) transformation matrix The scene mapping component 240 transforms the matrix The optimized pose is transmitted to the pose tracking component 210 so that future tracking can be performed relative to the optimized pose.

[0065] The mesh representation 270 can help visualize and calculate metrics related to the accuracy of the scene representation generated by the scene mapping component 240. The mesh representation 270 can be generated from the geometry of the scene 205 as learned by the neural implicit representation 250, which can involve simulating a lidar sensor 108A placed as an estimated keyframe pose. The neural implicit representation 250 can calculate weights along rays terminating at lidar measurement points, which can then be approximated in a 3D mesh. In response to the presence of multiple weights within the same grid cell, the maximum of the multiple weights can be retained. A computer graphics algorithm can then be used to form a mesh based on the results. In an example, in addition to the processing steps performed by the computer 104, such a process can be run offline for visualization and evaluation.

[0066] In an example, a Jensen-Shannon loss function (described with reference to expressions (6) and (7) below) can be used to measure the difference between the target distribution and the sample distribution of a lidar measurement point in response to a transmission signal from a lidar sensor (e.g., 108A). For example, a learning area can have similar target and sample distributions, which results in a smaller Jensen-Shannon divergence between the target distribution and the sample distribution. Therefore, Figure 3Ais a graph (300) showing sample weights 305 resulting from a query of a neural implicit representation and distances with respect to unlearned measurement points. Figure 3A The vertical axis of indicates the weight (w) predicted by the neural implicit representation 250 of a ray terminating at a lidar measurement point relative to the distance (d) from the lidar sensor 108A. The target distribution 310 represents the distribution where σ* has been assigned the minimum value ε 最小 / 3. Alternatively, the sample distribution (e.g., 315) can be defined as Indicates that and represents the mean and standard deviation of the predicted weights (e.g., predicted by the neural implicit representation 250) along the ray terminating at the lidar measurement point. The dynamic margin (ε) can then be defined according to the following expression (6): dyn ):

[0067] ε dyn =ε min (1+αJ * ) (6)

[0068] in

[0069]

[0070] Where α represents a constant scaling parameter. In an example, α may be assigned an integer value (e.g., 1, 2, 3, 5, etc.), or may be assigned a floating point value (e.g., 0.1, 0.2, 1.1, 1.2, etc.). During the training process, it is operable to enhance the target distribution 310 (e.g., and the sample distribution 315 (e.g., S = The convergence between , indicates that the weights calculated by the neural implicit representation 250 are approximately normal distributed. In the expression middle, represents the average value of the weights, and represents the standard deviation of the weight. The JS of expression (7) 最大 It can represent the upper bound of the Jensen-Shannon (JS) score, and JS 最小 In response to determining that the JS score is less than JS 最小 , we can use ε 动态 Set equal to ε 最小 .

[0071] exist Figure 3A In the example of , each of the sample weights 305 represents a response to a query (e.g., In response to the query, the neural implicit representation 250 transmits a value indicating the probability that the ray terminates at a lidar measurement point of the sample distribution 315 located at a distance (d) from the lidar sensor 108A. Figure 3A As shown in FIG. , the sample weights 305 returned or predicted by the neural implicit representation 250 appear to be relatively evenly distributed over the distance d 1 With distance d 2 This relatively uniform distribution of sample weights 305 returned by the neural implicit representation 250 indicates a low confidence in the ability of the neural implicit representation 250 to predict the distance between the lidar sensor 108A and, for example, a static object within the field of view of the lidar sensor 108A. The neural implicit representation 250 predicts the distance between the lidar sensor 108A and the measurement point Z. 1 This low confidence in the ability to predict the distance between * indicates that for a given sample distribution (e.g., sample distribution 315), the neural implicit representation 250 is not yet able to accurately predict the terminal distance from the lidar sensor 108A to the set of points in sample distribution 315.

[0072] Figure 3A Also depicted is a training distribution 320 (e.g., The training distribution 320 represents the distribution of training samples that operate to train the neural implicit representation 250 to achieve the target distribution 310. Such training may involve successive iterations of sampling the training distribution, obtaining prediction distances and weights from the neural implicit representation 250, computing a loss function, updating the weights of the neural implicit representation 250, and resampling the training distribution.

[0073] Figure 3B is a graph showing the sample weights generated by the query on the neural implicit representation and the learning measurement point Z 2 * is a graph of the distance between (350). Figure 3A Similar to the situation, Figure 3B The vertical axis indicates the axis ending at the lidar measurement point (Z 2 *) relative to the distance (d) from the lidar sensor 108A. In response to the query, the neural implicit representation 250 transmits a value to indicate the probability that the measurement point of the ray represented by the sample distribution 365 is located at the distance (d) from the lidar sensor 108A. Thus, in the example, each of the sample weights 355 represents a response to a query initiated by the scene mapping component 240 and transmitted to the neural implicit representation 250. In response to the query, the neural implicit representation 250 transmits a value to indicate the probability that the ray terminates at a lidar measurement point located at the distance (d) from the lidar sensor 108A. Figure 3BIn , a dynamic margin (ε) has been set for the training distribution 370, thus allowing refinement to the measurement point Z through the neural implicit representation 250 2 Thus, by expression (7), the neural implicit representation 250 has learned to accurately predict the distance from the lidar sensor 108A to the set of points in the training distribution 320 (such as the measurement point Z 2 *) end distance.

[0074] Therefore, reference Figure 3A and Figure 3B , the use of the Jensen-Shannon loss function operates to set a dynamic margin ε for each ray represented by a measurement point according to the similarity between the training distribution 320 and the target distribution 310. The Jensen-Shannon loss function operates to provide a scaled training distribution that sets a higher margin (ε) for measurement points of rays that terminate in unlearned regions. 动态 ), and set a lower margin for the ray terminating in the learning region to refine the neural implicit representation 250, thereby enhancing the convergence between the training distribution 320 and the target distribution 310. In the example, expression (7) operates to assign a minimum value to an unlearned measurement point (e.g., point Z) in response to the assigned margin being less than a first threshold. 1 *), and in response to the allocated margin being greater than a second threshold, the maximum value is allocated to the measurement point (e.g., Z 2 *).

[0075] Figure 4A Describes the scene outside the LiDAR measurement system. Figure 4A As shown in , scene 400 corresponds to an urban traffic scene. Building 405 is depicted on the left side of scene 400, while building 425 is depicted on the right side of scene 400. Street lamp posts 410, 420 and sky 415 are also depicted, which is depicted at a positive elevation angle relative to lidar sensor 108A in the global reference frame.

[0076] Figure 4B yes Figure 4A 4. In the example, reconstructed scene 450 depicts a reconstructed image obtained via lidar measurement points. In the example, reconstructed scene 450 depicts a reconstructed image obtained via lidar sensor 108A. In the example of processing utilizing lidar sensor measurements and neural implicit representation 250, a point representation of building 405 is shown as reconstructed building 455 at the left portion of reconstructed scene 450. A point representation of building 425 is depicted as building 475 at the right portion of reconstructed scene 450. It can be appreciated that street lamp post 410 and street lamp post 420 have been accurately rendered in reconstructed scene 450 (e.g., reconstructed lamp posts 460 and 470).

[0077] The reconstructed scene 450 additionally shows a sky 465, which can be seen as in FIG. Figure 1 , Figure 2 , Figure 3A and Figure 3B Reconstructed as described. In addition, as previously described with reference to expression (4), the depth loss of the ray terminating at the lidar measurement point is used as an additional term in calculating the loss function for the ray. In this context, the depth loss means the difference between the termination distance of the ray predicted or expected by the neural implicit representation 250 and the actual termination distance of the ray measured by the lidar sensor 108A. The depth loss can be calculated according to the following expression (8):

[0078]

[0079] Wherein reference expressions (3) and (4) describe and z * , and where the subscript "2" in expression (8) denotes the L2 (Euclidean) norm. In some instances, the depth loss may result in blurry reconstructions within a limited training time. However, in some instances, the depth loss may provide hole filling, where a set of points of a detected geometric shape appear to surround one or more points where there is no return signal from a lidar sensor (e.g., 108A). In an example, such as by replacing the weights (e.g., λ of expression (4) by 1 ) is set to a small value (such as about 10 -5 With about 10 -6 In an example, the scene mapping component 244 may assign a zero weight to any point in a set of measurement points based on the absence of a return signal received in response to a signal transmitted during a measurement scan of the lidar sensor 108A.

[0080] Moreover, as previously described with reference to expression (4), the sky loss of a ray terminating at a lidar measurement point is used as an additional term in calculating the loss function of the ray. In an example, the sky loss can be determined by observing holes in a measurement scan of a scene (e.g., scene 400) obtained via lidar sensor 108A. In an example, each scan can be converted into a depth image. The image can then be filtered by applying a small number (e.g., less than 10) of dilation and erosion operations. Any points that remain empty can reflect areas of the lidar scan for which no return has been received. In an example, in response to a ray terminating at a lidar measurement point that includes a positive elevation angle in a global reference frame, the ray can be determined to be directed toward the sky of the scene based on the lidar sensor (108A) being flush with the ground. In an example, for rays terminating at a lidar measurement point that is determined to be directed toward a sky-facing area of ​​the scene, then for all such rays, the sky loss can be calculated according to the following expression (9):

[0081]

[0082] Thus, reconstructed scene 450 may accurately depict sky 415 of scene 400. In an example, in response to the absence of a return lidar signal transmitted during a measurement scan of lidar sensor 108A, w of expression (9) may be assigned a weight of 0.

[0083] The techniques herein utilize the neural implicit representation 250 to extract geometric shape data from the scene 205, 400. In response to the training of the neural implicit representation 250, output signals of points within the scene 205, 400 are received from the lidar sensor 108A during a measurement scan of the lidar sensor 108A. Over time, the programming of the computer 104 implements the neural implicit representation to continuously or intermittently train and refine the neural implicit representation 250 as the vehicle 100 advances along the path 50.

[0084] Figure 5 is a flow chart of process 500 for generating a pose via lidar measurements. The memory of computer 104 stores executable instructions for performing the steps of process 500 and / or may be programmed such as the structures mentioned above. Process 500 may begin in response to vehicle 100 being turned on or shifted from park to drive. As a general overview of process 500, computer 104 receives data from lidar sensor 108A and initializes an occupancy grid map. Computer 104 may independently train neural implicit representation 250 for a preset number of iterations. For each measurement point obtained during a scan of lidar sensor 108A, computer 104 may calculate (T) according to expressions (1) and (2). i ) and the weight of the measurement point (w i). Computer 104 can also use expression (3) to estimate the expected termination distance of the ray: And estimate the termination distance of the ray representing the measurement point. Computer 104 can also calculate the dynamic loss margin according to expression (4): In an example, the loss function may be the Jensen-Shannon loss function according to expression (5) When calculating the Jensen-Shannon loss function, the computer 104 can calculate the dynamic headroom (ε) according to expressions (6) and (7). dyn ). Computer 104 can also calculate the depth loss according to expression (8) And the sky loss is calculated according to expression (9):

[0085] Process 500 begins at block 505, which may include generating a set of points from a measurement scan obtained by lidar sensor 108A. In an example, lidar sensor 108A may include a scanning lidar sensor mounted to vehicle 100 that performs a continuous side-to-side (e.g., left-to-right) scan of the environment external to vehicle 100. Lidar sensor 108A may include a vertical stack of lidar sensors to obtain data about numerous measurement points in a single side-to-side scan.

[0086] Process 500 continues at block 510, which may include generating an expected termination distance to a set of measurement points based on output signals from the neural implicit representation 250 of the set of measurement points. Figure 3A and Figure 3B In the example of the case of , the expected termination distance corresponds to the distance d returned by the neural implicit representation 250 and distributed from the lidar sensor 108A 1 With d 2 The sample weights between 305.

[0087] Process 500 continues at block 515, which may include responding to a point (eg, Z ) in the set of points (such as the sample points represented by sample weights 305). 2 *) previously learned by the neural implicit representation 250, based on the point (e.g., Z 2 *) and calculates a loss function by calculating the expected termination distance of the point (e.g., Z 2 *) are relatively low (ε), such as Figure 3A As indicated in .

[0088] Process 500 continues at block 520, which includes generating a keyframe from the set of points. Figure 3BAs described above, a keyframe may be generated in response to a plurality of rays terminating at measurement points having relatively small margins.

[0089] Process 500 continues at block 525 , which includes generating a pose of the lidar sensor 108A based on the updated weights of the neural implicit representation of the keyframe.

[0090] After block 525, process 500 ends.

[0091] In an example, in response to generating a posture of the lidar sensor 108A, the computer 104 may actuate one or more of a propulsion system of the vehicle component 110, a steering system of the component 110, or an HMI 112. For example, the computer 104 may actuate one or more of the components 110 by executing an advanced driver assistance system (ADAS). ADAS is an electronic technology that assists the driver in achieving driving functions and parking functions. Examples of ADAS include lane departure detection, blind spot detection, adaptive cruise control, and lane keeping assistance. The computer 104 may actuate the system of the vehicle 100 to stop the vehicle before reaching an object in the environment according to an algorithm that operates without human input. The computer 104 may autonomously operate the vehicle 100, i.e., the propulsion system and / or the steering system, based on the output signal from the neural implicit representation 250.

[0092] Generally speaking, the computing systems and / or devices described may employ any of a variety of computer operating systems, including but not limited to the following versions and / or varieties: Ford Application; AppLink / Smart Device Link middleware; Microsoft Operating system; Microsoft Operating system; Unix operating system (for example, distributed by Oracle Corporation of Redwood Shores, California operating systems); AIX UNIX operating system distributed by International Business Machines Corporation of Armonk, New York; Linux operating system; Mac OSX and iOS operating systems distributed by Apple Inc. of Cupertino, California; BlackBerry operating system distributed by BlackBerry Ltd. of Waterloo, Canada; and Android operating system developed by Google Inc. and the Open Handset Alliance; or provided by QNX Software Systems CAR Infotainment Platform. Examples of computing devices include, but are not limited to, an in-vehicle computer, a computer workstation, a server, a desktop, notebook, laptop or handheld computer, or some other computing system and / or device.

[0093] Computing devices typically include computer-executable instructions, wherein the instructions are executable by one or more computing devices such as those listed above. Computer-executable instructions may be compiled or interpreted from a computer program created using a variety of programming languages ​​and / or technologies, including, alone or in combination, but not limited to Java. TM , C, C++, Matlab, Simulink, Stateflow, Visual Basic, Java Script, Python, Perl, HTML, etc. Some of these applications can be compiled and executed on virtual machines such as Java virtual machines, Dalvik virtual machines, etc. Typically, a processor (e.g., a microprocessor) receives instructions from, for example, a memory, a computer-readable medium, etc., and executes these instructions to perform one or more processes, including one or more of the processes described herein. Such instructions and other data can be stored and transmitted using various computer-readable media. Files in a computing device are typically a collection of data stored on a computer-readable medium (such as a storage medium, a random access memory, etc.).

[0094] Computer-readable media (also referred to as processor-readable media) include any non-transitory (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media can take many forms, including, but not limited to, non-volatile media and volatile media. Instructions may be transmitted via one or more transmission media, including optical fiber, wires, wireless communications, including internals that make up a system bus coupled to a processor of a computer. Common forms of computer-readable media include, for example, RAM, PROM, EPROM, flash EEPROM, any other memory chip or cassette tape or any other medium from which a computer can read.

[0095] The database, data repository or other data storage described herein may include various mechanisms for storing, accessing and retrieving various data, including hierarchical databases, file sets in file systems, application databases in special formats, relational database management systems (RDBMS), non-relational databases (NoSQL), graphic databases (GDB), etc. Each such data storage is typically included in a computing device using a computer operating system such as one of the above-mentioned, and can be accessed via a network in any one or more of a variety of ways. The file system can be accessed from the computer operating system and can include files stored in various formats. In addition to the language (such as the above-mentioned PL / SQL language) for creating, storing, editing and executing stored programs, RDBMS also typically uses structured query language (SQL).

[0096] In some examples, system elements may be implemented as computer-readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) stored on computer-readable media associated therewith (e.g., disks, memories, etc.). A computer program product may include such instructions stored on a computer-readable medium for performing the functions described herein.

[0097] In the accompanying drawings, the same reference numerals indicate the same elements. In addition, some or all of these elements may be changed. With respect to the media, processes, systems, methods, heuristics, etc. described herein, it should be understood that although the steps of such processes, etc. have been described as occurring in a certain ordered sequence, such processes may be practiced by performing the steps in an order different from that described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. The operations, systems, and methods described herein should always be implemented and / or performed in accordance with applicable owner / user manuals and / or safety guidelines.

[0098] The present disclosure has been described in an illustrative manner, and it should be understood that the terminology that has been used is intended to be in the nature of descriptive words, rather than limiting. The adjectives "first" and "second" are used throughout this document as identifiers, and are not intended to indicate importance, order, or quantity. The use of "in response to" and "after determining..." indicates a causal relationship, not just a temporal relationship.

[0099] Many modifications and variations of the disclosure are possible in light of the above teachings, and the disclosure may be practiced otherwise than as specifically described.

[0100] According to the present invention, a system is provided, which has: a computer, the computer including a processor and a memory, the memory including instructions, the instructions executable by the processor to: generate a set of points from a measurement scan obtained by a lidar sensor; generate an expected termination distance of the set of points based on a neural implicit representation of the set of points; calculate a loss function based on the expected termination distance of a learning point, the loss function including a relatively low margin related to the variance or standard deviation of a training distribution centered on the learning point in the set of points, the learning point being learned by the neural implicit representation; generate a keyframe from the set of points; and generate a pose of the lidar sensor based on the keyframe.

[0101] According to an embodiment, the instructions for generating the pose of the lidar sensor further include instructions for modifying the pose of the lidar sensor to align with the neural implicit representation of the set of points.

[0102] According to an embodiment, the instructions for calculating the loss function include instructions for performing the following operations: based on the fact that the unlearned points in the set of points have not been learned by the neural implicit representation, a relatively high margin associated with the variance or standard deviation of the training distribution centered on the unlearned points.

[0103] According to an embodiment, the instructions are further used to: transmit the generated pose of the lidar sensor to an autonomous vehicle driving application.

[0104] According to an embodiment, the instructions are further for: performing motion planning by the autonomous vehicle driving application based on the generated pose.

[0105] According to an embodiment, the calculated loss is based on a combination of the main loss and the opacity loss of the learning point.

[0106] According to an embodiment, the instructions for calculating the loss function include instructions for calculating a depth loss for the learning point, the depth loss representing a difference between an expected distance to the learning point based on the neural implicit representation and a distance extracted from the measurement scan.

[0107] According to an embodiment, the instructions for calculating the loss function include instructions for performing the following operations: calculating the gradient of the loss function; and using the calculated gradient to update the generated pose estimate and weights of the neural implicit representation via gradient descent to reduce the magnitude of the loss function.

[0108] According to an embodiment, the instructions are further for calculating the margin of the learning point based on the neural implicit representation of the learning point and a weight of the learning point derived from the measurement scan.

[0109] According to an embodiment, the instructions are further for: allocating a minimum margin to the learning point in response to the allocated margin being less than a first threshold.

[0110] According to an embodiment, the instructions are further configured to allocate a maximum margin to the learning point in response to the allocated margin being greater than a second threshold.

[0111] According to an embodiment, the instructions are further for assigning a zero weight to any point in the set of points based on an absence of a return signal received in response to a signal transmitted during the measurement scan.

[0112] According to an embodiment, the instructions are further for: generating a mesh representation of the measurement scan based on the neural implicit representation of the set of points.

[0113] According to an embodiment, the neural implicit representation comprises a continuous function representing the geometry of the three-dimensional scene.

[0114] According to an embodiment, the neural implicit representation comprises expected weights along rays terminating at the set of points.

[0115] According to the present invention, a method includes: generating a set of points from a measurement scan obtained by a lidar sensor; generating an expected termination distance of the set of points based on a neural implicit representation of the set of points; calculating a loss function based on the expected termination distance of a learning point, the loss function including a relatively low margin associated with a variance or standard deviation of a training distribution centered on the learning point in the set of points, the learning point being learned by the neural implicit representation; generating a keyframe from the set of points; and generating a pose of the lidar sensor based on the keyframe.

[0116] In one aspect of the invention, the method includes assigning a relatively higher margin associated with a variance or standard deviation of a training distribution centered around an unlearned point in the set of points based on the unlearned point not being learned by the neural implicit representation.

[0117] In one aspect of the invention, the method includes transmitting the updated pose of the lidar sensor to an autonomous vehicle driving application.

[0118] In one aspect of the invention, the method includes performing, by the autonomous vehicle driving application, motion planning based on the generated pose.

[0119] In one aspect of the invention, the method comprises assigning a zero weight to any point in the set of points based on an absence of a return signal received in response to a signal transmitted during the measurement scan.

Claims

1. A method comprising: generating a set of points from a measurement scan obtained by a lidar sensor; generating an expected termination distance for the set of points based on a neural implicit representation of the set of points; calculating a loss function based on the expected termination distance of a learning point of the set of points, the loss function comprising a relatively low margin related to a variance or standard deviation of a training distribution centered at the learning point, the learning point being learned by the neural implicit representation; generating a keyframe from the set of points; as well as A pose of the lidar sensor is generated based on the keyframes.

2. The method of claim 1, further comprising: The pose of the lidar sensor is modified to align with the neural implicit representation of the set of points.

3. The method of claim 1, further comprising: Based on the unlearned points in the set of points not being learned by the neural implicit representation, a relatively higher margin associated with the variance or standard deviation of the training distribution centered on the unlearned points is assigned.

4. The method of claim 1, further comprising: The generated pose of the lidar sensor is transmitted to an autonomous vehicle driving application.

5. The method of claim 4, further comprising: Motion planning is performed by the autonomous vehicle driving application based on generating the pose.

6. The method of claim 1, wherein calculating the loss function is based on a combination of a primary loss and an opacity loss of the learning point.

7. The method of claim 1, wherein calculating further comprises: A depth loss is calculated for the learned point, the depth loss representing the difference between an expected distance to the learned point based on the neural implicit representation and a distance extracted from the measurement scan.

8. The method of claim 1, wherein calculating the loss function further comprises: Calculating the gradient of the loss function; as well as The computed gradients are utilized to update generated pose estimates and weights of the neural implicit representation via gradient descent to reduce the magnitude of the loss function.

9. The method of claim 1, further comprising: The margin of the learning point is calculated based on the neural implicit representation of the learning point and a weight of the learning point derived from the measurement scan.

10. The method of claim 9, further comprising: A minimum margin is allocated to the learning point in response to the allocated margin being less than a first threshold.

11. The method of claim 9, further comprising: A maximum margin is allocated to the learning point in response to the allocated margin being greater than a second threshold.

12. The method of claim 1, further comprising: Any point in the set of points is assigned a weight of zero based on an absence of a return signal received in response to a signal transmitted during the measurement scan.

13. The method of claim 1, further comprising: A mesh representation of the measurement scan is generated based on the neural implicit representation of the set of points.

14. A computer programmed to perform the method of any one of claims 1 to 13.

15. A vehicle comprising the computer according to claim 14.

Images