Object detection for rotation sensors

Abstract

In some aspects, a device may obtain point data from a LiDAR scanner. The point data may be associated with an angular subrange of a polar coordinate grid of the LiDAR scanner. The device identifies a set of points based at least in part on the angular subrange, analyzes the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid, and has a transformer model process the point data to indicate whether the set of points is associated with an object. The device may perform an action based at least in part on whether the transformer model indicates that the set of points is associated with an object. Numerous other aspects are described.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims priority to U.S. Non-Provisional Patent Application No. 17 / 249,967, filed March 19, 2021, entitled “OBJECT DETECTION FOR A ROTATIONAL SENSOR,” which is expressly incorporated herein by reference.

[0002] Aspects of the present disclosure relate generally to rotational sensors, for example, to object detection for rotational sensors. [Background technology]

[0003] A light detection and ranging (herein referred to as "LiDAR") scanner is a rotating sensor that uses light in the form of a pulsed laser to obtain point data that indicates the amount of time it takes for the light to return to the LiDAR scanner. Summary of the Invention [Means for solving the problem]

[0004] In some aspects, a method includes acquiring, by the device, point data from a LiDAR scanner, where the point data is associated with an angular subrange of a polar coordinate grid of the LiDAR scanner; causing, by the device, a transformer model to process the point data to identify a set of points based at least in part on the angular subrange, analyze the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid, and indicate whether the set of points is associated with the object; and performing, by the device, an action based at least in part on whether the transformer model indicates that the set of points is associated with the object.

[0005] In some aspects, a device includes one or more memories and one or more processors communicatively coupled to the one or more memories, wherein the one or more processors are configured to: acquire point data from a LiDAR scanner, where the point data is associated with an angular subrange of a polar coordinate grid of the LiDAR scanner; process the point data to identify a set of points based at least in part on the angular subrange; analyze the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid; and indicate whether the set of points is associated with the object; and perform an action based at least in part on whether the transformer model indicates that the set of points is associated with the object.

[0006] In some aspects, a non-transitory computer-readable medium having a set of instructions stored thereon includes one or more instructions that, when executed by one or more processors of a device, cause the device to acquire point data from a LiDAR scanner, where the point data is associated with an angular subrange of a polar coordinate grid of the LiDAR scanner; process the point data to identify a set of points based at least in part on the angular subrange; analyze the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid, and indicate whether the set of points is associated with the object; and perform an action based at least in part on whether the transformer model indicates that the set of points is associated with the object.

[0007] In some aspects, an apparatus includes means for acquiring point data from a LiDAR scanner, where the point data is associated with an angular subrange of a polar coordinate grid of the LiDAR scanner; means for causing a transformer model to process the point data to identify a set of points based at least in part on the angular subrange, analyze the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid, and indicate whether the set of points is associated with an object; and means for performing an action based at least in part on whether the transformer model indicates that the set of points is associated with the object.

[0008] Aspects generally include methods, apparatus, systems, computer program products, non-transitory computer-readable media, user devices, user equipment, wireless communication devices, and / or processing systems as fully described with reference to and as illustrated in the drawings and this specification.

[0009] The foregoing has outlined rather broadly the features and technical advantages of the embodiments according to the present disclosure so that the following detailed description may be better understood. Additional features and advantages are described below. The concepts and examples disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Such equivalent structures do not depart from the scope of the appended claims. The nature of the concepts disclosed herein, both their organization and method of operation, together with associated advantages, will be better understood from the following description when considered in conjunction with the accompanying drawings. Each of the figures is provided for the purpose of illustration and description, and not as a definition of the limits of the claims.

[0010] So that the above-listed features of the present disclosure can be understood in detail, further details of the above briefly summarized description may be obtained by referring to embodiments, some of which are illustrated in the attached drawings. However, since the present description may admit of other equally effective embodiments, it should be noted that the attached drawings illustrate only some typical embodiments of the present disclosure and therefore should not be considered as limiting its scope. The same reference numbers in different drawings may identify the same or similar elements. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates an example environment in which object detection for the rotational sensor described herein may be implemented in accordance with the present disclosure. [Diagram 2] 2 illustrates an example component of one or more devices shown in FIG. 1, such as a vehicle and a wireless communication device, in accordance with the present disclosure. [Diagram 3] FIG. 1 illustrates an example relating to object detection for a rotation sensor in accordance with the present disclosure. [Figure 4A] FIG. 1 illustrates an example relating to object detection for a rotation sensor in accordance with the present disclosure. [Figure 4B] FIG. 1 illustrates an example relating to object detection for a rotation sensor in accordance with the present disclosure. [Diagram 5] 4 is a flowchart of an example process associated with object detection for a rotation sensor in accordance with the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Various aspects of the present disclosure will be described more fully below with reference to the accompanying drawings. However, the present disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function presented throughout the present disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art should appreciate that the scope of the present disclosure encompasses any aspect of the present disclosure disclosed herein, whether implemented independently of or in combination with other aspects of the present disclosure. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects described herein. In addition, the scope of the present disclosure is intended to encompass such an apparatus or method practiced using other structures, functions, or structures and functions in addition to or other than the various aspects of the present disclosure described herein. It should be understood that any aspect of the present disclosure disclosed herein may be embodied by one or more elements of a claim.

[0013] Point data acquired by a rotating sensor, such as a Light Detection and Ranging (LiDAR) scanner, may be used for three-dimensional (3D) object detection. The LiDAR scanner may use light in the form of a pulsed laser to acquire the point data as the LiDAR scanner is rotated. The point data may correspond to reflections of the light from an object. The point data associated with the object may include a set of data points indicating the amount of time it takes for the light to return to the LiDAR scanner, the orientation of the object relative to the LiDAR scanner, the distance of the object from the LiDAR scanner, etc. The LiDAR scanner may acquire a complete point cloud of the point data based on a 360-degree rotation of the LiDAR scanner.

[0014] A neural network may be configured to interpret the LiDAR point data to identify objects within range of the LiDAR scanner. The neural network may analyze a complete point cloud of the point data and may identify objects in the environment based at least in part on interpreting the analysis of the point data. However, the same object may generate different point data based at least in part on the distance between the LiDAR scanner and the object (e.g., a vehicle at 10 meters may cause the LiDAR scanner to generate different point data than the point data generated for a vehicle at 50 meters). Furthermore, a complete point cloud of data includes a relatively large amount of data that needs to be processed before an object is detected, and the neural network may require a relatively large amount of processing resources to analyze the large amount of data.

[0015] Additionally, as the range from the LiDAR scanner increases, the point data acquired by the LiDAR scanner becomes sparser, typically with relatively few points of point data acquired beyond a distance of 80 meters compared to the amount of points of point data acquired at closer distances, which may prevent the neural network from detecting key features of the object.

[0016] Some implementations described herein relate to object detection for a rotation sensor. For example, a LiDAR system may obtain point data from a LiDAR scanner. The point data may be associated with an angular subrange of a polar coordinate grid associated with the LiDAR scanner. The LiDAR system identifies a set of points based at least in part on the angular subrange, analyzes the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid, and has a transformer model process the point data to indicate whether the set of points is associated with an object. The LiDAR system may perform an action based at least in part on whether the transformer model indicates that the set of points is associated with an object.

[0017] In this manner, the LiDAR system utilizes a transformer model (with a trained encoder and decoder) to detect a particular type of object based at least in part on the distance of point data associated with the object to the LiDAR scanner. Thus, the encoder may transform the point data differently according to the object's location relative to the LiDAR scanner. Furthermore, by detecting objects based on point data associated with angular subranges of the LiDAR scanner's polar coordinate grid, the LiDAR system may conserve computing resources (e.g., processing resources, memory resources, and / or communication resources) compared to a system configured to detect objects based on the entire range (e.g., 360 degrees) of the polar coordinate grid.

[0018] FIG 1 is a diagram of an example environment 100 in which systems and / or methods described herein may be implemented. As shown in FIG 1, environment 100 may include a vehicle 110 having a corresponding electronic control unit (ECU) 112, a wireless communication device 120, and a network 130. Although vehicle 110 is illustrated in FIG 1 with a single corresponding ECU 112 (e.g., ECU 112 is co-located with vehicle 110), a vehicle 110 in environment 100 may include two or more ECUs 112. The devices of environment 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0019] Vehicle 110 may include any vehicle capable of transmitting and / or receiving data associated with object detection for a rotational sensor as described herein. For example, vehicle 110 may be a consumer vehicle, an industrial vehicle, a commercial vehicle, etc. Vehicle 110 may be capable of traveling and / or providing transportation via public roads, may be capable of being used in operations associated with a workplace (e.g., a construction site), etc. Vehicle 110 may include a sensor system including one or more sensors used to generate and / or provide vehicle data associated with vehicle 110, and / or a LiDAR scanner used to obtain point data used for 3D object detection.

[0020] The vehicle 110 may be controlled by the ECU 112, which may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with object detection for the rotational sensors described herein. For example, the ECU 112 may include and / or be a component of a communication and / or computing device, such as an on-board computer, a control console, an operator station, or a similar type of device. The ECU 112 may be configured to communicate with other ECUs and / or other devices. For example, advances in communication technology have enabled vehicle-to-everything (V2X) communications, which may include vehicle-to-vehicle (V2V) communications, vehicle-to-pedestrian (V2P) communications, and the like. In some aspects, the ECU 112 may include and / or be used to provide vehicle data (e.g., identification information, sensor data, etc.) associated with the V2X communications, vehicle 110, as described herein.

[0021] Wireless communication device 120 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with object detection for a rotation sensor, as described elsewhere herein. For example, wireless communication device 120 may include a base station, an access point, etc. Additionally or alternatively, wireless communication device 120 may include a communications and / or computing device, such as a mobile phone (e.g., a smartphone, a radiotelephone, etc.), a laptop computer, a tablet computer, a handheld computer, a desktop computer, a gaming device, a wearable communications device (e.g., a smart watch, smart glasses, etc.), or a similar type of device.

[0022] Network 130 includes one or more wired and / or wireless networks. For example, network 130 may include a peer-to-peer (P2P) network, a cellular network (e.g., a long-term evolution (LTE) network, a code division multiple access (CDMA) network, a 3G network, a 4G network, a 5G network, another type of next generation network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad-hoc network, an intranet, the Internet, a fiber-optic based network, a cloud computing network, etc., and / or a combination of these or other types of networks. In some aspects, network 130 may include and / or be a P2P communication link that is directly between one or more of the devices of environment 100.

[0023] The number and arrangement of devices and networks shown in Figure 1 are provided as one or more examples. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown in Figure 1. Furthermore, two or more devices shown in Figure 1 may be implemented within a single device, or a single device shown in Figure 1 may be implemented as multiple distributed devices. Additionally or alternatively, a set of devices (e.g., one or more devices) of environment 100 may perform one or more functions described as being performed by another set of devices of environment 100.

[0024] 2 is a diagram illustrating example components of a device 200 in accordance with the present disclosure. The device 200 may correspond to the vehicle 110, the ECU 112, and / or the wireless communication device 120. In some aspects, the vehicle 110, the ECU 112, and / or the wireless communication device 120 may include one or more of the devices 200 and / or one or more components of the devices 200. As shown in FIG. 2, the device 200 may include a bus 205, a processor 210, a memory 215, a storage component 220, an input component 225, an output component 230, a communication interface 235, a sensor 240, a LiDAR system 245, etc.

[0025] The bus 205 includes components that enable communication between the components of the device 200. The processor 210 is implemented in hardware, firmware, software, or a combination of hardware, firmware, and software. The processor 210 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some aspects, the processor 210 includes one or more processors that can be programmed to perform functions. The memory 215 includes a random access memory (RAM), a read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by the processor 210.

[0026] The storage component 220 stores information and / or software related to the operation and use of the device 200. For example, the storage component 220 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of non-transitory computer-readable medium along with a corresponding drive.

[0027] Input components 225 include components that enable device 200 to receive information, such as through user input (e.g., a touchscreen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone). Additionally or alternatively, input components 225 may include components for determining the position or location of device 200 (e.g., a Global Positioning System (GPS) component, a Global Navigation Satellite System (GNSS) component, etc.), sensors for sensing information (e.g., an accelerometer, a gyroscope, an actuator, another type of position sensor or environmental sensor, etc.). Output components 230 include components that provide output information from device 200 (e.g., a display, a speaker, a haptic feedback component, an audio or visual indicator, etc.).

[0028] The communications interface 235 includes transceiver-like components (e.g., a transceiver and / or a separate receiver and transmitter, etc.) that enable the device 200 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communications interface 235 may enable the device 200 to receive information from another device and / or provide information to another device. For example, the communications interface 235 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency interface, a universal serial bus (USB) interface, a wireless local area interface (e.g., a Wi-Fi interface), a cellular network interface, etc.

[0029] Sensor 240 includes one or more devices capable of sensing a characteristic associated with device 200. Sensor 240 may include one or more integrated circuits (e.g., on a packaged silicon die) and / or one or more passive components of one or more flex circuits to enable communication with one or more components of device 200.

[0030] The sensor 240 may include an optical sensor having a field of view from which the sensor 240 may determine one or more characteristics of the environment of the device 200. In some aspects, the sensor 240 may include a camera. For example, the sensor 240 may include a low-resolution camera (e.g., a video flux array (VGA)) capable of capturing images of less than 1 megapixel, images of less than 1216×912 pixels, etc. The sensor 240 may be a low-power device (e.g., a device consuming less than 10 milliwatts (mW) of power) with always-on capability while the device 200 is powered on.

[0031] Additionally or alternatively, the sensors 240 may include magnetometers (e.g., Hall effect sensors, anisotropic magnetoresistive (AMR) sensors, giant magnetoresistive sensors (GMR), etc.), location sensors (e.g., Global Positioning System (GPS) receivers, Local Positioning System (LPS) devices (e.g., using triangulation, multilateration, etc.), etc.), gyroscopes (e.g., Microelectromechanical System (MEMS) gyroscopes or similar type devices), accelerometers, speed sensors, motion sensors, infrared sensors, temperature sensors, pressure sensors, etc.

[0032] The LiDAR system 245 may include a LiDAR scanner, which uses light in the form of a pulsed laser to measure distances from the LiDAR scanner to objects. The LiDAR system 245 may utilize data acquired by the LiDAR scanner to perform 3D object detection, as described herein.

[0033] The device 200 may perform one or more processes described herein. The device 200 may perform these processes based on the processor 210 executing software instructions stored by a non-transitory computer-readable medium, such as the memory 215 and / or the storage component 220. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.

[0034] Software instructions may be read into memory 215 and / or storage component 220 from another computer-readable medium or another device via communications interface 235. When executed, the software instructions stored in memory 215 and / or storage component 220 may cause processor 210 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, the aspects described herein are not limited to any specific combination of hardware circuitry and software.

[0035] In some aspects, device 200 includes means for performing one or more processes described herein and / or means for performing one or more operations of the processes described herein. For example, device 200 may include means for acquiring point data from a LiDAR scanner. The point data may be associated with an angular subrange of a polar coordinate grid of the LiDAR scanner. Device 200 may include means for identifying a set of points based at least in part on the angular subrange, analyzing the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid, and having a Transformer model process the point data to indicate whether the set of points is associated with an object. Device 200 may include means for performing an action based at least in part on whether the Transformer model indicates that the set of points is associated with an object. In some aspects, such means may include one or more components of device 200 described with respect to FIG. 2, such as bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, sensor 240, LiDAR system 245, etc.

[0036] The number and arrangement of components shown in Figure 2 are provided as an example. In practice, device 200 may include additional, fewer, different, or differently arranged components than those shown in Figure 2. Additionally or alternatively, a set of components (e.g., one or more components) of device 200 may perform one or more functions that are described as being performed by another set of components of device 200.

[0037] 3 is a diagram illustrating an example 300 relating to object detection for a rotation sensor in accordance with the present disclosure. As shown in FIG. 3, the example 300 includes a LiDAR system configured to generate and / or train a transformer model to perform object detection based on LiDAR scanner training data.

[0038] As indicated by reference numeral 310, the LiDAR system receives historical data associated with the LiDAR scanner. The historical data may include one or more complete point clouds of data points generated by the LiDAR scanner, and / or generated by a machine learning model, and / or input by a user, etc. For example, the historical data may include one or more complete point clouds of data points associated with one or more types of objects (e.g., vehicles, roadside objects, people, and / or animals, among others) at one or more distances from the LiDAR scanner, which the transformer model will be trained to detect. The historical data may be stored in a reference data structure associated with the LiDAR system and / or another device (e.g., a third-party server device). The LiDAR system may receive the historical data from the reference data structure and may provide a complete point cloud of data points associated with the objects and included in the historical data to a position encoder associated with the LiDAR system.

[0039] In some aspects, the historical data includes a set of ground truth objects that indicate the actual location of the object represented by the complete point cloud of data points. The ground truth objects may include a set of parameters of an oriented 3D bounding box. For example, the ground truth objects may include a first parameter indicating an x-axis Cartesian coordinate, a second parameter indicating a y-axis Cartesian coordinate, a third parameter indicating a z-axis Cartesian coordinate, a fourth parameter indicating a width of the bounding box, a fifth parameter indicating a length of the bounding box, a sixth parameter indicating a height of the bounding box, a seventh parameter indicating a sine of a heading angle of the bounding box, an eighth parameter indicating a cosine of a heading angle of the bounding box, and a ninth parameter indicating a class associated with the bounding box.

[0040] The position encoder may receive the complete point cloud of data points and may convert the historical data into a polar coordinate grid. An origin of the polar coordinate grid may correspond to a location of a LiDAR scanner (e.g., a LiDAR scanner used to acquire the complete point cloud of data points) associated with the complete point cloud of data points. The polar coordinate grid may be divided into multiple angular subranges emanating from the origin. For example, the polar coordinate grid may be divided into four angular subranges of 90 degrees, and / or eight angular subranges of 45 degrees, etc. Additionally, the polar coordinate grid may be divided into a series of concentric circles centered on the origin of the polar coordinate grid. For example, the polar coordinate grid may be divided into a series of three, four, five, six, etc. concentric circles. In some aspects, the quantity of concentric circles is determined based on the size of the geographic area represented by the polar coordinate grid and / or associated with the complete point cloud of data points. In some aspects, the radius of the concentric circles may be a multiple of the smallest radius of the first concentric circle. For example, the radius of a first concentric circle may be a first value X, and the radius of a second concentric circle may be twice the first value (eg, 2X).

[0041] The concentric circles may intersect with the angular ranges to define rings of polar cells. Each ring of polar cells may include polar cells in each angular range. For example, a polar grid may be divided into eight angular subranges and five concentric circles to define 40 polar cells of five different sizes (e.g., five rings of polar cells). A position encoder may convert a position of a data point included in a complete point cloud of data points to a position on a polar grid. The position encoder may associate a data point with information (e.g., a set of polar coordinates) that identifies the position of the data point on the polar grid. The position encoder may determine an angular subrange and / or a polar cell that includes the position of the data point on the polar grid, and may associate the data point with information identifying the angular subrange and / or the polar cell.

[0042] In some aspects, the position encoder may determine that the object is spread across multiple polar cells based on transforming the historical data into a polar coordinate grid. The position encoder may identify the polar cell that contains the largest portion of the object, a threshold percentage of the object (e.g., 40%, 45%, 50%, etc.), etc., and may associate data points associated with the object with information identifying the polar cell.

[0043] As indicated by reference numeral 320, the LiDAR system trains a transformer model according to the polar distance. The transformer model may include an encoder and a decoder. The encoder may identify an angular subrange associated with a plurality of polar cells. The encoder may identify a polar cell of the plurality of polar cells and obtain a set of data points associated with the polar cell. The encoder may transform the polar coordinates associated with the set of data points into object query data. The object query data may include three-dimensional data associated with a type of object that the transformer model is trained to detect and / or identify.

[0044] The decoder may predict whether the set of data points is associated with that type of object based at least in part on the object query data. The decoder may be configured to predict whether the set of data points is associated with that type of object based at least in part on a similarity analysis associated with the object query data and reference data associated with the type of object.

[0045] In some aspects, the prediction generated by the decoder includes information indicating that an object is detected and information indicating the type of object detected. For example, the decoder may generate an output that includes information identifying a bounding box associated with the detected object and / or information identifying the type of object detected.

[0046] The Transformer model may utilize an encoder and / or a decoder to find a bipartite matching between a prediction generated based on a set of data points and a ground truth object. The Transformer model may search for the lowest cost N-element permutation, where N is the quantity of predictions. In some aspects, the Transformer utilizes an algorithm (e.g., the Hungarian algorithm) to calculate the best match based on a pair-wise matching cost between the ground truth and the predictions associated with the ground truth.

[0047] In some aspects, prior to transforming the historical data into a polar coordinate grid, the LiDAR system may generate a global feature extraction layer that extracts features of the object represented by the complete point cloud of data points directly from a Cartesian space associated with the complete point cloud of data points to generate an extracted pseudo-image. The extracted pseudo-image may be discretized into a polar space of a combination of ring-like sectors of polar cells. A bounding box of a first size is computed to be the same for all of the polar cells included in the ring. A convolutional neural network (CNN) may transform the bounding box and associated features computed for each ring into one or more embedding shapes. In some aspects, the transformer model may utilize a different CNN for each ring of polar cells.

[0048] In some aspects, the historical data includes a set of data points representing objects at various distances. The transformer model may be trained to determine whether the set of data points included in the first polar cell represents an object based at least in part on the set of data points representing objects at various distances. In this manner, the transformer model may be trained to detect objects at various distances from the LiDAR scanner.

[0049] In some aspects, the historical data includes a set of data points associated with another LiDAR scanner that identified a first object in an area that is a physical distance from the other LiDAR scanner, and the transformer model is trained to determine whether the set of data points included in the first polar cell represents a second object that is that physical distance from the LiDAR scanner, based at least in part on the set of points associated with the other LiDAR scanner. In this manner, the transformer model is trained to detect an object based on the distance of the object from the LiDAR scanner. The LiDAR scanner and the other LiDAR scanner may be the same LiDAR scanner or different LiDAR scanners. The first object and the second object may be the same type of object (e.g., the same type of vehicle, person, animal, roadside object, etc.).

[0050] In some aspects, the transformer model is trained to determine whether a set of points represents a second object based at least in part on historical data associated with a first object in a feature space that corresponds to a polar cell of a polar coordinate grid defined by polar distances that correspond to physical distances.

[0051] In some aspects, a transformer model is trained to determine whether a set of points represents a second object based at least in part on a placement of the set of points within a polar cell of a polar coordinate grid. The transformer model may be trained at least in part on a historical set of points associated with the first object in a feature space corresponding to the polar cell.

[0052] As noted above, Figure 3 is provided as an example. Other examples may differ from those described with respect to Figure 3.

[0053] 4A-4B are diagrams illustrating an example 400 relating to object detection for a rotation sensor in accordance with the present disclosure. As shown in FIG. 4A-4B, the example 400 includes a LiDAR system to perform object detection based on LiDAR data received from a LiDAR scanner associated with a vehicle.

[0054] 4A, as indicated by reference numeral 410, a LiDAR scanner acquires LiDAR data (e.g., a set of point data) associated with an angular subrange of a polar coordinate grid. An origin of the polar coordinate grid may correspond to a location of the LiDAR scanner. The angular subrange of the polar coordinate grid may correspond to a range that is a threshold percentage (e.g., 10%, 12.5%, 20%, 25%, etc.) of 360 degrees.

[0055] In some aspects, the LiDAR scanner provides the LiDAR data associated with the angular subranges to the LiDAR system based on the acquisition of the LiDAR data (e.g., before the LiDAR scanner acquires a complete point cloud of the LiDAR data). Alternatively and / or additionally, the LiDAR scanner may provide the complete point cloud of the LiDAR data to the LiDAR system, and the LiDAR system may sequentially process the sets of points based on the angular subranges and / or polar cells associated with the sets of points.

[0056] The LiDAR system may provide the LiDAR data to the position encoder based on receiving the LiDAR data from the LiDAR scanner. The position encoder may convert the LiDAR data into a polar coordinate grid having an origin corresponding to the location of the LiDAR scanner. The polar coordinate grid may be divided into a plurality of angular subranges and a plurality of concentric circles to define a plurality of rings of polar cells in a manner similar to that described above with respect to FIG. 3. For a data point included in the LiDAR data, the position encoder may determine a set of polar coordinates for the data point based on the distance and orientation of the data point from the location of the LiDAR scanner. The encoder system may determine an angular subrange and / or a polar cell associated with the data point based on the set of polar coordinates.

[0057] As indicated by reference numeral 420, the LiDAR system has the transformer model process the point data according to the polar cells and / or angular subranges. The transformer model may process the point data based at least in part on a sequence associated with the polar coordinate grid. The sequence may indicate an order in which individual subsets of point data associated with corresponding polar cells of the polar coordinate grid are processed. In some aspects, the sequence indicates that individual subsets of point data are processed based on the distance of the corresponding polar cell from the origin of the polar coordinate grid. For example, a set of point data associated with a first polar cell that is closer to the origin compared to a second polar cell is processed before a set of point data associated with the second polar cell.

[0058] The Transformer model may identify a first polar cell based on the sequence. The Transformer model may identify a set of data points associated with the first polar cell. The Transformer model may process a set of point data associated with the set of data points based on the set of data points being associated with the first polar cell. The Transformer model may generate an output based on processing the set of point data. The output may include object data indicating whether the set of point data is associated with an object and / or a type of object associated with the set of point data.

[0059] In some aspects, the Transformer model may generate an output based on processing a set of point data. In some aspects, the Transformer model may generate an output based on processing a set of point data associated with a plurality of polar cells (e.g., two, three, all, etc. of the polar cells) in an angular subrange that includes a first polar cell. The Transformer may identify a second polar cell in an angular subrange that includes the first polar cell based on the sequence. The Transformer model may process point data associated with a set of data points associated with the second polar cell in a similar manner as described above. In some aspects, the Transformer model continues in a similar manner for each polar cell included in the angular subrange. The Transformer model may generate an output based on processing a set of point data associated with each polar cell included in the angular subrange.

[0060] In some aspects, the Transformer model is configured to selectively determine whether to process a particular subset of point data associated with a data point associated with a polar cell based at least in part on whether a polar cell associated with an already processed subset of point data included at least one data point. For example, the Transformer model may determine not to process a particular subset of point data based on a polar cell associated with an already processed subset of point data not including at least one data point. In some aspects, the Transformer model may generate an output based on determining not to process a particular subset of point data.

[0061] The LiDAR system may provide an output (e.g., object information) to the vehicle associated with the LiDAR data. As shown in FIG. 4B at reference numeral 430, the vehicle's ECU may determine whether an object is indicated or identified based on the object data. The ECU may analyze the object data included in the output provided by the LiDAR system and may determine whether an object is indicated or identified based on the analysis. The ECU may provide the object information via a user interface associated with the vehicle and / or a driver of the vehicle (e.g., a user interface of the driver's user device) and / or control the operation of the vehicle when an object is indicated or identified (e.g., cause the vehicle to issue a notification (e.g., sound and / or light), stop, slow down, speed up, and / or change lanes, among others).

[0062] The LiDAR scanner and LiDAR system are iteratively applied to the next angular subrange, as indicated by reference numeral 440. The transformer model may sequentially process sets of point data associated with data points in polar cells included in the remaining angular subranges in a similar manner based on processing the data points included in each polar cell of the angular subrange and / or based on determining not to process a particular subset of point data. The transformer model may generate outputs associated with the next angular range and may provide the outputs to the vehicle in a similar manner as described above.

[0063] The ECU indicates the object information via a user interface associated with the vehicle and / or controls the vehicle according to the location of the object, as indicated by reference numerals 450 and 460. The ECU may cause the object information to be displayed via a user interface associated with the vehicle and / or a driver of the vehicle, and / or control operation of the vehicle based on the object information indicating and / or identifying the object, the type of object, the location of the object relative to the vehicle, and / or an operating parameter associated with the vehicle (e.g., the speed of the vehicle, the acceleration of the vehicle, the current weather conditions, the type of road the vehicle is traveling on (e.g., paved, gravel, wet, and / or icy, among others), and / or another type of operating parameter).

[0064] As indicated by reference numeral 470, the LiDAR system updates the transformer model according to feedback input via a user interface. The feedback may indicate an accuracy associated with the output. For example, a user may input information indicating whether an object was at the location indicated by the object information, whether the object was of the type of object indicated by the object information, whether the action performed by the ECU was appropriate, and / or other types of feedback information. The LiDAR system may utilize the LiDAR scanner used to generate the feedback and output to retrain the transformer model. In this manner, the LiDAR system may increase the accuracy of the output generated by the transformer model compared to a transformer model that is not retrained based on the received feedback.

[0065] As noted above, Figures 4A-4B are provided as an example, other examples may differ from those described with respect to Figures 4A-4B.

[0066] FIG. 5 is a flowchart of an example process 500 associated with object detection for a rotation sensor. In some implementations, one or more processing blocks of FIG. 5 may be performed by a device (e.g., a LiDAR system). In some implementations, one or more processing blocks of FIG. 5 may be performed by another device or group of devices separate from or including a device, such as a vehicle (e.g., vehicle 110 and / or ECU 112), and / or a wireless communication device (e.g., wireless communication device 120). Additionally or alternatively, one or more processing blocks of FIG. 5 may be performed by one or more components of device 200, such as processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, sensor 240, and / or LiDAR system 245.

[0067] 5, process 500 may include acquiring point data from a LiDAR scanner (block 510). For example, the device may acquire the point data from the LiDAR scanner as described above. In some implementations, the point data is associated with an angular subrange of a polar coordinate grid of the LiDAR scanner.

[0068] 5, process 500 may include identifying a set of points based at least in part on the angular subrange, analyzing the set of points based at least in part on a polar distance between the set of points and an origin of a polar coordinate grid, and having a Transformer model process the point data to indicate whether the set of points is associated with the object (Block 520). For example, the device may identify a set of points based at least in part on the angular subrange, analyze the set of points based at least in part on a polar distance between the set of points and an origin of a polar coordinate grid, and have a Transformer model process the point data to indicate whether the set of points is associated with the object, as described above.

[0069] 5, process 500 may further include performing an action based at least in part on whether the Transformer model indicates that the set of points is associated with the object (Block 530). For example, the device may perform an action based at least in part on whether the Transformer model indicates that the set of points is associated with the object, as described above.

[0070] Process 500 may include additional implementations, such as any single implementation or any combination of implementations described below and / or in conjunction with one or more other processes described elsewhere herein.

[0071] In a first implementation, the transformer model is trained to determine whether a set of points represents an object based at least in part on historical data associated with another scanner that has identified another object in an area at a physical distance from the other scanner. The other object and the object may be of the same type, and the other scanner may be associated with a LiDAR scanner. The physical distance may be associated with a polar distance.

[0072] In a second implementation, the transformer model is trained to determine whether a set of points represents an object based at least in part on historical data associated with another object in the feature space corresponding to a polar cell of the polar coordinate grid defined by the polar distance. The historical data may include one or more historical sets of points representing another object.

[0073] In a third implementation, a transformer model is trained to determine whether a set of points represents an object based at least in part on the placement of the set of points within a polar cell of a polar coordinate grid. The transformer model may be trained at least in part on a historical set of points associated with another object in a feature space corresponding to the polar cell.

[0074] In a fourth implementation, the transformer model includes an encoder that converts polar coordinates of a set of points into object query data, and a decoder that predicts whether the set of points is associated with an object based at least in part on the object query data.

[0075] In a fifth implementation, the object query data includes three-dimensional data associated with one or more types of objects. The transformer model may be trained to identify the one or more types of objects, and the object type may be one of the one or more types of objects.

[0076] In a sixth implementation, the decoder is configured to predict whether a set of points is associated with an object based at least in part on a similarity analysis associated with the object query data and reference data associated with a type of object.

[0077] In a seventh implementation, the transformer model processes the point data based at least in part on a sequence associated with a polar coordinate grid, the sequence indicating an order in which individual subsets of the point data associated with corresponding polar cells of the polar coordinate grid are processed.

[0078] In an eighth implementation, the transformer model is configured to selectively determine whether to process a particular subset of the point data based at least in part on whether an already processed subset of the point data included at least one point.

[0079] In a ninth implementation, performing the action includes providing, via a user interface, an indication associated with the object based at least in part on the Transformer model indicating that the set of points is associated with the object. The indication may indicate at least one of a location of the object, or a type of the object.

[0080] In a tenth implementation, performing the action includes obtaining feedback from the user device associated with an indication that the set of points is not associated with the object based at least in part on the Transformer model indicating that the set of points is not associated with the object, and retraining the Transformer model based at least in part on the set of points and the feedback.

[0081] In an eleventh implementation, the angle sub-range corresponds to a range that is a threshold percentage of 360 degrees.

[0082] 5 illustrates example blocks of process 500, in some implementations process 500 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those illustrated in FIG 5. Additionally or alternatively, two or more of the blocks of process 500 may be performed in parallel.

[0083] The following provides a summary of several aspects of the disclosure.

[0084] Aspect 1: A method comprising: acquiring, by a device, point data from a LiDAR scanner, the point data being associated with an angular subrange of a polar coordinate grid of the LiDAR scanner; causing, by the device, a transformer model to process the point data to identify a set of points based at least in part on the angular subrange, analyze the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid, and indicate whether the set of points is associated with an object; and performing, by the device, an action based at least in part on whether the transformer model indicates that the set of points is associated with the object.

[0085] Aspect 2: The method of aspect 1, wherein the transformer model is trained to determine whether a set of points represents an object based at least in part on historical data associated with another scanner that has identified another object in an area at a physical distance from the other scanner, the other object and the object are of the same type, the other scanner is associated with a LiDAR scanner, and the physical distance is associated with a polar distance.

[0086] Aspect 3: The method of aspects 1 to 2, wherein the transformer model is trained to determine whether a set of points represents an object based at least in part on historical data associated with another object in a feature space corresponding to a polar cell of a polar coordinate grid defined by a polar distance, and the historical data includes one or more historical sets of points representing another object.

[0087] Aspect 4: A method according to any of aspects 1 to 3, wherein a Transformer model is trained to determine whether a set of points represents an object based at least in part on the placement of the set of points within a polar cell of a polar coordinate grid, and the Transformer model is trained based at least in part on a historical set of points associated with another object in a feature space corresponding to the polar cell.

[0088] Aspect 5: A method as described in any of aspects 1 to 4, wherein the transformer model includes an encoder that transforms polar coordinates of a set of points into object query data, and a decoder that predicts whether the set of points is associated with an object based at least in part on the object query data.

[0089] Aspect 6: The method of aspect 5, wherein the object query data includes three-dimensional data associated with one or more types of objects, the transformer model is trained to identify the one or more types of objects, and the object type is one of the one or more types of objects.

[0090] Aspect 7: The method of aspect 6, wherein the decoder is configured to predict whether a set of points is associated with an object based at least in part on a similarity analysis associated with the object query data and reference data associated with a type of object.

[0091] Aspect 8: A method as described in any of aspects 1 to 7, wherein the transformer model processes the point data based at least in part on a sequence associated with a polar coordinate grid, the sequence indicating the order in which individual subsets of the point data associated with corresponding polar cells of the polar coordinate grid are processed.

[0092] Aspect 9: The method of aspect 8, wherein the Transformer model is configured to selectively determine whether to process a particular subset of the point data based at least in part on whether an already processed subset of the point data included at least one point.

[0093] Aspect 10: A method as described in any of aspects 1 to 9, wherein the step of performing an action includes providing, via a user interface, an indication associated with the object based at least in part on the transformer model indicating that the set of points is associated with the object, the indication indicating at least one of a location of the object or a type of the object.

[0094] Aspect 11: A method as described in any of aspects 1 to 10, wherein the step of performing an action includes the steps of obtaining feedback from the user device associated with an indication that the set of points is not associated with the object based at least in part on the Transformer model indicating that the set of points is not associated with the object, and retraining the Transformer model based at least in part on the set of points and the feedback.

[0095] Aspect 12: The method of any of aspects 1 to 11, wherein the angular subrange corresponds to a range that is a threshold percentage of 360 degrees.

[0096] Aspect 13: An apparatus for wireless communication in a device, comprising a processor, a memory coupled to the processor, and instructions stored in the memory and executable by the processor to cause the apparatus to perform a method of one or more of aspects 1 to 12.

[0097] Aspect 14: A device for wireless communication comprising a memory and one or more processors coupled to the memory, wherein the memory and the one or more processors are configured to perform a method of one or more of aspects 1 to 12.

[0098] Aspect 15: An apparatus for wireless communication, comprising at least one means for performing the method of one or more of aspects 1 to 12.

[0099] Aspect 16: A non-transitory computer-readable medium storing code for wireless communication, the code including instructions executable by a processor to perform a method of one or more of aspects 1 to 12.

[0100] Aspect 17: A non-transitory computer-readable medium storing a set of instructions for wireless communication, the set of instructions comprising one or more instructions that, when executed by one or more processors of a device, cause the device to perform a method of one or more of aspects 1 to 12.

[0101] The above disclosure provides illustration and description, but is not intended to be exhaustive or to limit the embodiments to the precise form disclosed. Modifications and variations may be made in light of the above disclosure or acquired from practice of the embodiments.

[0102] The term "component" as used herein shall be broadly construed as hardware, firmware, and / or a combination of hardware and software. As used herein, a processor is implemented in hardware, firmware, and / or a combination of hardware and software. It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, and / or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limiting of the aspects. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code. It should be understood that software and hardware can be designed to implement the systems and / or methods based at least in part on the description herein.

[0103] As used herein, "meeting a threshold" can refer to a value being greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, not equal to the threshold, etc., depending on the context.

[0104] Although certain combinations of features are recited in the claims and / or disclosed herein, such combinations do not limit the disclosure of the various aspects. In fact, many of these features may be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim recited below may depend directly on only one claim, the disclosure of the various aspects includes each dependent claim in combination with every other claim in the set of claims. As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. By way of example, "at least one of a, b, or c" is intended to encompass a, b, c, ab, ac, bc, and abc, as well as any combination having multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other order of a, b, and c).

[0105] No element, act, or instruction used herein should be construed as critical or essential unless expressly described as such. Additionally, the articles "a" and "an" as used herein include one or more items and may be used interchangeably with "one or more." Additionally, the article "the" as used herein includes one or more items mentioned with the article "the" and may be used interchangeably with "one or more." Additionally, the terms "set" and "group" as used herein include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items) and may be used interchangeably with "one or more." When only one item is intended, the phrase "only one" or similar language is used. Additionally, terms such as "has," "have," and "having" as used herein are intended to be open-ended terms. Additionally, the phrase "based on" is intended to mean "based at least in part on," unless otherwise specified. Also, as used herein, the term "or" is intended to be inclusive when used in a consecutive manner, and may be used interchangeably with "and / or" unless otherwise noted (e.g., when used in combination with "either" or "only one of"). [Explanation of symbols]

[0106] 100 Environment 110 vehicles 112 Electronic Control Unit (ECU) 120 Wireless communication devices 130 Network 200 devices 205 Bus 210 Processor 215 Memory 220 Memory Components 225 Input Components 230 Output Components 235 Communication Interface 240 Sensors 245 LiDAR System

Claims

1. 1. A method comprising: acquiring, by a device, point data from a LiDAR scanner, the point data being associated with angular subranges of a polar coordinate grid of the LiDAR scanner; The device causes the transformer model to processing the point data to identify a set of points based at least in part on the angle subranges; analyzing the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid; indicating whether the set of points is associated with an object; and causing the performing, by the device, an action based at least in part on whether the Transformer model indicates that the set of points is associated with the object; A method comprising:

2. the transformer model is trained to determine whether the set of points represents an object based at least in part on historical data associated with another scanner that has identified the object in an area that is a physical distance from the other scanner; the other object and the object are of the same type, and the other scanner is associated with the LiDAR scanner; the physical distance is related to the polar distance; The method of claim 1.

3. the transformer model is trained to determine whether the set of points represents an object based at least in part on historical data associated with another object in a feature space corresponding to a polar cell of the polar coordinate grid defined by the polar distance; the historical data includes one or more historical sets of points representing the other object; The method of claim 1.

4. the transformer model is trained to determine whether the set of points represents the object based at least in part on a placement of the set of points within a polar cell of the polar coordinate grid; the transformer model is trained based at least in part on a historical set of points associated with another object in a feature space corresponding to the polar cell; The method of claim 1.

5. The transformer model is an encoder for converting the polar coordinates of the set of points into object query data; a decoder for predicting whether the set of points is associated with the object based at least in part on the object query data; 2. The method of claim 1, comprising:

6. the object query data includes three-dimensional data associated with one or more types of objects; the transformer model is trained to identify one or more types of the object; the type of object is one of the one or more types of object; The method of claim 5.

7. 7. The method of claim 6, wherein the decoder is configured to predict whether the set of points is associated with the object based at least in part on a similarity analysis associated with the object query data and reference data associated with the type of object.

8. the transformer model processes the point data based at least in part on a sequence associated with the polar coordinate grid; The method of claim 1 , wherein the sequence indicates an order in which individual subsets of the point data associated with corresponding polar cells of the polar coordinate grid are processed.

9. A non-transitory computer-readable storage medium storing a set of instructions, which when executed by one or more processors of a device, causes the device to: A non-transitory computer readable storage medium comprising one or more instructions for causing the method of any one of claims 1 to 8 to be performed.

10. An apparatus comprising: means for acquiring point data from a LiDAR scanner, the point data being associated with angular subranges of a polar coordinate grid of the LiDAR scanner; Transformer model, processing the point data to identify a set of points based at least in part on the angle subranges; analyzing the set of points based at least in part on a polar distance between the set of points and an origin of the polar coordinate grid; indicating whether the set of points is associated with an object; and a means for causing said means for performing an action based at least in part on whether the transformer model indicates that the set of points is associated with the object; 13. An apparatus comprising:

11. the transformer model is trained to determine whether the set of points represents an object based at least in part on historical data associated with another scanner that has identified the object in an area that is a physical distance from the other scanner; the other object is associated with the object and the other scanner is associated with the LiDAR scanner; the physical distance is related to the polar distance; 11. The apparatus of claim 10.

12. the transformer model is trained to determine whether the set of points represents an object based at least in part on historical data associated with another object in a feature space corresponding to a polar cell of the polar coordinate grid defined by the polar distance; the historical data includes one or more historical sets of points representing the other object; 11. The apparatus of claim 10.

13. the transformer model is trained to determine whether the set of points represents the object based at least in part on a placement of the set of points within a polar cell of the polar coordinate grid; the transformer model is trained based at least in part on a historical set of points associated with another object in a feature space corresponding to the polar cell; 11. The apparatus of claim 10.

14. The transformer model is an encoder for converting the polar coordinates of the set of points into object query data; a decoder for predicting whether the set of points is associated with the object based at least in part on the object query data; The apparatus of claim 10, comprising:

15. The means for performing the action comprises: means for providing, via a user interface, an indication associated with the object based at least in part on the transformer model indicating that the set of points is associated with the object; Including, The instructions are: the location of the object; or The type of object The apparatus of claim 10, exhibiting at least one of:

Images