Method and system for determining information related to an environment of a vehicle

Abstract

A computer-implemented method for determining information related to an environment of a vehicle comprises the following steps performed by a computer hardware component: determining a point cloud based on measurement data, the point cloud comprising a plurality of points; determining a corresponding feature of each point of the point cloud; determining a grid structure based on the point cloud; determining a processed grid structure based on processing the grid structure using an artificial neural network; and determining at least one final grid structure based on the point cloud, the respective features of each point of the point cloud, and the processed grid structure.

Description

Technical Field

[0001] The present disclosure relates to methods and systems for determining information related to an environment of a vehicle. Background Art

[0002] A car perception system may need to provide different kinds of outputs, which can be obtained through different tasks. For example, the object detection task can provide a list of objects of interest, while semantic segmentation can indicate the location of the drivable space.

[0003] However, obtaining such different kinds of output can be cumbersome.

[0004] Therefore, there is a need to provide efficient and accurate methods for obtaining such different kinds of outputs. Summary of the invention

[0005] The present disclosure provides a computer implemented method, a computer system, a vehicle and a non-transitory computer readable medium according to the independent claims. Embodiments are given in the dependent claims, the detailed description and the drawings.

[0006] In one aspect, the present disclosure relates to a computer-implemented method for determining information related to an environment of a vehicle, the method comprising the following steps performed (in other words, executed) by computer hardware components: determining a point cloud based on measurement data, the point cloud comprising a plurality of points; determining corresponding features for each point of the point cloud; determining a grid structure based on the point cloud; determining a processed grid structure based on processing the grid structure using an artificial neural network; and determining at least one final grid structure based on the point cloud, the corresponding features for each point of the point cloud, and the processed grid structure.

[0007] In other words, neural network architectures can be provided in methods for multi-task and multi-resolution neural networks, for example, for automotive perception systems.

[0008] By methods according to various embodiments, hybrid processing of grids and points can be provided. For example, point features can be enhanced by concatenating contextual grid features and then converted to a grid of any resolution. Hybrid processing can avoid the loss of information from gridding in early stages and the waste of computational work in sparse high-resolution grids. Connection can be understood as adding additional information. It should be understood that the connection of data can be provided by various data structures.

[0009] According to an embodiment, information related to the environment of a vehicle includes a first category of information and a second category of information; wherein determining at least one final grid structure includes determining a first final grid structure and a second final grid structure; wherein the first category of information is determined based on the first final grid structure; and wherein the second category of information is determined based on the second final grid structure. It has been found that using some information for both the first category of information and the second category of information, and using some information specific to each of the first category of information and the second category of information, can provide efficient and reliable processing.

[0010] According to an embodiment, the first final grid structure has a different resolution than the second final grid structure. This may allow providing results of different resolutions and / or results for different regions of interest.

[0011] According to an embodiment, the first final grid structure has a size different from the size of the second final grid structure. This may allow providing results of different resolutions and / or results for different regions of interest.

[0012] According to an embodiment, the first category of information is different from the second category of information.

[0013] According to an embodiment, the first category of information includes at least one of object detection information or semantic segmentation information; and the second category of information includes at least one of object detection information or semantic segmentation information.

[0014] According to an embodiment, the measurement data comprises at least one of lidar data or radar data.For example, the measurement data may be acquired by a sensor (eg mounted on a vehicle).

[0015] According to an embodiment, determining the corresponding features of each point of the point cloud includes time fusion. For time fusion, the points of (time) frame T-1 can first be moved to the vehicle coordinate system (VCS) of frame T (this can be called self-position compensation), and then fused with the points of frame T. At each frame, the number of fused points can be fixed (for example, fixed to an integer N); therefore, downsampling the fused points to a fixed number of points can be provided. Point fusion can then be performed by connecting the points of frame T-1 and frame T and then performing random sampling to maintain N points. Alternatively, point fusion can be performed by connecting the points of frame T-1 and frame T and then performing a clustering method to obtain N clusters, where each cluster can be regarded as a point (for example, by taking the cluster center as the point).

[0016] According to an embodiment, determining the processed grid structure includes time fusion. For time fusion, in the deep network, some layers for preprocessing of sensor information can be combined with a recurrent neural network. Self-motion compensation of hidden states can be performed, allowing the network to locally correlate past and current data and train the expected output of the occupancy grid, for example, using a lidar. Such an architecture allows learning sensor models, as well as the best combination method and weighting of data adapted to the actual situation. For example, an LSTM (Long Short-Term Memory) network can be used, or any other recurrent neural network can be used.

[0017] According to an embodiment, determining at least one final grid structure includes performing a point-to-grid conversion using deformable attention. For example, each grid cell can be linked to a fixed number of points, and the cell features can be calculated by determining a weighted sum of the point features. The weights can be determined by the deformable attention. Such deformable attention is widely used, for example, as described in "Vision Transformer with Deformable Attention", Zhuofan Xia et al., 2022, arXiv:2201.00520.

[0018] According to an embodiment, determining at least one final grid structure includes: supplementing the corresponding features of each point of the point cloud with features from the processed grid structure, and determining at least one final grid structure based on the corresponding supplemented features. Supplementation can be understood as adding additional information. It should be understood that the supplementation of data can be provided by various data structures.

[0019] In another aspect, the present disclosure relates to a computer system comprising a plurality of computer hardware components configured to perform several or all of the steps of the computer-implemented methods described herein.

[0020] The computer system may include a plurality of computer hardware components (e.g., a processor, such as a processing unit or a processing network, at least one memory, such as a memory unit or a memory network, and at least one non-transient data store). It should be understood that further computer hardware components may be provided and used to perform the steps of the computer-implemented method in the computer system. The non-transient data store and / or the memory unit may include a computer program for instructing a computer, such as using a processing unit and at least one memory unit, to perform several steps or aspects or all steps or aspects of the computer-implemented method described herein.

[0021] In another aspect, the present disclosure is directed to a vehicle comprising a computer system as described herein.

[0022] According to an embodiment, the claimed vehicle further comprises at least one sensor configured for acquiring measurement data (alternatively, in an example, the at least one sensor may be comprised in a computer system comprised in the vehicle).

[0023] In another aspect, the present disclosure relates to a non-transitory computer-readable medium, which includes instructions that, when executed by a computer system, cause the computer system to perform several steps or aspects or all steps or aspects of the computer-implemented method described herein. The computer-readable medium can be configured as: an optical medium, such as a compact disk (CD) or a digital versatile disk (DVD); a magnetic medium, such as a hard disk drive (HDD); a solid-state drive (SSD); a read-only memory (ROM), such as flash memory; or the like. In addition, the computer-readable medium can be configured as a data storage accessible via a data connection (such as an Internet connection). The computer-readable medium can be, for example, an online data repository or cloud storage.

[0024] The present disclosure also relates to a computer program for instructing a computer to perform several steps or aspects or all steps or aspects of the computer-implemented method described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Exemplary embodiments and functionality of the present disclosure are described herein in conjunction with the following drawings, which schematically illustrate:

[0026] Figure 1 is an illustration of a network architecture with task-specific resolution according to various embodiments.

[0027] Figure 2A and Figure 2B is a diagram of a network architecture with range specific resolution according to various embodiments;

[0028] Figure 3 is an illustration of a network architecture with flexible resolution according to various embodiments;

[0029] Figure 4 is a flow chart illustrating a method for determining information related to an environment of a vehicle according to various embodiments; and

[0030] Figure 5 A computer system having a plurality of computer hardware components configured to perform the steps of a computer-implemented method for determining information related to an environment of a vehicle according to various embodiments is shown. DETAILED DESCRIPTION

[0031] A car perception system may need to provide different kinds of outputs, which can be obtained by different tasks. For example, an object detection task can provide a list of objects of interest, while semantic segmentation can indicate the location of the drivable space. To achieve this goal, a common approach is to design a separate network for each task. However, this strategy has several disadvantages. For example, since multiple tasks may be related, some low-level features can be shared across multiple tasks. In addition, computing features separately from raw data may result in a waste of computational resources. Moreover, different tasks may be complementary. For example, different tasks may provide complementary supervision. Training them separately may not take advantage of the complementary supervisory information.

[0032] Multiple tasks may require different spatial resolutions. For example, object detection, which outputs bounding boxes of objects, may require a lower resolution than semantic segmentation, which outputs accurate boundaries of objects and roads. In addition, for lidar and radar, which are commonly used sensors in automotive perception, the spatial resolution may depend on the distance. For example, sensor data in close distances may have a higher resolution than sensor data in far distances (which may have a lower resolution).

[0033] According to various embodiments, a unified multi-task and / or multi-resolution neural network architecture is provided. The architecture according to various embodiments may have a common feature extraction portion shared by multiple tasks, thereby saving computational costs. In addition, the architecture according to various embodiments may flexibly calculate and combine multi-resolution features for different tasks and different distances.

[0034] The neural network architecture according to various embodiments can be used for perception systems in multi-tasking and latency-sensitive applications (e.g., autonomous driving). Through a unified network, the architecture according to various embodiments can give multiple outputs, such as object detection, semantic segmentation (SemSeg), instance segmentation, or motion prediction, which can be basic components of the perception system. As an example, object detection and motion prediction can be used in automatic emergency braking (AEB), and semantic / instance segmentation can be used in automatic valet parking (AVP). The architecture according to various embodiments can be computationally efficient and therefore can be suitable for use in low-end devices, such as embedded chips.

[0035] The multi-task and multi-resolution networks according to various embodiments can be implemented in different architectures, for example, Figure 1 The architecture shown in FIG. 2 or the architecture shown in FIG. Figure 3 The architecture shown in the figure.

[0036] Figure 1A diagram 100 of a network architecture with task-specific resolution is shown in accordance with various embodiments.

[0037] As mentioned above, different tasks may require different spatial resolutions. Figure 1 As illustrated in , object detection 112 and semantic segmentation 126 may be used as examples to illustrate a network architecture according to various embodiments that computes multi-resolution features for different tasks and fuses features of different resolutions.

[0038] like Figure 1 As illustrated in , the input data may be a point cloud 102 (e.g., from a lidar or radar), which may first be processed by a shared network architecture (which may be referred to as point processing 104) to obtain some common point features. This process may be shared by object detection and semantic segmentation, and thus waste of computational effort may be avoided.

[0039] After common point processing 104, the points may be converted to a grid more suitable for use in a convolutional network. Both tasks may share the same region of interest (ROI), but may have different grid resolutions (and therefore different grid sizes). Object detection may have a lower resolution (and therefore, the grid 106 used for object detection may have a lower resolution), while semantic segmentation may have a higher resolution (and therefore, the grid 116 used for semantic segmentation may have a higher resolution). Figure 1 , grids 106 and 116 are illustrated as boxes, where the length and width of the boxes illustrate the spatial components, and where the height of the boxes illustrates the feature channels represented in the grid.

[0040] Since the size of semantic segmentation grid 116 is much larger than the size of object detection grid 106, the number of feature channels of semantic segmentation grid 116 can be reduced to a smaller value. This can help reduce the high computational cost of high-resolution grid 116.

[0041] The low resolution mesh 106 may be processed using a convolutional neural network 108 (also referred to as a ConvNet or CNN) followed by temporal fusion 110. Based on the temporally fused data, object detection 112 may be performed.

[0042] The high-resolution grid 116 may be processed using a convolutional neural network 118 (also referred to as a ConvNet or CNN) followed by temporal fusion 120. The high-resolution features (obtained by temporal fusion 120) may be enhanced by fusion with the low-resolution features (obtained by temporal fusion 110). For example, the low-resolution features (obtained by temporal fusion 110) may be upsampled (114) to obtain upsampled features 124, and the upsampled features 124 may be concatenated with the high-resolution features 122 obtained by temporal fusion 120. The low-level features may encode more contextual information, which may be complementary to the high-resolution features. In other words, the upsampled low-resolution features 124 may enhance the high-resolution grid 122.

[0043] Time fusion may be performed for different grids (performed in time fusion 110 and time fusion 120 respectively) to better accumulate historical information and reduce noise. This step may be particularly important for radar perception.

[0044] As described herein, according to various embodiments, the method can provide data for multiple tasks. For example, a multi-task network can provide that different tasks can be linked to different resolutions, high-resolution grids can have fewer channels to reduce computational costs, and multiple output grids for semantic segmentation with different resolutions can be fused as one output.

[0045] Figure 2A and Figure 2B Diagrams 200 and 250 of network architectures with distance specific resolutions are shown according to various embodiments.

[0046] like Figure 2A and Figure 2B As shown in the figure, a distance-specific resolution is taken into account. Figure 2A An example with two different ROIs, long distance and short distance, is shown. Figure 2A As illustrated in FIG, the input data may be a point cloud 202 (e.g., from a lidar or radar), which may first be processed by a shared network architecture (which may be referred to as point processing 204) to obtain some common point features. This process may be shared by further processing for long and short distances, and thus waste of computational effort may be avoided.

[0047] The far-distance grid 206 may have a lower resolution than the near-distance grid 220. Since the size of the far-distance grid 220 is larger than the size of the near-distance grid 206, the number of feature channels of the far-distance grid 220 may be reduced to a smaller value. This may help reduce the high computational cost of the far-distance grid 220.

[0048] The close-range mesh 206 may be processed using a convolutional neural network 208 (also referred to as a ConvNet or CNN) followed by temporal fusion 210 .

[0049] The long-distance grid 220 can be processed using a convolutional neural network 222 (also referred to as a ConvNet or CNN) and then temporally fused 224.

[0050] The low-resolution features 212 (obtained by the temporal fusion 210) may then be cropped (e.g., only a sub-portion 214 of the low-resolution features 212 may be used) and upsampled to obtain cropped and upsampled features 226. The cropped and upsampled features 226 may be fused with the high-resolution features 228 obtained by the temporal fusion 224. The low-resolution features 212 and the high-resolution features 226, 228 may then be used to perform a plurality of perceptual tasks, such as object detection and semantic segmentation. For example, the low-resolution features 212 may be used for low-resolution object detection 216 and low-resolution semantic segmentation 218. For example, the high-resolution features 226, 228 may be used for high-resolution object detection 230 and high-resolution semantic segmentation 232.

[0051] It should be understood that it may not be necessary to link all tasks to every feature. This should be decided based on the requirements of the perceived task.

[0052] like Figure 2A As illustrated in , object detection and semantic segmentation can be linked to both the low-resolution grid and the high-resolution grid, which means that there are multiple outputs for each task. Figure 2B The figure in the middle shows the fusion of the output.

[0053] For object detection, since the output is in the form of object lists 252 , 256 , fusion can be concatenating the object lists from the two resolutions to obtain the final object list 260 .

[0054] For semantic segmentation, the output grids 254, 258 of different resolutions may be fused, for example by upsampling the low-resolution grid 254 to obtain an upsampled grid 262, and overlapping the upsampled grid 262 with the high-resolution grid 258. In the overlapping region, only the high-resolution grid 258 may be used.

[0055] Figure 3 A diagram 300 of a network architecture with flexible resolution according to various embodiments is shown. Figure 3As illustrated in FIG, the input data may be a point cloud 302 (e.g., from a lidar or radar), which may first be processed by a shared network architecture (which may be referred to as point processing 304) to obtain some common point features. This process may be shared by further processing for the low-resolution mesh and for the point features.

[0056] The low-resolution grid 306 may be determined based on the output of the point processing 304. For example, after the point processing 304, the points may be converted into the low-resolution grid 306, which encodes the context information at a relatively low computational cost.

[0057] The low-resolution grid 306 may be processed using a convolutional neural network 308 (also referred to as a ConvNet or CNN) followed by temporal fusion 310 to obtain grid features 312 .

[0058] The output of point processing 304 may be used for temporal fusion 314 to obtain point cloud 316, which may include a plurality of points 318 and corresponding point features 320. It should be appreciated that point cloud 316 may have the original resolution of point cloud 302 because point cloud 316 has not undergone any discretization into a grid.

[0059] Point cloud 316 may be supplemented by mesh features 312 to obtain further point cloud 322, which may include a plurality of points 318 and corresponding point features 320 and corresponding mesh features 324. In other words, each point 318 may be supplemented with additional features based on mesh features 312. It should be appreciated that since point cloud 322 has not undergone any discretization into a mesh, point cloud 322 may have the original resolution of point cloud 302.

[0060] As an illustration, Figure 3 As shown in the figure, the points are not converted to a high-resolution grid. Instead, temporal fusion and further feature extraction can be performed at the point level, which can maintain the original spatial resolution. Afterwards, the point features can be enhanced (or supplemented) by concatenating with low-resolution features.

[0061] The enhanced points of the point cloud 322 may have the original spatial resolution, and may integrate contextual features and point features. Using the enhanced points of the point cloud 322, meshes with different ROIs and spatial resolutions may be generated and linked to different tasks. For point-to-mesh conversion (e.g., from the point cloud 322 to meshes 326, 328, 330, where each of the meshes 326, 328, 330 may have a task-specific ROI and resolution), for example, a basic method (vanilla method) or an attention-based method may be used.

[0062] In the basic approach, points that fall into a grid cell can be merged (e.g., using average pooling) to construct the features of that grid cell. In the attention-based approach, the correspondence between point 322 and grid cell 334 can be learned by a neural network. A weighted sum can then be used to construct the features of the grid cell based on its corresponding points. The weights of the points can also be learned by a neural network.

[0063] Figure 4 A flow chart 400 is shown, which illustrates a method for determining information related to an environment of a vehicle according to various embodiments. In 402, a point cloud can be determined based on measurement data. The point cloud can include a plurality of points or can be a plurality of points. In 404, a corresponding feature of each point of the point cloud can be determined. In 406, a grid structure can be determined based on the point cloud. In 408, a processed grid structure can be determined based on processing the grid structure using an artificial neural network. In 410, at least one final grid structure can be determined based on the point cloud, the corresponding feature of each point of the point cloud, and the processed grid structure.

[0064] According to various embodiments, the information related to the environment of the vehicle may include or may be a first category of information and a second category of information. According to various embodiments, determining at least one final grid structure may include or may be determining a first final grid structure and a second final grid structure. According to various embodiments, the first category of information may be determined based on the first final grid structure. According to various embodiments, the second category of information may be determined based on the second final grid structure.

[0065] According to various embodiments, the first final mesh structure may have a resolution different from a resolution of the second final mesh structure.

[0066] According to various embodiments, the first final mesh structure may have a size that is different from a size of the second final mesh structure.

[0067] According to various embodiments, the first category of information may be different from the second category of information.

[0068] According to various embodiments, the first category of information may include or may be at least one of object detection information or semantic segmentation information. According to various embodiments, the second category of information may include or may be at least one of object detection information or semantic segmentation information.

[0069] According to various embodiments, the measurement data may include or may be at least one of lidar data or radar data.

[0070] According to various embodiments, determining the respective features of each point of the point cloud may include or may be temporal fusion.

[0071] According to various embodiments, determining the processed mesh structure may include or may be temporal fusion.

[0072] According to various embodiments, determining at least one final mesh structure may include or may be a point-to-mesh conversion using deformable attention.

[0073] According to various embodiments, determining at least one final mesh structure may include or may include: supplementing corresponding features of each point of the point cloud with features from the processed mesh structure; and determining at least one final mesh structure based on the corresponding supplemented features.

[0074] Each of steps 402, 404, 406, 408, and 410, and further steps described above, may be performed by computer hardware components.

[0075] Figure 5 A computer system 500 having a plurality of computer hardware components configured to perform the steps of a computer-implemented method for determining information related to an environment of a vehicle according to various embodiments is shown. The computer system 500 may include a processor 502, a memory 504, and a non-transitory data store 506. A sensor 508 may be provided as part of the computer system 500 (e.g., Figure 5 ), or may be provided external to the computer system 500.

[0076] Processor 502 may execute instructions provided in memory 504. Non-transitory data storage 506 may store computer programs including instructions that may be transferred to memory 504 and then executed by processor 502. Sensor 508 may be used to determine a point cloud as described herein.

[0077] The processor 502, the memory 504 and the non-transitory data storage 506 may be coupled to each other, for example, via an electrical connection 510 (such as, for example, a cable or a computer bus) or via any other suitable electrical connection to exchange electrical signals. The sensor 508 may be coupled to the computer system 500, for example, via an external interface, or may be provided as part of the computer system (in other words: inside the computer system, for example coupled via the electrical connection 510).

[0078] The terms "coupled" or "connected" are intended to include "direct coupled" (eg, via a physical link) or "directly connected" as well as "indirect coupled" or "indirectly connected" (eg, via a logical link), respectively.

[0079] It should be understood that what is described with respect to one of the above methods may be similarly applicable to the computer system 500 .

[0080] Reference numerals list

[0081] 100 Illustration of a network architecture with task-specific resolution according to various embodiments

[0082] 102 point cloud

[0083] 104 points processing

[0084] 106 grids for object detection (low resolution grids)

[0085] 108 Convolutional Neural Networks

[0086] 110 Time Fusion

[0087] 112 Object Detection

[0088] 114 Upsampling

[0089] 116 grids for semantic segmentation (high resolution grids)

[0090] 118 Convolutional Neural Networks

[0091] 120 Time Fusion

[0092] 122 Processed high-resolution grid obtained by time fusion 120

[0093] 124 Upsampled low-resolution grid

[0094] 126Semantic Segmentation

[0095] 200 Illustration of a network architecture with distance specific resolution according to various embodiments 202 Point cloud

[0096] 204 point processing

[0097] 206 long-distance grid (low-resolution grid)

[0098] 208 Convolutional Neural Network

[0099] 210 Time Fusion

[0100] 212 Low-resolution features

[0101] 214 Subsections of low-resolution features

[0102] 216 Low Resolution Object Detection

[0103] 218 Low-resolution semantic segmentation

[0104] 220 close-range grids (high-resolution grids)

[0105] 222 Convolutional Neural Network

[0106] 224 Time Fusion

[0107] 226 High Resolution Features

[0108] 228 high resolution features

[0109] 230 High Resolution Object Detection

[0110] 232 High Resolution Semantic Segmentation

[0111] 250 Illustration of a network architecture with distance specific resolution according to various embodiments

[0112] 252 Low-resolution object list

[0113] 254 low resolution grid maps

[0114] 256 High-resolution object list

[0115] 258 High-resolution object lists

[0116] 260 Final Object List

[0117] 262 Upsampled grid

[0118] 300 is a diagram of a network architecture with flexible resolution according to various embodiments

[0119] 302 Point Cloud

[0120] 304 point processing

[0121] 306 low-resolution meshes

[0122] 308 Convolutional Neural Network

[0123] 310 Time Fusion

[0124] 312 Mesh Features

[0125] 314 Time Fusion

[0126] 316 Point Cloud

[0127] 318 points

[0128] 320 features

[0129] 322 Point Cloud

[0130] 324 Mesh Features

[0131] 326 grids for task-specific ROIs and resolutions

[0132] 328 grids for task-specific ROIs and resolutions

[0133] 330 grids for task-specific ROIs and resolutions

[0134] 332 points

[0135] 334 grid cells

[0136] 400 is a flow chart illustrating a method for determining information related to the environment of a vehicle according to various embodiments.

[0137] Step 402: Determine a point cloud based on the measurement data, the point cloud including a plurality of points

[0138] Step 404: Determine the corresponding features of each point in the point cloud

[0139] Step 406: Determine the grid structure based on the point cloud

[0140] Step 408: Determine a processed grid structure based on processing the grid structure using an artificial neural network

[0141] Step 410: Determine at least one final grid structure based on the point cloud, the corresponding features of each point of the point cloud, and the processed grid structure

[0142] 500 Computer system according to various embodiments

[0143] 502 processor

[0144] 504 Memory

[0145] 506 Non-transient data storage

[0146] 508 Sensors

[0147] 510 connections

Claims

1. A computer-implemented method for determining information related to an environment of a vehicle, the method comprising the following steps performed by computer hardware components: - determining (402) a point cloud based on the measurement data, the point cloud comprising a plurality of points; - determining (404) a corresponding feature for each point of the point cloud; - determining (406) a grid structure based on the point cloud; - determining (408) a processed grid structure based on processing the grid structure using an artificial neural network; as well as - determining (410) at least one final mesh structure based on the point cloud, the corresponding features of each point of the point cloud and the processed mesh structure.

2. The computer-implemented method of claim 1, wherein: The information related to the environment of the vehicle includes a first category of information and a second category of information; wherein determining (410) the at least one final grid structure comprises determining a first final grid structure and a second final grid structure; wherein the first type of information is determined based on the first final grid structure; and The second type of information is determined based on the second final grid structure.

3. The computer-implemented method of claim 2, wherein: The first final grid structure has a resolution different from a resolution of the second final grid structure.

4. The computer-implemented method of claim 2, wherein: The first final mesh structure has a size different from a size of the second final mesh structure.

5. The computer-implemented method of any one of claims 2 to 4, wherein: The first category of information is different from the second category of information.

6. The computer-implemented method of any one of claims 2 to 4, wherein: The first category of information includes at least one of object detection information or semantic segmentation information; and The second type of information includes at least one of object detection information or semantic segmentation information.

7. The computer-implemented method of any one of claims 1 to 4, wherein: The measurement data includes at least one of lidar data or radar data.

8. The computer-implemented method of any one of claims 1 to 4, wherein: Determining (404) the corresponding features for each point of the point cloud includes temporal fusion.

9. The computer-implemented method of any one of claims 1 to 4, wherein: Determining (408) the processed mesh structure includes temporal fusion.

10. The computer-implemented method of any one of claims 1 to 4, wherein: Determining (410) the at least one final mesh structure includes performing a point-to-mesh conversion using deformable attention.

11. The computer-implemented method of any one of claims 1 to 4, wherein: Determining (410) the at least one final mesh structure includes supplementing the corresponding features of each point of the point cloud with features from the processed mesh structure and determining the at least one final mesh structure based on the corresponding supplemented features.

12. A computer system (500) comprising a plurality of computer hardware components configured to perform the steps of the computer-implemented method of at least one of claims 1 to 11.

13. A vehicle comprising the computer system (500) of claim 12.

14. The vehicle of claim 13, further comprising at least one sensor configured to acquire the measurement data.

15. A non-transitory computer-readable medium comprising instructions for executing the computer-implemented method of at least one of claims 1 to 11.

Images