COMPUTER IMPLEMENTED METHOD AND DATA PROCESSING HARDWARE FOR PROCESSING SENSOR DATA POINTS - Patent application

Abstract

COMPUTER IMPLEMENTED METHOD AND DATA PROCESSING HARDWARE FOR PROCESSING SENSOR DATA POINTS - Patent application The present invention relates to a computer-implemented method for processing sensor data points using data processing hardware, initializing a distribution buffer to store an initial plurality of Gaussian distributions, each Gaussian distribution of the initial plurality of Gaussian distributions comprising an initial plurality of sensor data points sequentially received from at least one sensor and having a corresponding predetermined distribution distance threshold, and sequentially updating the distribution buffer for n new sensor data points based on a distribution distance condition to generate updated multiple Gaussian distributions comprising either one of a new Gaussian distribution, an updated single Gaussian distribution, or an updated merged Gaussian distribution.

Description

[Technical field]

[0001] The present invention relates to a computer-implemented method and data processing hardware for processing sensor data points.

[0002] Terms used in the present invention Raw data points, also known as raw sensor data or sensor data points, represent data received from a sensor.

[0003] A lightweight environment map is a map that provides all the details of an environment with a small memory footprint (RAM usage), small central processing unit (CPU) usage, and overall small system resource usage. [Background technology]

[0004] Sensors providing various types of data are increasingly used in a wide range of activities and technical fields. Typically, the sensors provide a number of raw data points in each processing cycle, which are processed to provide relevant information. The processing of the raw data points includes creating some data structure according to the needs, which is used to output the data processing of the sensor data points in each processing cycle.

[0005] In some cases, the output of data processing of the raw data points is used to build an environmental map, for example describing static objects in the environment, where one or more sensors generate "dense" raw data points from objects in a scene, for example in the automotive industry, or in warehouse management, or other spaces where a large number of objects are stored, etc.

[0006] In other cases, the output of data processing of raw data points is used to transmit to decision makers without generating an environment map, for example in the medical field, where data from patients is collected by sensors and transmitted to medical devices for further analysis by specialists.

[0007] Disadvantages of the Prior Art The first disadvantage is that the distribution of data received from the sensor is not preserved during processing of a large number of raw points. The second disadvantage stems from the first disadvantage. Without preserving the distribution of data received from the sensor, storing a large number of raw points would result in a large amount of data, and processing this large amount of data would require increasingly larger memory and processing power of one or more hardware devices.

[0008] In the specific case of the automotive industry, sensor data comes from modern sensors, e.g. lidar, radar, ultrasonic sensors, which are used for environment mapping of static objects from the scene, providing a large number of points in each processing cycle. Typically, an environment map is constructed using these sensor data points, e.g., a grid map or a point cloud. Creating and updating the map requires large memory and computational resources, limiting options for building the architecture of the vehicle's electronic control unit. Summary of the Invention [Problem to be solved by the invention]

[0009] The problem that the present invention aims to solve is to provide a more efficient way to process sensor data points, in particular received from sensors that collect data from stationary objects in the environment, while preserving the distribution of the sensor data in a predefined data structure, using a smaller amount of memory and fewer computational resources compared to existing solutions. [Means for solving the problem]

[0010] To achieve this goal, the inventors have provided in a first aspect of the invention a computer-implemented method for processing sensor data points comprising four steps performed by data processing hardware: 1. Initial multiple Gaussian distributions g nwherein each Gaussian distribution g comprises an initial plurality of sensor data points Pr sequentially received from at least one sensor, and each Gaussian distribution g has a corresponding predetermined distribution distance threshold Dd min and 2. New sensor data points Pr from at least one sensor new Sequentially acquire new sensor data points Pr new calculating the distribution distance Dd between each Gaussian distribution g and 3. For each Gaussian distribution g, the distribution distance Dd satisfies the distribution distance condition Dd <Dd min Whether the distribution distance Dd<the corresponding predetermined distribution distance threshold Dd min A step of verifying whether 3.1.Distribution distance condition Dd <Dd min If there is no Gaussian distribution g that satisfies the new sensor data point Pr new A new Gaussian distribution g new Generate 3.2.Distribution distance condition Dd <Dd min If a single Gaussian distribution g1 satisfies the new sensor data point Pr new Update the single Gaussian distribution g1 by adding 1-updated Generate 3.3.Distribution distance condition Dd <Dd min Let g1, g2, …, g be a set of at least two Gaussian distributions. m If there are at least two Gaussian distributions g1,g2,…,g m Merge and merge Gaussian distribution g m-merged and generate new sensor data points Pr new Merge Gaussian distribution g by adding m-merged and update the updated merged Gaussian distribution g m-merged-updated generating a 4. New Gaussian distribution g new or The updated single Gaussian distribution g 1-updated or Updated merged Gaussian distribution g m-merged-updated By including one of the initial Gaussian distributions g n , and update the multiple Gaussian distributions g n-updated and generating Steps 1.2, 1.3 and 1.4 are repeated to generate n new sensor data points Pr n-new We have conceived a computer-implemented method for iterating over

[0011] In a second aspect of the invention, there is provided data processing hardware configured to perform the steps of a computer-implemented method for processing sensor data points.

[0012] The present invention extends to any novel aspect or feature described and / or illustrated in this disclosure. Any feature of one aspect of the invention may be applied to other aspects of the invention in any suitable combination. Particular combinations of the various features of the invention may be implemented and / or provided and / or used independently. Effect of the Invention

[0013] The main advantages of the present invention are: Generating a predetermined data structure for the sensor data points received as input from one or more sensors (e.g., ultrasonic, radar, etc.) during processing of the sensor data points for successive processing cycles, and storing the predetermined data structure. To conserve memory and computing power resources of data processing hardware by using a predefined data structure during processing of raw point clouds for successive processing cycles. The savings in memory and computing power resources allows the invention to be used to process raw sensor data with a wider range of data processing hardware, especially small data processing hardware, while this allows the invention to be used in a wider range of situations, for example by providing data processing hardware on some sensors instead of sending the raw sensor data somewhere to be processed. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 shows a schematic diagram of a method for processing sensor data points according to the present invention. [Diagram 2] FIG. 2 shows an example scene in one of the embodiments using the method of the present invention. [Diagram 3] FIG. 3 shows the result of applying the prior art method to the scene of FIG. [Figure 4] FIG. 4 shows, in the same embodiment using the example scene of FIG. 2, a schematic diagram of applying the method to construct an ellipse from the points shown in FIG. [Diagram 5] FIG. 5 shows a schematic diagram of the method according to the same embodiment for the scene of FIG. 2, ie, constructing an object (in this case a convex hull) from an ellipse. [Figure 6] FIG. 6 shows a schematic diagram of another preferred embodiment of the present invention for constructing an object based on an ellipse. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Detailed explanation and implementation example Raw sensor data typically embeds large amounts of unclean and useless data. Large amounts of unnecessary and useless data unnecessarily increase memory and computational power, which is undesirable in constrained sensor networks. Therefore, the raw sensor data needs to be first subjected to data cleaning processing, and the cleaned sensor data is output, and the data is processed to obtain relevant information. The proposed method also processes the raw sensor data using a Gaussian distribution. Since the Gaussian distribution is constructed sequentially, there is no need to store the raw points.

[0016] Referring to FIG. 1, the computer implemented method for processing sensor data points according to the present invention comprises four steps performed by the data processing hardware. In step 1.1, we use the initial Gaussian distributions g n , where each Gaussian distribution g comprises an initial plurality of sensor data points Pr sequentially received from at least one sensor. The sensor data points Pr used in the method may be received, for example, from one or more sensors, or as a result of fusion or other type of processing of the sensor data points Pr received from the sensors, from a processing device, or from any other source providing sensor data points. Non-limiting examples of sensors include ultrasonic sensors, radar sensors, position sensors, e.g., GPS, motion sensors, e.g., acceleration sensors, gyroscopes, camera sensors, light sensors, microphones, proximity sensors, magnetic sensors, sensors measuring temperature, pressure, humidity, body medical parameters, chemical and biochemical substances, and nerve signals, as well as infrared sensors, infrared cameras, motion detectors measuring distance to nearby objects, presence of smoke and gas, and humidity sensors. The sensor data points Pr include any of the following types: sensor data points Pr with two coordinates x,y, sensor data points Pr with three coordinates x,y,z, sensor data points Pr with two or three coordinates and further features, e.g., reflection amplitude, direction, radial velocity, acceleration. The sensor data points Pr also include errors, which have the same number of coordinates as the sensor data points Pr, i.e., two coordinates x,y or three coordinates x,y,z. Gaussian distribution g n is defined by a mean and a covariance, e.g., for a two-dimensional sensor data point with x and y coordinates, the mean is a vector of the two coordinates, and the covariance matrix is ​​a 2x2 symmetric matrix, i.e., in such a case, five floating points are needed to represent the distribution. Each Gaussian distribution g has a corresponding predefined distribution distance threshold Dd min has. In step 1.2., new sensor data points Pr new The data processing hardware sequentially acquires new sensor data points Pr new and the initial multiple Gaussian distributions g nCalculate the distribution distance Dd between each Gaussian distribution g from Furthermore, in step 1.3, the data processing hardware determines whether, for each Gaussian distribution g, the distribution distance Dd satisfies the distribution distance condition Dd <Dd min Whether or not the distribution distance Dd<the corresponding predetermined distribution distance threshold Dd min Verify whether it is. There are three mutually exclusive options: 1.3.1.Distribution distance condition Dd <Dd min If there is no Gaussian distribution g that satisfies the new sensor data point Pr new A new Gaussian distribution g new Generate. 1.3.2.Distribution distance condition Dd <Dd min If a single Gaussian distribution g1 satisfies the new sensor data point Pr new Update this single Gaussian distribution g1 by adding 1-updated Generate. 1.3.3.Distribution distance condition Dd <Dd min Let g1, g2, …, g be a set of at least two Gaussian distributions. m If there are at least two Gaussian distributions g1,g2,…,g m Merge and merge Gaussian distribution g m-merged and generate new sensor data points Pr new By adding this merge Gaussian distribution g m-merged and update the updated merged Gaussian distribution g m-merged-updated Generate. In the final step 1.4. New Gaussian distribution g new or The updated single Gaussian distribution g 1-updated or Updated merged Gaussian distribution g m-merged-updated By including one of the initial Gaussian distributions g from the distribution buffer G, n Update. Steps 1.2, 1.3 and 1.4 are repeated to generate n new sensor data points Pr n-newRepeat for . At the end of the method, the updated Gaussian distributions g n-updated in the distribution buffer G.

[0017] Updated multiple Gaussian distributions g n-updated The main advantage of the method is that during processing of the sensor data points for successive processing cycles, the predetermined data structure is distributed according to a Gaussian distribution and a predetermined distribution distance threshold Dd min and storing each predetermined data structure. The main advantage of this comes from removing the need to store multiple sensor data points by filtering them with Gaussian distribution features to remove outliers, as shown in the embodiment shown in FIG. Applying the method of the present invention has the advantage of saving memory and computational power resources of the data processing hardware by using a predefined data structure during processing of the raw point cloud for successive processing cycles. Furthermore, the present invention has the advantage that the savings in memory and computing power resources allows raw sensor data to be processed by a wider range of data processing hardware, particularly small data processing hardware, thereby enabling the present invention to be used in a wider range of situations.

[0018] Before detailing some examples of the use of the invention in various technical fields, it is convenient to present some preferred embodiments of the method of the invention, since the various embodiments make it possible to use the invention in a wide range of areas.

[0019] In a preferred embodiment, the distribution distance Dd is the new sensor data point Pr new and the mean μ of each Gaussian distribution g g is the Euclidean distance between A particular advantage of using Euclidean distance is that it conserves computing power resources, which allows the present invention to be used to process raw sensor data with more extensive data processing hardware. Conserving computing power resources has the further advantage of reducing computational costs.

[0020] In another preferred embodiment, to further save computational resources, the distribution distance is a norm that is one distance, i.e., for example, we use the l1 norm, which defines the distribution distance Dd=|x1-x2|+|y1-y2| in two dimensions.

[0021] In another preferred embodiment, the distribution distance Dd is calculated by dividing the initial Gaussian distributions g n For each Gaussian distribution g, a new sensor data point Pr new is the probability of

number

number

[0022] In another preferred embodiment, the calculation of the distribution distance Dd is performed using a plurality of initial Gaussian distributions g nFor each Gaussian distribution g, the point Pr new This is done by calculating the normalized probability of

number

[0023] In another preferred embodiment, the distribution distance Dd is calculated using the Mahalanobis distance formula. new Whether to define the distribution distance Dd as the probability of a new sensor data point Pr new and the mean μ of each Gaussian distribution g g The use of the Mahalanobis distance formula applies regardless of whether the distance is calculated as the Euclidean distance between Using the Mahalanobis distance formula is particularly advantageous because it is less expensive to compute than other distances except for the Euclidean distance.

[0024] In another preferred embodiment, n new sensor data points Pr n-new The updated single Gaussian distribution g by successively adding 1-updated and the updated merged Gaussian distribution g m-merged-updated The incremental update of is done according to the following formula:

number

[0025] In another preferred embodiment, n new sensor data points Pr n-new A more numerically stable updated single Gaussian distribution g by successively adding 1-updated and the updated merged Gaussian distribution g m-merged-updated The incremental update of is done according to the following formula:

number

number

number

[0026] In another preferred embodiment, the new sensor data point Pr new is the covariance matrix

number

number

number

[0027] In another preferred embodiment, at least two Gaussian distributions g1, g2, ..., g m The merge of at least two Gaussian distributions g1,g2,…,g m We successively merge pairs of two distributions from the m-merged and weighting the means and variances of the two distributions from each pair of two distributions. Successive merging is done as follows: At least two Gaussian distributions g1, g2, …, g m Select a pair of distributions from By weighting the means and variances of the two distributions from the pair, the first intermediate merged Gaussian distribution g m-merged-intermediary Generate At least two Gaussian distributions g1, g2, …, g m Select another distribution from and merge it with the intermediate Gaussian distribution g m-merged-intermediary and merge it into the second intermediate merged Gaussian distribution g m-merged-intermediary Generate At least two Gaussian distributions g1, g2, …, g m Further intermediate merging Gaussians g are added until all distributions from m-merged-intermediary Continue generating. The successive merging is performed according to the following formula:

number

number

[0028] The method of the present invention in any of the above embodiments is applied in various technical fields. Depending on the purpose of using the present invention, various types and numbers of sensors are used, and better results can be obtained from the above specific embodiments. For all possible uses of the method, the advantage of saving memory and computing power is possible, for example, by providing data processing hardware on some sensors instead of sending raw sensor data to be processed somewhere, allowing the use of small hardware processing devices.

[0029] As can be seen in Figs. 2, 4 and 5, the updated Gaussian distributions g n-updated The new sensor data points Pr can be used for intelligent parking management, smart transportation systems where drivers can easily check and find which parking lots have free spaces, or intelligent warehouse management where they can see where there are available free spaces. new is sequentially obtained from sensors that control these types of applications, e.g. GPS sensors for position, acceleration sensors for speed, gyroscopes for orientation, RFID for vehicle identification, infrared sensors for passenger and vehicle detection, cameras for recording vehicle movement, etc.

[0030] The updated Gaussian distributions g n-updated The method can be used in smart transportation applications, traffic monitoring and management applications that use sensors to manage daily traffic in a city, intelligent information processing systems to appropriately route traffic, locate drunk drivers, estimate traffic conditions and patterns so as to predict future traffic conditions, thereby minimizing traffic congestion, ensuring easy and hassle-free parking, and avoiding accidents, and can also implement vehicle tracking systems for traffic monitoring. The updated multiple Gaussian distributions g generated by the method can also be used in smart transportation applications, traffic monitoring and management applications that use sensors to manage daily traffic in a city, intelligent information processing systems that can appropriately route traffic, locate drunk drivers, estimate future traffic conditions, minimize traffic congestion, ensure easy and hassle-free parking, and avoid accidents, and can also implement vehicle tracking systems for traffic monitoring. n-updated can be used in applications for ensuring the safety of people travelling with their vehicles and in accident detection applications using corresponding sensors.

[0031] New sensor data points Pr from the sensor that collects data from the patient new are sequentially obtained, the updated Gaussian distributions g generated by the steps of the method n-updated can be used in specialized applications to continuously monitor and record a patient's health status and send an alert if any abnormal indicators are found.

[0032] 2, 3, 4 and 5, a general example of using the method of the present invention in any of the above embodiments to build an intelligent parking management or intelligent warehouse management is shown, in which a number of initial Gaussian distributions g from a distribution buffer G are obtained. n and the updated multiple Gaussian distributions g n-updated Using the above, multiple objects from the buffer O are n We create a lightweight representation of the static environment with multiple objects o from a buffer O. n can be of any mathematical type, for example a polygon, a convex hull, etc. The mathematical representation corresponds to a real object or a part of a real object. For example, a parked vehicle can be represented by one or more polygons. Each object is constructed from a similar distribution based on distance measurements.

[0033] Such lightweight representations of static environments are used, for example, when there is a need to locate free space between objects, such as in the automotive industry, or in warehouse management, or in other spaces where multiple objects are stored.

[0034] FIG. 2 shows an example scene of an embodiment in which an ego vehicle is scanning two side-by-side vehicles parked parallel to a wall with an open parking slot between them.

[0035] FIG. 3 shows "dense" data points from objects in a scene, ie, two-dimensional data points acquired from 12 ultrasonic sensors on the ego vehicle, according to the prior art.

[0036] In step 8.1, we use the initial Gaussian distributions g n From the distribution list of initial objects o n-initial Define {overscore (g)}, for example an object o with distributions g1, g2, g5, in particular o={g1, g2, g5}. In step 8.2, the updated Gaussian distributions g n-updated from, New Gaussian distribution g new or The updated single Gaussian distribution g 1-updated or Updated merged Gaussian distribution g m-merged-updated , which for ease of understanding will be referred to as the updated distribution. Here, we estimate the initial objects o from the updated distribution. n-initial Then, calculate the object distance Do to each object o in the set of distributions g. Thus, for the above example, the object distance Do is defined as the minimum distance between the updated distribution g and all distributions belonging to object o, i.e., distance(o,g)=min{distance(g1,g),distance(g2,g),distance(g5,g)},o={g1,g2,g5}, where the inter-distribution distance is calculated using the Kullback-Leibler divergence or Bhattacharya distance as defined above. In step 8.3, it is determined whether the updated distribution satisfies the object distance condition, i.e., whether the object distance Do<a predetermined object distance threshold Do min (Do <Do min ) and verify whether the initial objects o n-initial Update the updated objects o according to the update distribution type as follows: n-updated Generate The updated distribution is the new Gaussian distribution g new If so, then the initial objects o n-initial The update is done as follows: Early Objects n-initial For any object in, the updated distribution satisfies the object distance condition (Do <Do min ) is not satisfied, a new Gaussian distribution g new By assigning a new object o new and store a new object o in the object buffer O. new Save; For a single object o1, the updated distribution satisfies the object distance condition (Do <Do min ) the new Gaussian distribution g new Update a single object o1 by assigning 1-updated and stores the updated single object o in the object buffer O. 1-updated Save. For the above example where the object is defined as a list of Gaussian distributions o={g1, g2, g5}, after the Gaussian distribution assignment the object becomes o={g1, g2, g5, g}. As described in the preferred embodiment below, the convex hull representation may be updated by selecting specific points to represent each distribution from the corresponding object o using any convex hull construction algorithm. Early Objects n-initial Multiple objects of m Regarding the updated distribution, the object distance condition (Do <Do min ) is satisfied, multiple objects o m Merge and merge objects m-merged Generate a new Gaussian distribution g new Merge objects m-merged Merge objects by assigning m-merged Update the updated merge object o m-merged-updated and store the updated merged object o in the object buffer O. m-merged-updated Save. The first object is defined as a list of Gaussians o = {g1, g2, g5} and the second object is o' = {g3, g 10}, after merging the first object o with the second object o', o''={g1,g2,g5,g3,g 10}, we obtain a merge object defined as The updated distribution is a single Gaussian distribution g 1-updated If so, then the initial objects o n-initial The update of x is done by updating the representation of the corresponding object o with a single Gaussian distribution g1, and the updated object o updated and stores the updated object o in the object buffer O. updated Save; The updated distribution is the updated merged Gaussian distribution g m-merged-updated If so, then the initial objects o n-initial The update is done as follows: For a single object o1, the updated distribution is <Do min If it satisfies, update the representation of the single object o1 by including the updated distribution, and1-updated and stores the updated single object o in the object buffer O. 1-updated Save; Early Objects n-initial Multiple objects of m Regarding the object distance condition, the updated distribution is <Do min If multiple objects o m Merge and merge objects m-merged Generate an updated merged object o by including the updated distribution. m-merged-updated and store the updated merged object o in the object buffer O. m-merged-updated Save. In step 8.4. At least one novel object new or Updated Objects updated or Merge objects m-merged From the object buffer O, we create and continually update a lightweight representation of the static environment by exporting one of the objects, Repeat steps 8.2 to 8.4 to generate n new sensor data points Pr n-new Repeat for .

[0037] FIG. 4 shows the results of applying a prior art method for processing sensor data points to the example of FIG. 3; the 22,093 raw data points shown each have three coordinates (x, y, the "reflection direction" direction), but for simplicity only the x and y coordinates are shown.

[0038] The distributions shown in Figure 5 show the results of a computer-implemented method for processing sensor data points according to the present invention. Thus, 2093 raw data points are shown, with a total of 320 distributions. Each of the 320 distributions requires 5 floating points to be stored. This is because multiple objects nThis means that building a lightweight representation of a static environment with In the prior art, multiple objects n In total, (2 for each coordinate)*22093=44186 floating point values ​​are required to represent the low points of a static environment with As a result, multiple objects n The compressed map, specifically the lightweight representation, of a static environment with is only about 3.6%=1600*100 / 44186 in size compared to the uncompressed data. In this example, the use of the predefined data structure during processing of the raw data point cloud for successive processing cycles using the method according to the invention results in a saving of about 96.4% in size of memory and computing power resources of the data processing hardware. In other words, the compression ratio = before / after is approximately 28, meaning that the compressed data is approximately 1 / 28th the size of the uncompressed data.

[0039] It should be noted that the compression ratio depends on the concentration of sensor data points and the distribution of clutter and measurement errors: if the latter distribution is Gaussian, the computer-implemented method for processing sensor data points according to the present invention representing the sensor data points will result in a better compression ratio.

[0040] Various representations of maps can be created from objects created as distribution lists, which means grouping distributions into objects. Each concrete representation of a map depends on the concrete needs and the concrete program for creating this concrete representation of the map, for example, one concrete representation of the map may comprise polygonal shapes, another concrete representation of the map may comprise simple shapes, e.g., convex hulls, etc. The construction of concrete representations, such as but not limited to polygonal shapes, convex hulls, etc., from the distribution list is done by algorithms available in the prior art literature.

[0041] In one option, New Object new , Updated Object updated and Updated merge object m-merged-updated is the mean μ of the corresponding Gaussian distribution. n Created by. In another option, New Object new , Updated Object updated and Updated merge object m-merged-updated is made from two points chosen on the principal axis of the corresponding Gaussian distribution, which correspond to a certain probability, for example the 2 sigma points or any other multiple sigma of the principal axis. In one variation, New Object new , Updated Object updated and Updated merge object m-merged-updated If is constructed from 2 sigma points, then the corresponding Gaussian distribution has 2*k dimensions. In another option, New Object new , Updated Object updated and Updated merge object m-merged-updated is a certain number L of sensor data points Pr L where a certain number L is sampled from the hyperellipse corresponding to a certain probability. New point Pr new The number of is adjusted to take into account that the distribution in the final map is not overfitted. New point Pr new A particular advantage of increasing the number of is a more accurate concrete map representation while saving memory and computing power resources, thereby enabling the present invention to be used to process raw sensor data with a wider range of data processing hardware.

[0042] Figure 6 shows the multiple object o n 1 shows an illustrative image of the preferred embodiment illustrating the construction of a plurality of objects o n The use of constructed ellipses, e.g., convex hulls or polygons to construct or update can be done similarly to using points whose means of Gaussians represent the point locations. However, if the ellipse is large and corresponds to a large variance, more sensor data points will be used to construct the object, for example, two points from the major axis of the ellipse or more sensor data points from the contour of the ellipse. This ellipse is represented by a specific Gaussian distribution g n and the specific probability prob(Pr,g n ), where the probability determines which ellipse is selected from the Gaussian distribution. Choosing a specific probability is important because a very small probability, in the two-dimensional case, would result in an ellipse covering the entire x,y plane, whereas choosing a large probability would only give an average. n ) may be selected based on the cumulative distribution function CDF, and the percentage of points that should be included in the object may be selected.

[0043] In a second aspect of the invention, there is provided data processing hardware configured to perform the steps of the method according to any embodiment.

[0044] In a preferred embodiment, data processing hardware is further configured to be included in the sensor, taking advantage of the use of the sensor's processor to perform the method of the present invention. A lightweight representation of the static environment in any of the above variations may be transmitted to the vehicle's drive system in communication with the data processing hardware.

[0045] While the description of the method has been disclosed in detail with respect to the preferred embodiment, it will be understood by those skilled in the art that various modifications and variations of the present invention will be apparent without departing from the essential scope of the teachings of the present invention and without diminishing its advantages. Accordingly, it is intended that such modifications and variations be covered by the appended claims. [Explanation of symbols]

[0046] Pr new New sensor data points received from at least one sensor Pr n-new n new sensor data points Pr Initial multiple sensor data points g n Initial multiple Gaussian distributions g n-updated Updated Multiple Gaussian Distributions G A distribution buffer with multiple Gaussian distributions g Each Gaussian distribution of multiple Gaussian distributions G Dd New sensor data point Pr new and the distribution distance between each Gaussian distribution g Dd min A given distribution distance threshold g new New sensor data point Pr new New Gaussian distribution including g1 distribution distance condition Dd <Dd min A single Gaussian distribution that satisfies g 1-updated New sensor data point Pr new An updated single Gaussian distribution containing g1,g2,g m Distribution distance condition Dd <Dd min At least two Gaussian distributions that satisfy g m-merged At least two Gaussian distributions g1, g2, …, g m The merged Gaussian distribution of g m-merged-updated New sensor data point Pr new The merged Gaussian distribution g m-merged The updated merged Gaussian distribution of O Buffer with multiple objects o n-initial Initial multiple Gaussian distributions g n The initial objects are created as a distribution list from o n-updated Updated multiple Gaussian distributions g n-updated Updated multiple objects created based on o n Initial multiple Gaussian distributions g n and the updated multiple Gaussian distributions g n-updated Multiple objects created using Do the initial multiple objects from the updated distribution. n-initial The object distance is calculated as Do min A predefined object distance threshold o new New Gaussian distribution g new A new object is generated by assigning o1 Object distance condition Do <Do min A single object that satisfies o 1-updated Updated single object o1 with updated distribution o m Object distance condition Do <Do min Multiple objects that satisfy o m-merged Object distance condition Do <Do min Multiple objects that satisfy m Merged objects containing o m-merged-updated Updated merged object of merged objects with updated distributions

Claims

1. A computer-implemented method for processing sensor data points Pr to measure distances to nearby objects and generate a map of a static environment, comprising: The data processing hardware performs the following steps: 1.

1. Initial multiple Gaussian distributions g ｎ Each Gaussian distribution g comprises an initial plurality of sensor data points Pr sequentially received from at least one sensor, and each Gaussian distribution g corresponds to a corresponding predetermined distribution distance threshold Dd ｍｉｎ and 1.

2. A new sensor data point Pr from the at least one sensor ｎｅｗ Sequentially acquire the new sensor data point Pr ｎｅｗ Calculating the distribution distance Dd between each Gaussian distribution g and 1.

3. For each Gaussian distribution g, the distribution distance Dd satisfies the distribution distance condition Dd<Dd ｍｉｎ Whether the distribution distance Dd<the corresponding predetermined distribution distance threshold Dd ｍｉｎ A step of verifying whether 1.3.

1. The distribution distance condition Dd<Dd ｍｉｎ If there is no Gaussian distribution g that satisfies the new sensor data point Pr ｎｅｗ A new Gaussian distribution g ｎｅｗ Generate 1.3.

2. The distribution distance condition Dd<Dd ｍｉｎ Let g be a single Gaussian distribution １ If this is true, the new sensor data point Pr ｎｅｗ By adding １ , and the updated single Gaussian distribution g １－ｕｐｄａｔｅｄ Generate 1.3.

3. The distribution distance condition Dd<Dd ｍｉｎ Let g be the sum of at least two Gaussian distributions. １ , g ２ , ..., g ｍ If the at least two Gaussian distributions g １ , g ２ , ..., g ｍ and merge the Gaussian distribution g ｍ－ｍｅｒｇｅｄ and generating the new sensor data point Pr ｎｅｗ By adding ｍ－ｍｅｒｇｅｄ , and the updated merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ generating a 1.

4. The new Gaussian distribution g ｎｅｗ or The updated single Gaussian distribution g １－ｕｐｄａｔｅｄ or The updated merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ The initial Gaussian distributions g ｎ In the distribution buffer G, the updated Gaussian distributions g ｎ－ｕｐｄａｔｅｄ and generating n new sensor data points Pr ｎ－ｎｅｗ and the Gaussians g n are grouped to create static environment objects in the map.

2. The distribution distance Dd is the new sensor data point Pr ｎｅｗ and the mean μ of each Gaussian distribution g ｇ The computer-implemented method of claim 1 , wherein the Euclidean distance between

3. The distribution distance Dd is the new sensor data point Pr ｎｅｗ is the probability of [0010] Where: [0025] is the new point Pr ｎｅｗ The probability that is generated from each Gaussian distribution g; P ｎｅｗ is a vector with k components of new sensor data points; μ ｎ is the mean of each Gaussian distribution g, a vector with k components; Σ ｎ is the covariance of each Gaussian distribution, a (k x k) matrix; Ｔ The computer-implemented method of claim 1 , wherein x is a transpose.

4. The computer-implemented method of claim 1 , wherein the distribution distance Dd is calculated using a Mahalanobis distance formula.

5. n new sensor data points Pr ｎ－ｎｅｗ The updated single Gaussian distribution g １－ｕｐｄａｔｅｄ and the updated merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ The successive updates of are done according to the following formula: [0030] Where: μ ｇｎ is any single Gaussian distribution g １ and the merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ Average 、 Σ ｇｎ is any single Gaussian distribution g １ and the merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ The covariance of μ ｇｎ－ｕｐｄａｔｅｄ is any of the updated single Gaussian distributions g １－ｕｐｄａｔｅｄ and the updated merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ The average of Σ ｇｎ－ｕｐｄａｔｅｄ teeth 、 Any of the updated single Gaussian distributions g １－ｕｐｄａｔｅｄ and the updated merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ The covariance of Ｔ The computer-implemented method of claim 1 , wherein x is a transpose.

6. The new sensor data point Pr ｎｅｗ is the covariance matrix [0045] Gaussian distribution of sensor data points [0050] The distribution distance Dd is calculated by dividing each Gaussian distribution g and the sensor data point Gaussian distribution by the Kullback-Leibler divergence or the Bhattacharyya distance. [006] The computer-implemented method of claim 1 , wherein the calculation is performed between

7. The at least two Gaussian distributions g １ , g ２ , ..., g ｍ The merge of at least two Gaussian distributions g １ , g ２ , ..., g ｍ By successively merging pairs of two distributions from ｍ－ｍｅｒｇｅｄ 2. The computer-implemented method of claim 1 , wherein the method is performed by generating a weighted mean and a weighted variance of the two distributions from each pair of two distributions.

8. In the following steps, which are performed by the data processing hardware, the initial plurality of Gaussian distributions g are extracted from the distribution buffer G. ｎ and the updated multiple Gaussian distributions g ｎ－ｕｐｄａｔｅｄ and a plurality of objects o from the object buffer O are ｎ Create a lightweight representation of a static environment with: 8.

1. The initial Gaussian distributions g ｎ From the distribution list, the initial multiple objects o ｎ－ｉｎｉｔｉａｌ Define 8.

2. The updated Gaussian distributions g ｎ－ｕｐｄａｔｅｄ The updated distribution from, i.e., The new Gaussian distribution g ｎｅｗ or The updated single Gaussian distribution g １－ｕｐｄａｔｅｄ or The updated merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ from the distribution buffer G, and extracting the initial plurality of objects o from the updated distribution. ｎ－ｉｎｉｔｉａｌ Calculate the object distance Do to each object o in 8.

3. The updated distribution satisfies the object distance condition Do<Do ｍｉｎ In other words, whether the object distance Do<a predetermined object distance threshold value Do ｍｉｎ and verifying whether the initial plurality of objects o ｎ－ｉｎｉｔｉａｌ , and update the plurality of objects o according to the following update distribution type: ｎ－ｕｐｄａｔｅｄ Generate 8.3.

1. The updated distribution is the new Gaussian distribution g ｎｅｗ If, then, the initial plurality of objects o ｎ－ｉｎｉｔｉａｌ The update is done as follows: The initial plurality of objects o ｎ－ｉｎｉｔｉａｌ For any object, the updated distribution satisfies the object distance condition Do<Do ｍｉｎ If not satisfied, the new Gaussian distribution g ｎｅｗ By assigning a new object o ｎｅｗ and stores the new object o in the object buffer O. ｎｅｗ Save; Single object o １ Regarding the object distance condition Do<Do, the updated distribution ｍｉｎ If the new Gaussian distribution g ｎｅｗ By assigning １ , and the updated single object o １－ｕｐｄａｔｅｄ and stores the updated single object o in the object buffer O. １－ｕｐｄａｔｅｄ Save; The initial plurality of objects o ｎ－ｉｎｉｔｉａｌ Multiple objects o ｍ Regarding the object distance condition Do<Do, the updated distribution ｍｉｎ If the plurality of objects o ｍ and merge the merged object o ｍ－ｍｅｒｇｅｄ Generate a new Gaussian distribution g ｎｅｗ By assigning ｍ－ｍｅｒｇｅｄ , and the updated merge object o ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ and stores the updated merged object o in the object buffer O. ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ Save; 8.3.

2. The updated distribution is the updated single Gaussian distribution g １－ｕｐｄａｔｅｄ If, then, the initial plurality of objects o ｎ－ｉｎｉｔｉａｌ The update of the single Gaussian distribution g １ and updating the corresponding object o with the updated object o ｕｐｄａｔｅｄ and stores the updated object o in the object buffer O. ｕｐｄａｔｅｄ Save; 8.3.

3. The updated distribution is the updated merged Gaussian distribution g ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ If, then, the initial plurality of objects o ｎ－ｉｎｉｔｉａｌ The update is done as follows: Single object o １ Regarding the object distance condition Do<Do, the updated distribution ｍｉｎ If the single object o １ , and updating the single object o by including the updated distribution. １－ｕｐｄａｔｅｄ and stores the updated A single object o １－ｕｐｄａｔｅｄ Save; The initial plurality of objects o ｎ－ｉｎｉｔｉａｌ Multiple objects o ｍ If the updated distribution satisfies the object distance condition, ｍ and merge the merged object o ｍ－ｍｅｒｇｅｄ , and the merged object o ｍ－ｍｅｒｇｅｄ , and update the representation of the updated merged object o ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ and stores the updated merged object o in the object buffer O. ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ Save, 8.

4. The new object o ｎｅｗ or The updated single object o １－ｕｐｄａｔｅｄ or The updated merge object o ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ from the object buffer O to a user interface for immediate use, and steps 8.

2. to 8.

4. are ｎ－ｎｅｗ The computer-implemented method of claim 1 , further comprising: iterating for

9. The new object o ｎｅｗ , The updated object o ｕｐｄａｔｅｄ and The updated merge object o ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ is the mean μ of the corresponding Gaussian distribution. ｎ The computer-implemented method of claim 8 , produced by:

10. The new object o ｎｅｗ , The updated object o ｕｐｄａｔｅｄ and The updated merge object o ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ 9. The computer-implemented method of claim 8, wherein x is constructed from two points selected on a principal axis of a corresponding Gaussian distribution, the two points corresponding to a certain probability.

11. 11. The computer-implemented method of claim 10, wherein the corresponding Gaussian distributions have 2*k points for each Gaussian distribution g, where k is the dimension of the corresponding Gaussian distribution.

12. The new object o ｎｅｗ , The updated object o ｕｐｄａｔｅｄ and The updated merge object o ｍ－ｍｅｒｇｅｄ－ｕｐｄａｔｅｄ is a specific number L of sensor data points Pr Ｌ 9. The method of claim 8, wherein the particular number L is sampled from a hyperellipse corresponding to a certain probability.

13. The computer-implemented method of claim 8 , wherein the lightweight representation of the static environment is transmitted to a vehicle's drive system in communication with the data processing hardware.

14. Data processing hardware configured to perform the steps of the method according to any one of claims 1 to 13.

15. The data processing hardware of claim 14 , further configured to be included in a sensor.

Images