quasi-rotationally invariant shape descriptors

Abstract

Quasi-rotationally invariant shape descriptors. One aspect of the disclosure relates to a computer-implemented method of determining a quasi-rotationally invariant shape descriptor of point cloud data in an automotive system to monitor an environment of a vehicle. To determine the quasi-rotationally invariant shape descriptor, a method of determining a center point of the point cloud data is performed multiple times, each time using a different value for one or more starting parameters. The method can produce different center points depending on the selected value of the one or more starting parameters. A regression line of the multiple center points is computed such that a shape descriptor is computed based on a base form of the multiple center points.

Description

Technical Field

[0001] The present invention relates to a method and apparatus for determining quasi-rotational invariant shape descriptors of point cloud data in an automotive system to monitor the vehicle environment. Background Technology

[0002] In the fields of imaging systems and computer vision, there is a need for a method that can easily compute the position of the center point of point cloud data located in n-dimensional space. The term "point cloud data" (also known as an object) is understood here as a set of points that are sufficiently close to other points from a given set, such that the set of objects can be distinguished from other similar objects in the data space.

[0003] The location of the center point of a point cloud can depend on the details of the method used. The method can produce different results based on, for example, the values ​​of one or more initial parameters. Thus, the method used to determine the center point can be repeated to determine multiple center points. The multiple center points of a point cloud can also be interpreted as the skeleton of the point cloud.

[0004] The skeleton of point cloud data can provide a basis for calculating descriptive variables that indicate the target represented by the point cloud data. The expected characteristic of these descriptive variables is that they are invariant when the point cloud data is rotated; that is, point cloud data representing the same target but with different orientations (rotations) of the target relative to the imaging sensor should result in similar values ​​for the descriptive variables.

[0005] Various methods for calculating the location of an object's center point can be found in the literature. In most reported cases, these methods are used to find the center point of a light point in a two-dimensional data space. The center point can be defined in various ways depending on the application requirements. In most cases, the algorithms used calculate the so-called centroid of the point cloud data.

[0006] Reference [1] describes a method for locating the centroid (i.e., the center of gravity) of planar objects of different shapes by scanning images depicting different objects and locating their centroids using computer vision techniques. In this method, the entire area of ​​the object is considered while the position of its center point is calculated. As a result, if, for example, two or more smaller objects are connected together by a “bridge,” the center of gravity is located somewhere between these smaller objects. In a similar manner, if an unwanted object (the “stuck” part) is stuck together with the object of interest, the center of gravity shifts from the position in the absence of the unwanted object segment.

[0007] Reference [2] describes an algorithm that can be used to place labels and tooltips on polygons by finding hard-to-reach polygon poles, which are the interior points farthest from the polygon outline. However, this method is known to be computationally complex and is therefore not recommended for real-time applications. In this case, even hundreds of temporary points covering the points being processed are chosen. For each point, the distance to a specific segment of the boundary is calculated, which requires a large number of trigonometric operations.

[0008] Reference [3] describes an algorithm for calculating features used to describe the shape of an object in a simple way. Before starting the process described in Reference [3], the location of the center point of the object must be determined. This feature is then used in an object classification process performed using an artificial neural network. In such an application, the required accuracy for calculating the location of the center point of the light spot does not need to be very high.

[0009] Patent document [4] describes a method for identifying candidate points as possible feature points of a calibration pattern within an image of a calibration pattern.

[0010] References [5] through [8] provide examples for determining the skeleton of a graphic, with the methods described in references [5] and [6] based on traditional image processing techniques, and references [7] and [8] employing artificial neural networks (ANNs). The problem is that all of these methods are computationally complex.

[0011] The method in reference [5] works by passing the image through multiple consecutive passes to remove pixels on the object boundaries. This continues until no more pixels can be removed. The image is associated with a mask that assigns each pixel a number in the range of 0 to 255, which corresponds to each possible pattern of its eight neighboring pixels. A lookup table is then used to assign the values ​​0, 1, 2, or 3 to the pixels, which are selectively removed during the iteration.

[0012] The algorithm described in reference [6] uses an octree data structure to examine the 3×3×3 neighborhood of a pixel. The algorithm works by iteratively scanning the image and removing pixels at each iteration until the image stops changing. Each iteration consists of two steps: first, assembling a candidate list for removal; then, sequentially re-examining pixels from that list to better preserve the connectivity of the image.

[0013] In existing technical literature, various image descriptors have been proposed to extract specific features of a particular 2D object: Reference [9] describes the histogram of oriented gradients, Reference

[10] describes the local binary pattern, Reference

[11] describes the scale-invariant feature, Reference

[12] describes the acceleration robust feature, Reference

[13] describes the intensity feature, Reference

[14] describes the color information, and Reference

[15] describes the spot position.

[0014] [References]

[0015] [1]: Gahramanova, Amina (2019) "Locating Centers of Mass with ImageProcessing", Undergraduate Journal of Mathematical Modeling: One+Two: Vol.10: Iss.1, Article 1,

[0016] DOI: https: / / doi.org / 10.5038 / 2326-3652.10.1.4906

[0017] [2]: Vladimir Agafonkin (2016) "A new algorithm for finding a visual center of apolygon" [Blog post], taken from:

[0018] https: / / blog.mapbox.com / a-new-algorithm-for-finding-a-visual-center-of-a-polygon-7c77e6492fbc

[0019] [3]:EP 3872693A1

[0020] [4]:US10,776,953B2

[0021] [5]: TYZhang and CYSuen (March 1984) "A fast parallel algorithm forthinning digital patterns", Communications of the ACM, Volume 27, Number 3

[0022] [6]: T.-C. Lee, R. L. Kashyap, and C.-N. Chu (1994) "Building skeleton models via 3-D medial surface / axis thinning algorithms", computer vision, Graphics, and image processing, 56(6): 462-478

[0023] [7]: Yukang Wang, Yongchao Xu, Stavros Tsogkas, Xiang Bai, Sven Dickinson, Kaleem Siddiqi (2019) "DeepFlux for Skeletons in the Wild", Computer Vision and Pattern Recognition (CVPR), 10.1109 / CVPR.2019.00543

[0024] [8]: Chang Liu, Yunjie Tian, Jianbin Jiao, Qixiang Ye (May 2021) "Adaptive Linear Span Network for Object Skeleton Detection", IEEE Transactions on Image Processing, Vol. 30, pp. 5096-5108

[0025] [9]: Dalal, N., Triggs, B. (2006) "Object Detection using Histograms of Oriented Gradients", In: Pascal VOC Workshop, ECCV

[0026]

[10] : Ojala, Timo, Matti Pietikainen, and Topi Maenpaa (2002) "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns",

[0027] IEEE Transactions on pattern analysis and machine intelligence 24.7:971-987

[0028]

[11] : Lowe, David G. (1999) "Object recognition from local scale-invariant features", Proceedings of the International Conference on Computer Vision, 2, pp. 1150-1157

[0029]

[12] : Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool (2008) "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346-359

[0030]

[13] : P.F. Alcantarilla, L.M. Bergasa, et al. (September 2011) “Automatic Light Beam Controller for driver assistance”, Machine Vision and Applications, Vol. 22, pp. 819-835

[0031]

[14] : A. Lopez, J. Hilgenstock, et al. (2008) “Nighttime vehicle detection for intelligent headlight control”, 10th International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 113-124

[0032]

[15] : YLChen (March 2009) "Nighttime vehicle light detection on a moving vehicle using image segmentation and analysis techniques", WSEASTransactions onComputers, Vol.8, pp.506-515 Summary of the Invention

[0033] In the novel method proposed in this disclosure, a quasi-rotation-invariant shape descriptor is determined for point cloud data, which includes multiple center points of the point cloud data. The quasi-rotation-invariant shape descriptor can be used to describe the shape of an unknown object represented by the point cloud data. The term "point cloud data" (also referred to as an object) is understood here as a set of points sufficiently close to other points from a set, such that the point cloud data can be distinguished from any other point cloud data in the data space. The multiple center points of the point cloud data determined in the proposed method are included in the so-called skeleton of the point cloud data.

[0034] Compared to the existing technical solutions described above, the proposed quasi-rotation-invariant shape descriptor provides a much simpler description of point cloud data. It returns only three values. One of these values ​​is the tilt angle representing the rotation of the point cloud data. Based on this angle, the object is rotated to its "base" version, and then the horizontal and vertical stretching (or compression) parameters are calculated. The latter two values ​​are sufficient to determine whether the point cloud data has a simple (regular) or more complex (irregular) shape.

[0035] In the proposed method for determining the center point of point cloud data, the center point is understood to some extent as in reference [2]. However, contrary to reference [2], it is assumed that the boundaries of the point cloud data may be irregular, which is important from the perspective of the target application in the automotive system. Therefore, the purpose of the proposed method is to find a quasi-rotation-invariant shape descriptor for point cloud data, regardless of the shape and rotation of the point cloud data. In the following text, for the sake of brevity, the proposed quasi-rotation-invariant shape descriptor can be simply referred to as the shape descriptor.

[0036] Another objective of the proposed method is to provide an algorithm with significantly reduced computational complexity compared to existing technologies. This is particularly important for real-time applications in vehicles.

[0037] One aspect of the implementation relates to a computer-implemented method for determining a shape descriptor of point cloud data in an automotive system to monitor the vehicle environment. The shape descriptor represents the shape of the point cloud data. The point cloud data is generated by one or more sensors of the vehicle relative to a reference coordinate system. The point cloud data defines a connected subspace of the reference coordinate system. The method includes the steps of: a) determining a bounding box of the point cloud data; b) selecting multiple initial agent positions for agents within the bounding box; and c) selecting multiple coordinate systems relative to the bounding box. The method further includes step d): performing multiple agent movement operations for one of the selected multiple initial agent positions and one of the selected multiple coordinate systems. Each agent movement operation includes moving the agent from a current agent position to a new agent position parallel to a coordinate axis of the selected coordinate system. The new agent position is determined based on a through-line passing through the current agent position parallel to the coordinate axis. The method further includes step e): after step d) is completed, determining the new agent position as the center point of the point cloud data. The method further includes step f): repeating steps d) and e) for each combination of the multiple initial agent positions and the multiple coordinate systems to determine multiple center points of the point cloud data. The method further includes step g): determining a regression line by calculating a linear regression of multiple center points. The method further includes step h): performing one of steps h1) and h2): step h1) involves rotating the multiple center points to a base form such that the regression line corresponds to a predetermined direction; or step h2) involves rotating the point cloud data to a base form such that the regression line corresponds to a predetermined direction, and repeating steps a) to f) for the base form to obtain multiple center points of the base form. The method further includes step i): calculating a shape descriptor based on the multiple center points of the base form, wherein the shape descriptor includes one or more tilt angles and information indicating the stretching and compression of the base form of the point cloud data relative to one or more axes of a reference coordinate system.

[0038] Therefore, to determine the shape descriptor, the method for determining the center points of the point cloud data is executed multiple times, with different values ​​applied to one or more initial parameters each time. This method can generate different center points based on the values ​​chosen for one or more initial parameters. The resulting multiple center points are then used to calculate the shape descriptor.

[0039] Compared to the solutions of the prior art, the proposed method has significantly lower computational complexity. Furthermore, the method can be used iteratively, thus adapting its computational complexity to specific applications. A problem could be, for example, searching point cloud data with regular shapes (e.g., light sources) while rejecting point cloud data exhibiting irregularities (e.g., concave shapes). In this case, after repeating the method for determining the center points of the point cloud data only a few times, information about the distribution of the obtained center points can be obtained, and the iterative process can be discontinued. The measure of aggregation here is an indicator of the regularity of the objects being processed.

[0040] On the other hand, multiple starting agent locations are selected either at random locations within the bounding box or at predetermined locations within the bounding box.

[0041] On the other hand, information representing the basic form of stretching and compression is given by at least one of standard deviation, variance, skewness, and mean.

[0042] On the other hand, the reference coordinate system and the coordinate system relative to the bounding box are orthogonal.

[0043] On the other hand, the reference coordinate system and the coordinate system relative to the bounding box are 2-dimensional, and step c) includes selecting a rotation angle that defines the rotation of the reference coordinate system relative to the coordinate system.

[0044] On the other hand, the new agent position is determined based on the edge point given by the intersection of the line that passes through the current agent position parallel to the coordinate axis and the boundary of the subspace defined by the point cloud data.

[0045] On the other hand, the new agent position is determined based on the first and second edge points among the edge points given by the intersection of the through line that passes through the current agent position parallel to the coordinate axis and the boundary of the subspace defined by the point cloud data. The first and second edge points are determined as the edge points of the connecting line segment closest to the current agent position among a plurality of connecting line segments, or the first and second edge points are determined as the edge points of the largest connecting line segment among a plurality of connecting line segments. Each of the plurality of connecting line segments is defined as the range in which the through line intersects with the connecting subspace.

[0046] On the other hand, the new agent position is determined as the center point between the first edge point and the second edge point.

[0047] On the other hand, the new agent position is determined by dividing the difference between the first number and the second number by a positive real constant and adding the result to the current agent position relative to the coordinate axis, where the first number is the number of data points of the point cloud data in a first direction along the through line and the second number is the number of data points of the point cloud data in a second direction along the through line.

[0048] On the other hand, the chosen coordinate system has coordinate axes x1,…,x N The N-dimensional coordinate system, and step d) includes the coordinate axis x i The agent is moved from its current position to a new position in either the positive or negative direction of (i = 1, ..., N).

[0049] In another scenario, the number of agent movement operations is predetermined, or step d is completed if the distance between the current agent location and the new agent location is less than a threshold distance.

[0050] On the other hand, the method further includes the step of classifying the shape of the point cloud data as regular or irregular based on the shape descriptor.

[0051] On the other hand, one or more sensors in the vehicle include cameras, photoelectric sensors, infrared sensors, lidar sensors, radar sensors, and / or ultrasonic sensors.

[0052] On the other hand, the method further includes a step of performing predetermined operations on the vehicle based on the determined shape descriptor of the point cloud data after step i), wherein the predetermined operations include adaptive headlight control, traffic sign recognition, automatic emergency braking and / or object tracking.

[0053] In another aspect, this disclosure relates to a data processing apparatus that includes means for performing the steps of the methods described in any of the foregoing aspects.

[0054] On the other hand, this disclosure relates to vehicles including data processing equipment. Attached Figure Description

[0055] Figure 1a This is an example of an image showing photoelectric sensor data representing two bright spots in a car's illuminated headlights.

[0056] Figure 1b Two separate images are shown, each representing a different image. Figure 1a Point cloud data associated with one of the headlights in the image.

[0057] Figure 1c The bounding box is shown Figure 1b An example of point cloud data.

[0058] Figure 2 This is a flowchart of a method for determining the center point of point cloud data in an illustrative embodiment.

[0059] Figure 3 Examples of multiple agent movement operations for two-dimensional point cloud data are shown.

[0060] Figures 4a to 4d Point cloud data of different shapes are shown.

[0061] Figures 5a to 5c Point cloud data of different shapes are shown.

[0062] Figures 6a to 6c Point cloud data of different shapes are shown.

[0063] Figures 7a to 7c Point cloud data of different shapes are shown.

[0064] Figure 8 Point cloud data with a basic rectangular shape is shown.

[0065] Figures 9a to 9d Point cloud data with rectangular shapes and tilt angles of 11°, 22°, 33° and 66° are shown.

[0066] Figures 10a to 10d Point cloud data with rectangular shapes and tilt angles of 11°, 22°, 33° and 66° are shown.

[0067] Figure 11a and Figure 11b Point cloud data of different shapes are shown.

[0068] Figure 12a and Figure 12b Point cloud data of different shapes are shown.

[0069] Figure 13 This is a schematic diagram of the hardware structure of a data processing device. Detailed Implementation

[0070] The invention will now be described in conjunction with specific embodiments. These embodiments are provided to enable those skilled in the art to better understand the invention, but are not intended to limit the scope of the invention in any way, which is defined by the appended claims. In particular, the embodiments described independently throughout this specification may be combined to form other embodiments, provided that they are not mutually exclusive.

[0071] The point cloud data of a target can be based on data generated using one or more sensors, such as digital imaging sensors with rows and columns of pixels. For example, imaging sensors can include cameras, photoelectric sensors, infrared sensors, lidar sensors, radar sensors, ultrasonic sensors, etc. The one or more sensors that generate the point cloud data can be part of a vehicle and can, for example, provide information related to the vehicle's surrounding environment. Similarly, one or more sensors can provide information related to the vehicle's interior. A target refers to a real-world object whose center point needs to be determined in order to track / monitor it. A target could be, for example, a car's headlights.

[0072] Point cloud data can represent information about a target and can be used, for example, in subsequent data analysis methods such as target identification or target tracking methods. Target identification methods can be used to determine the category or type of a target represented by point cloud data. Target tracking methods can be used to track the temporal evolution of a target represented by point cloud data, which includes two or more point clouds representing the target at different times, thereby providing information related to the target's trajectory, direction of motion, velocity, acceleration, etc.

[0073] Point cloud data is given relative to a reference coordinate system. The reference coordinate system can be a spatial coordinate system, where each axis represents a spatial direction. The reference coordinate system can be, for example, 2D, 3D, or higher-dimensional. The reference coordinate system can be orthogonal.

[0074] An example of 2D point cloud data of a target (i.e., point cloud data represented by a 2D reference coordinate system) is point cloud data generated by a photoelectric sensor. For example, after preprocessing such as thresholding or binarization, a photoelectric sensor can provide a black-and-white image of a scene at a given time, where data points such as white pixels correspond to light sources or reflections from light sources, and no data points such as black pixels correspond to the absence of light. For example, the reference coordinate system is defined such that its axes are aligned along the edges of the image, i.e., the x-axis is aligned along the horizontal edge of the image, and the y-axis is aligned along the vertical edge.

[0075] An example of 3D point cloud data (i.e., point cloud data represented by a 3D reference coordinate system) is point cloud data generated by a lidar system such as a scanning lidar system or a flash lidar system. A target may be illuminated, for example, by one or more laser pulses generated by the lidar system, and the laser reflected by the target is received by the lidar system's laser sensor, where the laser sensor provides spatial and temporal information. The temporal information can be used to determine the distance between the lidar system and the target, thus providing the 3D point cloud data.

[0076] By adding a time axis to 3D data, the reference coordinate system can also be 4D. Furthermore, by merging data from multiple sensors, even higher-dimensional reference coordinate systems can be used.

[0077] Point cloud data can preferably be digitized, meaning that the data points in the reference coordinate system take on discrete values. For example, the reference coordinate system can be adapted to the structure of one or more digital imaging sensors, each with rows and columns of pixels. In this case, each cell of the discrete reference coordinate system can correspond to a pixel or readout time interval of one or more sensors.

[0078] The point cloud data of the target can be based on data generated by one or more sensors, which has been preprocessed according to the requirements of subsequent data analysis methods. For example, the point cloud data can be based on sensor data that has been filtered, scaled, binarized, and / or transformed relative to a predetermined threshold.

[0079] One or more sensors can provide information (sensor data) related to multiple targets and / or multiple features of a single target. In this case, the information can generate multiple point cloud data, each of which represents one of the multiple targets and / or one of the multiple features of a single target.

[0080] Point cloud data defines a connected subspace of a reference coordinate system. The connected subspace consists of multiple data points from the point cloud data, where each data point is adjacent to another data point within the same set of data points. In other words, a data point is considered adjacent if it is sufficiently close (i.e., within a certain range) to another data point within the same set of data points, thus distinguishing the multiple data points from other data points. For example, in a discrete reference coordinate system, for each data point, there is another data point in an adjacent cell.

[0081] Figure 1a This is an example of an image based on photoelectric sensor data, showing two bright spots representing the illuminated headlights of a car in a dark environment. In this case, reference coordinate system 1 is two-dimensional.

[0082] Figure 1b Two separate images are shown, each representing point cloud data 2 associated with (corresponding to) one of the headlights. In this example, point cloud data 2 corresponds to... Figure 1a The two highlighted binarized versions are shown. Binarization of photoelectric sensor data can be performed using a predetermined threshold, such that data points of photoelectric sensor data with values ​​greater than the predetermined threshold are assigned to point cloud data 2, while values ​​less than or equal to the predetermined threshold are not assigned to point cloud data 2.

[0083] When the point cloud data (target) has a regular shape, the centroid provides a good estimate of the location of the center point of the point cloud data. Figure 1a Several examples of this type of target are shown. However, the computational complexity is high in this case and also depends on the internal dimensions (number of pixels / dots) of the object being processed. This is especially true when the object has an irregular shape (see...). Figure 4c and Figure 4d(As shown in the example), the center of gravity can be located far from the desired center point. This can happen in, for example, AHC (Adaptive Headlight Control) algorithms designed to distinguish vehicle headlights from other light sources. In many observed cases, vehicle headlights appear "stuck" to other light sources (e.g., lights from other vehicles, streetlights, etc.) in images captured by a camera. In such cases, the unpredictable shape of the unwanted object stuck to the target of interest can lead to an incorrect description of that object's shape, potentially resulting in misclassification of the given light.

[0084] Figure 1a The shape of the light spots in the image serves as an indicator of the real-world target captured in the image. For example, car headlights typically have regular shapes, such as ellipses or rectangles, or slightly concave features due to noise or other minor distortions. Additionally, headlights are usually horizontally aligned. Different images of road signs will be obtained, for example. Many European traffic signs have a circular red border around a bright background. After simple preprocessing, such traffic signs will provide point cloud data with shapes resembling circles or rings. Pedestrians or cyclists will create point cloud data with irregular shapes. This point cloud data can also be classified using artificial neural networks (such as deep learning convolutional neural networks). The problem in this case is the large number of pixels in such point cloud data.

[0085] In the described scenario, the ability to determine a simplified dataset, which can be interpreted as a skeleton of point cloud data, may be useful for several reasons. On the one hand, the computed skeleton allows for a significant reduction in the amount of data to be processed during subsequent classification of targets, for example, using artificial neural networks. On the other hand, after some additional processing, the obtained skeleton can be used as supplementary data to aid in classification or to validate classification results.

[0086] To determine the skeleton, a method for determining the center points of point cloud data is executed multiple times, with different values ​​used for one or more initial parameters in each execution. Therefore, the proposed method for determining the center points of point cloud data aims to provide a method for determining... Figure 1b A quick and effective method for finding the center point of each of the two bright spots.

[0087] Figure 2 This is a flowchart of a method 100 for determining multiple center points of point cloud data in an illustrative embodiment. Method 100 will be described relative to point cloud data given by binarized photoelectric sensor data, although method 100 can be applied to other types of sensor data. Method 100 may include other steps not shown, and these steps may be performed in different orders (in particular, the order of steps S20 and S30 is arbitrary).

[0088] In the first step S10, the bounding box of point cloud data 2 is determined, wherein each side of the bounding box is a tangent to point cloud data 2. The bounding box can preferably have a rectangular shape, as this simplifies the calculation. Figure 1c The bounding box is shown Figure 1b Examples of each corresponding point cloud data 2. For example, a standard image cropping algorithm can be used to determine the bounding box, where the image cropping algorithm removes... Figure 1b The image contains no unnecessary outer regions without data points, while maintaining a rectangular shape.

[0089] For convenience, the space within the bounding box can be parameterized using normalized coordinate values. That is, the reference coordinate system is adjusted so that reference coordinate system 1 is spanned by the edges of the bounding box that intersect at the origin of the reference coordinate system. In this case, points within the bounding box along each axis can have normalized coordinate values ​​between 0 and 1 relative to a given axis.

[0090] In the second step S20, multiple initial agent positions are selected within the bounding box. The agent acts as a reference point for subsequent agent movement operations, and its position changes in each agent movement operation. Therefore, the agent corresponds to a point in a reference coordinate system within the bounding box (or a coordinate system selected in a subsequent step S30). Each of the multiple initial agent positions can be selected from different random locations or different predetermined locations within the bounding box. For example, the multiple predetermined locations of the multiple initial agent positions can be given by a set of equidistant points within the bounding box (e.g., points on a lattice, or a set of points with varying distances from each other).

[0091] In the third step S30, multiple coordinate systems are selected relative to the bounding box. Each of the selected coordinate systems relative to the bounding box determines the permissible movement direction of the agent in the agent movement operation. One or more (preferably all) of the selected coordinate systems can be orthogonal. The selected coordinate systems have the same dimensions as the reference coordinate system. The selected coordinate systems can also be rotated relative to the reference coordinate system; that is, the third step may include selecting one or more rotation angles that define the rotation of the reference coordinate system relative to each of the selected coordinate systems. Furthermore, the origin of the reference coordinate system and the origin of one or more of the selected coordinate systems can be the same.

[0092] The number of selected starting agent locations and coordinate systems can be based on the complexity of the shape of the point cloud data. The complexity of the point cloud data shape can, for example, be based on the complexity of the shape of previous point cloud data analyzed by method 100. The complexity can also be based on the type of sensor on which the information generating the point cloud data is based. The complexity can correspond to a quality setting used to control the granularity of the determined skeleton. The quality setting can be set by the driver or vehicle manufacturer, etc. The quality setting can depend on external conditions, such as weather, time of day, or the current amount of data generated by the sensor. Typically, the number of selected starting agent locations and coordinate systems is on the order of 10.

[0093] In the fourth step S40, multiple agent movement operations are performed for one of the selected multiple starting agent positions and one of the selected multiple coordinate systems. For each movement operation, the agent moves from its current position to a new position parallel to a coordinate axis of the selected coordinate system. In other words, the current agent position is given by the new agent position determined in a previous agent movement operation, or, in the case of the first agent movement operation, by the starting agent position. After each of the multiple agent movement operations, the coordinate axis parallel to the agent movement can be changed. Preferably, in step S40, the agent moves along all coordinate axes of the selected coordinate system.

[0094] The new agent position can be determined based on the edge points given by the intersection of a line parallel to the coordinate axes that passes through the current agent position and the boundary of the subspace defined by point cloud data2. The boundary of the point cloud data is not known in advance, but is determined only along the line of intersection, eliminating the need to pre-analyze the point cloud data to determine and store its boundaries. This further reduces computational complexity.

[0095] The new agent position can be determined based on the first and second edge points, and can be defined as the center point between the first and second edge points. In other words, the new agent position is given by the midpoint of the line connecting the first and second edge points, which is determined by dividing the distance between the first and second edge points by two. In the computer implementation of the agent movement operation, dividing by 2 can be performed by a bitwise shift operation that shifts the binary number one bit to the right, where the least significant bit is removed. Such an operation is therefore inexpensive for a computer, which contributes to the overall efficiency of method 100.

[0096] The first and second edge points can be determined as the edge points of the largest connecting segment among a plurality of connecting segments, where each connecting segment can be defined as the range where the penetrating line intersects the connecting subspace. In this method, the sum of data points for each of the plurality of connecting segments can be calculated.

[0097] Alternatively, the first edge point and the second edge point can be determined as the edge point of the connecting line segment closest to the current agent position among a plurality of connecting line segments, wherein each of the plurality of connecting line segments is defined as the range in which the through line intersects the connecting subspace.

[0098] Figure 3 Examples of multiple agent movement operations (M1, M2, M3) for defining a connected subspace of two-dimensional point cloud data 2 with an irregular boundary 3 are shown. The starting agent position is denoted by P0(x0, y0), where x0 and y0 are the coordinates of the starting agent position relative to the selected coordinate system relative to the bounding box.

[0099] For the first agent movement operation M1, a direction parallel to the x-axis of the selected coordinate system is chosen. The current agent position is P0(x0,y0). A first penetration line 4a, parallel to the x-axis and passing through P0(x0,y0), is drawn at the first edge point P. p1 (x p1 ,y p1 ) and the second edge point P k1 (x k1 ,y k1 The new agent position P1(x1,y1) intersects with boundary 3 at point ). The coordinates x1 and y1 of the new agent position P1(x1,y1) are then determined as the center points of the first and second edge points, such that...

[0100] x1=x p1 +(x k1 -x p1 ) / 2=x p1 +Δx1 / 2,

[0101] y1 = y0.

[0102] In this way, the agent moves from the current agent position P0(x0,y0) to the new agent position P1(x1,y1) in the positive direction of the x-axis.

[0103] For the second agent movement operation M2, a direction parallel to the y-axis relative to the bounding box coordinate system is chosen. The current agent position is P1(x1,y1), which is the new agent position determined by the previous agent movement operation M1. The second penetration line 4b, parallel to the y-axis and passing through P1(x1,y1), passes through the first edge point P. p2 (x p2 ,y p2 ) and the second edge point P k2 (x k2 ,y k2 The new agent position P2(x2,y2) intersects with boundary 3 at point ). The coordinates x2 and y2 of the new agent position P2(x2,y2) are then determined as the center points of the first and second edge points, such that...

[0104] x2 = x1,

[0105] y2=y p2 +(y k2 -y p2 ) / 2=y p2 +Δy2 / 2.

[0106] In this way, the agent moves from the current agent position P1(x1,y1) to the new agent position P2(x2,y2) in the positive direction of the y-axis.

[0107] For the third agent move operation M3, a direction parallel to the x-axis relative to the bounding box coordinate system is chosen. The current agent position is P2(x2,y2), which is the new agent position determined by the previous agent move operation M2. The third penetration line 4c, parallel to the x-axis and passing through P2(x2,y2), is respectively at the first edge point P. p3 (x p3 ,y p3 ) and the second edge point P k3 (x k3 ,y k3 The new agent position P3(x3,y3) intersects with boundary 3 at point 3. The coordinates x3 and y3 of the new agent position P3(x3,y3) are then determined as the center point between the first edge point and the second edge point, such that...

[0108] x3=x p3 +(x k3 -x p3 ) / 2=x p3 +Δx3 / 2,

[0109] y3 = y2.

[0110] In this way, the agent moves from the current agent position P2(x2,y2) to the new agent position P3(x3,y3) in the negative x-axis direction.

[0111] exist Figure 3 In the example shown, the agent is allowed to move only within the point cloud data; that is, once the agent enters the point cloud data, it cannot leave it. In this method, the algorithm is robust to unexpected objects.

[0112] As an alternative to determining the new agent location based on edge points, the new agent location can be based on a first number N1 of data points in the point cloud data along a first direction of the penetration line and a second number N2 of data points in the point cloud data along a second direction of the penetration line. The first and second directions can be positive and negative, respectively. In other words, the current agent location divides the penetration line into two parts. N1 counts the number of data points in the point cloud data located in one part of the penetration line, and N2 counts the number of data points in the point cloud data located in the other part of the penetration line.

[0113] Then, a new agent position can be determined as follows, that is, dividing the difference between N2 and N1 by a positive real-valued constant K (i.e., the distance S given by the following formula dist ).

[0114] S dist =(N1 - N2) / K,

[0115] and adding it to the current agent position with respect to the selected coordinate axis, such that if N1 > N2, the agent moves in the first direction, if N1 < N2, the agent moves in the second direction, and if N1 = N2, the agent does not move. The constant K can be set to a value greater than 1. The constant K can be set to a value less than 3. The constant K can preferably be set to 2.

[0116] In the above example, the new agent position P for the movement operation in the direction parallel to the x-axis of the coordinate system with respect to the bounding box can be given by the following formula i (x i , y i ):

[0117] x i =x i-1 +(N1 - N2) / 2 = x i-1 +S dist ,

[0118] y i =y i-1 ,

[0119] where P i-1 (x i-1 , y i-1 ) is the current agent position, N1 and N2 are the number of pixels in the positive x-direction and the negative x-direction respectively, and the constant K is set to 2.

[0120] In the above example, the new agent position P for the movement operation in the direction parallel to the y-axis of the coordinate system with respect to the bounding box can be given by the following formula i (x i , y i ):

[0121] x i =x i-1 ,

[0122] y i =y i-1 +(N1 - N2) / 2,

[0123] where P i-1 (x i-1 , y i-1) is the current agent position, N1 and N2 are the number of pixels in the positive y direction and negative y direction, respectively, and the constant K is set to 2.

[0124] According to an alternative approach, the agent can be allowed to temporarily leave the point cloud data. This method can be used as a computationally less complex alternative to calculating the centroid.

[0125] Step S40 can be terminated in several ways. The number of agent movement operations can be predetermined, for example, limited to those mentioned above. Figure 3 The example described includes three agent movement operations. Alternatively, if the distance between the current agent position and the new agent position is less than a threshold distance D, that is, if the distance between the current agent position and the new agent position is less than the threshold distance D, then it can be determined that the fourth step S40 has been completed.

[0126] |P i -P i-1 | <D。

[0127] The advantage of the proposed agent movement operation is that calculating consecutive agent positions only requires simple arithmetic operations, such as addition, subtraction, increment, and shift (when divided by 2 or a power of 2).

[0128] It should be noted that any combination of the above methods can be used for the proxy movement operations in step S40, which involves performing multiple proxy movement operations. In other words, referring to the above methods, a first subset of the multiple proxy movement operations can be based on the first and second edge points of the connecting line segment closest to the current proxy position, determined from among the multiple connecting line segments; a second subset of the multiple proxy movement operations can be based on the first and second edge points of the largest connecting line segment among the multiple connecting line segments; and a third subset of the multiple proxy movement operations is based on the difference between the first and second quantities divided by a positive real-value constant. The first proxy movement operation performed in step S40 can be a proxy movement operation based on the first and second quantities divided by a positive real-value constant.

[0129] In the fifth step S50, the new agent position determined by the final agent movement operation in the previous fourth step S40 is identified as the center point of point cloud data 2. The identified center point is included in the skeleton.

[0130] The center point of the point cloud data determined by this method is different from the centroid or centroid of the point cloud data. Figures 4a to 4d Point cloud data of different shapes are shown, and two methods are used to calculate the center point of the point cloud data. The first method follows the traditional approach and is based on determining the centroid of the point cloud. The centroid is represented by a solid cross. The second method is proposed method 100, which generates symbols using hollow symbols. The center point of the marker. Depending on the shape of the point cloud data, the calculated position of the center point can differ significantly between the two methods.

[0131] exist Figure 4a and Figure 4b The image shows point cloud data with regular shapes, whose shapes correspond to... Figures 1a to 1c The headlights are shown in the image. Here, the difference between the center points determined by the two methods is relatively small.

[0132] Figure 4c The diagram shows a point cloud with an irregular shape, specifically, an undesirable elongated segment on the left side connecting to the principal point on the right. As a result, the difference between the centroid location and the point determined using the proposed method 100 is relatively large. Figure 4c As shown, the proposed method 100 is able to find the location of the center point of the main part of the light spot, without considering the visible unwanted segments.

[0133] exist Figure 4d In the example, the point cloud data is represented by the shape of three points connected by a bridge. When calculating the centroid, it is found that the centroid is outside the point cloud data. In the case of the proposed method 100, the location of the center point can depend on the details of a specific implementation of the proposed method. For example, the location of the center point can depend on the randomly selected starting agent location, the orientation of the coordinate system relative to the bounding box, the number of agent movement operations, or the way the first and second edge points are selected. Figure 4d The diagram shows two results of center point calculations based on an implementation of the proposed method. The two center points are located at the centers of the two largest segments of the point cloud data.

[0134] In the sixth step S60, the previous two steps S40 and S50 are repeated for each combination of multiple initial proxy positions and multiple coordinate systems. This determines multiple center points. The multiple center points are included in the skeleton of the point cloud data.

[0135] For example, in step S20, a plurality of starting proxy positions, number A, are selected, where A is a positive integer. In step S30, a plurality of coordinate systems, number B, are selected, where B is another positive integer. After performing multiple proxy movement operations for one of the selected multiple starting proxy positions and one of the selected multiple coordinate systems in step S40, steps S40 and S50 are repeated for the remaining combinations of the multiple starting proxy positions and multiple coordinate systems in step S60. Thus, after step S60 is completed, a total of A × B multiple proxy movement operations have been performed, resulting in a skeleton containing a plurality of A × B center points.

[0136] If the center points included in the skeleton are located close to or not close to each other, the resulting skeleton of the point cloud data can be used to classify the shape of the point cloud data as regular or irregular. Here, "close" can be defined using, for example, a predetermined value of radius. In this case, if there exists a sphere with a predetermined radius such that all (or a predetermined percentage) of the center points are located within that sphere, then the center points included in the skeleton are considered close to each other, wherein the sphere has the same dimension as the reference coordinate system. Alternatively, "close" can be defined using a measure of the statistical deviation of the multiple center points included in the skeleton, such as the average distance between the center points, the standard deviation relative to the mean of the center points, the interquartile range relative to the median of the center points, etc.

[0137] If point cloud data has a regular convex shape, such as an ellipse or rectangle, the center point will be located close to the centroid of the point cloud data or its principal axis, with relatively small deviation. In the case of irregular shapes, such as point cloud data composed of a series of regions connected by "bridges," the center point will focus near the local centroid of a specific region of the point cloud data. This conclusion is important because it allows us to distinguish between regular and irregular objects.

[0138] Alternatively, the aforementioned metric of statistical deviation can be calculated relative to a specific reference line for two-dimensional point cloud data or relative to a specific reference plane for three-dimensional point cloud data. The reference line or reference plane can be an axis or plane that is (approximately) symmetrical to the point cloud data. Figure 5b The image shows an example of a reference line for point cloud data with a rectangular shape. Here, multiple center points are distributed along the axis of symmetry of the rectangle.

[0139] Figures 5a to 7c Point cloud data of different shapes are shown, where each determined center point is marked with a solid dot. Marked, with the center of gravity marked with a solid cross. mark.

[0140] Figures 5a to 5c The process of constructing a skeleton for a point cloud with a regular shape (i.e., a shape close to a convex polygon) is shown. Figure 5a and Figure 5b The results are shown for point cloud data with rectangular shapes rotated at different angles. In this case, the set of initial proxy locations comprises 5 points, with 4 points near the corners of the bounding box and 1 point in the center. Method 100 ensures that the resulting center point lies on the long axis of the rectangular point cloud data. A large set of these points around this axis is typical for objects with straight edges.

[0141] Figures 6a to 6cPoint cloud data with irregular shapes is shown. Specific examples have been selected to illustrate some characteristic trends in the obtained results. Figures 6a to 6c This illustrates the case of connecting approximately regular line segments together. (Using...) Figures 5a to 5c The arrangement of multiple starting proxy positions within similar bounding boxes used in the objects shows that the center points are concentrated near the center of specific line segments of the objects, and also on the "bridges" connecting these line segments. This principle is visible regardless of the object's rotation angle (comparative). Figure 6b and Figure 6c ).

[0142] Compared to point cloud data with regular shapes, the point cloud data with irregular shapes discussed in the above examples is characterized in that, for point cloud data with irregular shapes, the center points are concentrated near several separate line segments. This allows for easy differentiation between point cloud data with regular or irregular shapes.

[0143] Figures 7a to 7c It shows a polygon with a convex shape ( Figure 7a ) and concave polygons ( Figure 7b and Figure 7c The comparison of point cloud data is as follows: In the first case, the center points cluster near the object's center of gravity, producing an image with small closed loops around that center. In the second case, a more spatially dispersed skeleton is formed, and no closed loops are formed. Figure 7b and Figure 7c The illustration shows how the algorithm can capture the general shape of an object and return a similar arrangement of center points. A similar situation occurs... Figures 5a to 5c In the case shown.

[0144] When observing the results of the proposed method, a clear trend can be observed. When the point cloud data has a convex polygonal shape resembling a circle or ellipse, or a slightly concave polygonal shape, the algorithm outputs multiple center points (skeleton) with a high density factor. When the point cloud data has convex polygonal and elongated shapes, the resulting multiple center points are dispersed around a straight line. In the case of irregular shapes composed of smaller line segments connected to each other, the algorithm provides multiple center points concentrated around a line that serves as the axis of symmetry for a specific line segment of the processed point cloud data.

[0145] In the seventh step S70, a regression line is calculated by performing a linear regression of the multiple center points determined after step (S60) is completed. The orientation of the regression line relative to the reference coordinate system represents the orientation (rotation) of the target represented by the point cloud data.

[0146] In step S80, rotation is performed based on the determined regression line. The object of rotation can be multiple center points or the point cloud data itself. The object of rotation is rotated in a reference coordinate system such that the regression line corresponds to a predetermined direction. The predetermined direction can, for example, be parallel to (or even the same as) one of the coordinate axes of the reference coordinate system. The resulting rotated object (i.e., the multiple rotated center points or the rotated point cloud data) is referred to as the basic form in both cases. Therefore, only one of the sub-steps S81 and S82 described below is performed in step S80.

[0147] In substep S81, multiple center points are rotated to the basic form so that the regression line corresponds to a predetermined direction.

[0148] In sub-step S82, the point cloud data is rotated to the base form so that the regression line corresponds to a predetermined direction, and steps S10 to S60 are repeated on the base form to obtain multiple center points of the base form.

[0149] Compared to substep S82, substep S81 is computationally more efficient because the number of center points is typically much smaller than the number of data points belonging to the point cloud data. Regardless of which substep is executed, the result of step S80 is multiple center points of the basic form. These multiple center points of the basic form can also be referred to as the skeleton of the basic form.

[0150] In the ninth step S90, a shape descriptor is calculated based on multiple center points of the basic form, wherein the shape descriptor includes one or more tilt angles and information indicating the stretching and compression of the basic form of the point cloud data relative to one or more axes of the reference coordinate system.

[0151] One or more tilt angles correspond to one or more angles of the regression line relative to the coordinate axes of the reference coordinate system.

[0152] Information indicating the tension and compression of the base form can be given by at least one of the standard deviation, variance, skewness, mean, or different normalized statistical measures of multiple center points of the base form.

[0153] The reason for rotating the original shape to its basic form is to ensure that the resulting shape descriptors are always identical or at least very similar, regardless of the orientation (rotation) of the point cloud, i.e., regardless of the orientation (rotation) of the target represented by the point cloud. Point cloud data of the same shape should produce comparable and reproducible results, regardless of the tilt angle. This also guarantees that point cloud data with similar geometric features can be easily matched to each other, which is necessary, for example, in image classification tasks. Furthermore, the tilt angle can provide important information about the target and help distinguish it from other targets.

[0154] The shape classification of point cloud data can be based on shape descriptors. For example, based on information representing the stretching and compression of the basic shape, the shape can be classified as regular or irregular.

[0155] Figure 8 An example of point cloud data with a basic rectangular shape is shown. Note that the point cloud data has not yet been placed within a bounding box.

[0156] Figures 9a to 9d It shows the relationship with Figure 8 The same rectangular point cloud data is shown, where the point cloud data is rotated by angles of 11°, 22°, 33° and 66° relative to the base form, respectively. Note that the point cloud data has not yet been placed within the bounding box.

[0157] Figures 10a to 10d The corresponding elements placed in the bounding box are shown respectively. Figures 9a to 9d The point cloud data. Furthermore, multiple center points, marked with solid dots (·), have been identified according to the proposed method. Figures 10a to 10d In each of these, multiple center points (skeleton) converge along the principal axis of the rectangle, which (approximately) correspond to the regression line. The regression line is determined to have a 10.99° ( Figure 10a ), 21.98° Figure 10b ), 32.97° Figure 10c ) and 65.93° Figure 10d The inclination angles of these (approximately) correspond to 11°. Figure 10a ), 22° Figure 10b ), 33° Figure 10c ) and 66° Figure 10d The actual rotation angle is calculated. The calculated value shows that the tilt angle successfully reproduces the rotation angle. Accuracy can depend on the number of initial agent positions and the number of coordinate systems relative to the bounding box.

[0158] Figure 11a and Figure 11b Examples of point cloud data with different shapes are shown. Figure 11a The point cloud data shown has an elliptical shape, while Figure 11b The point cloud data shown has the shape of an airplane. Note that the point cloud data has not yet been placed within a bounding box.

[0159] Figure 12a and Figure 12b The corresponding elements placed in the bounding box are shown respectively. Figure 11a and Figure 11b The point cloud data. Furthermore, multiple center points, marked with solid dots (·), have been identified according to the proposed method. Figure 12a In the data, multiple center points are concentrated near the centroid (not shown) of the point cloud data indicating a regular shape, while... Figure 12b In the diagram, multiple center points are aligned with the irregularly shaped fuselage and wings of the aircraft. The straight lines correspond to the calculated regression lines.

[0160] It should be understood that Method 100 is not limited to two-dimensional point cloud data. For example, the point cloud data can be three-dimensional, such as point cloud data generated by a LiDAR sensor. In this case, the chosen coordinate system is a three-dimensional coordinate system with axes x, y, and z, and agent movement operations parallel to any of these axes will be performed similarly to those described above. For example, for the i-th agent movement operation parallel to the z-axis, the new agent position P i (x i ,y i ,z i The coordinates x) i y i and z i Then they were respectively identified as the first edge point P. pi (x pi ,y pi ,z pi ) and the second edge point P ki (x ki ,y ki ,z ki The center point of ) makes

[0161] x i =x i-1 ,

[0162] y i =y i-1 ,

[0163] z i =z pi +(z ki -z pi ) / 2.

[0164] In this way, the agent moves from the current agent position P in either the positive or negative z-axis direction. i-1 (x i-1 ,y i-1 ,z i-1 Move to the new agent location P i (x i ,y i ,z i ).

[0165] Similarly, point cloud data can be N-dimensional. In this case, the chosen coordinate system has coordinate axes x1,…,x N The N-dimensional coordinate system, and the fourth step S40 includes moving the agents i = 1, ..., N from the current agent position to the new agent position in the positive or negative direction of the coordinate axis x1.

[0166] As an alternative to determining the new agent location based on the center point of the first and second edge points, different methods can be used. For example, the distance between the first and second edge points can be a positive value other than 2. If the point cloud data includes non-binary data, such as grayscale or color image data, a weighted average can be used to determine the center point, where the weights are given by the grayscale or color values ​​of the data points on the line connecting the first and second edge points.

[0167] In addition, method 100 may include the following additional steps: repeating steps S10 to S60 for point cloud data representing targets at different points in time, thereby determining multiple skeletons that provide information indicating the temporal evolution of the targets.

[0168] Furthermore, method 100 may include, after step S60, additional steps to perform predetermined operations of the vehicle based on the determined shape descriptor of the point cloud data, wherein the predetermined operations include adaptive headlight control, traffic sign recognition, automatic emergency braking, and / or object tracking.

[0169] Figure 13 This is a schematic diagram of the hardware structure of a data processing device, which includes means for performing the steps of the method described in any of the above embodiments.

[0170] The data processing device 200 has an interface module 210 that provides means for sending and receiving information. The data processing device 200 also has a processor 220 (e.g., a CPU) for controlling the data processing device 200 and, for example, for performing the steps of the methods of any of the embodiments disclosed above. It also has a working memory 230 (e.g., random access memory) and an instruction memory 240, the instruction memory 240 storing computer programs having computer-readable instructions that, when executed by the processor 220, cause the processor 220 to perform the methods of any of the embodiments disclosed above.

[0171] Instruction memory 240 may include a ROM (e.g., in the form of electrically erasable programmable read-only memory (EEPROM) or flash memory) preloaded with computer-readable instructions. Alternatively, instruction memory 240 may include RAM or a similar type of memory, and computer-readable instructions may be input therefrom from a computer program product, such as a computer-readable storage medium like a CD-ROM.

[0172] In the foregoing description, several aspects have been described with reference to several embodiments. Therefore, the specification should be considered illustrative rather than restrictive. Similarly, the figures shown in the accompanying drawings, which highlight the functionality and advantages of the embodiments, are presented for illustrative purposes only. The architecture of the embodiments is flexible and configurable enough that it can be utilized in ways other than those shown in the accompanying drawings.

[0173] In one exemplary embodiment, the software implementations presented herein may be provided as computer programs or software, such as one or more programs having instructions or sequences of instructions, which are included in or stored in an article of art, such as a machine-accessible or machine-readable medium, an instruction store, or a computer-readable storage device, each of which may be non-transient. Programs or instructions on non-transient machine-accessible media, machine-readable media, instruction stores, or computer-readable storage devices can be used to program computer systems or other electronic devices. Machine or computer-readable media, instruction stores, and storage devices may include, but are not limited to, floppy disks, optical disks, and magneto-optical disks, or other types of media / machine-readable media / instruction stores / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They can be found in any computing or processing environment. As used herein, the terms “computer-readable,” “machine-accessible medium,” “machine-readable medium,” “instruction store,” and “computer-readable storage device” shall include any medium capable of storing, encoding, or transmitting instructions or sequences of instructions for execution by a machine, computer, or computer processor, and enabling the machine / computer / computer processor to perform any of the methods described herein. Furthermore, in this art, software is often referred to as taking an action or causing a result in one form or another (e.g., program, procedure, process, application, module, unit, logic, etc.). Such expressions are merely shorthand ways of describing how a processing system executes software to cause the processor to perform actions to produce a result.

[0174] Some implementations can also be achieved by preparing application-specific integrated circuits, field-programmable gate arrays, or by using appropriate networks to interconnect conventional component circuits.

[0175] Some implementations include computer program products. A computer program product can be a storage medium, instruction memory, or storage device having instructions stored thereon or therein, which can be used to control or cause a computer or computer processor to execute any program of the exemplary implementations described herein. Storage media / instruction memory / storage devices can include, for example, but not limited to, optical discs, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAID, remote data storage / archiving / warehousing, and / or any other type of device suitable for storing instructions and / or data.

[0176] Some implementations stored on any of a computer-readable medium, instruction memory, or storage device include hardware for controlling the system and software for enabling the system or microprocessor to interact with human users or other mechanisms using the results of the embodiments described herein. Such software may include, but is not limited to, device drivers, operating systems, and user applications. Finally, such a computer-readable medium or storage device also includes software for performing the exemplary aspects described above.

[0177] Included in the system's programming and / or software are software modules used to implement the processes described herein. In some example embodiments herein, the modules comprise software, although in other example embodiments herein, the modules comprise hardware or a combination of hardware and software.

[0178] While various embodiments of this disclosure have been described above, it should be understood that they are presented as examples and not as limitations. It will be apparent to those skilled in the art that various changes in form and detail can be made therein. Therefore, the exemplary embodiments described above are not restrictive.

Claims

1. A computer-implemented method for determining a shape descriptor of point cloud data in an automotive system to monitor the vehicle's environment, the shape descriptor indicating the shape of the point cloud data, the point cloud data being generated by one or more sensors of the vehicle relative to a reference coordinate system, the point cloud data defining a connected subspace of the reference coordinate system, the method comprising the following steps: a) Determine the bounding box of the point cloud data (S10); b) Select multiple starting agent locations for the agents within the bounding box described in (S20); c) Select (S30) multiple coordinate systems relative to the bounding box; d) For one of the selected multiple starting agent positions and one of the selected multiple coordinate systems, perform (S40) multiple agent movement operations, wherein each agent movement operation includes moving the agent from the current agent position to a new agent position parallel to the coordinate axis of the selected coordinate system, wherein the new agent position is determined based on an edge point given by the intersection of a through-line parallel to the coordinate axis passing through the current agent position and the boundary of the subspace defined by the point cloud data, and wherein the agent acts as a reference point for the agent movement operation and the position of the agent changes in each agent movement operation; e) After step d) is completed, the new agent location is determined (S50) as the center point of the point cloud data; f) For each combination of the plurality of initial agent positions and the plurality of coordinate systems, repeat steps d) and e) of (S60) to determine the plurality of center points of the point cloud data; g) Determine the (S70) regression line by calculating the linear regression of the plurality of center points; h) Perform one of the following steps (S80) h1) and h2): h1) Rotate the plurality of center points (S81) to the basic form such that the regression line corresponds to a predetermined direction; or h2) Rotate the point cloud data (S82) to the base form such that the regression line corresponds to a predetermined direction, and repeat steps a) to f) for the base form to obtain multiple center points of the base form; i) Calculate (S90) the shape descriptor based on multiple center points of the basic form, wherein the shape descriptor includes one or more tilt angles and information indicating the stretching and compression of the basic form of the point cloud data relative to one or more axes of the reference coordinate system. Wherein, the one or more tilt angles correspond to one or more angles of the regression line relative to the coordinate axes of the reference coordinate system.

2. The method according to claim 1, wherein, The plurality of initial agent locations are selected at random locations within the bounding box or at predetermined locations within the bounding box.

3. The method according to claim 2, wherein, The information indicating the tension and compression of the basic form is given by at least one of standard deviation, variance, skewness, and mean.

4. The method according to claim 1, wherein, The reference coordinate system is orthogonal to the coordinate system relative to the bounding box.

5. The method according to claim 4, wherein, The reference coordinate system and the coordinate system relative to the bounding box are 2-dimensional, and Step c) includes selecting a rotation angle that defines the rotation of the reference coordinate system relative to the coordinate system.

6. The method according to claim 1, wherein, The new agent location is determined based on the first and second edge points among the edge points. The first edge point and the second edge point are determined as the edge point of the connecting line segment closest to the current agent position among a plurality of connecting line segments, or the first edge point and the second edge point are determined as the edge point of the largest connecting line segment among a plurality of connecting line segments, wherein each connecting segment among the plurality of connecting line segments is defined as the range where the through line intersects with the connecting subspace, and The new agent position is determined to be the center point between the first edge point and the second edge point.

7. The method according to claim 1, wherein, The new proxy position is determined by dividing the difference between a first quantity and a second quantity by a positive real constant and adding the result to the current proxy position relative to the coordinate axis, wherein the first quantity is the number of data points of the point cloud data in a first direction along the through line, and the second quantity is the number of data points of the point cloud data in a second direction along the through line.

8. The method according to claim 1, wherein, The chosen coordinate system has coordinate axes x1,…,x N N dimensions, and Step d) includes the following steps on the coordinate axis x i Move the agent from the current agent position to the new agent position in the positive or negative direction of (i=1,…,N).

9. The method according to claim 1, wherein, The number of agent mobile operations is predetermined, or If the distance between the current agent location and the new agent location is less than the threshold distance, then step d) is completed.

10. The method according to claim 1, further comprising the step of: The shape of the point cloud data is classified as regular or irregular based on the shape descriptor.

11. The method according to claim 10, wherein, The one or more sensors of the vehicle include cameras, photoelectric sensors, infrared sensors, lidar sensors, radar sensors, and / or ultrasonic sensors.

12. The method according to claim 1, further comprising the following steps: Following step i), the predetermined operation of the vehicle is performed based on the determined shape descriptor from the point cloud data. The predetermined operations include adaptive headlight control, traffic sign recognition, automatic emergency braking, and / or object tracking.

13. A data processing apparatus comprising means for performing the steps of the method according to claim 1.

14. A vehicle comprising the data processing equipment according to claim 13.

15. A non-transient machine-readable medium storing a computer program containing instructions that, when executed by a processor of a computer, cause the processor to perform the method according to claim 1.

Images