Preprocessing point clouds to optimize data processing in networks

Abstract

A computer (500) identifies property data (PROPERTY) belonging to a physical object having a relationship with a technical device (400). A pre-processor module (510) of the computer receives a point cloud dataset (200) from a sensor associated with the device (400). The point cloud dataset (200) has N data points. The pre-processor module (510) applies a transformation to the dataset, the transformation including mapping computed function values to a derived dataset according to a predefined mapping rule. A previously trained machine learning tool processes the derived dataset to identify the property data (PROPERTY).

Description

Technical Field

[0001] In general, this disclosure relates to data processing via a network, and more specifically, to computer systems, methods, and computer program products for processing sensor data, the objective of which is to identify attributes of an object and also for correspondingly controlling technical devices. Background Technology

[0002] Typically, computers can represent objects in the physical world using data. Object data can come from sensors, and the computer can arrange this data according to a data structure. For example, an object can reflect light, and the computer can receive object data from an image sensor (such as a camera). The data will be constructed as an image with pixels. The light image can be a 2D image with (x, y) position coordinates, or it can be a 3D image with (x, y, z) position coordinates (pixels will become "voxels").

[0003] Other types of sensors can provide additional or alternative information. For example, radar or lidar sensors can provide data indicating whether an object (or part of an object) is physically moving away from or closer to the sensor. Simply put, radar images (2D or 3D) map data points to coordinates within a 2D or 3D mesh structure.

[0004] Technicians are familiar with other sensor types, such as temperature sensors (used to measure, for example, thermal radiation), light intensity sensors, vibration sensors, and acoustic sensors (used, for example, to measure sound distribution via a directional microphone).

[0005] Technicians are also familiar with using sensor data (from a single sensor, or from multiple sensors of the same or different types) to draw conclusions. For example, an object may perform a certain movement pattern (such as a human or animal making a gesture whose semantics a computer should recognize).

[0006] Technically, computers receive data in an order or sequence specified by the data structure, such as "line by line" for 2D images and "2D by 3D images".

[0007] Feature extraction techniques allow computers to process object data to identify the object's attributes. These techniques can be based on machine learning, and technicians are familiar with neural networks (or similar structures) implemented by computers.

[0008] Network technology is adapted to the type of object data: extracting features from light images or radar images can differ in how the neural network is configured.

[0009] Computers can use the identified attributes for specific and predefined purposes, where they control technological devices.

[0010] For example, a computer can be associated with the device, and the object can be a living organism that might interfere with the device (or vice versa). The computer will monitor the object to identify attributes related to, for example, the object's movement.

[0011] Based on the identified object attributes, the computer can act on the device. In this example, interference may be undesirable because the object might collide with the device: for example, an animal might collide with a vehicle, or a human operator might enter the operating area of ​​an industrial machine. The computer will instruct the device to stop or move away from the object.

[0012] However, processing data to act on devices presents two conflicting challenges (in terms of data and time). Simply put, the amount of data a computer receives can be relatively large, making it difficult for the computer to act in a timely manner.

[0013] Data constructed as an image allows a computer to identify object attributes, but the computer will have to process the image completely. Because the computer will process the data entirely, it will process all pixels (or voxels) of the image (i.e., a light image, a radar image, or both). Many pixels will be "empty," but still must be processed.

[0014] Therefore, data processing can take a relatively long time (even too long). For example, object-device collisions may be unavoidable because the computer will not be able to identify the attributes in time. This late prediction of collisions will not allow the computer to trigger appropriate preventative measures.

[0015] Technicians can address these challenges by upgrading computers to use additional memory, additional processing cores, and so on. More hardware resources allow for parallel data processing, thus speeding up processing. However, for some devices, such hardware upgrades may not be possible.

[0016] In recent years, new data structures have become available, including what is known as point clouds.

[0017] In a specific scene (or space), individual elements of an object (i.e., real-world points) are associated with data, but other elements are not. Metaphorically speaking, multiple data points form a point cloud. The order in which the computer receives the data can vary.

[0018] Computer architectures for processing data from point clouds to classify, segment, etc., of objects are known in the art, for example, from “Pointnet: Deep learning on pointsets for 3D classification and segmentation” by Qi, Charles R. et al., Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.

[0019] US 11,798,173 B1 describes data processing for point cloud data. Point cloud data is acquired from sensors mounted on a vehicle and represents the vehicle's surrounding environment, such as streets. Data processing aims to generate street maps, etc. Summary of the Invention

[0020] A computer-implemented method enables computers to identify attribute data belonging to physical objects associated with technological equipment.

[0021] Through the preprocessor module, the computer is receiving point cloud datasets from one or more sensors associated with a technological device. A point cloud dataset represents elements of a physical object. A point cloud dataset has N data points, where each data point includes a multi-value tuple with D values. The number D corresponds to the number of dimensions.

[0022] Using a preprocessor module, the computer is applying transformations to the point cloud dataset by applying specific values ​​of each dimension to each data point. This produces a transformed dataset, which is a matrix S with R intermediate N-tuples.

[0023] Using the preprocessor module, the computer maps the intermediate N-tuples to the derived Q-tuples of the derived dataset according to predefined mapping rules.

[0024] Using previously trained machine learning tools, the computer is processing the derived dataset to identify attribute data.

[0025] Optionally, receiving the point cloud dataset includes receiving the point cloud dataset as an input matrix X, which is a D x N matrix, wherein applying the transformation includes multiplying the transformation matrix P as an R x D matrix with the input matrix X to obtain the matrix S as an R x N matrix.

[0026] Optionally, receiving a point cloud dataset from one or more sensors includes receiving a point cloud dataset from a sensor arrangement having multiple channels, where the dimensions correspond to the multiple channels.

[0027] Optionally, mapping R intermediate N-tuples to derived N-tuples according to predefined mapping rules includes: mapping the function values ​​computed as intermediate N-tuples with tuple elements to feature expressions, such that intermediate N-tuples with the same number and type of tuple elements are mapped to the same feature expressions.

[0028] Optionally, the mapping includes selectively sorting the elements of each row in matrix S in ascending or descending order.

[0029] Optionally, the mapping includes calculating the offset and normalizing the elements of the intermediate N-tuple accordingly.

[0030] Optionally, processing of the derived dataset can be performed through classification to identify attribute data.

[0031] Optionally, the computer may also provide control commands to the technical device based on the identified attribute data.

[0032] Optionally, the derived dataset is processed in a cascaded manner by previously trained machine learning tools.

[0033] Optionally, after grouping the data channels from one or more sensors, the steps of applying the transformation to the point cloud dataset and the mapping are performed separately for each dimension group.

[0034] Optionally, the preprocessor module receives point cloud datasets with different data subsets, where the data subsets correspond to multiple parts of objects that are different in time or space, and applies transformations to the different subsets respectively.

[0035] Optionally, the method identifies the attributes of the physical object relative to the technical device, wherein the relationship and the preprocessing are selected from the following: the physical object is moving relative to the technical device; the technical device is moving relative to the physical object; and both the technical device and the physical object are moving.

[0036] A method implemented by a computer is disclosed to determine the properties of a physical object relative to a technical device. The computer is associated with the technical device to control the device based on the determined properties of the physical object.

[0037] A computer system is associated with a technical device and adapted to perform a computer-implemented method to identify attribute data belonging to a physical object related to the technical device, and to use the attribute data to control the technical device based on the identified attributes of the physical object.

[0038] A computer program product, when loaded into the memory of a computer and executed by at least one processor of the computer, causes the computer to perform the steps of the method. Attached Figure Description

[0039] Further details and advantages of the invention will become apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings, in which: Figure 1 The physical objects associated with the technical equipment are shown, and the data representing the elements of the physical objects are characterized. Figure 2 The illustration shows a technological device sending data to a computer equipped with machine learning (ML) tools to provide object attribute data; Figure 3 A flowchart is shown showing a computer-implemented method for identifying attribute data belonging to physical objects; Figure 4 An example of a point cloud dataset transformed by matrix notation X, transformation matrix P, and intermediate matrix S is shown. Figure 5 The mapping steps of the method are illustrated by showing an example in which the intermediate matrix can be sorted; Figure 6 The objects are shown, along with the point cloud dataset and derived dataset; Figure 7 The diagram illustrates the grouping of data channels and the individual processing of each data channel; and Figure 8 An object is shown that is divided into multiple objects. Detailed Implementation Writing Agreement

[0040] This description uses uppercase letters (e.g., D, N, R) for numbers, such as the number of elements in a set (i.e., the cardinality of the set), and lowercase letters (e.g., d, n, r) for indices. The term "Ω-tuple" (where Ω = D, N, Q, R, etc.) refers to a set of numbers with cardinality Ω (i.e., a finite sequence in which each element appears exactly once). Tuples can also be viewed as vectors.

[0041] This description also uses uppercase letters for matrices (e.g., X, P, S), and indicates the number of rows and columns by writing "row x column". If needed, the matrix can be transposed for computation or to simplify understanding (e.g., ...). Figure 4 P in T ).

[0042] Sometimes, a matrix is ​​written as X = { [ ].1, [ ].2, ..., [ ].n, ..., [ ].N}, where [ ] denotes columns (n ​​= 1 to N). Alternatively, a matrix can be written as, for example, P .

[0043] Since the mathematical elements of a matrix can be identified by double indices, this description separates the index n by the small dot (".n") on the left.

[0044] Figure 1 A physical object 100 associated with the technical device 400 is shown. At least one sensor 410 (i.e., a single sensor or an arrangement of multiple sensors) is associated with the technical device 400. The sensor 410 provides a point cloud dataset 200, which has data representing elements of the physical object 100 through multidimensional data points [x].n. In a simplified example, Figure 1 The physical object 100 is symbolically represented by a circle, and the elements E1, E2, and E3 are represented by black dots.

[0045] The number of 3 elements in object 100 is merely a simplification. Since data point [x].n corresponds to a specific element, data point [x].1 corresponds to E1, data point [x].2 corresponds to E2, and data point [x].3 corresponds to E3. In other words, real-world points or elements (at least some, but not necessarily all) are represented by data points.

[0046] Sensor 410 repeatedly provides data. As used herein, sensor 410 provides a first point cloud dataset 200|t1 at time point t1, a second point cloud dataset 200|t2 at time point t2, and more generally, a point cloud dataset 200|tm at time point tm. Time points can occur at predefined intervals delta_t = t(m+1) - tm (i.e., equidistant intervals), but time points can represent any other time interval. Sensor 410 can even provide data only for certain cases (i.e., not at predefined intervals).

[0047] As the attribute “point cloud” already implies, dataset 200 may lack an inherent structure, mainly because sensor 410 can provide data points in different orders.

[0048] As shown in square brackets [], each data point [x].n is constructed as a D-tuple with tuple elements (e.g., coordinates and / or measurement data), and this structure is predefined and remains unchanged.

[0049] In contrast, as shown in curly braces {}, data from multiple data points can arrive from sensor 410 in any order as a point cloud dataset 200 at a single time point tm.

[0050] The order can be a quasi-standard arrival order {[x].1, [x].2, [x].3} or an alternative arrival order {[x].2, [x].1, [x].3}, or {[x].2, [x].3, [x].1}, etc. The arrival order does not have to be deterministic, but it can be a random order.

[0051] As sensor 410 continues to repeat the operation, the arrival order can be changed randomly. For example, dataset 200|t arrives in the first order (e.g., a quasi-standard order in which the index corresponds to the order), and the next dataset 200|t2 arrives in the second, different order.

[0052] The point cloud dataset 200 is forwarded to a computer for processing, with the overall goal of extracting one or more features from the data. In other words, the goal is to identify one or more attributes of object 100, as the object may interact with the technology device 400. However, the random order of arrival, as a technological constraint, does affect this processing.

[0053] To simplify the explanation, Figure 1 An arrow pointing to the right indicates data forwarding, but the computer is omitted.

[0054] Figure 2 A technical device 400 is shown that sends a point cloud dataset 200 to a computer 500. The computer 500 includes a machine learning (ML) tool 520 that provides object attribute data (or simply PROPERTY). PROPERTY can be data representing the state of the device 400 (as part of a technical system for interaction between objects and devices) and can also be data sent to control the device 400 (according to feedback, etc., indicated by a left arrow).

[0055] Technicians can choose the type of tool based on their interests. In implementations, the machine learning (ML) tool can be an artificial neural network, a tree-based model, a support vector machine (SVM), or another tool capable of providing attribute data. ML tool 520 applies machine learning techniques. ML tool 520 is a previously trained tool (i.e., training has been completed in advance, also known as "pre-trained"). For simplicity, the description illustrates that ML tool 520 is an implementation of a neural network, such as a convolutional neural network (CNN).

[0056] In this case, ML tool 520 can be a network having, for example, the following from input to output: - Two layers used to perform 3D convolution - Flattened layer, - The first dense layer has a relatively high number of nodes. -A second dense layer with a relatively high number of nodes, and - A softmax layer used for prediction (which can be classification).

[0057] Technicians can use other configurations. Since such neural networks with multiple layers are known, no graph is presented.

[0058] Simply put, computer 500 takes a certain time interval ΔT from receiving point cloud dataset 200 to providing PROPERTY. This interval ΔT is a delay that can be associated with control device 400. (Technical personnel know that ΔT is also related to delta_t). ΔT can be analogous to the reaction time that humans take to react to a sudden event.

[0059] will utilize Figures 3 to 5 This describes the details of computer 500 (which executes computer-implemented methods to identify the characteristics of physical objects). Computer 500 uses preprocessor module 510 (in... Figure 2 (Highlighted in bold) Dataset 200 is preprocessed before being processed by ML tool 520. Preprocessing results in derived dataset 250, which is equivalent in meaning to the data preserved (from dataset 200) at least in terms of property quality.

[0060] Although ΔT is the sum of ΔT1 (preprocessing) and ΔT2 (processing to identify PROPERTY), preprocessing allows ΔT2 to remain relatively short, making ΔT relatively short as well.

[0061] At first glance, adding a preprocessing step (with ΔT1) seems contradictory. However, preprocessing compensates for the lack of structure in the point cloud (i.e., the change in arrival order) by transforming the point cloud into an order-independent data structure (i.e., derived dataset 250), which can be processed in a relatively short time interval ΔT2. In other words, high-speed processing is possible despite the sparse data structure. Furthermore, preprocessing keeps the amount of data (i.e., point cloud dataset 200, derived dataset 250) relatively constant. The ML tool 520 does not need to handle computationally intensive voxel data, etc. Scene

[0062] For illustrative purposes only, this description provides an overview of some scenarios.

[0063] In the first scenario, the technical device 400 could be an industrial machine (such as a robot) or a vehicle. The motion of object 100 (e.g., a person or a group of people) relative to the machine or vehicle should be classified. For example, ML tool 520 would provide PROPERTY as a binary classification: - "No one is near the machine / vehicle", and - "There are people near the machine / vehicle".

[0064] Based on predefined rules, PROPERTY will ultimately allow the device to continue operating, or will stop its operation (e.g., to prevent an expected collision with a person).

[0065] For example, a vehicle should have sensors and some computer-supported control mechanisms. If a physical object or person (i.e., an "object") is present in the vehicle's intended direction of travel, the vehicle should slow down or even stop moving. The same principle applies to the moving parts of industrial machinery.

[0066] The first scenario can have other aspects, such as gesture recognition. PROPERTY will be a classification corresponding to the semantics of the gesture. Although a person makes a gesture with some intention, the detection algorithm for events related to the person but not intentionally will be similar (e.g., if a person falls or falls into an emergency).

[0067] In the second scenario, the technical equipment 400 and computer 500 can belong to a monitoring system. It can be installed at the entrance (or exit) of a building to count visitors, installed on the road to count vehicles, and so on.

[0068] In the third scenario, technology device 400 could be an aircraft. This scenario will also serve as a fictional example, described further below.

[0069] The scenarios involve different devices and objects, as well as different relationships between them. However, there is a common aspect: the accuracy of computer recognition of properties is related to the use of the property. For example, gesture recognition needs to distinguish more categories than just binary absence / presence, resulting in different levels of accuracy.

[0070] The first scenario is relevant to industrial and traffic safety, but it is irrelevant if the person in motion is, for example, male or female. The accuracy of detecting the presence of a person (i.e., an object) within the reach of a machine or vehicle is sufficient.

[0071] The second scenario is usually related to security, but again, it doesn't matter what kind of visitors are arriving, or whether the vehicle is powered by an internal combustion engine or otherwise.

[0072] In other words, processing accuracy may be less relevant than processing speed (relatively short ΔT).

[0073] Technicians can reduce processing speed by providing computer hardware to run tasks in parallel, for example. However, such methods may not be desirable or feasible. Generally, additional computational work should be avoided. Details of point clouds

[0074] This description now turns to the details, please refer again. Figure 1 .

[0075] As already mentioned, the point cloud dataset 200 includes N data points [x].n. The index n identifies a specific data point (i.e., n = 1 to N). In this example, there are N = 3 data points [x].1, [x].2, and [x].3 (as mentioned above, corresponding to object elements E1, E2, and E3).

[0076] Each data point [x].n can be written as a D tuple [x1, x2, ..., xD].n, where each element in the tuple represents a dimension. Therefore, there are D dimensions.

[0077] The term "dimension" has a technical equivalent in the term "data channel," and the sensor will be a multi-channel sensor (i.e., a sensor arrangement with multiple channels). In this example, there are D = 3 dimensions (i.e., channels) of spatial coordinates x1, x2, and x3. In implementations, there may be D > 3 dimensions. Additional dimensions (i.e., data channels) may be, for example, the following: - Indicates the value of so-called Doppler data (if an object element is moving closer to or away from the sensor). - Indicates one or more values ​​of color data (e.g., color encoding via RGB values ​​results in three additional RGB dimensions for color). - One or more values ​​for the attribute that has already been identified by sensor 410 (in Figure 1 In the example, the elements identified by the point cloud (matrix X) can have certain attributes, such as color = black, size = standard size, etc. - For example, one or more values ​​of signal strength (or intensity) from a radar sensor.

[0078] The tuple elements within [] are ordered invariant. For example, the sensor will always provide coordinates in a specific order and will provide Doppler data at position d = 4. In other words, the order of index d within [] does not change because sensor 410 provides data points that do not change. For example, the sensor provides position data where x1, x2, and x3 are Cartesian coordinates (see […]). Figure 1 And such coordinates are applied to each data point.

[0079] Point cloud dataset 200 can also be written into matrix X: X = { [x1, x2, ..., xD].1, [x1, x2, ..., xD].2, ..., [x1, x2, ...,xD].n, ..., [x1, x2, ..., xD].N}。

[0080] A more concise representation of this set could be X = { [x].1, [x].2, ..., [x].n, ..., [x].N}.

[0081] In the same representation, a matrix X with D rows (from d=1 to D) and N columns (from n=1 to N, i.e., columns []) can be written out in full. This D x N convention is convenient for illustration in this paper, but different conventions can also be applied. Formula (1) x1.1 x1.2 ... x1.n ... x1.N x2.1 x2.2 ... x2.n ... x2.N ... xd.1 xd.2 ... xd.n ... xd.N xD.1 xD.2 ... xD.n ... xD.N

[0082] As already mentioned, indices d = 1 to D identify dimensions, and indices n = 1 to N identify specific data points [x].n.

[0083] However, as mentioned above, the situation is different for dataset 200, so index n does not identify the order of the data points [x].n provided by sensor 410.

[0084] A point cloud matrix X' can include the same data points as a point cloud matrix X, and therefore represents the same point cloud (and the same objects), but the arrangement or order of the data is different (e.g., in columns). The dashed lines here symbolize permutations.

[0085] For example, data points [x].1 and [x].2 could be swapped. The arrival order would be different, and the computer (such as a neural network or ML tool) would have to process data from column 2 before data from column 1. Alternatively, the network could receive data in parallel, with the data residing in different memory addresses, etc.

[0086] There could be many reasons why sensor 410 might simply provide [x].2 before [x].1. In other cases, sensor 410 might revert to the original order [x].1, [x].2, etc. Note that some sensors provide data only when they detect something; otherwise, they remain silent. As already mentioned, the order can be random.

[0087] It should be noted that sensor operation can be related to signal propagation time (e.g., radar signals) and internal sensor processing delays. Phase shifts between radio signals, etc., may also become relevant. Signal reflections (e.g., radio waves or sound waves passing through a device or other structures such as buildings) can also affect data arrival. In another example, sound waves have different propagation speeds in different media (e.g., air and water), so the order of arrival may also be affected for this reason.

[0088] If an ML tool (such as a neural network) receives matrix X, its output will be PROPERTY, and if it receives matrix X', its output will be PROPERTY'. Since X and X' both represent the same object, the ML tool should provide PROPERTY = PROPERTY', but it can provide PROPERTY ≠ PROPERTY'. Therefore, the responses (i.e., CONTROL, FEEDBACK, etc.) may be different.

[0089] Alternatively, depending on the training, the ML tool may correctly provide PROPERTY=PROPERTY', but it will take more time (both when it processes the data and during training) to adapt to the permutation. Explore ways to highlight one aspect of the problem

[0090] The description here focuses on written communication between people. There is a certain analogy between data points and letters, and between point cloud data and words. Individual letters will retain their internal structure [], for example, the letter "R" will not become "Я", and the letter sequence {} within a word should not be changed.

[0091] For example, a reader of this text will easily interpret the sequence {[M][A][T][R][I][X]} in terms of its mathematical conventions. A reader might similarly interpret a permutation such as {[M][I][T][R][A][X]}, but this interpretation will be based on experience (similar to how training a neural network is error-agnostic), and of course, the reader will need some additional time to grasp its meaning. Other readers may not understand the permutation at all, and might even misunderstand it. preprocessor

[0092] Regarding point cloud data, technicians will apply voxelization techniques to avoid this situation, but there is additional computational effort involved.

[0093] The preprocessing in module 510 is helpful here; module 510 provides input to ML tool 520, and this input is derived dataset 250, which is: (i) Not specific to column order (i.e., permutations in arrival order, etc.) but (ii) It is still relatively easy to process by ML tool 520 (no voxelization required).

[0094] To explain the operation of the preprocessor module 510, this description relates to the point cloud matrix X (a D x N matrix of formula (1)). Data for a specific point [x].n is available in columns with index n across all D dimensions.

[0095] Point cloud dataset 200, if based on computer implementation methods ( Figure 3 The 600 in the middle is processed to convey one or more characteristics of the object 100 associated with the technical equipment 400.

[0096] In a simplified example, object 100 could be a flock of birds flying in an industrial setting such as a manufacturing hall (i.e., equipment 400).

[0097] This example addresses some aspects of the scenario described above: collision avoidance safety and potential surveillance. In a very simplified manner, the presence of a data point [x].n in the point cloud dataset 200 will indicate the presence of a bird. (Other dimensions such as Doppler can be considered, but they are omitted for simplicity in this description). This is certainly not a property.

[0098] To process a point cloud dataset of 200, a computer can, in principle, use a variety of methods to represent the data.

[0099] In the first method, a voxel model provides a coordinate system with D dimensions, where each dimension is scaled with 5 coordinate values ​​(e.g., 0, 1, 2, 3, 4): 5 5 5 = 125 (i.e., 125 coordinate units). The computer will process 125 values, of which 3 are logical "yes" and 122 are logical "no". The numbers in this simplified example may seem very low, but more practical implementations offer better accuracy and potentially additional dimensions. For example, if sensor 410 will only measure N = 200 points (within a certain time frame or at time point tm), creating a reasonably fine 3D mesh (i.e., a voxel model) might require up to 1000... 1000 1000 units, but most of them will remain empty.

[0100] In the second method, the same information can be given by vectors (or arrows) in space. For example, starting from point [x].1 with coordinates (2, 2, 1), there will be a first vector (+2, -2+1) reaching data point [x].2, and a second vector (-3, +2, +2) reaching data point [x].3. The computer will have to process these vectors in a specific order. Data points and vectors can be viewed as a graph with nodes (points) and edges (vectors). In a variation of the second method, the (inverse) distance between points can be used as the weight of the edges.

[0101] In the third method, and this is the more relevant one here, the computer can simply receive a point cloud with data points (in the example here with D=3 spatial coordinates). [2, 2, 1].1, [4, 0, 2].2 and [1, 2, 4].3 The first order is [].1, [].2, and [].3. The number to the right of the brackets is the index n.

[0102] However, sensor 410 does not necessarily provide data in the first order [].1[].2[].3 (quasi-standard), but sensor 410 can provide data in a second order (e.g., [].1[].3[].2 or [].2[].3[].1 etc.) as a different order. The second order can be regarded as a permutation of the first order, and the third order can be regarded as a permutation of the second order, and so on. Any order can also be represented in matrix X of formula (1).

[0103] The first and second methods are generally more computationally demanding than the direct processing of point clouds in the third method (through ML tools, such as neural networks).

[0104] However, tools such as neural networks are sensitive to transformations in the input data. Suppose a neural network is directly received with a point cloud dataset of 200 (refer to the matrix), the permutations may have undesirable effects (e.g., different properties, need for further training, etc.).

[0105] However, the preprocessing in module 510 compensates for this, and the preprocessing makes the identification of PROPERTY tolerant of changes in order. Methods for identifying attribute data

[0106] Figure 3A flowchart is shown of a computer-implemented method 600 for identifying property data belonging to physical object 100 in relation to technical device 400. Method 600 is executed by computer 500 (steps 610, 620, and 630) with preprocessor module 510 and by ML tool 520 (step 640).

[0107] Method 600 includes (by module 510) receiving a point cloud dataset 200 from one or more sensors 410 associated with device 400. This description has illustrated that the point cloud dataset 200 represents the physical object 100, the elements of the dataset (200), and several representations have been explained, including a representation as a point cloud matrix X.

[0108] Method 600 also includes a sequence of steps 620 / 630, also performed by module 510, to preprocess the point cloud dataset 200 into a derived dataset 250 (or a preprocessed dataset). The preprocessing sequence includes: - Apply a 620 transformation to obtain the intermediate N-tuple (in the transformed dataset 220 or matrix S), and - Map the function value 630 to (obtain) a derived dataset 250 (with Q tuples) according to one or more predefined rules (e.g., in the rule implemented by sorting).

[0109] Figure 3 Preprocessing is symbolized by showing dataset 200 in the variants “abc”, “bca”, and “cab” (i.e., some permutations) and by showing the derived dataset 250 as a single symbol ①. What the derived dataset 250 looks like is not important, but it contains information from dataset 200 to derive the PROPERTY, regardless of whether dataset 200 shows permutations.

[0110] Finally, ML tool 520 processes (step 640) the derived dataset to identify attribute data (PROPERTY). As will be explained, point cloud dataset 200 can be matrix X or matrix X', but PROPERTY = PROPERTY'.

[0111] It should be noted that technicians will be able to implement a method that receives a point cloud dataset and uses ML tools (e.g., neural networks) to identify attribute data, as outlined by Qi et al.

[0112] However, preprocessing (within the transformations and mappings) addresses issues related to data structures, ΔT, permutations, etc. Since preprocessing involves some computation, these will be explained below. Preprocessing

[0113] Figure 3 The preprocessor module 510 is also shown in steps 620 and 630 for preprocessing. Figure 4 An example of step 620 (transformation) is given. Figure 5 An example of step 630 (mapping, for example, by sorting) is given.

[0114] Preprocessing may include activities such as removing outliers and data inconsistencies. Technicians will clean the data as early as possible, for example, upon receiving the point cloud dataset X (in step 610). However, such cleaning activities can also be performed later to clean N-tuples or Q-tuples. As an example, such activities may include pruning (e.g., setting values ​​above a threshold as a threshold) or normalization. Split data in optional methods

[0115] On its right, Figure 3 An alternative method is also shown. A computer-implemented method 600-δΔ for identifying attribute data includes the steps of method 600, but introduces the splitting of the point cloud dataset X according to criteria such as a time-progress dimension. This splitting can also be considered as grouping or clustering.

[0116] Steps 620 and 630 can be executed separately: steps 620-δ and 630-δ are executed for the data in group δ to obtain the first derived dataset, and steps 620-Δ and 630-Δ are executed for the data in group Δ to obtain the second derived dataset (i.e., the first feature expression and the second feature expression). Step 640 will be executed after the associated derived dataset (step 641).

[0117] The details of this data subset-specific processing will be explained below, such as splitting or grouping along dimensions ( Figure 7 ), and time-based splitting (“Dynamic Adaptation Feature Function” section). Computer program

[0118] Figure 3 A computer program or computer program product is also shown. When loaded into the computer's memory and executed by at least one processor of the computer, the computer causes the computer to perform the steps of the computer-implemented method 600. In other words, Figure 3 The boxes in the diagram illustrate that the method can be implemented by a computer under program control, and show that the computer has a module for receiving the point cloud dataset (see step 610), a module for applying the transformation (see step 620, belonging to preprocessor 510, see...). Figure 2 The module used to perform the mapping (step 630, also part of preprocessor 510, see...) Figure 2 ) and modules for processing data (i.e., Figure 2ML tools 520 (in the text).

[0119] Figure 4 An example is shown. - Let the point cloud dataset be denoted as point cloud matrix X, which is 200. - Transformation matrix 210 (also denoted as transformation matrix P) and The transformed dataset 220 (also denoted as the transformed matrix S, which serves as an intermediate matrix and has N-tuples 221-r).

[0120] The details regarding step 610 have already been explained. In an equivalent representation, point cloud dataset 200 can be represented as... Where D is the dimension of each point [x].n, and N is the number of points in the point cloud (e.g., ...). Figure 1 (As explained). As already explained, D=3 can be an implementation with only spatial coordinates, while D=4 can add another dimension, such as Doppler, to the spatial coordinates.

[0121] exist Figure 4 In the example, dataset 200 is Figure 1 The dataset has D=3.

[0122] Regarding the use of function value (f) in step 620 r To obtain the transformation of the intermediate N-tuple, module 510 performs the following operations:

[0123] Characteristic functions can be defined. , where R is the number of such characteristic functions. Each characteristic function f r Each results in a scalar value (e.g., implemented by a computer as an integer, a real number, etc.). Since the function has scalar values, these values ​​can be considered to be in 1D (one dimension). The resulting intermediate N-tuples (with these scalar values) are shown by reference numerals 221-1, 221-2, 221-3, 221-4, 221-5, where r = 1 to R = 5, or more generally denoted by 221-r.

[0124] However, this is not dimensionality reduction, but rather a transformation. This is achieved by computing multiple (i.e., R) features f. r There is essentially no information loss (which would prevent the acquisition of property). In this embodiment, the transformation is a linear transformation (through a linear transformation matrix P).

[0125] The index r identifies a specific function (in...) In the case of (within), the following applies: In other words, a data point [x] with D tuple elements (i.e., dimensions) is processed by a predefined function for each tuple element, and the result is an integer (i.e., 1D feature). This computation is performed separately for each of the N data points.

[0126] Module 510 generates R 1D features. Predefined functions. Formula (2) It can be represented by the transformation matrix P.

[0127] Transformation matrix 210 or It is an R x D matrix, and This is data point [x] (point cloud 200). (An index n can be added because the predefined function in the transformation matrix P is applied to each data point [x].n.)

[0128] As already mentioned, data points [x].n are represented by columns of the point cloud matrix X. In this representation, the transformation of the point cloud matrix X (i.e., the dataset 200 as a whole) is written as... Formula (3) S=

[0129] The transformation matrix S ( ) represents the output of the transformation, i.e., the transformation matrix 220 or This is used for linear transformations. Nonlinear transformations are also envisioned. The output of this transformation will be the transformation matrix S, and the matrix elements will be s(r,n).

[0130] For example, for D = 3 dimensions (i.e., only spatial coordinates x1, x2, and x3, such as...) Figure 1 For the point cloud dataset 200 (shown), the first feature function f1 can be the sum of values: f1 = x1 + x2 + x3 (i.e., column summation). With regard to the transformation matrix P in formula (3), x1, x2 and x3 will have to be multiplied by factors (i.e. p11 = 1, p12 = 1 and p13 = 1) and summed.

[0131] More in detail, Figure 1 The point cloud dataset 200 will be transformed into an intermediate dataset S, where s(r, n) is given for r = 1. - (1, 1) = 2 + 2 + 1 = 5, - (1, 2) = 4 + 0 + 2 = 6 - (1, 3) = 1 + 2 + 4 = 7

[0132] In the case of point cloud dataset 200 arriving at permutations (e.g., ( )1, ( )3, (2), X'), the following applies: s'(r, n) s'(1,1) = 2 + 2 + 1 = 5 s'(1,3) = 1 + 2 + 4 = 7 s'(1,2) = 4 + 0 + 2 = 6

[0133] The second characteristic function can be defined, for example, f2 = 2. x1 - x2 + x3, or in the example: s(2,1) = 2 2 - 2 + 1 = 3, s(2,2) = 2 4 - 0 + 2 = 10 s(2,3) = 2 1 - 2 + 4 = 4

[0134] In the case of permutations, for matrix S', the eigenfunction must be calculated: s'(2,1) = 2 2 - 2 + 1 = 3, s'(2,3) = 2 1 - 2 + 4 = 4 s'(2,2) = 2 4 - 0 + 2 = 10

[0135] This description is simplified; the S for f1 and the S for f2 will be different. Note that a function of size R can be defined (see [link to documentation]). Figure 4 (R=5).

[0136] Figure 4 An example of a matrix S with the s(r, n) values ​​just presented, having R rows and N columns, is also shown. Since the terms "row" and "column" are interchangeable, this description uses the term "tuple," and matrix S comprises R N-tuples 2^21-r. These N-tuples are intermediate N-tuples, as they will be processed in the next method step. General mapping and sorting as an example

[0137] Figure 5 The arrows / / → | | indicate the method step mapping 630, and also show two matrices S and S'.

[0138] The computed values ​​of s(r, n) in intermediate dataset 220 (matrix S) and s'(r, n) in intermediate dataset 220' (matrix S') will be N-tuples (because both matrices are R×N matrices). They can be written as / / , / / ', and for further permutations, even as / / ”, etc.

[0139] In this example, For r=1, the N-tuples are / 5, 6, 7 / (see 221-1) and / 5, 6, 7 / ' (as in the first row of S and S'). For r = 2, the N-tuples are / 3, 10, 4 / (see 221-2) and / 3, 10, 4 / ' (as in the second row of S and S').

[0140] The elements of these N-tuples are scalar values. Computers represent them as integer values ​​(depending on the function f). r (Some values ​​can be zero or negative).

[0141] The tuple elements in / / and / / ' are the same in type and number. For example, in the N-tuples 221-1 and 221-1', the tuple elements are: for / / , an element of type "5", an element of type "6", and an element of type "7", and for / / ', an element of type "3", an element of type "4", and an element of type "10".

[0142] In other words, the tuples / / and / or / / are equivalent to a deck of cards. In a deck of 32 cards, there are, for example, 4 "suits", and each "suit" has 8 values. In that sense, the tuple is shuffled.

[0143] Computers apply predefined mapping rules (i.e., generalized mathematical functions and / or mapping algorithms) that map various types and quantities of tuples (at the input / / , / / ', N-tuples) to feature data or feature expressions 231-r, such as Q-tuples ||, which are features of type and quantity (at the output), see [link to relevant documentation]. Figure 5 231-1, 231-2, etc.

[0144] Symbolically, the / / symbol is reconciled only by being given in an upright position | |. In other representations, the mapping can be represented as / / →| |.

[0145] Mapping can also be viewed as encoding (or even as providing a "signature").

[0146] Although the (intermediate) N-tuples each have N elements, the feature expression can be a Q-tuple, where Q>N (mapping to a larger target), Q<N (mapping to a smaller target), and Q=N (mapping to a target of equal size).

[0147] Intermediate N-tuples 221-r that have tuple elements identical in number (as the term already implies, this will be N) and type (for example, N-tuple 221-1 with N=3 elements, one type "5", one type "6", one type "7", and N-tuple 221-1' in the first row of S') can be mapped to the same feature expression (for example, term 231-1 with |5, 6, 7|).

[0148] The mathematical function for Q=N can be sorting, for example, sorting into ascending order |3, 4, 10| and |5, 6, 7|. For sorting, the number of elements in the tuple remains constant: Q=N.

[0149] Other algorithms can also be applied. For example, the tuples / 2, 3, 6, 1, 5, 4 / and / 2, 6, 3, 1, 4, 5 / ' (not shown) can be sorted into | 1, 2, 3, 4, 5, 6| (ascending order), | 6, 5, 4, 3, 2, 1| (descending order) | 1, 3, 5, 6, 4, 2| (odd numbers in ascending order, even numbers in descending order), | 6, 4, 2, 1, 3, 5, | (even numbers in descending order on the left, odd numbers in ascending order on the right)

[0150] For example, the tuples / 4, 0, 6, 0, 4, 4 / and / 4, 0, 6, 4, 4, 0 / can be mapped to |2, 0,0, 0, 3, 0, 1| (two 0s, zero 1s, zero 2s, zero 3s, three 4s, zero 5s, one 6, by indicating the count of different values).

[0151] There are further possibilities. For example, / 3, 4, 10 / can be normalized to a rounded average (here: the average of 3, 4 and 10 is approximately 5), and represented as the difference from this average |-2, -1, 5|. Normalization to a rounded average can be regarded as a mapping with a compensation offset (for example, offset 5).

[0152] For multiple N-tuples, such as / 3, 4, 10 / , / 4, 5, 11 / , / 2, 3, 9 / , a common offset can be calculated, so that the N-tuples can be mapped to |0, 1, 7| by normalization.

[0153] Although the intermediate N-tuples are defined by numerical values (as they are generated by computation), the mapped Q-tuples (i.e., 231-r) may be characters, such as characters of the Latin alphabet (Q=1, A to Z), or even non-Latin characters (see Figure 2 and Figure 7 ① in). A person skilled in the art is familiar with Unicode character encoding (i.e., Q<N, Q=2 or two bytes for one character). There is no need to associate the N-tuples with human-comprehensible semantics, they only need to be distinguishable by the ML tool (see 520).

[0154] In other words, for the ML tool (i.e., the neural network), the form of the feature expression does not matter, as long as the mapping function remains the same (for training, as well as for other phases, such as when the method is to be executed).

[0155] In this sense, the mapping is a general method, but sorting is a more illustrative method. The abbreviation "sort" is merely an example, indicating that: the mapping has been performed.

[0156] Figure 5 Two matrices S and S' (which have been calculated by transformation before mapping, items 220 and 220') and derived matrices S_sort and S'_sort (after mapping, items 230 and 230') are shown.

[0157] In a subsequent step, the mapping 630 function values are used to obtain the derived dataset 250, and the module 510 maps the entries of each row in the matrix S to equivalent entries. As shown in the figure, the mapping may be, for example, sorting in descending or ascending order, thereby generating S_sort. The sorted matrix S_sort has the same number of R×N as the transformed matrix S. This also applies to S'_sort.

[0158] Figure 5 The mapping 630 from the matrix S' to the matrix S'_sort is illustrated by directed arrows. Of course, the mapping applies to all matrices S regardless of whether permutation is applicable or not.

[0159] This description uses the abbreviation "sort" for each pair of mapping / / → | | and the | | result of the mapping. In Figure 5 In the example, the mapping algorithm sorts row by row (ascending order) from the smallest element to the largest element.

[0160] It is important to note that the mapping (e.g., by sorting) is performed independently for each row of S, which means that the column-by-column correspondence between the columns and individual points in the original point cloud is lost. In this sense, the mapping is a many-to-one relationship (here for the two matrices S and S') to a sorted matrix S_sort = S'_sort. The mapping does not need to be invertible or reversible.

[0161] In the Figure 5 example, the rows in matrices S and S' are sorted in descending order. It makes no difference whether the sorting order is "descending" or "ascending".

[0162] However, the key effect achieved by this processing is that S_sort is independent of the sorting of the points in the point cloud matrix X. That is, if the columns of the point cloud matrix X are permuted to give another point cloud matrix X' representing the same point cloud, the resulting feature matrix S'_sort will be identical. In other words, S_sort = S'_sort.

[0163] In the example, the sorting order will be 7, 6, 5 (for the first feature) and 10, 4, 3 (for the second feature), regardless of the permutation in which the data points arrive.

[0164] This data (in the cascade) can be considered as the derived dataset 250 processed by the ML tool 520, as already described. For example, the derived dataset 250 will be a vector having the first row, the second row, the third row, etc. in the matrix S_sort (or S'_sort). Implementation Aspects of Transformation Matrix

[0165] In the Figure 4 example, the transformation matrix P is presented with examples having arbitrary integers. The present description now discusses some implementation options: - Since the transformation matrix P is an R×D matrix, D may correspond to the number of dimensions D. - Alternatively, an R×D~ matrix with D~<D may be used, where not all dimensions are used. In other words, D~ indicates that the method 600 may be applied to the data only partially (or differently per dimension). Examples of two matrices with D1<D and D2<D are given further below. - The number of functions R is selected according to the accuracy with which the PROPERTY must be provided, but is limited by the computation (to calculate matrix S). - A suitable order of magnitude for R is between R=3 and R=N / 100. For example, for a point cloud expected to have N=1,000 data points, R will be about 10. - The numerical values in the transformation matrix P can be positive or negative. - The numerical values can be integers (to keep computational resources low). - You can set a range of values ​​from the minimum to the maximum, for example, the range [-100, +100]. Figure 4 The example uses the range [-10, +10]. - These values ​​can be selected randomly. Random selection can result in "all zeros", "all one", diagonal matrices, sparse matrices (mostly zeros), etc. However, this can be limited by setting rules for checking the random matrix and eventually ignoring some of them. - To avoid this random effect, the distribution of values ​​can be predefined as, for example, a uniform distribution of matrix elements (with a given tolerance). - Formula (3) presents matrix multiplication, where the transformation matrix T defines a linear transformation. This does not preclude the possibility of introducing nonlinear calculations for some data points. Dynamic systems

[0166] Device 400 and object 100 can be considered as a system. Changes can be expected in this system for the method 600 used to identify property data (belonging to the physical object 100 related to device 400), but such changes should not be a constraint on the method. This system can be dynamic because, for example, the following: - The relative positions of device 400 and object 100 can be changed (i.e., the distance changes if the device and / or object moves). - Device 400 and / or object 100 can change orientation and / or change how they move. - Device 400 may be a vehicle and / or industrial robot that is moving toward or away from object 100 (which will be an obstacle).

[0167] The property of the expected object also changes over time. For example, some or all of its elements (e.g., Figure 1 (E1, etc.) will move, change color, etc.

[0168] This description now introduces some optional concepts (or variations) to address these and other constraints: - By using a relative dataset at the input of the preprocessor, - By grouping feature functions within the preprocessor (see...) Figure 2 Project 510 (in China). - By processing data of different dimensions (d) separately.

[0169] This description explains the details of these concepts individually, but it should be noted that these concepts can be combined.

[0170] Technicians can use readily available tools to provide certain auxiliary data. Some of these tools are even widely used in consumer products (such as digital cameras) and / or are being developed for autonomous driving. Here are some examples: Object trackers can identify the physical location of objects and even predict future changes in their location. Trackers can provide location data or positioning data. An object separator can separate data associated with an object, allowing, for example, location data to be used for different parts of the object. For instance, if image data of the object is available (not necessarily a point cloud dataset), the individual parts of the object can be identified. The separator can also provide location data. In other words, location data can be used for a portion of the object. Absolute and relative datasets

[0171] This can be achieved by repeating method 600 ( Figure 3 This allows for the detection of changes, and for each repetition, the result of the method execution (property data) will be different. Technicians can easily monitor how the property evolves.

[0172] However, changes can already be considered at the input of the preprocessor 510.

[0173] For example, [x].1 = [2, 2, 1], [x].2 = [4, 0, 2], and [x].3 = [1, 2, 4] at the first time point t1 can be changed to [x].1 = [2, 3, 1], [x].2 = [4, 1, 2] and [x].3 = [1, 3, 4] at the second time point t2.

[0174] Assuming the arrival order is preserved (no permutation), the alternative dataset can be computed as the differential point cloud dataset 200 by simply calculating the differences: [x].1 = [0, 1, 0], [x].2 = [0, 1, 0], and [x].3 = [0, 1, 0]. The point cloud dataset 200 is transformed from a set with absolute values ​​to a set with relative values, but the method steps remain the same for the preprocessor module 510.

[0175] This differential point cloud dataset 200 can also be the input to the preprocessor 510.

[0176] Two methods can also be combined: using an absolute-value-based point cloud dataset 200 and using a relative-value-based point cloud dataset 200. Note that the image sensor can provide continuous images using both absolute and relative values ​​as a video stream. Multiple objects and different characteristic functions

[0177] Figure 6 The objects are shown, along with the point cloud dataset and derived dataset.

[0178] It can be assumed that a single sensor (such as Figure 1 Sensor 410 (multiple channels) provides a single point cloud dataset 200. Elements of object 100 are displayed in the "perceived scene" that the sensor can monitor. In the figure, N is approximately 30.

[0179] As shown on the left side of the figure, data point [x] will belong to a single object, and derived dataset 250 will also belong to that single object. For example, a single object could be multiple people located near the device (see...). Figure 1 Alternatively, given the flock example, derived dataset 250 would be derived from that flock.

[0180] As shown on the right side of the figure, the space can be divided into subspaces. This is not necessarily determined by the preprocessor (see [link to preprocessor]). Figure 2 Instead of being executed by computer vision tools or similar techniques that can be applied by those skilled in the art, this can be achieved through computer vision tools or similar techniques that can be applied by those skilled in the art.

[0181] The subspace is occupied by different objects 100-1, 100-2, and 100-3. The dashed lines merely symbolize this distinction. These different objects are physical terms that cannot coexist in the same space. They are spatially disjoint.

[0182] Feature functions can be divided into groups, such as a first group with R1 functions, a second group with R2 functions, and so on. Each group can be assigned to a spatial region in the perceived scene. That is, the first group is only applied to points located within a certain region (i.e., points representing elements in that region), while the second group is only applied to points located within another region.

[0183] Therefore, the derived dataset 250 will be a set of datasets 250-1, 250-2, and 250-3 (or 250-K, where K is usually the number of groups).

[0184] This concept can be useful in the following use case scenario: it can be expected that points in different spatial regions correspond to different people. Different people can have different attributes. For example, the first person (i.e., object 100-1) will be wearing protective clothing (yellow, etc.), while the second and third people will not be wearing protective clothing.

[0185] Relative to the device, object attributes can be expected to differ in different spaces. For example, near the device, a person wearing protective clothing might be allowed. The feature set used for that space will be specifically designed to detect the type of clothing (or at least distinguish protective clothing from normal clothing). For other spaces, other detection methods may be more important.

[0186] Given the bird flock example, some birds (e.g., the birds in object 100-3) will be near the device. Doppler data can also be processed, which results in the transformation matrix P taking the "Doppler dimension" into account (for object 100-3) and results in a simpler transformation matrix that does not take the Doppler dimension into account (for objects 100-1 and 100-2 at non-critical distances). Dynamic adaptation features

[0187] Specializing the feature function to certain spatial regions (e.g.) Figure 4 (In the middle) is not the only way to specialize functions.

[0188] Characteristic function It can be defined as depending on the time step in a dynamic point cloud scene. That is, for tm = t1 to tM, the feature function can be written as f1|tm ... f R |tm. For each tm (e.g., for M time slots from t1 to t=tM), the function can be different. This concept can be useful in scenarios where certain objects in the scene are being tracked, allowing the function to be dynamically adapted to the predicted location.

[0189] The concept may also be relevant if the system itself undergoes some dynamic changes. For example, the system could be mounted on a vehicle or robot, and the functions could then be adapted to different positions, orientations, or speeds of the vehicle.

[0190] Certain dimensions, such as example Doppler data or intensity data, are typically found in radar point clouds. The data in these dimensions conveys characteristics different from spatial information.

[0191] Therefore, in some cases, it may be more adequate to deal with these dimensions individually.

[0192] Figure 7 This illustrates the grouping of data channels and the individual processing of each data channel. Figure 7 Further explanation by Figure 3 The right side (method 600-δΔ) supports this.

[0193] On the left, Figure 7 Matrix X (from) was repeated Figure 1 It has a spatial dimension), but it extends it with another dimension, such as Doppler data (e.g., "5" for n=1, "4" for n=2, and "4" for n=3). In other words, the data is divided into a first group and a second group, which are matrices X1 and X2 (i.e., subsets).

[0194] These dimensions can initially be separated, given Where D1 + D2 = D, such that X1 and X2 coincide with X along the concatenation of the second dimension. Then X1 is processed as described above to obtain S_sort_1. The correspondence between the features in a single row of S_sort_1 and the original points in X or X2 is stored. Subsequently, the remaining dimensions can be concatenated to the rows of S_sort_1 accordingly to obtain S_sort.

[0195] As shown in the figure, we can assume the existence of a first-dimensional set δ (e.g., spatial coordinates, see Figure 1). Figure 1 The dimensions x1, x2, and x3 in the data and the second dimension set Δ (e.g., Doppler data, the intensity of radar reflection, etc.).

[0196] The total number of dimensions is D = δ + Δ. Matrix X (see formula (1)) can be simply split into an upper part (X1, δ dimensions) and a lower part (X2, Δ dimensions): Formula (4) x1.1 x1.2 ... x1.n ... x1.N x2.1 x2.2 ... x2.n ... x2.N ... xδ.1 xδ.2 ... xδ.n ... xδ.N Formula (5) x1.1 x1.2 ... x1.n ... x1.N x2.1 x2.2 ... x2.n ... x2.N ... xΔ.1 xΔ.2 ... xΔ.n ... xΔ.N

[0197] There exist separate transformation matrices (not shown) with R xδ and R xΔ (R can also be different) for the upper part resulting in S_sort(δ), and for the lower part resulting in S_sort(Δ), see steps 620 and 630 performed respectively. Variations of this method are possible, for example, to S_sort(δ) and then concatenate the values ​​of S_sort(Δ) to S_sort(δ), see Figure 3 Step 641 of the related process.

[0198] Alternatively, you can process data of different dimensions separately. For example, if the dimension in each part of the matrix will be the same (i.e., δ = Δ), the intermediate sorted results S_sort can be added together, for example, element by element (the first element in S_sort(δ) is added to the first element in S_sort(Δ), and so on).

[0199] Introducing predefined weights is another option, for example, giving S_sort(δ) weights more (or less) than S_sort(Δ) when processed by ML tool 520.

[0200] But return to Figure 7 The upper part (e.g., having spatial data with δ = 3 dimensions) can be processed as already described (method 600, as performed by preprocessor 510-δ). Figures 4-5 The operations in the middle), and the lower part (e.g., Δ = 1 dimension from the Doppler channel) can be processed in other ways, here symbolized by the preprocessor 510-Δ.

[0201] At the output of the two preprocessors, the derived Q-tuples (i.e., the characteristic expressions) will be concatenated and forwarded to ML tool 520 (more generally, see...). Figure 3 (Related step 641).

[0202] exist Figure 7 In the example, the derived Q-tuple would be "①" (from spatial data) and "4" (from Doppler). In this example, the preprocessing uses a transformation (in step 620, or more precisely, step 620-Δ), which is greatly simplified to obtain the most frequently occurring data (i.e., 4). For example, the Doppler data "5" and "4" indicate an element (see...). Figure 1 Elements E1, E2, and E3 in the sample have a measurement rate of 5, and two elements have a measurement rate of 4. In this example, ML tool 520 will receive the cascaded "①4".

[0203] For any arrival order, the derived value "4" is an indication of the specific speed associated with most elements in object 100.

[0204] It is also possible to apply further rules: the Doppler data may be incorrect, causing outliers to be ignored.

[0205] Alternatively, you can bypass the dimension Δ data (no matrix processing or mapping required; the method is from steps 610 to 640). Figure 3 The bypass is shown by the dashed arrows from step 610 to step 641. Continue describing

[0206] The preprocessing of point cloud dataset 200 into derived dataset 250 has been described for static datasets with relatively few dimensions (e.g., D=3). However, as object 100 changes over time (relative to device 400), point cloud dataset 200 will also change over time.

[0207] Therefore, complexity increases. This description continues with a hypothetical example to illustrate how Computer 500 can adapt to further complexity. Fictional example

[0208] The technology device 400 could be a drone (UAV) flying over farmland. The UAV does not fly alone, but is accompanied by birds that occasionally land on the ground to enjoy the food provided by the farmland.

[0209] In the use case scenario of "birdwatching," a UAV carries a camera to capture light images of farmland and birds. A light image processing computer (located at the UAV or remotely) processes the images to identify the bird species. This knowledge allows farmers to take actions such as emitting bird-specific sounds to invite them to feed elsewhere. Such onboard image processing should be available to those skilled in the art.

[0210] However, UAVs are suitable for another scenario: "collision avoidance." UAVs should have radar / liDAR sensors that capture radar / liDAR data from their surroundings. Figure 1 Sensor 410 in the UAV. The UAV has been programmed to remain within the boundaries of the farmland, so that it should not come into contact with trees or walls located on the outside. However, the UAV faces the risk of collision. Birds will be "elements of the object". The onboard computer executes the computer-implemented method 600 ( Figure 3 (Details, as described, are used to identify the properties of these objects.) In this simplified example, the property data PROPERTY indicates: (a) The bird is flying away from the drone, or (b) The bird is approaching the UAV.

[0211] Airborne ML tools (see) Figure 2 The dataset (520 in the dataset) will be classified into two classes (derived dataset 250) to distinguish between bird-sized elements (which may cause unwanted collisions) and mosquito-sized elements (which allow collisions). In other words, this is a simple binary classification.

[0212] ML tools (or auxiliary tools as described above) also provide approximate location data of the object. Therefore, computer 500 can detect the presence of birds and instruct the UAV to modify its movement (control signals derived from PROPERTY to avoid collisions).

[0213] In other words, the computer processes data differently depending on the scenario. When there's no risk of collision, the computer has ample time to observe the bird and even choose some bird-specific measures. However, if a bird is approaching the UAV, the computer can take immediate action.

[0214] Figure 8The diagram shows object 100 split into multiple objects 100-A and 100-B. Continuing with the example of birds and drones, object 100 could be a flock of birds at a first time point. However, the flock is splitting, and at a second time point, objects 100-A and 100-B should exist (i.e., two flocks of birds).

[0215] The dynamics described above begin to take effect: the onboard computer will detect a positional change, resulting in object 100-A being closer to the UAV than object 100-B. There are two point clouds to process: one for object 100-A and one for object 100-B. Since computation time is a valuable resource, the computer can apply predefined rules to consider the Doppler data of the closer object 100-A and ignore the Doppler data of object 100-B. In other words, the δ-Δ splitting described above can be used. This saves computation.

[0216] Assuming that the assessment of Doppler data for object 100-A will indicate the movement of an object with a collision hazard, the UAV can be guided away from object 100-A.

[0217] Other rules can be applied, such as also checking the Doppler data of object 100-B, but doing so later (because the object is further away). Other aspects

[0218] This description has described a computer-implemented method 600 for identifying property data PROPERTY belonging to physical object 100 (which is related to technical device 400). This description concludes by discussing other aspects of method 600.

[0219] The preprocessor module 510 of computer 500 is receiving a point cloud dataset 200 from one or more sensors 410 associated with the technical device 400. The point cloud dataset 200 represents elements E1, E2, and E3 of the physical object 100. The point cloud dataset 200 can be represented by a matrix X = {[x1, x2, ..., xD].1, [x1, x2, ..., xD].2, ..., [x1, x2, ..., xD].n, ..., [x1, x2, ..., xD].N}. As already mentioned, there are N data points ([x].n), where each data point ([x].n) consists of a multi-value tuple ([x1, x2, ..., xd, ... xD].n) with a number of D values. The number D corresponds to the number of dimensions.

[0220] The preprocessor module 510 of computer 500 processes the point cloud dataset X (see R) according to R feature functions. Figure 4 Project 200) Application (see Project 200) Figure 3Step 620) Transformation. The transformation produces a transformed dataset (see item 220). The transformation is unidirectional because the resulting intermediate N-tuples are not transformed back to X.

[0221] By applying transformations, the preprocessor module 510 can derive a single expression with one dimension for each data point [x].n. This expression can be a quantity (i.e., a one-dimensional quantity). In this example, for data point [x].1 (the left column in matrix 200), the expression is 5 for the first eigenfunction (identified by r = 1), 3 for the second eigenfunction (r = 2), and so on.

[0222] According to predefined mapping rules, the preprocessor module 510 can transform the dataset (e.g., Figure 4 The transformed dataset (item 220) is mapped (step 630) to the derived dataset (e.g., see items 230 and 250). The mapping is permutation-invariant: the order of elements in the transformed dataset 220 (e.g., elements in the N-tuples 221-1 to 221-5 before the mapping) is irrelevant. In other words, the permutation has no effect on the derived dataset regarding the order of the N point cloud data (i.e., the matrix X received by module 510).

[0223] According to predefined mapping rules, preprocessor module 510 can map (step 630) the intermediate N-tuples (see items 221-1, 221-2, 221-3, 221-4, 221-5) to derived Q-tuples of derived dataset 250, respectively. As mentioned, mapping 630 is permutation-invariant: the order in which the elements are arranged within the N-tuples is irrelevant, but the elements in the derived Q-tuples (e.g., rows in matrix S_sort, S'_sort, ...) are not important. Figure 5 Items 230 in the list have the same order. For example, there are N-tuple mappings (orderings) from 5, 6, 7 to 5, 6, 7, from 3, 10, 4 to 3, 4, 10, and so on. (In cases where, for example, the N-tuple 221-2 in row r=2 would be 3, 4, 10 or 3, 10, 4 or 4, 3, 10 or 4, 10, 3, etc., such permutations would be irrelevant: 3, 4, 10 would be the only Q-tuple).

[0224] It is important to note that the mapping is performed separately (i.e., within the N-tuple): item 221-1 is mapped to item 231-1 (e.g., by sorting), item 221-2 is mapped to item 231-2, and so on.

[0225] As explained, by applying a transformation, preprocessor module 510 derives a single expression with one dimension for each data point [x].n. Each feature function results in its own single expression, but these expressions are all one-dimensional. In other words, the feature function defining the transformation results in a one-dimensional feature (i.e., an expression that no longer distinguishes the dimension of the received point cloud dataset). Although the expression is one-dimensional, preprocessor module 510 can use multiple steps (or mathematical operations) to obtain the expression in its derivation (of the feature function applied to the data point). In other words, the multiple steps can be viewed as a predefined algorithm.

[0226] Each feature function may have a first mathematical operator OP1 and a predefined dimension-specific value as the first operand (see the R×D transformation matrix P for the operand). Figure 4 Item 21 in the description. This description has already explained the details of the example, where OP1 is multiplication. and the values ​​in the transformation matrix P (see Figure 4 In this example, the value in P is a factor.

[0227] The second mathematical operator OP2, which is addition in this example, can produce an expression. Since OP1 and OP2 can be applied multiple times, there are multiple expressions, and these multiple expressions can be mapped relative to each other (permutation-invariant mapping).

[0228] See the example. Figure 4 Examples of matrices S in the example are expressions that are integers: 5, 6, 7, 3, 10, ... -8, ..., 6 (i.e., R × N expressions in matrix S, shown as matrix elements). Expressions can be mapped relative to each other: after mapping, the expressions remain unchanged (neither expressions are added nor removed), but their order can be changed. For example, see [from...] Figure 5 For items 220 to 230, the mapping can be sorting: expressions 5, 6, and 7 can be mapped to 5, 6, 7; expressions 3, 10, and 4 can be mapped to 3, 4, and 10. Technicians can use other mappings or sorting methods. For example, 1, 9, 6, 5, 7, 2 can be sorted by ascending odd numbers and descending even numbers: 1, 5, 7, 9, 6, 2. Expressions can be characters A, B, C, ..., Z, and they can be sorted by ABC. Technicians know that characters are encoded; for example, code can place N-tuples (of N characters) into some sorted permutation. Note that the number of characters is not limited to 26 (like Latin characters), but there are thousands of available non-Latin characters. Sorting (e.g., Unicode character encoding) has been discussed in further detail in this description.

[0229] More specifically, for each of the R characteristic functions (see r = 1 to R, such as in...), Figure 4 In the description, matrix P is given. (and +), the preprocessor module 510 uses the first mathematical operator (e.g., OP1) This is used to process the D values ​​of each multi-valued tuple ([x1, x2, ..., xd, ... xD].n) and subsequent predefined dimension-specific values ​​to obtain D preliminary results. For example, for column n=1 in matrix X, the D=3 point cloud values ​​2, 1, and 1 are multiplied by 1, 1, and 1 respectively (see column r=1 in matrix P from d=1 to d=D). The preliminary results are 2, 2, and 1.

[0230] The preprocessor module 510 uses a second mathematical operator (e.g., OP2+) to aggregate D preliminary results into R intermediate N-tuples (221-1, 221-2, 221-3, 221-4, 221-5, where matrix S is an R×N matrix) containing expressions specific to each data point ([x].n), where each N-tuple has an expression for each feature function (an element of matrix S). Figure 4 In the example, the initial results will be aggregated by addition: 2 + 2 + 1 = 5 (see 5 at row r = 1, column d = 1 in matrix S / 220). There are R intermediate N-tuples (one tuple for each feature function, e.g., Figure 4 (R=5).

[0231] As already described, the previously trained machine learning tool 520 processes (step 640) the derived dataset 250 to identify the attribute data PROPERTY.

[0232] Although the transformation may increase the amount of data that the ML tool must process, the differences between data points are still preserved. The dimensionality of the point cloud dataset X is reduced (i.e., by a second mathematical operator). Using the first and second operators results in expressions that can be mapped to each other (due to the dimensionality change). The mapping reconciles intermediate results (N-tuples become Q-tuples), making the order in which points are organized in X irrelevant.

[0233] Because the dimensionality is being reduced while the distinctions are still maintained, intermediate results can be mapped (i.e., ordered). ML tools can handle the reconciled intermediate results. When the same (or similar) objects appear as point cloud data, the order of the point cloud dataset may be different (i.e., permuted), but the reconciled intermediate results will correspond to the data that the ML tool already knows.

[0234] Furthermore, the step transformation can be viewed as an initial mapping from the point cloud dataset X to the transformation in matrix S. A one-to-one relationship exists from X to S. The mapping step can be viewed as a subsequent mapping (within N-tuples).

[0235] exist Figure 4 In the example, matrix X has D×N = 3x3 elements (item 200), and the result of the transformation (step 630) is matrix S with R×N = 5x3 elements (item 220). At first glance, this might seem like increasing the amount of data (X has 9 elements, S has 15 elements, which preserves injectivity), but technically this preserves the differences between the data points. It is still possible to distinguish the individual data points (since the R components of each data point ([x].n) in matrix S provide distinction in different formats).

[0236] This description uses an example of a characteristic function with two mathematical operators: the first operator is multiplication (… The second operator is addition (+). The resulting expressions (integers in this example) can be mapped relative to each other, for example, by sorting (i.e., in the sense of sorting, arranging expressions in order according to a predefined criterion). The expressions are preserved. The expressions can be real numbers.

[0237] For example, the element s(2,1) of matrix S is calculated from the point cloud values ​​[x1, x2, ..., xD].1 with D = 3 dimensions (see column n = 1 in matrix X): the value x1 = 2 (for d = 1, n = 1) is processed by multiplication (multiplied by 2, d = 1, r = 2) to obtain 2. 2 (preliminary result with d = 1). The value x² = 2 (for d = 2, n = 1) is processed by multiplication (multiplied by -1, d = 2, r = 2) to obtain 2. (-1) (Preliminary result for d=2). The value x3=1 (for d=3, n=1) is processed by multiplication (multiplied by 1, d=3, r=2) to obtain 1. 1 (Preliminary result of d=4). The preliminary result (e.g., product) 4, -2, and 1 are aggregated (e.g., by summation + to obtain the expression): 2 2 - 2 + 1 = 3 (see matrix S, columns n = 1 and r = 2).

[0238] This description involves matrix multiplication, such as S = PX (i.e., S, P, and X are matrices with multiplication and addition, as described in the standard text book, see Bronstein, Algebra chapter). However, other operators can be used whenever the point cloud dataset is transformed.

[0239] The rows in matrix S' (items 231-1, 231-2, etc. after mapping) can be considered as feature expressions. Processing the 640 derived dataset (see the last step) to identify attribute data is crucial because the order of N data points [x].n in the point cloud dataset 200 is unknown. In other words, the aforementioned permutations no longer matter. Other examples of characteristic functions

[0240] As explained, by applying the transformation, the preprocessor module 510 derives a single expression with one dimension for each data point [x].n. An example of a characteristic function involving two steps (multiplication and addition) has been given in this description, where the values ​​are defined in the transformation matrix P (item 210).

[0241] Many other options exist for performing transformations, and this description continues with examples. For simplicity, examples are presented individually, but combinations are possible.

[0242] In Example 1, the values ​​in the multi-valued tuple [x1, x2, ..., xd, ..., xD].n are simply concatenated. Continuing with [x]1 as an example, the concatenation of 2, 2, and 1 yields the expression "221". For [x]2, the concatenation of 4, 0, and 2 yields the expression "402". With R = 1, only a single characteristic function will exist: the concatenated values.

[0243] In Example 2, another feature function (R=2) could be a concatenation of values ​​in reverse order: 122, 204, etc. This would result in a dataset with a two-row transformation (see matrix S): an expression for the concatenated numbers in the first row and an expression for the reverse concatenation of numbers in the second row.

[0244] In Example 3, the numbers can be replaced (first operation), and the initial results can be summed (for each of the D dimensions). For example, with [x]1 = [21, 23, 17].1, the numbers can be flipped to 12, 32, 71 (first operation), and the expression will be 12 + 32 + 71 = 115 (second operation).

[0245] In Example 4, the example is modified so that the second operation is multiplication (resulting in the expression "27264").

[0246] In Example 5, the expression can be a minimum value (e.g., MIN[x]1 = 1) or a maximum value (e.g., MAX[x]1 = 2), the MIN-MAX difference, or some other result that a computer can compute in a short time. There will be 3 characteristic functions, resulting in 3 expressions (MAX, MIN, and the difference).

[0247] In Example 6, the expression can correspond to the total length of a vector line drawn from the origin (i.e., 3D coordinates, Euclidean distance, etc.) to a point (see element) in the coordinate system.

[0248] In Example 7, the expression is the average or median value.

[0249] In Example 8, an expression can be obtained by manipulating the numerical representation. For example, the measurement value 9 in the decimal system will be written as the number "11" in the octal system (first operation). The two numbers (1 and 1) can be considered "decimal" and can be interpreted as decimal (in the second operation).

[0250] Although the example may appear to create random values, the transformation is deterministic. See also Figure 1 When another object appears in front of the sensor, point E1 at [2, 2, 1] will also be transformed. Data from similar objects will be converted to similar data. Since the transformation is unidirectional, there is no reverse transformation back to the original [x1, x2, ..., xD].n.

[0251] Another method of transformation is to assign a label or other identifier to specific data points. For example, see Figure 1 [2, 2, 1], [4, 0, 2], and [1, 2, 4] can be given specific alphanumeric names (in step 620). When the method is repeated, for example when [2, 2, 1] is displayed again (received at a later point in time in another example), the name is retrieved, and that name is used for the next step mapping 630 and processing 640.

[0252] There's a kind of analogy in astronomy: some well-known stars have names, while most only have numbers. For example, "Betelgeuse" and "Rigel" can be identified based on multidimensional data (not just position coordinates) but also spectral data. The first person might report seeing both Betelgeuse and Rigel, and the second person might report seeing them in a different order (permutation, i.e., seeing Rigel and Betelgeuse), but an ABC (permutation-invariant) order would result in "Betelgeuse, Rigel". For entities to be trained (e.g., ML tools), this coordinated order allows subsequent processing to identify objects as Orion with minimal training and real-time computational effort. This example may seem overly simplistic, but ML tools can handle alphanumeric names (strings, etc.) even if they are not human-readable. computer

[0253] Embodiments of the present invention can be implemented in digital electronic circuits, or in computer hardware, firmware, software, or combinations thereof. The invention can be implemented as a computer program product, such as a computer program tangibly embodied in an information carrier (e.g., a machine-readable storage device), for execution by or control of a data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). The claimed computer program can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for a computing environment. The computer program can be deployed to execute on one or more computers at a single site, or distributed across multiple sites and interconnected via a communication network. The described methods can all be executed by corresponding computer products on respective devices (e.g., first and second computers, trusted computers, and communication devices).

[0254] The method steps of this invention can be executed by one or more programmable processors that execute a computer program to perform the functions of this invention by manipulating input data and generating output. The method steps can also be executed by dedicated logic circuitry, and the apparatus of this invention can be implemented as dedicated logic circuitry, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0255] As an example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors in any kind of digital computing device. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic elements of a computer are at least one processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, receiving data from or transferring data to, or both to, one or more mass storage devices (e.g., hard disks, magneto-optical disks, optical disks, or solid-state drives) for storing data. Such storage devices can also be provided on demand and accessed via the Internet (e.g., cloud computing). Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; hard disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0256] To provide interaction with the user, the present invention can be implemented on a computer having a display device for displaying information to the user, such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, and an input device through which the user can provide input to the computer, such as a keyboard, touchscreen or touchpad, or indicating device, such as a mouse or trackball. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0257] This invention can be implemented in a computing system that includes backend components (e.g., as a data server), middleware components (e.g., an application server), or frontend components (e.g., a client computer having a graphical user interface or web browser through which a user can interact with an implementation of the invention), or any combination of such backend, middleware, or frontend components. The client computer can also be a mobile device, such as a smartphone, tablet PC, or any other handheld or wearable computing device. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet or wireless LANs or telecommunications networks.

[0258] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship is established through computer programs running on the respective computers and having a client-server relationship with each other. Reference Symbol List / / → | | Mapping [x1, x2, ..., xD].n data points d-dimensional index D represents the number of dimensions, the number of dimensions themselves, and thus the tuple elements for each data point. The partial use of dimensions D~, D1, and D2; for D1 it is also δ, and for D2 it is also Δ. n is the index of data points used to identify a specific data point. The number of data points in dataset N P transformation matrix Number of R characteristic functions S matrix t1, t2, tm time points [x].n data points X is a dataset written in set notation {}, used as the input matrix. 100 objects 100-A and 100-B are objects belonging to object 100. 100-1, 100-2 Different objects 200-point cloud dataset 210 Transformation Matrix 220. Transformed dataset, transformation matrix S 221-1, 221-r… Middle N-tuples 231-r derived Q-tuple, characteristic expression 230, 230' Derivation Matrix 250 derived datasets 400 Technical Equipment 410 sensor 500 computers 510 Preprocessor Module 520 Machine learning (ML) tools, such as neural networks 600, 6xx methods, method steps

Claims

1. A computer-implemented method (600) for identifying property data belonging to a physical object (100) associated with a technical device (400), the method (600) comprising: The preprocessor module (510) of the computer (500) receives (610) a point cloud dataset (200) from one or more sensors (410) associated with the technical device (400), the point cloud dataset (200) representing elements (E1, E2, E3) of the physical object (100), the point cloud dataset (200) X = {[x1, x2, ..., xD].1, [x1, x2, ..., xD].2, ..., [x1, x2, ..., xD].n, ..., [x1, x2, ..., xD].N} having N data points ([x].n), wherein each data point ([x].n) includes a multi-value tuple ([x1, x2, ..., xd, ...xD].n) with a number of D values, wherein the number D corresponds to the number of dimensions; The preprocessor module (510) of the computer (500) applies (620) transformation to the point cloud dataset (X, 200) according to the number of R feature functions to obtain the transformed dataset (220), wherein the transformation is a one-way transformation; The preprocessor module (510) maps the transformed dataset (220) to a derived dataset (230, 250) according to a predefined mapping rule, wherein the mapping (630) is permutation invariant; The derived dataset (230, 250) is processed (640) by a previously trained machine learning tool (520) to identify the property data (PROPERTY).

2. The method (600) according to claim 1, wherein, Each feature function has at least a first mathematical operator and a predefined dimension-specific value as a first operand, and has a second mathematical operator that produces expressions that can be mapped relative to each other; wherein, for each of the R feature functions (r = 1 to R), the preprocessor module (510) uses the first mathematical operator to process D values ​​of each multi-value tuple ([x1, x2, ..., xd, ... xD].n) and the predefined dimension-specific value to process D preliminary results, and then uses the second mathematical operator to aggregate the D preliminary results into the transformed dataset (220), the transformed dataset (220) having expressions in R intermediate N-tuples (221-1, 221-2, 221-3, 221-4, 221-5), wherein the expressions are specific to each data point ([x].n), and wherein each N-tuple has an expression for each feature function.

3. The method (600) according to any one of claims 1 or 2, wherein, Receiving (610) the point cloud dataset (200) includes receiving the point cloud dataset (200) as an input matrix X, the input matrix X being a D×N matrix, wherein applying (620) the transformation includes multiplying the transformation matrix P (210) as an R×D matrix with the input matrix X to obtain the transformed dataset (220) as a matrix S (220), the matrix S (220) being an R×N matrix.

4. The method (600) according to any one of claims 1 to 3, wherein, Receiving (610) the point cloud dataset (200) from one or more sensors (410) includes receiving the point cloud dataset (200) from a sensor arrangement having multiple channels, wherein the dimension corresponds to the multiple channels.

5. The method (600) according to claim 3, wherein, Mapping the transformed dataset (210) to the derived dataset (230, 250) is to map the matrix S (220) with the R intermediate N-tuples to a sorted matrix (S_sort) with derived Q-tuples, wherein the predefined mapping rule includes mapping the calculated function value of the intermediate N-tuple (221-r, / / ) with tuple elements to the feature expression (231-r, ||), such that intermediate N-tuples with the same number and type of tuple elements are mapped to the same feature expression, thereby processing (640) the derived dataset to identify that the attribute data for the N data points ([x].n) in the point cloud dataset (200) is unknown.

6. The method (600) according to any one of claims 3 to 5, wherein, The mapping (630) includes selectively sorting the elements of each row in the matrix S (220) in ascending or descending order.

7. The method (600) according to any one of claims 3 to 6, wherein, The mapping (630) includes calculating the offset and normalizing the elements of the intermediate N-tuple (221-r) accordingly.

8. The method (600) according to any one of claims 1 to 7, wherein, Processing (640) the derived dataset (230, 250) to identify the attribute data is performed by classification.

9. The method (600) according to any one of claims 1 to 8, wherein, The computer (500) also provides control commands to the technical device (400) based on the identified attribute data.

10. The method (600) according to any one of claims 1 to 9, wherein, The derived dataset (250) is processed in a cascaded manner by the previously trained machine learning tool (520).

11. The method (600) according to any one of claims 1 to 10, wherein, The steps of applying the transformation (620) and mapping (630) to the point cloud dataset (200) are performed separately for each dimension group after the data channels (D1, D2) from the one or more sensors (410) are grouped.

12. The method (600) according to any one of claims 1 to 11, wherein, The preprocessor module (510) receives (610) the point cloud dataset (200) with different data subsets, wherein the data subsets correspond to multiple parts (100-A, 100-B) of the object (100) that are different in time or space, and applies (620) the transformation to the different subsets respectively.

13. The method (600) according to any one of claims 1 to 12, wherein, The method identifies the attributes of the physical object (100) relative to the technical device (400), wherein the relationship and the preprocessing are selected from the following: The physical object (100) is moving relative to the technical device (400); The technical device (400) is moving relative to the physical object (100); and Both the technical device (400) and the physical object (100) are in motion.

14. The method (600) according to any one of claims 1 to 13, wherein, The processing (640) of the derived dataset (250) to identify the property data is performed by a machine learning tool (520) selected from the following: artificial neural networks, tree-based models, and support vector machines.

15. A method (600) implemented by a computer (500) according to any one of claims 1 to 14 for determining the properties of a physical object (100) relative to a technical device (400), wherein, The computer (500) is associated with the device (400) to control the technical device (400) based on determined attributes of the physical object (100).

16. A computer system (500) associated with a technical device (400), the computer system (500) being adapted to perform a computer-implemented method (600) according to any one of claims 1 to 14 to identify property data belonging to a physical object (100) having a relationship with the technical device (400), and to control the technical device (400) using the property data based on determined properties of the physical object (100).

17. A computer program product, when loaded into the memory of a computer and executed by at least one processor of the computer, causes the computer to perform the steps of the method according to any one of claims 1 to 14.

Images