Apparatus and method for predicting observations
Through the method of pivoting Cholesky decomposition, M input points are selected for pivoting index determination, which solves the problem of too long running time of Cholesky decomposition and achieves more efficient calculations.
Patent Information
- Application Number
- CN202510075000.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-01-19
- Filing Date
- 2025-01-17
- Publication Date
- 2025-07-22
AI Technical Summary
When predicting observation results in the prior art, the running time of Cholesky decomposition is proportional to the matrix dimension O(N3), resulting in inefficient computing efficiency.
The method of pivoting Cholesky decomposition is used to determine the approximation of Cholesky decomposition in the O(N) operation, and the pivoting index is determined by selecting M input points to reduce the running time.
It effectively reduces the running time of Cholesky decomposition from O(N3) to O(MN), and improves the computing efficiency.
Smart Images
Figure CN120355965A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an apparatus and method for predicting observation results. Background Art
[0002] Observation results can be predicted for control, classification, or regression tasks.
[0003] Predicting observation results may require numerically determining the Cholesky decomposition of a symmetric and positive definite N×N matrix. The running time for determining the Cholesky decomposition is proportional to the cube of the matrix dimension O(N 3 )
[0004] Applying pivoted Cholesky decomposition determines an approximation of the Cholesky decomposition in M iterations and reduces the running time to O(MN). Summary of the Invention
[0005] An apparatus and computer-implemented method according to independent claims further reduce the running time by determining an approximation of the Cholesky decomposition in O(N) operations.
[0006] A computer-implemented method for predicting observation results includes providing an input for predicting observation results and training data including pairs of the input and the observation results, where the input characterizes a technical system, particularly a robot, preferably an autonomous vehicle, a manufacturing machine, a power tool, a household appliance, a personal assistance system, a medical imaging device, where the observation results characterize the technical system or the environment of the technical system, and the method includes determining, for the input, a covariance matrix of the input; determining a pivoted Cholesky decomposition of the covariance matrix; and determining a prediction for the observation results based on the pivoted Cholesky decomposition of the covariance matrix, where determining the pivoted Cholesky decomposition includes determining a covariance matrix of selected inputs and a covariance matrix of remaining inputs, which represents the covariance matrix of the input; determining a pivot index based on a measure of the difference between the covariance matrix of the remaining inputs and an approximation of the covariance matrix of the remaining inputs.
[0007] The method may include determining a control signal or classification or regression for the technical system based on the predicted observation results, particularly characterizing the health state, wear, or fatigue of a material or component of the technical system, or classification, which is particularly an object in the environment of the technical system, preferably a traffic sign, a traffic participant, an infrastructure, a tool, a workpiece.
[0008] The method may include determining a classification and a control signal for the technical system based on the classification, or determining a regression and a control signal for the technical system based on the regression.
[0009] Providing training data can include providing at least one pair of an input and an observation of a control signal or a classification or regression observation for a technical system observed for the input, where the input is particularly captured by a sensor, preferably by a camera, lidar sensor, radar sensor, motion sensor, infrared sensor, ultrasonic sensor, speed sensor, acceleration sensor, yaw rate sensor, steering angle sensor, temperature sensor, voltage sensor, current sensor, power sensor.
[0010] Providing the input can include capturing the input, particularly by a sensor, preferably by a camera, lidar sensor, radar sensor, motion sensor, infrared sensor, ultrasonic sensor, speed sensor, acceleration sensor, yaw rate sensor, steering angle sensor, temperature sensor, voltage sensor, current sensor, power sensor.
[0011] Determining a pivot index can include, for an input of a covariance matrix for remaining inputs (particularly a row of the covariance matrix for remaining inputs), determining a respective sum of elements of the covariance matrix of the remaining inputs associated with the same input (particularly elements of the respective row of the covariance matrix of the remaining inputs); for an approximated input (particularly for an approximated row), determining a respective sum of elements of the approximation associated with the same input; and determining the pivot index based on an elementwise difference between the sums of elements (particularly rows having the same index in the covariance matrix of the remaining inputs and the approximation). The respective sums can be stored in respective vectors. The difference can be determined as a norm between the vectors. the approximated input (particularly for the approximated row), determining the approximation (particularly the approximated respective row) of elements; and according to the elements associated with the same input (particularly rows having the same index in the covariance matrix of the remaining inputs and the approximation). The pivot index can be determined based on the elementwise difference between the sums of elements (particularly rows having the same index in the covariance matrix of the remaining inputs and the approximation). The respective sums can be stored in respective vectors. The difference can be determined as a norm between the vectors.
[0012] Preferably, a pivoted Cholesky decomposition is determined in an iteration, where the sum of elements of the approximation is determined in the iteration, and where the sum of elements of the covariance matrix of the remaining inputs is determined once or not in all iterations. Determining the sum of elements of the covariance matrix once or not in all iterations further reduces the running time. In the first iteration, the sum of elements of the approximation is initialized to a predetermined value, particularly zero, where the first approximation is determined in the initial iteration, and where the sum of elements of the approximation is determined in the second iteration based on the first approximation.
[0013] In the first iteration, the sum of elements of the approximation is initialized to a predetermined value, particularly zero, where the first approximation is determined in the initial iteration, and where the sum of elements of the approximation is determined in the second iteration based on the first approximation. In the first iteration, the sum of elements of the approximation is initialized to a predetermined value, particularly zero, where the first approximation is determined in the initial iteration, and where the sum of elements of the approximation is determined in the second iteration based on the first approximation. the sum of elements of the approximation.
[0014] Weighted according to at least one of the examples, and the difference between the observation associated with the same input as the example and the prediction for the observation.
[0015] The apparatus for predicting an observation includes at least one processor and at least one memory, wherein the at least one processor is configured to execute instructions that, when executed by the at least one processor, cause the apparatus to perform the method, and wherein the at least one memory is configured to store the instructions.
[0016] The data structure includes data fields for the input for predicting an observation, data fields for the prediction of the observation, data fields for the training data of the pair including the input and the observation, data fields for the covariance matrix of the input, data fields for the covariance matrix of the selected input of the covariance matrix of the input, data fields for the covariance matrix of the remaining input of the covariance matrix of the input, and for the covariance matrix Data fields for the approximation, data fields for the pivot indices of the pivoted Cholesky decomposition of the covariance matrix, and for the difference between the covariance matrix of the remaining input and The approximation.
[0017] The data structure may include data fields for the respective sums of the elements of the covariance matrix of the remaining input associated with the same input. This enables the storage and reuse of the sums to reduce the running time. Description of the Drawings
[0018] Further embodiments can be derived from the following description and the drawings. In the drawings:
[0019] Figure 1 Schematically depicts an apparatus for processing observations,
[0020] Figure 2 Depicts a flowchart with the steps of a method for processing observations. Detailed Description
[0021] Figure 1 Schematically depicts an apparatus 100 for processing observations. In this example, the observations relate to a technical system 102.
[0022] The technical system 102 can be a robot, preferably an autonomous vehicle, a manufacturing machine, a power tool, a household appliance, a personal assistance system, a medical imaging device.
[0023] The apparatus 100 includes at least one processor 104.
[0024] The apparatus 100 includes at least one memory 106.
[0025] The device 100 includes at least one interface 108.
[0026] The interface 108 is configured to receive observation results.
[0027] The device 100 is configured to process the observation results.
[0028] According to an example, the device 100 is configured to determine a control signal for a technical system based on the observation results.
[0029] According to an example, the device 100 is configured to determine a classification of the observation results.
[0030] According to an example, the device 100 is configured to determine a regression of the observation results.
[0031] The interface 108 is configured to output control, classification, or regression.
[0032] According to an example, the device 100 includes at least one sensor 110, which is configured to capture the observation results.
[0033] The interface 108 may include at least one sensor 110. The interface 108 may be configured to receive the observation results from at least one sensor 110.
[0034] The observation results in this context may refer to an image of the technical system 102 (in particular, a digital image, a lidar image, a radar image, a motion sensor image, an infrared image, an ultrasonic image) or the environment of the technical system 102. The observation results in this context may refer to a physical quantity of the technical system 102, such as speed, acceleration, yaw rate, steering angle, temperature, voltage, current, power.
[0035] The at least one sensor 110 may include a camera, a lidar sensor, a radar sensor, a motion sensor, an infrared sensor, an ultrasonic sensor, a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, a temperature sensor, a voltage sensor, a current sensor, a power sensor.
[0036] The at least one processor 104 is configured to execute instructions that, when executed by the at least one processor 104, cause the device 100 to perform a method for processing the observation results. The at least one memory 106 includes at least one non-volatile memory. The at least one memory 106 is configured to store the instructions.
[0037] The at least one memory 106 is configured to store the observation results.
[0038] Figure 2 A flowchart depicting the steps of a method for processing observation results is shown.
[0039] The method is described by way of an example of adapting a function f that represents a Gaussian process (GP) for probabilistic modeling. The method adapts the function f based on observations.
[0040] In active learning for classification or regression (e.g., determining a control signal), the method may include prior knowledge about the function f and a known covariance structure to help guide the exploration.
[0041] In the example, a prior p(f) ∼ GP(0, k) is evaluated on a function with a stationary kernel k (which has non - negative covariance) at N points X. According to the example, the kernel k is an exponentiated quadratic kernel with automatic relevance determination: where θ and Λ represent the parameters of the kernel.
[0042] The method uses the joint distribution of input points x ∈ X, x * ∈ X * and observations y = f(x) to determine one or more predictions ŷ * for at least one input point x * = f * (x * )
[0043]
[0044] where N is the normal distribution, and K XX = k(X, X), is the covariance matrix that includes pairwise evaluations of k(·, ·) applied to the points in X and / or X * .
[0045] The predictive distribution for one or more predictions f̂ * is:
[0046] p(f̂ * , X * , X, f) = N(f̂ * | μ * , Σ * )
[0047]
[0048] If the observations y = f(x) are corrupted by independent and identically - distributed Gaussian noise , then by the conjugacy of the Gaussian distribution, the distribution of one or more predictions f̂ * is given by:
[0049] p(f̂ * | X * , X, y) = N(f̂ * | μ * , Σ* )
[0050]
[0051] where I is the identity matrix of appropriate dimension, and K y = K XX + σ 2 I is the Gram matrix, i.e., symmetric and positive definite.
[0052] The method is described by way of examples of observations corrupted by independent and identically distributed Gaussian noise.
[0053] The method includes step 202.
[0054] Step 202 includes providing a data set (X; Y), where X ∈ R N×D . The data set (X; Y) includes N pairs of input points X i = x i ∈ X and observations y i ∈ Y. The data set (X; Y) is an example of training data. In this notation, any dimensional indices of the input points X i = x i are ignored. As a shorthand notation, K ij = k(x i , x j ) = Cov[f i , f j = Cov[f(x i ), f(x j ), and K XX = Cov[f X , f X is used.
[0055] The method includes step 204.
[0056] Step 204 includes determining an approximation of the covariance matrix K i for the input point x XX ∈ X. Approximation.
[0057] To determine the distribution of one or more predictions f * , the covariance matrix K XX is approximated. In the example, the covariance matrix K XX is approximated using Approximation for approximation
[0058]
[0059] where and is the covariance matrix of the function f evaluated at each data point.
[0060] Points can be sampled uniformly from the set {1,..., N} to obtain an unbiased estimator. Points can be sampled with a biased sampling strategy (i.e., a biased estimator), especially for the statistic of interest.
[0061] The method includes step 206.
[0062] Step 206 includes determining the pivoted Cholesky decomposition of the covariance matrix K XX The pivoted Cholesky decomposition is determined in the Cholesky routine.
[0063] Step 206 includes defining a permuted data set where I represents the M < N indices of the input points that have been selected, and R represents the N - M indices of the remaining input points.
[0064] According to the example, for the pivoted Cholesky decomposition, especially in active learning, M << N.
[0065] After selecting M points, the resulting Cholesky factor is:
[0066]
[0067] After selecting M points, the following relationship is applied:
[0068]
[0069] where the matrix P is a permutation matrix implicitly constructed during the Cholesky routine.
[0070] This means that step 206 includes determining the covariance matrix K of the selected inputs II and the covariance matrix K of the remaining inputs RR , which represents the covariance matrix K of the inputs XX .
[0071] Using approximation, the relationship is
[0072]
[0073] This means that the covariance matrix K of the remaining input points RR is approximated with approximation as:
[0074] K RR = K RI (K II ) -1 K IR
[0075] In the Cholesky routine, the pivoted Cholesky factorization is determined numerically in an iteration, where in iteration m, a pivot index π is selected m ∈ m,..., N, and the corresponding rows and columns of the system are swapped
[0076] According to an example, instead of explicitly constructing the matrix P, the method can determine and use a vector of pivot indices π.
[0077] For example, the pivot index π is swapped in the vector. This corresponds to swapping the rows and columns of the matrix P
[0078] Index π m is determined using an acquisition function.
[0079] According to an example, a new pivot index π is determined in iteration m using an acquisition function m
[0080]
[0081] where K RR is the covariance matrix and K RI (K II ) -1 K IT is an approximation of the covariance matrix that depends on the covariance matrix K II where I includes the m indices π that were selected as pivot indices in previous iterations and where 1 m is a vector of 1s of dimension N. N
[0082] This means that step 206 includes determining the pivot index π based on a measure of the difference between the covariance matrix K RR of the remaining input and the covariance matrix K RR of the remaining input approximation K RR - K RI (K II ) -1 K IR m .
[0083] Projected covariance:
[0084] According to an example, is the first vector, and is the second vector. The first vector can be calculated before performing the method with a minimum viable product (MVP). The second vector can be initialized for the first iteration m = 0 of the method The initial second vector of the method can be
[0085] In iteration m, determine the second vector for the next iteration m + 1 For example, the second vector in iteration m
[0086] The projected covariance is an efficient approximation of the greedy A-optimal selection. Compared with the greedy A-optimal selection, the projected covariance reduces the computational cost, such as energy or running time, from O(N 2 M) to O(NM 2 ) + N 2 .
[0087] Weighted projected covariance:
[0088] According to the example, is the first vector, and is the second vector. The first vector and the second vector include the initial residual (y - μ X ) instead of the 1 N vector. The first vector can be calculated before performing the method with a minimum viable product (MVP).
[0089] According to the example, a greedy least squares selector is used, such as matching pursuit or forward regression. The weighted projected covariance and using the greedy least squares selector result in a reduction in running time, which is similar to the reduction achieved with the projected covariance.
[0090] The iteration is continued, for example, until a predetermined amount of iterations m is reached or the last index π is collected by the acquisition function.
[0091] This means that determining the pivot index includes: for the input, determining the corresponding sum of the elements of the covariance matrix associated with the same input and the corresponding sum of the elements of the approximate, and determining the pivot index based on the element-wise difference between the covariance matrix and the sum of the elements associated with the same input in the approximation.
[0092] In the example, the elements of the covariance matrix associated with the same input are the elements of the corresponding row of the covariance matrix.
[0093] In the example, the elements of the approximation associated with the same input are the elements of the corresponding row of the approximation.
[0094] In the example, determining the covariance matrix and The element-wise difference between the sums of elements associated with rows having the same index in the approximation.
[0095] Column indices can be used to determine the covariance matrix and the sums and differences of elements having the same column index in the approximation, rather than using row indices.
[0096] This means that in the weighted projection covariance, at least one of the sums is weighted by the difference between the observations associated with the same input as that sum and the predictions for the observations.
[0097] The method includes step 208.
[0098] In step 208, one or more predictions f are determined * .
[0099] Linear system:
[0100] For modeling a linear system with a GP, the prediction f * represents the GP. According to a linear system of N equations, the prediction f * is fit to the observations x i , y i by
[0101]
[0102] This is done by forward substitution and solving
[0103]
[0104] and back substitution and solving
[0105]
[0106] where is the Cholesky factor.
[0107] According to the example, in the iteration, the prediction f is numerically fit to the * by using a numerical solver According to the example, the Cholesky factor
[0108]
[0109] which is available from step 206 after M input points have been selected in step 206, is used as the initial Cholesky factor for the solver.
[0110] Sparse modeling:
[0111] According to the example, the prediction f * represents new points.
[0112] For example, in step 206, M input points x have been selected. i After that, based on the intermediate quantity the selected input points x i , i ∈ I, the new point f * for the new input point x * (x * ) is determined as
[0113]
[0114] According to the example, the intermediate quantity α I is based on the least squares approximator of the output points
[0115]
[0116] According to the example, the intermediate quantity α I is based on the regularized least squares approximator of the output points
[0117]
[0118] where I M represents the identity matrix of the matching size.
[0119] The method includes step 210.
[0120] Step 210 may include outputting one or more predictions f * .
[0121] The method in the example predicts one or more predictions y * for at least one input point x * = f * (x * ).
[0122] Step 210 may include determining a control signal for the technical system 102 based on one or more predictions y * .
[0123] Step 210 may include determining a classification or regression characterizing the technical system 102 based on one or more predictions y * . The classification or regression may be the health state, wear or fatigue of the materials or components in the technical system 102 or the environment of the technical system 102.
[0124] Step 210 may include determining a classification of at least a part of the environment of the technical system 102 based on one or more predictions y * .
[0125] Step 210 may include determining, based on one or more predictions y * the classification of an object, preferably a traffic sign, a traffic participant, an infrastructure, in particular a tool of the technical system 102, in particular an artifact in the environment of the technical system 102.
[0126] Step 210 may include determining a control signal for the technical system 102 based on classification or regression.
[0127] According to an example, the input is an image of the technical system 102 or the environment of the technical system 102.
[0128] According to an example, the technical system 102 is a robot.
[0129] According to an example, the image includes an object.
[0130] According to an example, the object is a tool of the technical system 102 or an artifact in the environment of the technical system 102.
[0131] According to an example, the prediction is the health state, wear or fatigue of an object in the image.
[0132] According to an example, the technical system 102 is configured to use or process an object. According to an example, the technical system 102 prevents the use of the object or the processing of the object based on the prediction. For example, if the health state, wear or fatigue of the artifact indicates that the tool is unavailable, the technical system 102 is controlled to stop using the tool.
[0133] For example, if the health state, wear or fatigue of the artifact indicates that the processing of the artifact is completed, the technical system 102 is controlled to stop processing the artifact.
[0134] According to an example, the object is a traffic sign, a traffic participant or an infrastructure.
[0135] According to an example, the prediction is the class of the object, such as the type of traffic sign or traffic participant or infrastructure. The type of traffic sign may indicate stop, give way to traffic, or have priority in traffic.
[0136] According to an example, the prediction is the trajectory of the object or the technical system 102.
[0137] According to an example, the technical system 102 is controlled to start, continue or stop moving according to the class of the object. According to an example, if the class of the object indicates stop or give way to traffic, the technical system 102 is controlled to stop. According to an example, if the class of the object indicates having priority in traffic, the technical system 102 is controlled to continue moving.
[0138] According to an example, based on the trajectory of the object or the technical system 102, the technical system 102 is controlled to start, continue or stop moving.
[0139] For example, in the case where the detected trajectory of the object causes a collision with the technical system 102 or the trajectory of the technical system 102 causes a collision with the object, the movement of the technical system 102 along the trajectory stops or continues on a different trajectory.
[0140] The method may include determining a corresponding control signal for the technical system 102 based on classification or regression.
Claims
1. A computer-implemented method for predicting an observation result, characterized in that, The method includes providing (202) an input for predicting an observation, and training data including pairs of the input and the observation, wherein the input characterizes a technical system (102), in particular a robot, preferably an autonomous vehicle, a manufacturing machine, a power tool, a household appliance, a personal assistance system, a medical imaging device, wherein the observation characterizes the technical system (102) or the environment of the technical system (102), wherein the method includes determining (204) the covariance matrix of the input for the input; determining (206) the pivoted Cholesky decomposition of the covariance matrix; and determining (208) a prediction for the observation according to the pivoted Cholesky decomposition of the covariance matrix, wherein determining (206) the pivoted Cholesky decomposition includes determining the covariance matrix of the selected input and the covariance matrix of the remaining input, which represents the covariance matrix of the input; according to the difference between the covariance matrix of the remaining input and the approximation of the covariance matrix of the remaining input to determine a pivot index by a measure of the difference.
2. The method according to claim 1, wherein The method includes determining (210) a control signal or classification or regression for the technical system (102) based on the predicted observations, in particular characterizing the health state, wear or fatigue of the materials or components of the technical system (102), or classification, in particular of an object in the environment of the technical system (102), preferably a traffic sign, a traffic participant, infrastructure, a tool, a workpiece.
3. The method according to claim 2, characterized in that, The method includes determining (210) the classification and a control signal for the technical system (102) based on the classification, or determining (210) the regression and a control signal for the technical system (102) based on the regression.
4. The method according to any one of the preceding claims, characterized in that, Providing (202) the training data includes providing at least one pair of an input and an observation of a control signal or classification or regression for the technical system (102) observed for the input, the input being captured in particular by a sensor (10), preferably by a camera, a lidar sensor, a radar sensor, a motion sensor, an infrared sensor, an ultrasonic sensor, a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, a temperature sensor, a voltage sensor, a current sensor, a power sensor.
5. The method according to any one of the preceding claims, characterized in that, Providing (202) the input includes capturing the input, in particular by a sensor (110), preferably by a camera, a lidar sensor, a radar sensor, a motion sensor, an infrared sensor, an ultrasonic sensor, a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, a temperature sensor, a voltage sensor, a current sensor, a power sensor.
6. The method according to any one of the preceding claims, characterized in that, Determine (206) that the pivot index includes inputs to the covariance matrix for the remaining inputs, in particular rows of the covariance matrix for the remaining inputs, determine the corresponding sums of the elements of the covariance matrix of the remaining inputs associated with the same input, in particular the elements of the corresponding rows of the covariance matrix of the remaining inputs; for the approximate inputs, in particular for approximate rows, determine the corresponding sums of the elements of the approximate corresponding rows associated with the same input; and determine the pivot index based on the element-wise difference between the elements associated with the same input, in particular the sums of the elements of the rows with the same index in the covariance matrix of the remaining inputs and the approximate.
7. The method according to claim 6, wherein Determine the pivoted Cholesky factorization in an iteration, wherein the sum of the elements of the approximation is determined in the iteration, and the sum of the elements of the covariance matrix of the remaining inputs is determined in one or not all iterations. 8. The method according to claim 7, wherein In the first iteration, the sum of approximate elements is initialized to a predetermined value, specifically zero, where in the initial iteration a first approximation is determined, and where in the second iteration a sum of approximate elements is determined based on the first approximation.
9. The method according to any one of claims 6 to 8, characterized in that At least one of the sums is weighted based on the difference between the observation associated with the same input as the sum and the prediction for the observation.
10. An apparatus (100) for predicting an observation result, characterized in that, The device (100) includes at least one processor (104) and at least one memory (106), wherein the at least one processor (104) is configured to execute instructions which, when executed by the at least one processor (104), cause the device (100) to perform the method according to any one of claims 1 to 9, and wherein the at least one memory (106) is configured to store the instructions.
11. A data structure, characterized in that, The data structure includes data fields for input data used to predict an observation result, data fields for a prediction of the observation result, data fields for training data for pairs including the input and the observation result, data fields for a covariance matrix of the input, data fields for a covariance matrix of selected inputs of the covariance matrix of the input, data fields for a covariance matrix of remaining inputs of the covariance matrix of the input, and for the covariance matrix of the remaining inputs data fields for an approximation, data fields for pivot indices for a pivoted Cholesky factorization of the covariance matrix, and for a difference between the covariance matrix of the remaining inputs and the approximation.
12. The data structure according to claim 11, wherein The data structure includes, for the input, a data field for the respective sum of the elements of the covariance matrix of the remaining inputs associated with the same input.