Apparatus and computer-implemented methods for determining multi-dimensional quantity, for operating technical system and for providing quantity prediction system

Abstract

A method for determining a multi-dimensional quantity characterizing an object in a technical system or an environment of the technical system, comprising: providing at least one quantity characterizing the technical system or the environment of the technical system; predicting a first prediction of the multi-dimensional quantity depending on the at least one quantity, providing a multi-dimensional parameter defining a prediction interval, determining a second prediction of the multi-dimensional quantity depending on the first prediction of the multi-dimensional quantity and the multi-dimensional parameter, wherein providing the multi-dimensional parameter comprises providing a multi-dimensional prediction and a multi-dimensional reference of a plurality of pairs, providing a plurality of scores respectively associated with a dimension of the multi-dimensional quantity and a pair of the plurality of pairs, determining a plurality of rankings of the scores respectively associated with the dimension of the multi-dimensional quantity and the pair of the plurality of pairs depending on a magnitude of the scores of the plurality of scores, a ranking in the set of rankings is selected, a score in the set of scores is selected depending on the selected ranking, and a value of the multi-dimensional parameter in the same dimension is determined depending on the selected score.

Description

Technical Field

[0001] The invention relates to an apparatus and a computer-implemented method for determining multidimensional quantities, for operating a technical system and for providing a quantity forecasting system. Summary of the invention

[0002] A computer-implemented method for determining a multidimensional quantity characterizing a technical system or an object in an environment of a technical system, comprising: providing at least one quantity characterizing a technical system or an environment of a technical system, predicting a first prediction of the multidimensional quantity depending on the at least one quantity, providing a multidimensional parameter defining a prediction interval; determining a second prediction of the multidimensional quantity depending on the first prediction of the multidimensional quantity and the multidimensional parameter, wherein the second multidimensional prediction determines the multidimensional quantity, wherein providing the multidimensional parameter comprises providing a plurality of pairs of predictions of the multidimensional quantity and references to the multidimensional quantity, providing a plurality of scores respectively associated with dimensions of the multidimensional quantity and with one of the plurality of pairs, wherein the second predictions of the multidimensional quantity are associated with the dimensions of the multidimensional quantity, and wherein the second predictions of the multidimensional quantity are associated with the multidimensional quantity. The score of the association indicates the degree of conformity of the predicted multidimensional quantity of the pair with the reference multidimensional quantity of the pair at least in the dimension associated with the score, wherein the multiple scores include a score set, wherein the scores in the score set are associated with the same dimension, multiple rankings of the scores associated with the dimensions of the multidimensional quantity and one of the multiple pairs are determined depending on the magnitude of the scores in the multiple scores, a ranking in the ranking set is selected, wherein the ranking set includes rankings, in particular the maximum ranking, associated with different dimensions of the multidimensional quantity in the multiple rankings, a score in the score set is selected depending on the selected ranking, and a value of a multidimensional parameter on the same dimension is determined depending on the selected score. The first prediction of the multidimensional quantity is changed by the multidimensional parameter defining a prediction interval. The second prediction of the multidimensional quantity defines a prediction interval containing the true multidimensional quantity. The ranking takes into account correlation information between the dimensions of the multidimensional quantity.

[0003] Selecting the ranking may include determining a cut point for partitioning the ranking set into a first ranking set and a second ranking set based on the magnitude of the ranking, wherein the first ranking set includes a predetermined quantile of the ranking, wherein the cut point defines the selected ranking, and / or wherein selecting the score includes determining a cut point for partitioning the score set into a first score set and a second score set based on the magnitude of the score, wherein the first score set includes a predetermined quantile of the score, wherein the cut point for partitioning the respective score set is a score in the respective score set having the selected ranking. The predetermined quantile defines the guarantee established. Thus, the method provides a guarantee corresponding to the predetermined quantile. The score provides a measure of uncertainty associated with the prediction of the multidimensional quantity.

[0004] The method may include determining, for the plurality of pairs, a corresponding distance between the predicted multidimensional quantity and the reference multidimensional quantity in a pair, and determining a score dependent on the corresponding distance.

[0005] Determining a second prediction of a multidimensional quantity may include determining values ​​on a dimension of the second prediction of the multidimensional quantity, wherein determining the values ​​on a dimension of the second prediction of the multidimensional quantity includes: adding or subtracting the value of the parameter on the dimension with the value of the dimension of the first prediction of the multidimensional quantity corresponding to the second prediction of the multidimensional quantity and the dimension of the parameter.

[0006] According to an example, the multidimensional quantity and the reference to the multidimensional quantity characterize a technical system or an object in the environment of a technical system, in particular a physical quantity of the technical system, or wear or expected life.

[0007] A computer-implemented method for operating a technical system, in particular a computer-controlled machine, preferably a robot, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, an access control system, or a system for transmitting information, in particular a monitoring system or a medical imaging system, the method comprising determining a physical quantity characterizing the technical system, in particular a multidimensional quantity characterizing the technical system, or a wear or life expectancy, by means of a method according to one of the preceding claims, and operating the technical system in dependence on the multidimensional quantity determined by a second multidimensional prediction, in particular granting access, transmitting information about the system, or transmitting information about the multidimensional quantity, or stopping the technical system or operating the technical system in a safe mode upon detecting that the multidimensional quantity is within or outside a predetermined tolerance, or moving at least a part of the technical system in dependence on the multidimensional quantity. The method operates the technical system taking into account the correlation information between the dimensions and in particular operating the technical system in dependence on a guarantee established based on a predetermined quantile.

[0008] A method for operating a technical system may include determining a value on a dimension of a first prediction of a multidimensional quantity, and adding the value of a parameter on a dimension corresponding to a dimension of a second multidimensional prediction to the value on the dimension of the first prediction of the multidimensional quantity corresponding to the dimension to define a first boundary, and / or subtracting the parameter from the value on the dimension of the first multidimensional prediction to define a second boundary, and operating at least a portion of the technical system within the first boundary, within the second boundary, or within both boundaries.

[0009] A computer-implemented method for providing a quantity prediction system, which is used to predict multidimensional quantities that characterize a technical system or an object in the environment of the technical system, in particular a physical quantity that characterizes the technical system, or wear or expected life. The method may include: providing multiple pairs of multidimensional quantity predictions and multidimensional quantity references, providing multiple scores associated with the dimensions of the multidimensional quantity and one of the multiple pairs, respectively, wherein the scores associated with the dimensions of the multidimensional quantity indicate the degree of conformity of the predicted multidimensional quantity of the pair with the reference multidimensional quantity of the pair at least in the dimensions associated with the scores, wherein the multiple scores include a score set, wherein the scores in the score set are associated with the same dimension, determining multiple rankings of the scores associated with the dimensions of the multidimensional quantity and one of the multiple pairs respectively depending on the magnitude of the scores in the multiple scores, selecting a ranking in the ranking set, wherein the ranking set includes rankings associated with different dimensions of the multidimensional quantity in the multiple rankings, in particular the maximum ranking, selecting a score in the score set depending on the selected ranking, and determining the value of a multidimensional parameter that defines a prediction interval of the quantity prediction system on the same dimension depending on the selected score, and a vector prediction system provides the multidimensional parameter. The provided volume prediction system takes into account the correlation information between dimensions and in particular provides guarantees established based on predetermined quantiles.

[0010] In a method for providing a quantity prediction system, selecting a ranking may include determining a cutoff point for dividing a ranking set into a first ranking set and a second ranking set based on a magnitude of the ranking, wherein the first ranking set includes a predetermined quantile of the ranking, wherein the cutoff point defines the selected ranking, and / or wherein selecting a score includes determining a cutoff point for dividing a score set into a first score set and a second score set based on a magnitude of the score, wherein the first score set includes a predetermined quantile of the score, wherein the cutoff point for dividing the corresponding score set is a score in the corresponding score set having the selected ranking.

[0011] The method for providing a quantity prediction system may include determining, for the plurality of pairs, a corresponding distance between the predicted multidimensional quantity and the reference multidimensional quantity in a pair, and determining a score depending on the corresponding distance.

[0012] A device comprises at least one processor and at least one memory, wherein the at least one processor is configured to execute instructions which, when executed by the at least one processor, cause the device to perform a method for determining a multidimensional quantity, for operating a technical system, or for providing a quantity prediction system, wherein the at least one memory is configured to store the instructions.

[0013] A computer program, characterized in that the computer program comprises computer-readable instructions which, when executed by a computer, cause the computer to execute a method for determining a multidimensional quantity, for operating a technical system or for providing a quantity prediction system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Further advantageous embodiments emerge from the following description and the accompanying drawings. In the drawings: Figure 1 The device is schematically depicted, Figure 2 depicts a flowchart comprising the steps of a method for providing a volume forecasting system, Figure 3 depicts a flow chart comprising the steps of a method for determining a multidimensional quantity, Figure 4 The results of determining the multidimensional quantity are schematically depicted, Figure 5 A flow chart comprising the steps of a method for operating a technical system is depicted. DETAILED DESCRIPTION

[0015] Figure 1 The device 100 is schematically depicted. The device 100 comprises at least one processor 102 and at least one memory 104.

[0016] According to an exemplary embodiment of the device 100, the device 100 is configured to operate a technical system 106. The technical system 106 is a physical system.

[0017] According to an exemplary embodiment of the device 100, the device 100 is configured to capture at least one quantity, in particular a digital image 108, characterizing the technical system 106 or the environment of the technical system 106, or to read at least one quantity, in particular a digital image 108, characterizing the technical system 106 or the environment of the technical system 106 from at least one memory 104. According to the example, the at least one quantity characterizes a physical quantity that influences the wear or life expectancy of the technical system 106. According to the example, the digital image 108 comprises pixels.

[0018] The technical system 106 may be a computer-controlled machine, preferably a robot, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, an access control system, or a system for transmitting information, in particular a surveillance system or a medical imaging system.

[0019] According to an example, the device 100 includes a sensor 116. The sensor 116 may be configured to capture at least one quantity characterizing the technical system 106 or the environment of the technical system 106. The sensor 116 may be a camera, a radar sensor, a lidar sensor, a thermal imaging sensor, an ultrasonic sensor, an infrared sensor, or a motion sensor. In this example, the sensor 116 is configured to capture the digital image 108.

[0020] According to the example, the device 100 includes an actuator 118, which is configured to move the technical system 106 or a part of the technical system 106, or stop the technical system 106 or a part of the technical system 106, or operate the technical system 106 in a safe mode when a multidimensional quantity (in particular, a position) is detected to be within or outside a predetermined area.

[0021] According to an example, the device 100 is configured for determining the multidimensional quantity depending on at least one quantity characterizing the technical system 106 or an environment of the technical system 106 .

[0022] According to an example, the device 100 is configured for determining the position of the object depending on the digital image 108. According to an example, the device 100 is configured for providing a quantity prediction system capable of detecting a multidimensional quantity, in particular the position of the object, at least depending on at least one quantity characterizing the technical system 106 or the environment of the technical system 106, in particular the digital image 108. According to an example, the device 100 is configured to operate the technical system 106 using the quantity prediction system depending on the multidimensional quantity determined by the quantity prediction system, in particular on the position of the object determined by the object detection system.

[0023] Device 100 may be configured to operate technical system 106 to grant access, to communicate information about an object, or to communicate information about a multi-dimensional quantity, particularly location.

[0024] The device 100 may be configured to move at least a part of the technical system 106 as a function of a multidimensional variable, in particular as a function of the position of an object, or in order to avoid the position of an object.

[0025] At least one processor 102 can be configured to execute instructions which, when executed by at least one processor 102, cause the device 100 to perform a method for providing a quantity prediction system, or a method for determining a multidimensional quantity, in particular an object position, or a method for operating the technical system 106.

[0026] The at least one memory 104 is configured to store instructions.

[0027] The multidimensional quantity characterizes, for example, an object in the environment of the technical system 106 , in particular an object in the real-world environment, a physical quantity of the technical system 106 , or wear or expected life.

[0028] An example of a multi-dimensional quantity characterizing an object in the environment of the technical system 106 is a bounding box.

[0029] The device 100 is, for example, configured to predict a first bounding box 110 identifying a first set of pixels of the digital image 108. The device 100 is configured to predict the first bounding box 110 depending on the pixels of the digital image 108.

[0030] The device 100 is, for example, configured to predict a second bounding box 112 identifying a second set of pixels of the digital image 108. The device 100 is configured to predict the second bounding box 112 dependent on the first bounding box 110.

[0031] The second bounding box 112 defines a prediction interval for the location of the object. The second bounding box 112 defines the location of the object.

[0032] The device 100 may be configured to predict a third bounding box 114 identifying a third set of pixels of the digital image. The device 100 may be configured to predict the third bounding box 114 depending on the first bounding box 110. The third bounding box 114 defines a prediction interval for the location of the object. The third bounding box 114 defines the location of the object.

[0033] Preferably, the second bounding box 112 defines one end point of the prediction interval and the third bounding box 114 defines one end point of the prediction interval. The location of the object may be defined as being in the prediction interval or at one of the end points.

[0034] At least one dimension of second bounding box 112 and / or third bounding box 114 may be greater than a corresponding dimension of first bounding box 110. At least one dimension of second bounding box 112 and / or third bounding box 114 may be smaller than a corresponding dimension of first bounding box 110. Second bounding box 112 and / or third bounding box 114 may overlap with first bounding box 112. First bounding box 110 may contain second bounding box 112 and / or third bounding box 114. Second bounding box 112 and / or third bounding box 114 may contain the first bounding box. Second bounding box 112 may contain third bounding box 114. Third bounding box 114 may contain second bounding box 112. Third bounding box 114 may overlap with second bounding box 112. At least one dimension of third bounding box 114 may be greater than the second bounding box. At least one dimension of third bounding box 114 may be smaller than the second bounding box.

[0035] The bounding box in this example has a rectangular shape. Other shapes can be used, such as an ellipse or a circle. The bounding box in this example has a width and a height. According to this example, the two-dimensional coordinates define the location of a pixel in the digital image. According to this example, the pixel coordinates of the lower left pixel and the upper right pixel of the corresponding bounding box define the location of the corresponding bounding box. The set of pixels identified by the bounding box are the pixels that are within the location of the corresponding bounding box according to their coordinates.

[0036] The two-dimensional coordinates of the lower left pixel and the upper right pixel include a value of a first dimension of the two-dimensional coordinate and a value of a second dimension of the two-dimensional coordinate. Device 100 is, for example, configured to determine a value of a first dimension of a lower left pixel of second bounding box 112 and / or third bounding box 114 depending on the value of the first dimension of the lower left pixel of the first bounding box. Device 100 is, for example, configured to determine a value of a first dimension of an upper right pixel of second bounding box 112 and / or third bounding box 114 depending on the value of the first dimension of an upper right pixel of the first bounding box.

[0037] The device 100 is, for example, configured to determine the value of the second dimension of the lower left pixel or the upper right pixel of the second bounding box 112 and / or the third bounding box 114 as described for the corresponding value of the first dimension. The device 100 is, for example, configured to determine the value of the second dimension of the second bounding box 112 and / or the third bounding box 114 as described for the value of the first dimension.

[0038] The bounding box can be defined by other means, for example, by the coordinates of different corners, or the coordinates of a corner and the length or direction of a diagonal, or the length of two sides starting from the corner. The bounding box of a circle can be defined by the coordinates of the center of the circle and the value of the radius or diameter of the circle. The bounding box of an ellipse can be defined by the coordinates of the center of the ellipse and the values ​​of the lengths of the semi-major axis and the semi-minor axis.

[0039] According to an example, the device 100 is configured to add the dimension values ​​of the multi-dimensional parameter to the corresponding values ​​of the first bounding box 110 so as to determine the corresponding values ​​of the second bounding box 112. According to an example, the device 100 is configured to subtract the dimension values ​​of the multi-dimensional parameter from the corresponding values ​​of the first bounding box 110 so as to determine the corresponding values ​​of the third bounding box 114.

[0040] This means that the multidimensional parameters define prediction intervals.

[0041] The bounding box may be defined by a three-dimensional value, namely a two-dimensional value and a depth value of the corresponding bounding box.

[0042] The device 100 may be configured to multiply the corresponding value by the multi-dimensional parameter, or divide the corresponding value by the multi-dimensional parameter, so as to determine the corresponding values ​​of the second bounding box 110 and the third bounding box 114, respectively.

[0043] The device 100 may be configured to provide category-specific prediction intervals.

[0044] The multidimensional parameter may be associated with a category from a plurality of object categories. According to an example, the device 100 is configured to select a multidimensional parameter associated with a category predicted for the object from a plurality of multidimensional parameters defining respective prediction intervals and associated with respective categories from the plurality of object categories.

[0045] According to an example, the device 100 is configured to detect an object class from a plurality of object classes of an object. The device 100 is for example configured to predict a class from a plurality of object classes.

[0046] In inference, the digital image 108 may be a previously unseen digital image. In training, the digital image 108 may be a digital input image from a dataset.

[0047] In the dataset, a digital input image and a reference bounding box of at least one object depicted in the digital input image are provided and associated with a class of the object from a plurality of object classes of objects.

[0048] In this example, the reference bounding box is defined in the same manner as the bounding box that device 100 is configured to predict.

[0049] In this example, the reference bounding box identifies a subset of pixels of the digital image as the ground truth location of the object.

[0050] The reference bounding box is associated with a ground truth category of the object from multiple object categories.

[0051] Figure 2 A flow chart including steps for providing a method for a quantity prediction system is depicted. The method is described by the example of a rectangular digital image and a rectangular bounding box. The digital image and the bounding box can have different shapes. The method for providing a quantity prediction system is explained by the example of an object detection system, i.e., a quantity prediction system configured to predict a multi-dimensional quantity of an object in an environment of the characterization technology system 106.

[0052] The method is equally applicable to providing a quantity prediction system for predicting a multidimensional quantity which characterizes the technical system 106 , in particular a physical quantity which characterizes the technical system 106 , or a wear or expected life.

[0053] The volume forecasting system includes the volume forecasting model According to this example, the object prediction system includes an object prediction model

[0054] Volume Forecast Model Or object prediction model It can be a pre-trained neural network model. It can be configured to receive as input a physical quantity of the technical system 106, or wear or expected life, and output a mean prediction of the dimensional values ​​of the multi-dimensional quantity. For example, it is configured to receive a digital image x iAs input, it outputs the mean prediction of the lower left corner coordinates u0, v0 and the upper right corner coordinates u1, v1 of the bounding box, and the category c of the object included in the bounding box i According to the example, the digital image x is identified in the Cartesian coordinate system i The pixel position and digital image x i The position of the bounding box in , i.e., the coordinates.

[0055] A bounding box is an example of a multidimensional quantity. Coordinates u0, v0, u1, v1 are examples of dimensions of a multidimensional quantity.

[0056] The bounding box can be defined by other means, for example, by the coordinates of different corners, or by the coordinates of a corner and the length or direction of a diagonal, or the lengths of two sides starting from that corner. The bounding box of a circle can be defined by the coordinates of the center of the circle and the value of the radius or diameter of the circle. The bounding box of an ellipse can be defined by the coordinates of the center of the ellipse and the values ​​of the lengths of the semi-major and semi-minor axes.

[0057] Object Detection Model Can be configured to output the corresponding mean.

[0058] Volume Forecast Model or object detection models A trained quantile regression head may be included, the quantile regression head configured to produce quantile predictions.

[0059] The method includes step 202 .

[0060] Step 202 may include loading a dataset D or accessing a preloaded dataset D. The dataset includes at least one quantity characterizing the technical system 106 or the environment of the technical system 106 associated with corresponding ground truth values ​​of multidimensional quantities characterizing the technical system 106 or objects in the environment of the technical system 106 .

[0061] Step 202 comprises providing an input, in particular a digital input image x from a dataset D i .

[0062] According to an example, the method comprises providing a different input, for example a digital input image.

[0063] In this example, the dataset D is split into calibration data D cal ={(x i , Y i )} i＝1，...，n and test data

[0064] Calibration data D cal ={(x i , Y i )}i＝1，...，n It includes n pairs of at least one quantity and the corresponding ground truth for the multidimensional quantity, for example, n pairs of digital images x i And for the digital image x i The ground truth instances Y of the k reference bounding boxes of the k objects depicted in i ={(u0, v0, u1, v1; c k ) k} i . Test data Include t At least one quantity of and the corresponding ground truth for multidimensional quantities, e.g., n t The digital image x j And for the digital image x j The ground truth instances Yj = {(u0, v0, u1, v1; c l ) l} j .

[0065] According to this example, the method comprises for at least one category c from a predetermined plurality of object categories C o Perform class conditional conformalization. This is done below for an example class c o Preferably, as for the exemplary category c o As described, class-conditional conformalization is performed for a plurality of classes in a predetermined plurality of object classes C.

[0066] According to an example, the method comprises a class-agnostic conformalization. This means that the multi-dimensional parameters are the same for classes from a predetermined plurality of object classes C.

[0067] The method may include step 204 .

[0068] Step 204 includes providing a category c from a predetermined plurality of object categories. o The processed category c can be selected randomly or successively from a predetermined plurality of object categories C. o .

[0069] The method may include step 206 .

[0070] Step 206 includes identifying and collecting o Associated calibration data D cal The input (x i , Y i )’s category associated ground truth instances.

[0071] The method includes step 208 .

[0072] Step 208 includes: for the calibration data D cal Ground truth examples of the amount of use prediction model Determine a predicted multidimensional quantity that characterizes the technical system 106 or the environment of the technical system 106. The predicted multidimensional quantity is derived from the quantity prediction model Predicted from at least one physical quantity characterizing the technical system 106 , or wear or expected lifetime.

[0073] Step 208 includes determining a predicted bounding box for the collected ground truth instances, for example The predicted bounding box is generated by the object detection model From a digital input image x i This means that step 208 includes depending on the digital input image x i The pixels of the predicted digit input image x i The predicted bounding box of the first set of pixels

[0074] Using categories is optional. According to the example, the method does not use any categories. This means that the ground truth instance does not require any category, and steps 204 to 208 are not necessary.

[0075] Step 208 includes providing multiple pairs of predicted multidimensional quantities and reference multidimensional quantities, for example, predicted bounding boxes and reference bounding boxes b o This is done below for the calibration data D cal The digital input image x i Category c in o The reference bounding box of the object b o =(u0, v0, u1, v1) o，i and predicted bounding box An exemplary pair of o to describe.

[0076] Predicting bounding boxes and the reference bounding box b o P o are examples of pairs of corresponding prediction multidimensional quantities and corresponding reference multidimensional quantities.

[0077] Digital input image x i Can include different objects of the same category. Predicted bounding box and a reference bounding box b can be matched using a matching algorithm (e.g., an algorithm based on the intersection-over-union score of the boxes) to refer to the digital image x i The same object in .

[0078] For example, the corresponding predicted bounding boxes in multiple pairs are obtained from the digit input image x i is predicted and matched to the corresponding reference bounding box.

[0079] According to an example, at least a portion of the respective predicted bounding boxes in the plurality of pairs are determined from the same digital input image. According to an example, at least a portion of the pairs are determined from different digital input images.

[0080] The method includes step 210 .

[0081] Step 210 includes determining corresponding score sets for dimensions of the multidimensional quantity.

[0082] The corresponding score set is associated with the corresponding dimension.

[0083] Depending on the corresponding pair, a score in the corresponding score set is determined.

[0084] The score of a dimension of the multidimensional quantity determined depending on the corresponding pair indicates the degree to which the dimension of the predicted multidimensional quantity of the pair conforms to the same dimension of the reference multidimensional quantity of the pair.

[0085] Step 210 includes, for example, determining a score using a scoring function S. The score indicates: depending on the bounding box pair p o , from the p o The predicted bounding box With the pair o The reference bounding box b in o degree of compliance.

[0086] Step 210 includes determining from the calibration data set D cal The corresponding reference bounding box b o and the corresponding predicted bounding box Multiple pairs of p o Scores o .

[0087] Step 210 may include determining, for the plurality of pairs, a corresponding distance between the predicted multidimensional quantity and the reference multidimensional quantity in a pair, and determining a score depending on the corresponding distance, the corresponding distance being determined depending on the pair, depending on which the score is determined.

[0088] The method may include determining, for the plurality of pairs po, corresponding distances between a predicted bounding box and a reference bounding box in a pair, and determining a score depending on the corresponding distances.

[0089] The value of the scoring function S may depend on the distance between the coordinates of the bounding boxes. The scoring function S may be:

[0090] Object Detection Model A regression head may be included that is configured to depend on the cal The digits of the input image are used to predict the o The standard deviation σ of the distribution of predicted bounding boxes predicted by their associated reference bounding boxes.

[0091] The value of the scoring function S may depend on the distance between the coordinates of the bounding boxes normalized by the standard deviation σ. The scoring function S may be:

[0092] Object Detection Model A regression head may be included that is configured to depend on the cal The digital input image is used to predict the lower conditional quantile (e.g., 5%) and the upper conditional quantile (e.g., 95%).

[0093] The scoring function S can be:

[0094] The method includes step 212 .

[0095] Step 212 includes determining a respective rank of the scores in the respective score set depending on the magnitude of the scores in the respective score set, and determining for the respective pair, for the plurality of pairs, a maximum rank associated with the scores determined depending on the respective pair.

[0096] The method includes step 214 .

[0097] Step 214 includes determining a cutoff point for partitioning a ranked set including the maximum ranking determined for the respective pair into a first ranked set and a second ranked set according to the magnitude of the rankings, wherein the first ranked set includes a predetermined quantile of the rankings.

[0098] Ranking takes into account the correlation information between the dimensions of the quantity. Using ranking-based information improves the accuracy of predictions for the related dimensions.

[0099] According to the example, determine the matrix It includes the calibration data set D cal Determine the score s.

[0100] According to the example, determine the matrix It includes the corresponding score ranking. n represents the number of bounding boxes, and m represents the test dimension, e.g., coordinates. In the four coordinates u of the bounding box m=4 o , v o, u1, v1, the matrix R includes four columns, where the ranking of the scores within the score is determined depending on the value of the score, where rank 1 is associated with the lowest score, i.e., the score with the least uncertainty. The other scores are ranked depending on the magnitude of the score, where the higher the magnitude (i.e., the higher the uncertainty), the higher the ranking. Scores of different dimensions are ranked separately.

[0101] The correction in this example consists of applying max|·| row-wise to the matrix R across the test dimension: max 1≤l≤m |·|, where l indicates a row in the matrix R. This means that the maximum rank It is determined row by row based on the ranking of each test dimension l∈m. The maximum ranking row by row A magnitude that corresponds to a selected score among the scores in that row that are assigned to different test dimensions.

[0102] Determining max|·| over any subset of the test dimension is also possible and corresponds to an alternative coverage optimization criterion.

[0103] This example includes the maximum rank across different bounding boxes Determine conformal quantiles to obtain specific rankings As the largest ranking For example, a cutoff point of a predetermined quantile (e.g., 90% quantile) is determined. Associated with a specific score.

[0104] The method includes step 216 .

[0105] The quantiles depend on the calibration dataset D cal The number of digital images n in is defined as: Where 0<α<1 is a false coverage threshold, for example, a selectable or learnable parameter. For the 90% quantile in the first score set, the threshold α may be 0.1, or for the 95% quantile in the first score set, the threshold α may be 0.05.

[0106] The first ranked set includes, for example, the 90% or 95% quantile of the scores.

[0107] Step 216 includes determining the corresponding cutoff points for dividing the corresponding score sets.

[0108] In this example, the corresponding score set has a ranking of The score of is used as the cutoff point for the corresponding score set This means that the cutoff point used to divide the corresponding score set are the scores in the corresponding score set that have a rank that matches the cutoff point used to partition the ranked set.

[0109] The cutoff point of the corresponding score set The corresponding score set is divided into a first score set and a second score set according to the magnitude of the score, wherein the first score set includes a predetermined quantile of the score.

[0110] According to an example, the higher the score, the higher the uncertainty.The first score set, for example, comprises scores indicating a lower uncertainty than the second score set.

[0111] Volume prediction model or object detection model A regression head may be included that is configured to depend on the input (e.g., on the cal The digital input image) is used to predict the quantile cutoff point

[0112] An exemplary algorithm for determining the score is based on a score matrix S = (s1,,,.s n )=[s ik ]∈R n×m And the ranking matrix R = (r1,,,.r n )=[r ik ]∈N n×m , where 1≤i≤n, 1≤k≤m.

[0113] 1: Input: score matrix S, ranking matrix R 2: Output: coordinate-by-coordinate quantiles 3: Procedure: 4: Put ||·|| ∞ Apply row by row to R 5: Sort r in ascending order max Sort and select As an index Ranking at the site to ensure minimum desired coverage 6: Sort S column by column so that 7: Select the rank for each column in S The score at is taken as the conformal quantile, that is, 8: End the program.

[0114] The method includes step 218 .

[0115] Step 218 includes determining a multidimensional parameter depending on the score.The multidimensional parameter may be associated with a category.

[0116] Step 218 comprises determining, for at least one dimension of the multidimensional quantity, values ​​of multidimensional parameters on the dimensions of the multidimensional parameter corresponding to the dimension of the multidimensional quantity depending on the magnitude of the cutoff point used to partition the score sets comprising the scores of the respective at least one dimension.

[0117] This means that the value of the multidimensional parameter on a dimension depends on the corresponding cutoff point used to divide the corresponding score set determined for that dimension. To be determined.

[0118] Step 218 may include determining, for a respective category of the plurality of object categories, a multidimensional parameter associated with the respective category depending on the pair associated with the respective category.

[0119] Steps 206 to 218 may be specifically directed to another class c from the plurality of object classes C. o Preferably, steps 206 to 218 are performed for multiple categories of multiple object categories C.

[0120] Figure 3 A flow chart is depicted including the steps of a method for predicting a multidimensional quantity characterizing a technical system 106 or an object in the environment of the technical system 106, the multidimensional quantity particularly characterizing a physical quantity of the technical system 106, or wear or expected life, in particular for determining the position of an object in a digital image 108. The method for determining the position of an object is described for exemplary categories from a plurality of object categories. The method may be applicable to determining the position of objects of other categories in the same manner as described for the exemplary categories. The use of categories is optional. The method for determining the position of an object is described for exemplary objects. The digital image 108 may include more than one object. The method is applicable to other objects in the digital image as described for the exemplary objects.

[0121] The method for predicting a multidimensional quantity includes step 302 .

[0122] Step 302 comprises providing at least one quantity characterizing the technical system 106 or the environment of the technical system 106, for example a digital image 108. The at least one quantity (for example a digital image 108) is captured using a sensor 116, for example.

[0123] Thereafter, step 304 is executed.

[0124] In step 304 , a first prediction of a multi-dimensional quantity, for example, the first bounding box 110 , is predicted depending on the pixels of the digital image 108 .

[0125] Predicting the first bounding box 110 includes, for example, determining values ​​in dimensions of the first bounding box 110 that distinguish pixels in the first set of pixels from other pixels of the digital image 108 by locations of corresponding pixels in the digital image 108 .

[0126] In this example, the coordinates of the lower left pixel and the upper right pixel of the first bounding box 110 are predicted.

[0127] In case categories are used, optional step 306 is then performed.

[0128] In step 306 , a class from among a predetermined plurality of object classes is predicted for the first bounding box 110 based on at least a portion of the pixels of the digital image 108 .

[0129] Thereafter, step 308 is executed.

[0130] In step 308 , a method for providing a volume prediction system, in particular an object detection system, is utilized to provide multi-dimensional parameters for determining a second prediction of the multi-dimensional volume (eg, the second bounding box 112 and / or the third bounding box 114 ).

[0131] In this example, the multi-dimensional parameter associated with the category is provided from a plurality of multi-dimensional parameters associated with corresponding categories of a plurality of object categories.

[0132] A plurality of multidimensional parameters associated with respective classes of a plurality of object classes are determined, for example, using a method for providing a quantity prediction system, in particular an object detection system.

[0133] Thereafter, step 310 is executed.

[0134] Step 310 includes determining a second prediction of the multidimensional quantity.

[0135] Determining the second prediction of the multidimensional quantity includes determining a value of the second prediction of the multidimensional quantity at a dimension of the second prediction of the multidimensional quantity. The value of a specific dimension of the second prediction of the multidimensional quantity is determined depending on a value of a specific dimension of a multidimensional parameter corresponding to the specific dimension of the determined second prediction of the multidimensional quantity. Determining the value of a specific dimension of the second prediction of the multidimensional quantity includes adding or subtracting the value of the multidimensional parameter at the specific dimension from the value of the specific dimension of the first prediction of the multidimensional quantity corresponding to the second prediction of the multidimensional quantity and the specific dimension of the multidimensional parameter.

[0136] For example, in step 310 , the second bounding box 112 is determined depending on the first bounding box 110 and the multi-dimensional parameters.

[0137] Determining second bounding box 112 may include determining, for a dimension of second bounding box 112 , values ​​in the dimension of second bounding box 112 that distinguish pixels in the second set of pixels from other pixels of digital image 108 by locations of corresponding pixels in digital image 108 .

[0138] Determining the values ​​of the dimensions of the second bounding box 112 may include adding or subtracting the values ​​of the multi-dimensional parameters from the values ​​of the dimensions of the first bounding box 110 .

[0139] According to this example, determining the second bounding box 112 includes extending the first bounding box 110 depending on the multi-dimensional parameter.

[0140] In this example, the coordinates of the lower left pixel of the second bounding box 112 are determined by subtracting the value of the multidimensional parameter from the coordinates of the lower left pixel of the first bounding box 110. In this example, the coordinates of the upper right pixel of the second bounding box 112 are determined by adding the parameter to the coordinates of the upper right pixel of the first bounding box 110.

[0141] In step 310, the third bounding box 114 may be determined depending on the first bounding box 110 and the multi-dimensional parameter. Determining the values ​​of the dimensions of the third bounding box 114 may include adding or subtracting the values ​​of the multi-dimensional parameter from the values ​​of the dimensions of the first bounding box 110. According to this example, determining the third bounding box 114 includes reducing the first bounding box 100 depending on the parameter.

[0142] In this example, the coordinates of the lower left pixel of the third bounding box 114 are determined by adding the value of the multidimensional parameter to the coordinates of the lower left pixel of the first bounding box 110. In this example, the coordinates of the upper right pixel of the third bounding box are determined by subtracting the value of the multidimensional parameter from the coordinates of the upper right pixel of the first bounding box 110.

[0143] According to an example, both the second bounding box 112 and the third bounding box 114 are determined using the same multi-dimensional parameters.

[0144] According to an example, the second bounding box 112 and the third bounding box 114 are determined using different multi-dimensional parameters.

[0145] The plurality of multidimensional parameters associated with a corresponding category of the plurality of object categories may include a pair of multidimensional parameters associated with the category, wherein one multidimensional parameter is provided for determining the second bounding box 112 and the other parameter of the pair is provided for determining the third bounding box 114 .

[0146] The second bounding box 112 determines a first position of the object.

[0147] The third bounding box 114 determines a second position of the object.

[0148] The true position of the object is between the second bounding box 112 (ie, the first position) and the third bounding box 114 (ie, the second position). This is guaranteed for any correctly classified object with the marginal likelihood provided by the quantile.

[0149] The predetermined plurality of object categories may include categories such as "person", "vehicle", "road", "infrastructure", "animal", etc. This may be used if the technical system 106 is used for an autonomous mobile vehicle. The predetermined plurality of object categories may include categories such as "solid", "fragile", etc. This may be used if the technical system 106 is a robot.

[0150] Figure 4 The result of determining the positions of objects in a digital image 400 is schematically depicted. In this example, four objects of the same category "person" are identified. Figure 4 , a corresponding first bounding box 110, a second bounding box 112, and a third bounding box 114 are depicted for each object.

[0151] Figure 5 A flow chart including steps for a method of operating a technical system 106 is depicted.

[0152] The method for operating a technical system 106 includes step 502 .

[0153] Step 502 includes determining a multidimensional quantity using the above-described method.

[0154] For example, values ​​on a first predicted dimension of a multidimensional quantity are determined, and values ​​of multidimensional parameters on dimensions corresponding to the first predicted dimension of the multidimensional quantity are added to values ​​on the first predicted dimension of the multidimensional quantity corresponding to the dimension to define a first boundary.

[0155] For example, the values ​​on the dimensions of the first prediction of the multidimensional quantity are determined, and the values ​​of the multidimensional parameters on the dimensions corresponding to the dimensions of the first prediction of the multidimensional quantity are subtracted from the values ​​on the dimensions of the first prediction of the multidimensional quantity corresponding to the dimensions to define a second boundary.

[0156] Step 502 includes determining the location of an object in a digital image using the method described above.

[0157] Thereafter, step 504 is executed.

[0158] Step 504 includes operating at least a portion of technical system 106 as a function of the multidimensional quantity within the first boundary, within the second boundary, or within both boundaries.

[0159] Step 504 includes, for example, operating technical system 106 depending on the position of the object determined by second bounding box 112 and / or third bounding box 114 .

[0160] Operating the technical system 106 includes, for example: granting access, transmitting information about a multidimensional quantity (e.g., an object), or transmitting information about a multidimensional quantity (e.g., a location), or stopping the technical system 106 when the multidimensional quantity (e.g., a location) is detected to be within or outside a predetermined area, or operating the technical system 106 in a safe mode, or moving at least a part of the technical system 106 depending on the multidimensional quantity, in particular depending on the location of the object or in order to avoid the location of the object.

[0161] The method may include determining values ​​of dimensions of a second prediction of the multidimensional quantity (e.g., second bounding box 112), adding values ​​of the multidimensional parameters to values ​​of dimensions of the first prediction of the multidimensional quantity (e.g., first bounding box 110), and using the second prediction of the multidimensional quantity (e.g., second bounding box 112) to move at least a portion of the technical system 106 depending on the multidimensional quantity, such as the location of an object, so as to avoid the location of the object. In this case, the location may be a traffic participant or infrastructure, and the technical system 106 may be a vehicle that is moved to avoid a collision with the traffic participant or infrastructure.

[0162] The method may include determining values ​​in the dimensions of third bounding box 114 and subtracting the values ​​of the multi-dimensional parameters from the values ​​in the dimensions of first bounding box 110, and using third bounding box 114 as the location of the object to move at least a portion of technology system 106 to the location. In this case, the location may be a parking space and technology system 106 may be a vehicle being moved to the parking space.

[0163] The multidimensional parameters can be provided depending on a plurality of pairs of corresponding predicted multidimensional quantities and corresponding reference multidimensional quantities using the following steps: corresponding score sets are determined for dimensions of the multidimensional quantity. The corresponding score sets are associated with the corresponding dimensions. Scores in the corresponding score sets are determined depending on corresponding pairs in the plurality of pairs. Scores for dimensions of the multidimensional quantity determined depending on corresponding pairs indicate the degree of conformity of the dimensions of the predicted multidimensional quantity of the pair with the same dimensions of the reference multidimensional quantity of the pair.

[0164] The respective rankings of the scores in the respective score sets are determined depending on the magnitudes of the scores in the respective score sets.

[0165] A respective maximum ranking associated with a score determined dependent on the respective pair is determined for the respective pair of the plurality of pairs.

[0166] A cutoff point for dividing the ranked set including the maximum rank into the first ranked set and the second ranked set is determined according to the magnitude of the rankings.

[0167] The first set of rankings includes predetermined quantiles of rankings.

[0168] According to the magnitudes of the scores, respective cutoff points for partitioning respective score sets into first score sets and second score sets are determined for the dimensions of the multidimensional quantity.

[0169] The first score set includes quantiles of scores defined by a cutoff point used to divide the corresponding score set.

[0170] The cutoff point for partitioning the corresponding score set is determined as a score in the corresponding score set having a ranking matching the cutoff point for partitioning the ranking set.

[0171] Depending on the magnitude of the cutoff point used to partition the score sets associated with the respective dimension, values ​​of parameters on respective dimensions of the multidimensional parameters corresponding to the dimensions of the multidimensional quantity are determined for respective dimensions of the multidimensional quantity.

[0172] In this context, a physical quantity may refer to a temperature, acceleration, speed, revolutions per minute, electrical tension or current, force or torque at at least a part of the technical system 106, in particular a drive or a gearbox, or an energy source (e.g. a battery). In this context, wear or life expectancy may refer to wear or life expectancy of at least a part of the technical system 106, in particular a drive or a gearbox, or an energy source (e.g. a battery).

[0173] In this context, a physical quantity may refer to a material property, such as thickness, stiffness or geometry, of the technical system 106, in particular a component of the technical system 106. In this context, wear or life expectancy may refer to the wear or life expectancy of a component.

Claims

1. A computer-implemented method for determining a multidimensional quantity characterizing a technical system (106) or an object in the environment of a technical system (106), characterized in that The method comprises: providing (302) at least one quantity characterizing a technical system (106) or an environment of the technical system (106), predicting (304) a first prediction of a multidimensional quantity depending on the at least one quantity, providing (308) a multidimensional parameter defining a prediction interval, determining (310) a second prediction of the multidimensional quantity depending on the first prediction of the multidimensional quantity and the multidimensional parameter, wherein the second multidimensional prediction determines the multidimensional quantity, wherein providing (308) the multidimensional parameter comprises providing (208) a plurality of pairs of multidimensional quantity predictions and multidimensional quantity references, providing (210) a plurality of scores respectively associated with dimensions of the multidimensional quantity and with one of the plurality of pairs, wherein the scores associated with the dimensions of the multidimensional quantity indicate at least The degree of conformity of the predicted multidimensional quantity of the pair with the reference multidimensional quantity of the pair on the dimension associated with the score, wherein the multiple scores include a score set, wherein the scores in the score set are associated with the same dimension, determining (212) multiple rankings of the scores respectively associated with the dimension of the multidimensional quantity and one of the multiple pairs depending on the magnitude of the scores in the multiple scores, selecting (214) a ranking in the ranking set, wherein the ranking set includes rankings, in particular the maximum ranking, associated with different dimensions of the multidimensional quantity in the multiple rankings, selecting (216) a score in the score set depending on the selected ranking, and determining (218) the value of the multidimensional parameter on the same dimension depending on the selected score.

2. The method according to claim 1, characterized in that: Selecting (214) a ranking includes determining (214) a cutoff point for partitioning the ranking set into a first ranking set and a second ranking set based on the magnitude of the ranking, wherein the first ranking set includes a predetermined quantile of the rankings, wherein the cutoff point defines the selected ranking, and / or selecting (216) a score includes determining a cutoff point for partitioning the score set into a first score set and a second score set based on the magnitude of the score, wherein the first score set includes a predetermined quantile of the scores, wherein the cutoff point for partitioning the respective score sets is a score in the respective score set having the selected ranking.

3. The method according to one of the preceding claims, characterized in that The method comprises determining, for the plurality of pairs, a corresponding distance between the predicted multidimensional quantity and the reference multidimensional quantity in a pair, and determining (210) a score depending on the corresponding distance.

4. The method according to one of the preceding claims, characterized in that Determining (310) a second prediction of a multidimensional quantity includes determining values ​​on dimensions of the second prediction of the multidimensional quantity, wherein determining values ​​on dimensions of the second prediction of the multidimensional quantity includes: adding or subtracting values ​​of the parameters on the dimensions with values ​​on dimensions of the first prediction of the multidimensional quantity corresponding to the second prediction of the multidimensional quantity and the dimensions of the parameters.

5. The method according to one of the preceding claims, characterized in that The multidimensional quantity and the reference to the multidimensional quantity characterize a technical system (106) or an object in the environment of the technical system (106), in particular a physical quantity of the technical system (106), or wear or expected life.

6. A computer-implemented method for operating a technical system (106), in particular a computer-controlled machine, preferably a robot, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, an access control system, or a system for transmitting information, in particular a monitoring system or a medical imaging system, characterized in that The method comprises determining (502) a multidimensional quantity characterizing a technical system, in particular a physical quantity characterizing the technical system, or wear or expected life, using a method according to one of the preceding claims, and operating (504) the technical system (106) depending on the multidimensional quantity determined by a second multidimensional prediction, in particular granting access, transmitting information about the system, or transmitting information about the multidimensional quantity, or stopping the technical system (106) or operating the technical system (106) in a safe mode when it is detected that the multidimensional quantity is within or outside a predetermined tolerance, or moving at least a part of the technical system (106) depending on the multidimensional quantity.

7. The method according to claim 6, characterized in that The method includes determining (502) values ​​on a dimension of a first prediction of a multidimensional quantity, and adding values ​​of the parameter on a dimension corresponding to a dimension of a second multidimensional prediction to the values ​​on the dimension of the first prediction of the multidimensional quantity corresponding to the dimension to define a first boundary, and / or subtracting the parameter from the values ​​on the dimension of the first multidimensional prediction to define a second boundary, and operating (504) at least a portion of the technical system (106) within the first boundary, within the second boundary, or within both boundaries.

8. A computer-implemented method for providing a quantity prediction system for predicting a physical quantity characterizing a technical system (106) or an object in the environment of the technical system (106), in particular a multidimensional quantity characterizing the technical system (106), or a wear or expected life, characterized in that The method comprises: providing (208) a plurality of pairs of multidimensional quantity predictions and multidimensional quantity references, providing (210) a plurality of scores respectively associated with the dimensions of the multidimensional quantity and one of the plurality of pairs, wherein the scores associated with the dimensions of the multidimensional quantity indicate the degree of conformity of the predicted multidimensional quantity of the pair with the reference multidimensional quantity of the pair at least in the dimensions associated with the scores, wherein the plurality of scores comprises a score set, wherein the scores in the score set are associated with the same dimension, determining (212) a plurality of rankings of the scores respectively associated with the dimensions of the multidimensional quantity and one of the plurality of pairs depending on the magnitudes of the scores in the plurality of scores, selecting (214) a ranking in the ranking set, wherein the ranking set comprises rankings, in particular the maximum ranking, associated with different dimensions of the multidimensional quantity in the plurality of rankings, selecting (216) a score in the score set depending on the selected ranking, and determining (218) a value of a multidimensional parameter defining a prediction interval of the quantity prediction system on the same dimension depending on the selected score, and providing the multidimensional parameter to the quantity prediction system.

9. The method according to claim 8, characterized in that Selecting (214) a ranking includes determining (214) a cutoff point for partitioning the ranking set into a first ranking set and a second ranking set based on the magnitude of the ranking, wherein the first ranking set includes a predetermined quantile of the rankings, wherein the cutoff point defines the selected ranking, and / or selecting (216) a score includes determining a cutoff point for partitioning the score set into a first score set and a second score set based on the magnitude of the score, wherein the first score set includes a predetermined quantile of the scores, wherein the cutoff point for partitioning the respective score sets is a score in the respective score set having the selected ranking.

10. The method according to claim 8 or 9, characterized in that: The method comprises determining, for the plurality of pairs, a corresponding distance between the predicted multidimensional quantity and the reference multidimensional quantity in a pair, and determining (210) a score depending on the corresponding distance.

11. A device (100), characterized in that The device (100) comprises at least one processor (102) and at least one memory (104), wherein the at least one processor (102) is configured to execute instructions which, when executed by the at least one processor (102), cause the device (100) to perform a method according to one of the preceding claims, wherein the at least one memory (104) is configured to store the instructions.

12. A computer program, characterized in that The computer program comprises computer readable instructions which, when executed by a computer, cause the computer to perform the method according to one of claims 1 to 10 .

Images