Device and computer-implemented method for testing technical system, in particular for determining virtual sensor signals
By constructing a surrogate model and using simplex detection technology, combined with generalized Pareto distribution and tolerance threshold, the accuracy problem of virtual sensor signal testing was solved, and reliable matching and deviation identification between virtual sensor signals and real physical processes were achieved, thereby improving the reliability of the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2025-10-22
- Publication Date
- 2026-04-24
AI Technical Summary
Existing technologies struggle to effectively test and verify the accuracy of virtual sensor signals, especially in safety-related production environments, where it is impossible to ensure that the matching and deviation between virtual sensor signals and real physical processes are within tolerable limits.
Machine learning models such as artificial neural networks or Gaussian processes are used to construct surrogate models to describe real physical processes. Simplex detection and probability distribution are used to evaluate the prediction bias of virtual sensor signals. By combining generalized Pareto distribution and tolerance threshold, the reliability of virtual sensor signals can be tested.
It improves the accuracy and reliability of virtual sensor signal testing, ensures the matching between virtual sensor signals and real physical processes, can quickly identify areas where deviations exceed the tolerance range, and improves the reliability of the system.
Smart Images
Figure CN121920171A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an apparatus and a computer-implemented method for testing technical systems, particularly for determining virtual sensor signals. Background Technology
[0002] There is great interest in solutions that enable the use of artificial neural networks in safety-related production. Summary of the Invention
[0003] According to the computer-implemented method for testing a technical system, particularly for determining virtual sensor signals such as virtual sensors, as described in independent claim 1, the system includes a model that models a physical process, wherein physical input parameters of the physical process result in physical output parameters of the physical process, wherein the input space includes input values of the physical input parameters, wherein the output space includes output values of the physical output parameters, wherein a surrogate model is provided for the model, wherein the surrogate model is constructed to describe the real physical process, wherein input values are detected, wherein a simplex is determined in the input space by the detected input values, wherein for each simplex, particularly in relation to the user... In machine-computer interaction, at least one input value is determined from the input space, or at least one input value is randomly sampled from the input space on a simplex according to a pre-given probability distribution, particularly a uniform distribution. An output value for that input value is determined using the surrogate model. The model maps this input value to a prediction for the output value, particularly for determining the virtual sensor signal, and determines the distance between the prediction and the output value. The result of this test includes the probability that the model maps the input value of the physical input parameter from the input space to the prediction, which in particular deviates from the output value of the physical output parameter of the physical process for the physical input parameter having that input value by a deviation greater than a pre-given amount. This qualitatively evaluates the region in the input space of the system defined by the simplex. Here, input values in the input space that can be expanded by the simplex, i.e., data points, such as training data, are detected.
[0004] The model can include machine learning oracle functions, such as artificial neural networks or Gaussian processes.
[0005] It can be specified that the distance is determined based on the numerical value of the difference between the prediction and the output value.
[0006] It can be stipulated that the probability is determined based on the ratio of the number of distances greater than the tolerance to the number of distances determined in the test. This allows the evaluation to be performed when there are distances outside the tolerance.
[0007] It can be specified that the probability is determined using a generalized Pareto distribution, wherein the generalized Pareto distribution is matched to a distance less than the tolerance determined for the simplex. This allows the evaluation to be achieved when there are no distances outside the tolerance.
[0008] It can be specified that, during testing, it is checked whether at least one distance greater than the tolerance has been determined, wherein if at least one distance is greater than the tolerance, the probability is determined according to the ratio, and otherwise the probability is determined using the generalized Pareto distribution.
[0009] It can be specified that the surrogate model is constructed to describe a real physical process with tolerances, wherein the tolerances are determined based on tolerable deviations, specifically as a fraction of the deviation, for example, as half of the deviation. This makes the evaluation match the tolerances.
[0010] It can be specified that the probability is determined using the same tolerance for the simplex, particularly by overlapping or simultaneous time intervals, or by using different tolerances for at least two simplexes. Using the same tolerance enables rapid evaluation. Different tolerances provide a tolerable deviation matching to regions of the input space.
[0011] For example, for each simplex, multiple distances can be determined, for example, for different input values. This increases the persuasiveness of the probability.
[0012] It can be specified that the model learns based on a measured training dataset, which includes pairs of input values from the input space and output values from the output space, wherein if the probability for at least one simplex is greater than a pre-given boundary, new training data is measured, specifically with input values from one or more simplexes having probabilities greater than the boundary, and the new training data is added to the training dataset, and the model learns based on the training dataset supplemented with the training data, or if there is no simplex whose probability is greater than the pre-given boundary, the model is delivered for reliable physical process prediction.
[0013] An apparatus for testing a technical system, particularly for determining virtual sensor signals such as virtual sensors, is specified, the apparatus comprising at least one processor and at least one memory, wherein the at least one processor is configured to execute instructions stored in the memory, and the apparatus performs the method when the instructions are executed by the at least one processor.
[0014] A computer program may be specified, wherein the computer program includes instructions executable by a computer, and the method operates when the instructions are executed by the computer. Attached Figure Description
[0015] Further advantageous embodiments can be seen from the following description and accompanying drawings. In the drawings: Figure 1 A schematic diagram of the apparatus used for testing is shown. Figure 2 A flowchart showing the steps of a method for testing is provided. Figure 3 A schematic diagram of the input space is shown. Figure 4 This diagram illustrates the output values of the model used to determine the output values of the physical process. Figure 5 This is a schematic diagram showing the histogram of the distance between the predicted and output values. Detailed Implementation
[0016] exist Figure 1 The device 100 is shown schematically in the figure.
[0017] The apparatus 100 includes at least one processor 102 and at least one memory 104. The apparatus 100 is configured to test a technical system 106. In this example, the technical system 106 is arranged within the apparatus 100 for testing. The technical system 106 may also be arranged outside the apparatus 100 for testing.
[0018] Technical system 106 is, for example, a system for determining virtual sensor signals. Technical system 106 is, for example, a virtual sensor.
[0019] Technical system 106 includes a model. This model may include, for example, machine learning functions, such as artificial neural networks. Or a Gaussian process. The technical system 106 is configured to detect input values 108 from the machine 110. The machine 110 is, for example, an internal combustion engine or an electric motor.
[0020] At least one processor 102 is configured to execute instructions stored in memory 104, and when the instructions are executed by at least one processor 102, the apparatus 100 implements a method for testing the technology system 106.
[0021] This model is constructed, for example, to predict output values used to determine virtual sensor signals.
[0022] The technical system 106 is, for example, configured to determine and output virtual sensor signals based on predictions.
[0023] It can be specified that the technology system 106 is constructed for human-computer interaction with users.
[0024] It can be specified that the input parameter 108 can be one-dimensional or multi-dimensional.
[0025] The apparatus 100 for testing the technology system 106 is connected, for example, to a test bench. This test bench is configured to specifically measure real-world physical systems. The test bench is configured to determine the output values of physical processes operating on the physical system for input values from a given range in the input space, particularly for the simplex, and especially for given input values. The test bench is configured to specifically generate new pairs consisting of input and output values. The test bench is, for example, configured to detect pairs used to expand datasets, such as the training dataset required for testing the technology system 106.
[0026] The apparatus 100 for testing the technology system 106 is configured, for example, to perform tests in machine learning (ML) according to the following sequence: 1. Models, such as ML functions, learn based on a measured training dataset, which consists of pairs of input values from the input space and output values from the output space. This model, for example, is achieved through artificial neural networks. Or represented as a Gaussian process.
[0027] 2. Test the model according to this method.
[0028] The output of this method is the probability that the model—more precisely, for each individual simplex—makes a prediction with a large deviation from the expected physical output value.
[0029] 3. Repeat the test.
[0030] If the probability for at least one simplex is greater than the previously defined boundary p*, then new training data is requested from the testbed, specifically from simplexes with input values having a probability p > p*. This new training data is added to the existing training dataset, and steps 1 and 2 are repeated.
[0031] 4. Delivery Model.
[0032] If there is no simplex whose probability is greater than the boundary p*, the delivered model is used for reliable prediction of physical processes.
[0033] 5. Criteria for termination.
[0034] If step 4 is not achieved, for example, at the test bench after a predetermined time, or if the training dataset reaches or exceeds a predetermined size, the tested model is not adopted and other models are tested.
[0035] For example, this other model has an alternative model architecture. For example, in the case of neural networks, in neural networks... Multiple neurons can be set up within a process. For example, in the case of a Gaussian process, additional nuclei can be set up within the Gaussian process.
[0036] For example, this other model uses one or more alternative input parameters.
[0037] The sequence was then restarted using this other model.
[0038] exist Figure 2 The flowchart illustrates exemplary steps of this method. The method utilizes artificial neural networks. For example, it is applied accordingly in other models, such as those with Gaussian processes.
[0039] Artificial Neural Networks physical processes g Modeling: X→Y .
[0040] In this physical process, from the physical process g input space X The physical input parameters of this physical process x ∈ X This results in the output space of the physical process. Y The physical output parameters of this physical process y ∈ Y .
[0041] Input space X Including physical input parameters x Input values. Output space. Y Including physical output parameters y The output value.
[0042] Artificial Neural Networks Physical input parameters x The input values are mapped to the physical output parameters. y Prediction of output values ( x ).
[0043] Input parameter 108 is for physical input parameters. x Example. Virtual consumption is for physical output parameters. y Examples.
[0044] For testing purposes, the model, in this example, is an artificial neural network. Provide a proxy model. In this example, the proxy model is a function. : X → Y .
[0045] Proxy Model Constructed to describe real physical processes g For example, this function is constructed for applications with tolerances. / 2 describes the real physical process.
[0046] The method includes step 202.
[0047] In step 202, the input value is detected.
[0048] The method includes step 204.
[0049] In step 204, in the input space X The simplex is determined by the input values detected.
[0050] For example, one-dimensional simplex S The two input values detected are shown below: ,in Regarding the provided proxy model Applicable to: This means that the proxy model Within all simplexes, that is, with the characteristics of... Defined tolerances describe real physical processes g .
[0051] The method includes step 206.
[0052] In step 206, for each simplex, a) Determine or randomly select at least one input value from the input space. x k ; b) Utilizing the proxy model For input values x k Determine the output value ( x k ); c) Utilizing artificial neural networks Input value x k Mapped to the prediction used for the output value.
[0053] For example, determining the corresponding input values in human-computer interaction with users. x k .
[0054] For example, based on a pre-given probability distribution ( S In particular, the corresponding input values are randomly drawn from a uniform distribution on the simplex. x k , .
[0055] The method includes step 208.
[0056] In step 208, for each prediction, the prediction is determined. ( x k ) and the output value of the proxy model ( x k Distance between ) d k .
[0057] This distance is determined, for example, based on the numerical difference between the predicted and output values. .
[0058] It can be stipulated that, for each simplex, different input values... x k Determine multiple distances.
[0059] The method includes step 210.
[0060] In step 210, based on the simplex or multiple simplexes determined... N distance d k Determine artificial neural networks Mapping the input values of physical input parameters from the input space to the predicted probabilities p This prediction, in particular, involves a deviation greater than predefined, for example / 2 deviates from the output value of the physical output parameter of the physical process for the physical input parameter with that input value.
[0061] The results of the test include, for example, the probability.
[0062] probability p For example, it can be determined based on the ratio of distances greater than the tolerance to the number of distances determined in the test: probability p For example, using a density function γ The generalized Pareto distribution is determined by matching the generalized Pareto distribution to a distance less than the tolerance determined for the simplex.
[0063] For example, based on deviation To determine tolerances, it can be stipulated that tolerances are deviations. The fraction is determined. For example, the tolerance is half of the deviation, i.e. / 2 has been determined.
[0064] It can be stipulated that, during testing, it is checked whether at least one distance greater than the tolerance has been determined. If at least one distance is greater than the tolerance, then the probability is determined, for example, based on the ratio. Otherwise, the probability is determined, for example, using a generalized Pareto distribution.
[0065] This method can specify that the probability is determined using the same tolerance for the simplex. For example, the probability can be determined for the simplex in overlapping or simultaneous time intervals.
[0066] This method can specify that the probability is determined using the different tolerances for at least two simplexes.
[0067] exist Figure 3 An exemplary input space is schematically shown in the diagram.
[0068] The input space includes the detected input value 302. The detected input value 302 is, for example, the result of a measurement of the input parameter 108 at machine 110.
[0069] exist Figure 3 Simplex 304 is shown in the figure. Three simplexes are shown in this example.
[0070] exist Figure 3 In the simplex 304, an input value 306 that is determined or extracted in the test is shown as an example.
[0071] exist Figure 4 In the example, for an exemplary one-dimensional simplex spanned by two exemplary input values 302, the use of a function is shown. For example input values from a one-dimensional simplex x Determined exemplary output values y = ( x A schematic diagram of ) and in Figure 4 It is indicated by reference numeral 402 in the attached figure. In this example, the output value is... y within tolerance Within / 2. This means that the absolute distances, not only upwards but also downwards, are less than [a certain value]. / 2.
[0072] exist Figure 5 The diagram illustrates a histogram 500 showing the distance between the predictions of an artificial neural network and the output values of a function.
[0073] Beyond the boundary The generalized Pareto distribution 502 is shown. The generalized Pareto distribution 502 matches the tolerance determined for the simplex. / 2 and greater than the boundary The distance.
[0074] One exemplary implementation relates to a proxy model. This proxy model is provided and constructed to describe real physical processes.
[0075] Proxy Model For example, by determining the input value belonging to step 202 on each simplex in the simplex. x k The measured output value y k It is formed by linear interpolation.
[0076] If a single or given simplex is determined by the input value x 1. x 2、…、 x s Open, and thus give any point from the simplex. x = λ1* x 1+λ2* x 2+…+λ s * x s Where the coefficients λ1, λ2, ..., λ s If the sum of the numbers is a non-negative real number, then the proxy model... Defined as ( x ) = λ1* y 1+λ2* y 2+…+λ s * y s ,in y 1. y 2、…、 y s It belongs to the input value x 1. x 2、…、 x s The physical measurement output value.
[0077] Other proxy models can also be used. For example, Gaussian processes or physical models.
Claims
1. A computer-implemented method for testing a technology system (106), particularly for determining virtual sensor signals, such as those of a virtual sensor, characterized in that, The system includes a model for modeling a physical process, wherein physical input parameters of the physical process result in physical output parameters of the physical process, wherein the input space includes input values of the physical input parameters, wherein the output space includes output values of the physical output parameters, wherein a surrogate model is provided for the model, wherein the surrogate model is constructed to describe the real physical process, wherein input values (302) are detected (202), wherein a simplex (304) opened by the detected input values (302) is determined (204) in the input space, wherein for each simplex, particularly in human-computer interaction with a user, at least one input value (306) is determined from the input space, or according to a pre-defined... At least one input value is randomly drawn from the input space on the simplex (306) by a given probability distribution, particularly a uniform distribution. An output value for the input value is determined using the surrogate model. The input value is then mapped (206) to a prediction for the output value, particularly for determining the virtual sensor signal, using the model. The distance between the prediction and the output value is determined (208). The result of the test includes the probability that the model maps the input value of the physical input parameter from the input space to the prediction, which in particular deviates from the output value of the physical output parameter of the physical process for the physical input parameter having the input value by more than a pre-given deviation (210).
2. The method according to claim 1, characterized in that, The distance is determined based on the numerical value of the difference between the prediction and the output value.
3. The method according to any one of the preceding claims, characterized in that, The probability is determined based on the ratio of the number of distances greater than the tolerance to the number of distances determined in the test (210).
4. The method according to any one of claims 1 to 3, characterized in that, The probability is determined (210) using a generalized Pareto distribution (502), wherein the generalized Pareto distribution (502) is matched to a distance less than the tolerance determined for the simplex.
5. The method according to claims 3 and 4, characterized in that, In the test, it is examined whether (210) has been determined to at least one distance greater than the tolerance, wherein if at least one distance is greater than the tolerance, the probability is determined according to the ratio, and otherwise the probability is determined using the generalized Pareto distribution.
6. The method according to claim 3, 4 or 5, characterized in that, The proxy model is constructed for use with tolerances ( The tolerance describes the real physical process in a way that allows for tolerances based on tolerable deviations, specifically as a fraction of the deviation, for example, as half of the deviation. / 2), was determined (210).
7. The method according to any one of claims 3 to 6, characterized in that, The probability (210) is determined using the same tolerance for the simplex, particularly by overlapping or simultaneous time, or the probability (210) is determined using different tolerances for at least two simplexes.
8. The method according to any one of the preceding claims, characterized in that, For each simplex, multiple distances are determined, for example, for different input values (208).
9. The method according to any one of the preceding claims, characterized in that, The model learns based on a measured training dataset comprising pairs of input values from the input space and output values from the output space. If the probability for at least one simplex is greater than a pre-given boundary, new training data is measured, specifically with input values from one or more simplexes having probabilities greater than the boundary, and this new training data is added to the training dataset. The model learns based on the training dataset supplemented with the training data. Alternatively, if there is no simplex with a probability greater than the pre-given boundary, the model is delivered for reliable predictions of the physical process.
10. An apparatus (100) for testing a technology system (106), particularly for determining a virtual sensor signal, such as a virtual sensor, characterized in that, The apparatus (100) includes at least one processor (102) and at least one memory (104), wherein the at least one processor (102) is configured to execute instructions stored in the memory (104), and when the instructions are executed by the at least one processor (102), the apparatus (100) implements the method according to any one of claims 1 to 9.
11. A computer program, characterized in that, The computer program includes instructions executable by a computer, which, when executed by the computer, operate according to the method of any one of claims 1 to 9.