Method and device for generating test data points to excite a system under test
The method generates uniformly distributed measurement points using step-like functions with varied step lengths and heights across input variables, addressing the incomplete coverage issue and ensuring comprehensive and dynamic excitation modes in test data spaces.
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2015-03-11
- Publication Date
- 2026-04-30
AI Technical Summary
Existing methods for generating measurement points in a test data space often fail to achieve a comprehensive and dynamic coverage of the input data space, particularly when multiple input variables are involved, leading to incomplete and uneven distribution of excitation modes.
A method involving the generation of measurement points using step-like functions with uniformly distributed step lengths and heights, combined across multiple input variables, ensuring a uniform distribution and complete coverage of the input data space.
The method ensures that measurement points uniformly cover the input data space, achieving a high number of excitation modes and precise definition of coverage, thereby enhancing the effectiveness of test procedures.
Smart Images

Figure 00000009_0000 
Figure 00000009_0001 
Figure 00000010_0000
Abstract
Description
State of the art
[0001] The invention relates to test methods, and in particular methods for providing measurement points with which a system to be tested can be tested. In particular, the present invention relates to methods for providing measurement points that are provided in a test data space that is as space-filling and dynamically comprehensive as possible.
[0002] When measuring a physical system with measurement points, it is necessary to arrange the measurement points in such a way that as many combinations as possible of values of input variables in different excitation modes, i.e. combinations of gradients of the input variables, are measured, so that a space- and dynamics-filling allocation of the input data space with measurement points is obtained.
[0003] A space-filling occupancy of the input data space occurs when the distance of each measurement point from its nearest neighboring measurement point within the input data space is less than a predefined distance value. Similarly, a dynamic-filling occupancy of the input data space occurs when, for each measurement point, the distances of the gradients relating to that measurement point within the input data space are less than a predefined further distance value.
[0004] To create a set of measurement points, the curves of each input variable can be based on step-shaped signals. To ensure a spatially and dynamically uniform distribution of the measurement points, the parameters step length, step height, and jump height between the steps of the step-shaped curves can be varied randomly.
[0005] It is desirable to provide a method for generating measurement points from step-like functions in such a way that they are as spatially and dynamically comprehensive as possible. When creating measurement points with multiple input variables, step-like functions are combined for each input variable, exhibiting different step widths, step heights, and jump heights between successive steps. Each combination of these parameters can be understood as an excitation mode.
[0006] Especially when there are multiple input variables defining the measurement points, a random combination of these parameters cannot reliably guarantee that the resulting measurement points will fully occupy the input data space in terms of both space and dynamics, as they may only provide a limited number of excitation modes.
[0007] Patent application US 2005 / 0 281 333 A1 discloses a method for generating a quality measure for a video signal encoded using a compression algorithm employing a variable quantization step size and a two-dimensional transformation, such that the encoded signal contains a quantization step size parameter and transformation coefficients for image blocks, wherein the method comprises generating a first quality measure that is a function of the quantization step size parameter, generating a second quality measure that is a function of the number of blocks with a single transformation coefficient; and combining the first and second measures.
[0008] German patent application DE 10 2005 063 273 A1 discloses a method for generating stochastic random variables to emulate interference pulses on a motor vehicle's electrical system. During operation of the motor vehicle, measurement data of at least one quantity describing the behavior and / or characteristics of the interference pulses are recorded. The recorded measurement data and corresponding frequency distributions are stored in a distribution table. The random variables for the at least one quantity are generated with a frequency distribution corresponding to the probability density function of the recorded measurement data, based on the measurement data stored in the distribution table. The random variables are generated for the at least one quantity by a quantile transformation of a distribution function of the stored measurement data.
[0009] Publication CN 104 297 626 A discloses a fault location device for locating a fault in a power supply system. Disclosure of the invention
[0010] According to the invention, a method for determining a set of measuring points, in particular for carrying out a test procedure according to claim 1, and the corresponding device according to the dependent claim are provided.
[0011] Further details are provided for in the dependent claims.
[0012] According to a first aspect, a method for providing a sequence of measurement points, in particular for carrying out a test procedure for measuring a physical unit, is provided, wherein each measurement point is defined by a combination of one input value from several input variables according to the sequence of input values, the sequence of input values of each input variable corresponding to a step function with steps having a step height, a jump height between the steps and a step length; comprising the following steps: - Providing a step height value sequence for each of the input variables, wherein the step height value sequence corresponds to a uniformly distributed value sequence and specifies a sequence of step heights along the course of the step function of the input variable in question; - Providing a step length value sequence for each of the input variables, wherein the step length value sequence corresponds to a uniformly distributed value sequence and specifies a sequence of step lengths along the course of the step function; - Generating sequences of input values for each of the input variables depending on the sequence of step heights and depending on the sequence of step lengths, wherein the input values of each of the input variables are generated in a sequence resulting from a concatenation of the step heights according to their order in the step height value sequence assigned to the input variable in question, wherein each of the step heights is provided successively in a number that results from the sequence of the step lengths assigned to the input variable in question in the step length value sequence; - Generating the sequence of measurement points by combining the corresponding input variable values of the sequences of multiple input variables.
[0013] One idea behind the above method is to achieve a uniform distribution of measurement points such that they completely and dynamically cover a given input data space. This allows the coverage of the input quantity space by the measurement points and their excitation modes to be precisely defined. Since the parameters relating to the variation of the measurement points, such as the step length and step height, are determined from uniformly distributed sequences of values, there is a uniform distribution with respect to all parameters, and thus a complete and dynamically covering coverage of the input data space by the measurement points can be achieved.
[0014] Furthermore, generating sequences of input values for each of the input variables can include providing at least several of the step heights consecutively in a number that corresponds to a step length from the sequence of step lengths assigned to the input variable in question in the step length value sequence.
[0015] Furthermore, the respective step height value sequence can be provided by providing a step height value sequence as a uniformly distributed value sequence for each input variable, whereby, starting from an initial value, the step height values are successively applied to the initial value or the previously determined value, in particular by addition or subtraction, in order to obtain the step height value sequence, whereby the application is carried out in such a way that each value of the step height value sequence is restricted to a value within a specified permissible value range.
[0016] It may be provided that at least some of the step height value sequences assigned to the input variables are generated by permutation of subsets from a uniformly distributed value sequence.
[0017] In particular, the size of the subsets from the uniformly distributed sequence of values can be iteratively reduced as long as an evaluation measure for a uniform distribution of jump heights resulting from the permuted sequence of values assumed to be step heights indicates a deviation from an ideal uniform distribution for one or more of the input variables that is greater than a given threshold.
[0018] It may be provided that the size of the subsets from the uniformly distributed sequence of values is iteratively reduced, as long as an evaluation measure for a uniform distribution of jump heights resulting from the permuted sequence of values assumed to be step heights yields or indicates a smaller deviation from an ideal uniform distribution at each iteration.
[0019] According to one embodiment, the size of the subsets from the uniformly distributed sequence of values can be iteratively reduced until a predetermined maximum number of iterations has been performed.
[0020] The rating measure can depend on a variation in the frequencies of classified jump height values.
[0021] Furthermore, the evaluation measure can correspond to a distance measure with respect to the jump heights of the jump height value sequence and can in particular be specified as a sum of the absolute or square distances between any two jump heights that are closest with respect to their value.
[0022] It may be provided that the evaluation measure corresponds to a divergence measure of the jump heights of the jump height value sequence and, in particular, is determined by a discrete Kulback-Leibler divergence. Brief description of the drawings
[0023] The embodiments are explained in more detail below with reference to the accompanying drawings. These show: Fig. 1. A representation of a test system for measuring a physical system; Fig. 2 a course of measurement points of an input quantity with the parameters of the jump height, the step length and the step height; Fig. 3. A flowchart illustrating a procedure for determining a sequence of measurement points for carrying out a test procedure; Fig. 4. A flowchart illustrating a further procedure for determining a sequence of measurement points for carrying out a test procedure; and Fig. 5. A flowchart to illustrate another procedure for determining a sequence of measurement points for carrying out a test procedure. Description of embodiments
[0024] Fig. Figure 1 shows a schematic representation of a test or inspection system 1 designed for measuring a physical unit 2. A physical unit 2 could, for example, be an internal combustion engine of a motor vehicle or subsystems thereof. A measurement unit 3 controls the physical unit 2 with a sequence of measuring points M, which lead to specific operating points of the physical unit 2. The measuring points comprise one, but usually several, input variables, each with a permissible value range. Furthermore, the control of the physical unit 2 results in one or more output variables A, which are also measured, and the corresponding measured values can be communicated to the measurement unit 3.
[0025] Typically, to fully measure the physical unit 2, the measurement points are varied over a large range within the permissible value ranges to achieve the most comprehensive possible coverage of the input data space. However, simply selecting a sequence of measurement points for comprehensive coverage does not guarantee a dynamically comprehensive coverage of the input data space. Therefore, it is necessary to provide a method that reliably generates a sequence of uniformly distributed measurement points whose gradients are also uniformly distributed within the permissible value range.
[0026] Fig. Figure 2 shows an example of a desired progression of the values of an input variable in a sequence of measurement points. It has been found that a step-like progression of the input variables at the measurement points is advantageous. The parameters step height H (absolute value of the input variable), step length L (time interval between the steps of the step-like progression, or the number of identical consecutive values of step height H), and step height S (height between two step plateaus) are each arbitrary and, in particular, uniformly distributed. To achieve a suitable sequence of measurement points, the following procedure involves using a suitable combination of several input variables, each exhibiting a step-like progression, as shown in Figure 2. Fig. 2 shown, to generate in order to obtain a uniform spatial distribution of the measurement points in the input data space and a high number of excitation modes.
[0027] In Fig. Figure 3 is a flowchart illustrating a first procedure for generating measurement points that dynamically and spatially fill the input data space.
[0028] In step S1, a first sequence of values is generated for each input dimension, i.e., for each input variable of the measurement points, using suitable methods. This first sequence consists of a series of numbers that are as uniformly distributed as possible. For example, the first sequence of values can be provided in the form of a first Sobol sequence. The values of the respective first sequence serve to define the step heights of the corresponding input variable and correspond to a step height value sequence. Instead of the Sobol sequence, other number sequences, preferably number sequences that provide uniformly distributed values, can also be used for one or more of the input variables.
[0029] In step S2, a second sequence of values is generated as a sequence of uniformly distributed random numbers. For example, the second sequence can be provided in the form of a second Sobol sequence. Instead of the Sobol sequence, other sequences of numbers that provide uniformly distributed values can also be used for one or more of the input variables. The values of the second sequence serve to define the step lengths L and correspond to a step length value sequence. The step lengths can be specified as integers and indicate how many times a value corresponding to a step height is provided for the respective input variable before the next value in the step height value sequence is repeated, according to the number of times specified by the next value in the step length value sequence.
[0030] For both sequences of values, these are provided scaled to the permissible range of values specified for the respective input variable. Alternatively, the sequences of values can be further subdivided into subsets and permuted to obtain a better mixing of the values.
[0031] In step S3, starting from an initial value for each input variable, a first value (or, in subsequent loop iterations, a next value) is taken from the first sequence of values and applied to the specified initial value or the last reached value of the respective input variable. This application can be performed using a calculation method, such as addition or subtraction.
[0032] If, in step S4, it is determined that a value obtained from one of the input variables lies outside the permissible range specified for that input variable (alternative: Yes), then in step S8 the value of the input variable in question is modified to a value that lies within the permissible range for that input variable. This can be done, for example, by mirroring the value against the exceeded limit of the permissible range for that input variable. If, in step S4, it is determined that the value obtained from one of the input variables lies within the permissible range specified for that input variable (alternative: No), then the procedure continues with step S5.
[0033] In step S5, it is checked whether a number of consecutive measurement points, each defined as a combination of the corresponding values of the input variables, corresponds to a predefined maximum number or whether a predefined number of step heights has been reached. If this is the case (alternative: Yes) (termination condition), the procedure continues with step S6. In this way, a sequence of step height values is generated. Otherwise (alternative: No), the process returns to step S3. Steps S3, S4, and, if applicable, S8 are then repeated until the termination condition of step S5 is met.
[0034] In step S6, the step lengths L of the step length sequence assigned to each input variable are assigned to the individual step heights of the step height value sequences associated with the respective input variable. This assignment can be made in a sequence determined by the order of the step height value sequences and / or the step length value sequence. For example, for each or a subset of input variables, the corresponding first, second, etc. step length in the step length value sequence assigned to the respective input variable can be assigned to the first, second, etc. step heights of the step height value sequences of the respective input variables. The step lengths correspond to or specify the number of consecutive constant values of the step heights.
[0035] For example, if a specific step height is assigned a step length of 4, then the value of that specific step height will appear four times consecutively in the resulting curve of the input variable to be determined. This yields a step-like function for each input variable, defined by the sequences of step heights and their corresponding step lengths. In this way, the combination of sequences of input variable values generates measurement points that correspond to a uniform distribution in the input data space with as many excitation modes as possible.
[0036] In general, generating sequences of input values for each input variable can involve generating the input values of each input variable in a sequence determined by concatenating the step heights according to their order in the step height value sequence assigned to the respective input variable. Each step height can be assigned a number of successive steps, which results from the sequence of step lengths assigned to the respective input variable in the step length value sequence.
[0037] In Fig. Figure 4 shows a flowchart to illustrate another method for generating measurement points.
[0038] In step S11, an initial sequence of values is generated for each input variable of the measurement points, consisting of a sequence of numbers that are as uniformly distributed as possible. For example, this initial sequence can be provided in the form of a first Sobol sequence. The values of the respective initial sequence serve to define the step sizes of the corresponding input variable. Instead of the Sobol sequence, other number sequences that provide uniformly distributed values can also be used for one or more of the input variables.
[0039] In step S12, a second sequence of values is generated as a sequence of uniformly distributed random numbers. For example, the second sequence can be provided in the form of a second Sobol sequence. Instead of the Sobol sequence, other number sequences that provide uniformly distributed values can also be used for one or more of the input variables. The values of the second sequence serve to define the stage lengths L and correspond to a stage length value sequence. The stage lengths can be specified as integers and indicate how many times a value from the associated first sequence, which is to correspond to a stage, is repeatedly provided for the respective input variable before the next value of the stage length is repeated according to the number specified by the next value of the second sequence (stage length value sequence).
[0040] In an optional step S13, each of the level-height value sequences is subdivided into subsets, and a permutation, i.e., an exchange or mixing of the individual subsets of the value sequence in question, is performed and adopted as the new level-height value sequence.
[0041] In step S14, the jump heights between the values of each of the permuted step height sequences are determined from the uniformly distributed values of the step height sequences obtained for the input variables. This results in a jump height sequence for each of the input variables.
[0042] In step S15, a distribution of jump heights is determined for each of the input variables, for example using a histogram.
[0043] In step S16, a threshold comparison is performed regarding the uniform distribution of the step heights. If it is determined that a deviation from a uniform distribution of the step heights (difference between a maximum value in the histogram and a minimum value in the histogram) exceeds a certain threshold (alternative: Yes), in the optional step S17, the previously performed permutation from step S13 is undone for the relevant input variable(s), and the process returns to step S13, decreasing the size of the subsets, i.e., the number of values of the step height sequence assigned to the relevant subset, for the relevant input variable(s). If it is determined that a deviation from a uniform distribution does not exceed a certain threshold (alternative: No), the procedure continues with step S18.
[0044] In step S18, the sequence of measurement points is determined for the step height value sequences of the input variables obtained most recently through the permutation. This sequence is derived from the step length value sequences assigned to each input variable. That is, in step S18, as in step S6, the step lengths L of the step length value sequence assigned to each input variable are assigned to the individual step heights of the (most recently permuted) step height value sequences assigned to the respective input variable. The assignment can be made in a sequence determined by the order of the step height value sequences and / or the step length value sequence. For example, for each or a subset of the input variables, the corresponding first, second, etc. step length in the step length value sequence assigned to each input variable can be assigned to the first, second, etc. step heights of the step height value sequences of the respective input variables.The step lengths correspond to, or specify, the number of consecutive constant values of the step heights.
[0045] This method ensures that the step heights occupy the input data space in an even distribution and that the excitation modes are also distributed as evenly as possible.
[0046] In Fig. Figure 5 shows a flowchart illustrating another method for generating measurement points.
[0047] In step S21, a first sequence of values is generated for each input dimension, i.e., for each input variable of the measurement points, using suitable methods. This first sequence consists of a series of numbers that are as uniformly distributed as possible. For example, the first sequence of values can be provided in the form of a first Sobol sequence. The values of the respective first sequence serve to define the step heights of the corresponding input variable and correspond to a step height value sequence. Instead of the Sobol sequence, other number sequences, preferably sequences that provide uniformly distributed values, can also be used for one or more of the input variables.
[0048] In step S22, a second sequence of values is generated as a sequence of uniformly distributed random numbers. For example, the second sequence can be provided in the form of a second Sobol sequence. Instead of the Sobol sequence, other sequences of numbers that provide uniformly distributed values can also be used for one or more of the input variables. The values of the second sequence serve to define the step lengths L and correspond to a step length value sequence. The step lengths can be specified as integers and indicate how many times a value corresponding to a step height is provided for the respective input variable before the next value in the step height value sequence is repeated, according to the number of times specified by the next value in the step length value sequence.
[0049] In an optional step S23, each of the level-height value sequences is subdivided into subsets, and a permutation, i.e., an exchange or mixing of the individual subsets of the value sequence in question, is performed and adopted as the new level-height value sequence.
[0050] In step S24, the jump heights between the values of each of the permuted value sequences are determined from the uniformly distributed values of the step height value sequences obtained for the input variables. This results in a jump height value sequence for each of the input variables.
[0051] In step S25, the jump heights of each input variable are sorted, for example, in ascending order, and a weighting measure between the jump heights is determined, which can represent a measure of uniformity. This weighting measure serves to assess uniformity between the jump heights and to make the distributions of the jump heights comparable. The weighting measure can be defined as a distance measure. The distance measure of the jump heights is calculated as the sum of the squared distances between each adjacent jump height, i.e., the ∑ i (h i+1 - h i ) 2 This represents the distance measure. Alternatively, the maximum distance between any two adjacent jump heights in the sorted list of jump heights can be used as the distance measure.
[0052] In step S26, a threshold comparison is performed with respect to the jump height rating. If it is determined that a deviation from a rating exceeds a certain threshold (alternative: Yes), in step S27 the previously performed permutation in steps S23 and S24 for the relevant input variable(s) is reversed, and the process returns to step S23, reducing the size of the subsets, i.e., the number of values in the sequence assigned to the relevant subset, for the relevant input variable(s). If it is determined that a deviation from a rating does not exceed a certain threshold (alternative: No), the procedure continues with step S28.
[0053] According to an alternative embodiment, in step S25 the evaluation measure can be determined based on the occurring jump heights across all value sequences of the input variables. In step S26, a threshold comparison can then be performed with respect to the evaluation measure of the jump heights. If it is determined that a deviation from the evaluation measure exceeds a certain threshold (alternative: Yes), in step S27 the previously performed permutation in steps S23 and S24 is reversed for all input variables, and the process returns to step S23, reducing the size of the subsets, i.e., the number of values of the value sequence assigned to the subsets, for all input variables. If it is determined that a deviation from an evaluation measure does not exceed a certain threshold (alternative: No), the procedure continues with step S28.
[0054] Alternatively or additionally, another termination criterion for the iteration can be the maximum number of iterations, so that if in step S27 it is determined that a certain number of repetitions of steps S23-S27 have been performed, the procedure continues with step S28.
[0055] Alternatively or additionally, a further termination criterion for the iteration can be checked to see whether the distance from a jump height distribution, determined by the calculated evaluation measure, increases again in successive loop iterations S23-S27. If the determined evaluation measure increases, the last permutation performed in step S24 can be reversed and the procedure continued with step S28.
[0056] In step S28, the step lengths of the input variables are assigned to the value sequences obtained through the permutation, based on the second value sequences assigned to each input variable, according to the order of their respective value sequences, in order to determine the sequence of measurement points. That is, the first, second, etc., step heights of each input variable are assigned the corresponding first, second, etc., step length of the respective input variable. This results in a step-like function for each input variable, defined by the step height value sequences and their corresponding step lengths. In this way, the combination of input value sequences generates measurement points that correspond to a uniform distribution in the input data space with as many excitation modes as possible.
[0057] The evaluation measure can be determined as a divergence measure instead of a distance measure. The divergence measure can be a known discrete Kulback-Leibler divergence measure between a given desired distribution and the distribution of jump heights obtained according to the sequence of values for jump heights in step S25. The desired distribution of jump heights can correspond to an equally spaced distribution of jump heights or a random distribution of jump heights.
Claims
[1] A method for providing a sequence of measurement points, in particular for carrying out a test procedure for measuring a physical unit, wherein each measurement point is defined by a combination of one input value from several inputs according to the sequence of input values, wherein the sequence of input values of each of the inputs corresponds to a step function with steps having a step height (H), a jump height (S) between the steps and a step length (L); comprising the following steps: - Providing (S3, S4, S5; S11, S21) a step height value sequence for each of the input variables, wherein the step height value sequence corresponds to a uniformly distributed value sequence and specifies a sequence of step heights (H) along the course of the step function of the input variable in question; - Providing (S2, S12, S22) a step length value sequence for each of the input variables, wherein the step length value sequence corresponds to a uniformly distributed value sequence and specifies a sequence of step lengths (L) along the course of the step function; - Generating (S3, S4, S5; S13, S14, S15, S16; S23, S24, S25, S26) sequences of input values for each of the inputs depending on the sequence of step heights (H) and depending on the sequence of step lengths (L), wherein generating sequences of input values for each of the inputs comprises generating the input values of each of the inputs in a sequence resulting from concatenating the step heights according to their order in the step height value sequence assigned to the input in question, wherein each of the step heights is provided successively with a number resulting from the sequence of step lengths assigned to the input in question in the step length value sequence; - Generating (S6, S18, S28) the sequence of measurement points by combining the corresponding input values of the sequences of input values of the multiple input variables. [2] Method according to claim 1, wherein generating sequences of input variable values for each of the input variables comprises providing at least several of the step heights consecutively in a number corresponding to a step length from the sequence of step lengths assigned to the input variable in question in the step length value sequence. [3] Method according to one of claims 1 to 2, wherein the respective step height value sequence is provided by providing a step height value sequence as a uniformly distributed value sequence for each input variable, wherein, starting from an initial value, the step height values are successively applied to the initial value or the previously determined value, in particular by addition or subtraction, in order to obtain the step height value sequence, wherein, in particular, the application is carried out in such a way that each value of the step height value sequence is limited to a value within a predetermined permissible value range. [4] Method according to one of claims 1 to 3, wherein at least a part of the step height value sequences assigned to the input variables is generated by permutation of subsets from a uniformly distributed value sequence (S13, S23). [5] Method according to claim 4, wherein the size of the subsets from the uniformly distributed sequence of values is iteratively reduced as long as an evaluation measure for a uniform distribution of jump heights resulting from the permuted sequence of values assumed to be step heights indicates for one or more of the input variables a deviation from an ideal uniform distribution that is greater than a predetermined threshold. [6] Method according to claim 4 or 5, wherein the size of the subsets from the uniformly distributed sequence of values is iteratively reduced as long as, based on an evaluation measure for a uniform distribution of jump heights resulting from the permuted sequence of values assumed to be step heights, each iteration results in a smaller deviation from an ideal uniform distribution. [7] Method according to any one of claims 4 to 6, wherein the size of the subsets from the uniformly distributed sequence of values is iteratively reduced until a predetermined maximum number of iterations has been performed. [8] Method according to any one of claims 4 to 7, wherein the evaluation measure depends on a variation of frequencies of classified jump height values. [9] Method according to any one of claims 4 to 7, wherein the evaluation measure corresponds to a distance measure with respect to the jump heights of the jump height value sequence and is specified in particular as a sum of the absolute or square distances between any two jump heights that are closest with respect to their value. [10] Method according to one of claims 4 to 7, wherein the evaluation measure corresponds to a divergence measure of the jump heights of the jump height value sequence and is determined in particular by a discrete Kulback-Leibler divergence. [11] Device, in particular computing unit, configured to perform the method according to any one of claims 1 to 10. [12] Computer program which is configured to perform all steps of a method according to any one of claims 1 to 10. [13] Machine-readable storage medium on which a computer program according to claim 12 is stored.
Citation Information
Patent Citations
CN000104297626A
Stochastic random variables generating method for use in e.g. power line communications of motor vehicle, involves generating random variables by quantile transformation of distribution function of stored measuring data
DE102005063273A1
Video quality measurement
US20050281333A1
Method for generating a sequence of random numbers of A 1 / f-noise
US6795840B1