Method and device for training classifier

Abstract

A computer-implemented method (700) for determining a quality criterion (g) for a data set, where the method comprises the steps of: a. Obtaining the data set (701), where the data set comprises a plurality of sensor data, and for sensor data of the plurality of sensor data, respectively, an environmental state (802), wherein the environment state (802) corresponds to the sensor data and contains information about a state of the environment in which the sensor data is recorded; b. Determining (702) an envelope (800) of the environmental state (802) in the space of the environmental state; c. Determining (703) the coverage of the envelope; d. Providing (704) the coverage as a quality criterion (g).

Description

BACKGROUND

[0001] Advantages of the invention The system design, training or validation of sensors for environmental perception, such as for automated object recognition in autonomous vehicles, requires the availability of sensor data, often even annotated sensor data. It would be desirable for these sensor data to cover the possible space of environmental states as completely as possible in order to thereby obtain as good a statement as possible about the system design, training or validation. Semantic context can be understood as the environmental state under which the data has been recorded. For example, for a camera image as data the corresponding environmental state prevailing at the time of recording the camera image can be recorded, such as the time, the weather conditions, the position of the light source and its lighting properties, etc.

[0002] However, in known methods for generating a dataset of sensor data these environmental states are not detected together or, if at all, only partially and unsystematically.

[0003] The invention proposes that specific environmental data are detected together purposefully when creating a dataset of sensor data. It is further proposed that a quality criterion is determined on the basis of the environmental data detected together, which can be understood as a degree of maturity of the dataset. On the basis of the degree of maturity it can then be assessed how suitable the dataset is for system design, training or validation.

[0004] The developer can then advantageously assess to what extent the dataset is suitable for system design, training or validation. On the basis of this the dataset can then be used or, if the degree of maturity is assessed to be too small, the dataset can be enriched with additional sensor data or metadata. SUMMARY

[0005] In a first aspect the invention relates to a computer-implemented method for determining a quality criterion of a dataset, wherein the method comprises the following steps: a. obtaining the dataset, wherein the dataset comprises a plurality of sensor data and comprises for a sensor data of the plurality of sensor data respectively an environmental state, wherein the environmental state corresponds to the sensor data and contains information about the state of an environment under which the sensor data has been recorded; b. determining an envelope of the environmental states in the space of the environmental states; c. determining a coverage of the envelope; d. providing the coverage as a quality criterion.

[0006] The data set can be understood as being set up for developing a machine learning system, training the machine learning system and / or validating a machine learning system. The quality criterion can be understood as a degree of maturity which characterizes to what extent a data set is suitable for developing, training and / or validating a machine learning system.

[0007] The data set comprises sensor signals. For at least a portion of the sensor signals, the data set comprises a corresponding environment state. Preferably, for all sensor signals comprised by the data set, there is a corresponding environment state.

[0008] The environment state can be understood as characterizing one or more conditions of the environment under which the corresponding sensor data has been recorded. Here, the sensor data can be understood as a record of a specific property of the environment. For example, an image can characterize a segment of the environment of a camera. For example, the environment state can characterize the daytime point at which the sensor data was recorded, or the physical conditions present at the recorded point in time.

[0009] In particular, the sensor data can be an image recorded or having been recorded by a corresponding optical sensor. Here, the sensor can in particular be a camera, a lidar sensor, an imaging radar sensor, an ultrasonic sensor or a thermal imager. In this embodiment, images are provided in the data set, wherein the environment states are additionally provided together in the data set respectively. In the case of images, the environment state can in particular comprise the lighting or radiation properties of the light source at the recording time of the image, the relative position of the light source with respect to the optical sensor, the object properties of the objects imaged in the image, the properties of the optical sensor, the radio meteorological properties and / or the position of the light emitting or light emitting objects.

[0010] At least a portion of the environment state can be recorded by other sensors. It is also possible to derive a corresponding portion of the environment state on the basis of the sensor data. For example, by object recognition, corresponding objects can be detected on the recorded images and their properties in the environment state are provided. It is also possible for at least a portion of the environment state to be determined manually by a person.

[0011] The environment state can in particular be understood as a numerical representation of the corresponding conditions of the environment. The environment state is preferably present in the form of a vector, the individual dimensions of which respectively characterize a corresponding condition of the environment. The space of the environment state can therefore in particular be understood as a mathematical space.

[0012] For determining the quality criterion, in the method an envelope is determined in the space of the environmental states. The envelope can be understood as, inter alia, a desired range of values within which the environmental states are contained in the data set. Since the sensor data corresponds to the environmental states, the envelope can be understood as a representation of the desired complete sensor data in the sensor data with respect to its environmental states.

[0013] The determination of the envelope can be carried out in a variety of ways. For example, it is possible that the envelope is defined by a specification requirement and thus determined from a required range of values of a corresponding specification, for example a regulation or an industry standard. Alternatively, it is also possible that the envelope is defined on the basis of other developments.

[0014] In other words, the envelope preferably characterizes a desired coverage of the dimensions of the environmental states and thus of the situation at the time of the recording of the sensor data.

[0015] The envelope is preferably a convex envelope.

[0016] Advantageously, the coverage of the envelope can be determined more quickly in the case of a convex envelope than in the case of a concave envelope and thus the resource requirement of the method is reduced.

[0017] Preferably, the determination of the coverage can comprise the following steps: e. determining regions of a predefined size, wherein the midpoints of the regions respectively correspond to an environmental state; f. determining the volume of the regions within the envelope; g. providing the ratio of the determined volume to the volume of the envelope as the coverage.

[0018] The method can be understood such that the environmental states respectively act as representatives of regions of the space of the environmental states. Here, the regions can preferably be hypercubic hexahedrons or hyperspheres. However, other shapes of the regions are also possible, the hypercubic hexahedrons or hyperspheres being merely preferred.

[0019] The volume of the entire region can then be determined. This can be carried out simply by choosing a hypersphere or hypercubic hexahedron, in particular in the case of overlapping regions. The ratio of this volume of all regions to the volume of the envelope can then be provided as the coverage.

[0020] In another aspect, the invention relates to a method for determining a quality criterion of a data set, comprising the following steps: h. recording the data set, wherein the recording comprises the following steps: • recording a plurality of sensor data by means of a sensor; • documenting an environmental state for sensor data of the plurality of sensor data; • providing sensor data and environmental states of the data set; i. determining the quality criterion based on the data set according to one of the embodiments of the method for determining a quality criterion.

[0021] The method is essentially the same as the computer-implemented method for determining a quality criterion, wherein the data set is additionally documented in the method before the data set is provided to the computer-implemented method.

[0022] In another aspect, the present invention relates to a computer-implemented method for determining a first data set, comprising the following steps: j. determining a quality criterion for a second data set according to one of the preceding embodiments; k. if the quality criterion reaches or exceeds a predefined threshold: • providing the second data set as the first data set; l. else: • determining at least one region within the envelope, the filling of which leads to an improvement of the quality criterion; • enriching the data set with at least one additional data, wherein the environmental state corresponding to this data falls in the region; • repeating step j. and either step k. or l. with the enriched data set.

[0023] The method can be understood as follows, such that in view of a sufficient coverage with regard to the environmental states, an existing second data set is checked. If the coverage is sufficiently large, the second data set can be used directly as the first data set.

[0024] Otherwise, it can be provided that the second data set is enriched. In this regard, enrichment can be understood as an extension (sometimes also referred to as augmentation) of the second data set.

[0025] Here, the enrichment can be performed in different ways. For example, it can be possible that in the enrichment step, at least one data is an output of a sensor, which is provided to the method, and for which a corresponding environmental state is provided. In other words, the data set can be extended by another data documented by a sensor.

[0026] Alternatively or additionally, it can be possible that in the enrichment step, data and an environmental state corresponding to the data are synthesized, and the data set is extended by the synthesized data and the corresponding environmental state.

[0027] In other words, the necessary data for enriching can also be generated "artificially". It is possible, for example, to train a generative machine learning model on the basis of the existing data of the data set and then to determine new data points by means of the generative model. For this purpose, various models are conceivable. For example, a conditional generative model can be used, for example a conditional generative adversarial network or a conditional normalizing flow or a conditional diffusion model. BRIEF DESCRIPTION OF DRAWINGS

[0028] Embodiments of the application are explained in more detail below with reference to the accompanying drawings. In the drawings: Figure 1 A flow chart of a method for determining a quality criterion of a data set is shown; Figure 2 A space of environmental states is schematically shown; Figure 3 A method for determining a data set is shown; Figure 4 A training system is schematically shown; Figure 5 The structure of a control system for maneuvering an effector is schematically shown; Figure 6 An embodiment for controlling an at least partially autonomous robot is schematically shown; Figure 7 An embodiment for controlling a manufacturing system is schematically shown; Figure 8 An embodiment for controlling an access control system is schematically shown; Figure 9 An embodiment for controlling a surveillance system is schematically shown; Figure 10 An embodiment for controlling a personal assistant is schematically shown; Figure 11 An embodiment for controlling a medical imaging system is schematically shown. DETAILED DESCRIPTION

[0029] Figure 1 A flow chart of a method for determining a quality criterion of a data set is shown.

[0030] In a first step (701) a data set is received, which comprises sensor data and the environmental state respectively corresponding in the sensor data. Preferably, the sensor data and the environmental state can also be logged with the corresponding sensor as part of the first step. The sensor data is preferably an image and thus the sensor is an optical sensor. The environmental state preferably comprises one or more of the following conditions: • the illumination or radiation characteristics of the light source at the time of recording of the image, in particular: ° the intensity or propagation characteristics (e.g. measured by a calibrated radiation or brightness camera or from the response to a calibrated test target) ° the spectral composition of the radiation measured by the sensor • the relative position of the light source with respect to the optical sensor (e.g. determined by detecting the light source in the image and subsequently determining the position by means of intrinsic and extrinsic sensor parameters) • object characteristics of the objects imaged in the image, in particular: ° the size or scale of the objects ° the surface structure of the objects (e.g. determined by manual annotation or by automated methods as for example from the field of machine learning) ° the reflectivity ° the orientation with respect to the light source and / or the sensor • characteristics and working point of the optical sensor (as can be seen from the specifications of the sensor and the set parameters), for example: ° the dynamic range of the sensor ° the contrast recognition probability for brightness and / or chromaticity contrasts ° the geometric resolution of the sensor ° the opto-electric transfer function • the radio meteorological characteristics ° the size, density and velocity of particles in the air (e.g. available from meteorological stations with precipitation measurement and classification) ° reference sensor systems (e.g. laser-based systems for making distance determinations, global diffuse radiation sensors or "sky cameras", i.e. cameras with a 180° aperture for observing the visibility of the sky) • the position of light emitting or luminous objects (e.g. determined from the sensor data or by GPS coordinates).

[0031] In a second step (702) an envelope (Hülle) of the environmental state is determined in the space of the environmental state. Figure 2 Exemplary representations of such an envelope (800) are shown. For example, as in Figure 2As shown in the middle, the envelope (800) can be a surrounding convex envelope of the environmental states of the data set. In other embodiments (not shown), the envelope (800) can also have other shapes, for example, be wider than the outermost environmental state of the set of environmental states of the data set.

[0032] In a third step (703) of the method, the coverage (Abdeckung) of the envelope (800) is determined. For this, preferably for each environmental state (802) a region (801) around the corresponding environmental state (802) is determined. This region can be, for example, a hyper-sphere (see Figure 2 a) or a hyper-rectangular hexahedron, for example, a hyper-cube (see Figure 2 b). Subsequently, the ratio of the sum of the volumes included by the regions (801) to the volume of the envelope (800) can then be used as the determined coverage.

[0033] In a fourth step (704), the coverage is then provided as the quality criterion (g).

[0034] Figure 3 The flow of the method (900) for determining the first data set is shown schematically. In this method, in a first step, the quality criterion (g) of the second data set is determined according to an embodiment of the method (700) for determining the quality criterion (g).

[0035] If the quality criterion (g) exceeds a predefined threshold value, the second data set can be provided directly (902) as the first data set. This can be understood as the second data set already having a sufficient coverage of the space of environmental states.

[0036] Otherwise, the second data set can be enriched with further sensor data and corresponding environmental states. For this, in a further step, first the regions within the envelope (800) can be determined (903) which are not already covered by the regions (801) around the environmental states (802).

[0037] Subsequently, in a further step (904), the sensor data which can have led to the corresponding environmental state (802) can be determined. For this, for example, the environmental state (802) can be examined and the as far as possible matching situation in the real world can be mapped and the corresponding sensor data in this situation can be documented.

[0038] However, the sensor data can preferably also be synthesized. For this purpose, inter alia, a machine learning model, in particular a conditional generative model, such as a conditional generative adversarial network (English: conditional generative adversarial networks), a conditional normalizing flow (English: conditional normalizing flows) or a conditional diffusion model (English: conditional diffusion models) can be used. The model can be trained, for example, such that it synthesizes (also referred to as generates or samples) sensor data for a given environment state. Thereby, in the method, an uncovered region of the envelope can be determined, for which a midpoint can be determined and which can be passed as a condition input to the generative model, wherein the output of the generative model then characterizes a possible sensor signal for the passed environment state. The environment state and the synthesized sensor data can then be admitted together to the dataset.

[0039] With the enriched dataset, the method can then be repeated (905) from scratch. Thereby, the dataset can be iteratively always further enriched until the quality criterion has finally exceeded a predefined threshold, after which the method ends.

[0040] Figure 3 An embodiment of a training system (140) for training a machine learning system (60) by means of a dataset which has been determined in accordance with an embodiment from Figure 3 The dataset can in this case be understood as a training dataset (T). The training dataset (T) comprises a plurality of sensor data, which are also referred to below as input signals (x i ). The training dataset (T) furthermore comprises, for each input signal (x i ), an expected output signal (t i ) which corresponds to the input signal (x i ) and which characterizes a desired result of a regression analysis and / or a classification with respect to the input signal (x i ). The output signal (t i ) can be admitted to the dataset, for example, by means of manual annotation or an automated annotation method.

[0041] For the training, the training data unit (150) accesses a computer-implemented database (St2), wherein the database (St2) provides the training dataset (T). The training data unit (150) determines at least one input signal (x i ) and the expected output signal (t i ) which corresponds to the input signal (x i), and an input signal (x i ) is transmitted to a machine learning system (60). The machine learning system (60) determines an output signal (y i ) on the basis of the input signal (x i ).

[0042] The desired output signal (t i ) and the determined output signal (y i ) are transmitted to a change unit (180).

[0043] The new parameters (Φ') are then determined for the machine learning system (60) by the change unit (180) on the basis of the desired output signal (t i ) and the determined output signal (y i ). To this end, the change unit (180) compares the desired output signal (t i ) and the determined output signal (y i ) by means of a loss function. This loss function determines a first loss value which characterizes to what extent the determined output signal (y i ) deviates from the desired output signal (t i ). In this embodiment, a negative log-likehood function is selected as the loss function. In alternative embodiments, other loss functions are also conceivable.

[0044] Furthermore, it is conceivable that the determined output signal (y i ) and the desired output signal (t i ) each comprise a plurality of sub-signals, for example in the form of tensors, wherein the sub-signals of the desired output signal (t i ) each correspond to a sub-signal of the determined output signal (y i ). For example, it is conceivable that the machine learning system (60) is configured for object detection, and that a first sub-signal each characterizes the probability of occurrence of an object with respect to a portion of the input signal (x i ), and a second sub-signal characterizes the precise position of the object. For the case that the determined output signal (y i ) and the desired output signal (t i ) comprise a plurality of corresponding sub-signals, a second loss value is preferably determined for the respective corresponding sub-signals by means of a suitable loss function, and the determined second loss values are suitably aggregated into the first loss value, for example by means of a weighted sum.

[0045] The changing unit (180) determines new parameters (F) based on the first loss value. In this embodiment, this takes place by means of a gradient descent method, preferably Stochastic Gradient Descent Adam or AdamW. In other embodiments, the training can also be based on evolutionary algorithms or second-order optimization (English: econd-order optimization).

[0046] The determined new parameters (F) are stored in the model parameter memory (Sti). Preferably, the determined new parameters (F) are provided as parameters (F) to the machine learning system (60).

[0047] In other preferred embodiments, the described training is repeated iteratively for a predefined number of iteration steps, or iteratively until the first loss value does not exceed a predefined threshold. Alternatively or additionally, it is also conceivable to end the training when the average first loss value with respect to a test or validation data set does not exceed a predefined threshold. In at least one of the iterations, the new parameters (F) determined in the previous iteration are used as parameters (F) of the machine learning system (60).

[0048] Furthermore, the training system (140) can comprise at least one processor (145) and at least one machine-readable storage medium (146), which contains instructions which, when executed by the processor (145), cause the training system (140) to carry out the training method according to one of the aspects of the application.

[0049] Figure 5 A control system (40) is shown, which is set up for determining a control signal (A) for an effector (10) in an environment (20) of the effector (10), wherein the control signal (A) is determined on the basis of an output of a machine learning system (60). The environment (20) is detected in a sensor (30), in particular an imaging sensor, for example a camera sensor, at preferably regular time intervals, which can also be given by a plurality of sensors (for example a stereo camera). The sensor signal (S) of the sensor (30) (or in the case of a plurality of sensors each sensor signal (S)) is transmitted to the control system (40). The control system (40) thus receives a sequence of sensor signals (S). From this, the control system (40) determines a control signal (A), which is transmitted to the effector (10).

[0050] The control system (40) receives a sequence of sensor signals (S) of the sensor (30) in an optional receiving unit (50), which converts the sequence of sensor signals (S) into a sequence of input signals (x) (alternatively, the sensor signals (S) can also be accepted directly as input signals (x), respectively). For example, the input signals (x) can be a section of the sensor signals (S) or a further processing. In other words, the input signals (x) are determined from the sensor signals (S). The sequence of input signals (x) is fed to the machine learning system (60).

[0051] The machine learning system (60) is preferably parameterized by parameters (Φ), which are stored in a parameter store (P) and provided by the parameter store.

[0052] The machine learning system (60) determines an output signal (y) from the input signals (x). The output signal (y) is fed to an optional transformation unit (80), which determines therefrom a control signal (A), which is fed to the actuator (10) in order to control the actuator (10) accordingly.

[0053] The actuator (10) receives the control signal (A), is controlled accordingly, and performs a corresponding action. In this case, the actuator (10) can comprise (not necessarily structurally integrated) control logic, which determines a second control signal from the control signal (A) and then controls the actuator (10) with the second control signal.

[0054] In other embodiments, the control system (40) comprises the sensor (30). In yet other embodiments, the control system (40) alternatively or additionally also comprises the actuator (10).

[0055] In other preferred embodiments, the control system (40) comprises at least one processor (45) and at least one machine-readable storage medium (46), on which commands are stored, which, when executed on the at least one processor (45), cause the control system (40) to perform the method according to the application.

[0056] In alternative embodiments, instead of or in addition to the actuator (10), a display unit (10a) is provided.

[0057] Figure 6 It is shown how the control system (40) can be used for controlling an at least partially autonomous robot, here an at least partially autonomous motor vehicle (100).

[0058] The sensor (30) can be, for example, a video sensor, which is preferably arranged in the motor vehicle (100).

[0059] The machine learning system (60) is set up to identify objects identifiable on the basis of the input signals (x).

[0060] The actuators (10) which are preferably arranged in the motor vehicle (100) can be, for example, brakes, drive devices or steering devices of the motor vehicle (100). The control signals (A) can then be determined in such a way that the actuator or actuators (10) are controlled in such a way that the motor vehicle (100) is prevented from colliding with the objects identified by the machine learning system (60), in particular when this is an object of a specific class, for example a pedestrian.

[0061] Alternatively or additionally, the display unit (10a) can be controlled with the control signals (A) and, for example, represents the identified objects. It is also conceivable that the display unit (10a) is controlled with the control signals (A) in such a way that it outputs an optical or acoustic warning signal when it is determined that the motor vehicle (100) is threatened with a collision with one of the identified objects. The warning by means of the warning signal can also take place by means of a haptic warning signal, for example by means of a vibration of the steering wheel of the motor vehicle (100).

[0062] Alternatively, the at least partially autonomous robot can also be another mobile robot (not depicted), for example a robot which moves forwards by means of flight, swimming, diving or walking. The mobile robot can be, for example, an at least partially autonomous lawnmower or an at least partially autonomous cleaning robot. Even in these cases, the control signals (A) can be determined in such a way that the drive devices and / or steering devices of the mobile robot are controlled in such a way that the at least partially autonomous robot is, for example, prevented from colliding with the objects identified by the machine learning system (60).

[0063] Figure 7 An embodiment is shown in which the control system (40) is used to control the manufacturing machines (11) of the manufacturing system (200) in such a way that actuators (10) of the manufacturing machines (11) are controlled. The manufacturing machines (11) can be, for example, machines for punching, sawing, drilling and / or cutting. It is furthermore conceivable that the manufacturing machines (11) are configured to grasp the manufacturing products (12a, 12b) by means of grippers.

[0064] The sensor (30) can thus for example be a video sensor which for example detects a conveying surface of a conveyor belt (13) on which the manufactured products (12a, 12b) can be located. The machine learning system (60) can for example be set up to determine a position of the manufactured products (12a, 12b) on the conveyor belt. The actuator (10) of the manufacturing machine (11) can then be manipulated in accordance with the determined position of the manufactured products (12a, 12b). For example, the actuator (10) can be manipulated such that it stamps, saws, drills and / or cuts the manufactured products (12a, 12b) at predetermined positions of the manufactured products (12a, 12b).

[0065] Furthermore, it is conceivable that the machine learning system (60) is configured to determine further properties of the manufactured products (12a, 12b) instead of or in addition to the position. In particular, it is conceivable that the machine learning system (60) determines whether the manufactured products (12a, 12b) are defective and / or damaged. In this case, the actuator (10) can be manipulated such that the manufacturing machine (11) sorts out defective and / or damaged manufactured products (12a, 12b).

[0066] Figure 8 An embodiment is shown in which the control system (40) is used for controlling an access control system (300). The access control system (300) can comprise a physical access controller, for example a door (401). The sensor (30) can in particular be a video sensor or a thermal imaging sensor which is set up to detect an area in front of the door (401). The detected images can be interpreted by means of the machine learning system (60). In particular, the machine learning system (60) can detect persons on the conveyed input images (x). If several persons are detected at the same time, the identity of the persons can be determined particularly reliably by assigning the persons, i.e. objects, to one another, for example by analyzing the movements of the persons.

[0067] The actuator (10) can be a lock which releases or does not release the access controller, for example opens or does not open the door (401), in accordance with the manipulation signal (A). For this purpose, the manipulation signal (A) can be selected in accordance with the output signal (y) determined by means of the machine learning system (60) for the input images (x). For example, it is conceivable that the output signal (y) comprises information characterizing the identity of the persons detected by the machine learning system (60) and that the manipulation signal (A) is selected on the basis of the identity of the persons.

[0068] Instead of a physical access controller, a logical access controller can also be provided.

[0069] Figure 9An embodiment is shown, in which a control system (40) is used for controlling a surveillance system (400). This embodiment differs from the embodiment shown in Figure 8

[0070] Figure 10 An implementation is shown, in which a control system (40) is used for controlling a personal assistant (250). The sensor (30) is preferably an optical sensor, for example a video sensor or a thermal imaging camera, which receives an image of a gesture of a user (249).

[0071] The control system (40) determines a control signal (A) for the personal assistant (250) from the signal of the sensor (30), for example in that a machine learning system (60) performs gesture recognition. This determined control signal (A) is then transmitted to the personal assistant (250) and thereby controls the personal assistant accordingly. In particular, the determined control signal (A) can be selected such that it corresponds to an assumed desired control by the user (249). This assumed desired control can be determined from the gesture recognized by the machine learning system (60). The control system (40) can then select a control signal (A) for transmission to the personal assistant (250) in accordance with the assumed desired control and / or select a control signal (A) for transmission to the personal assistant that corresponds to the assumed desired control (250).

[0072] For example, such a corresponding control can include the personal assistant (250) calling up information from a database and reproducing the information in a manner acceptable to the user (249).

[0073] Instead of a personal assistant (250), a domestic appliance (not depicted), in particular a washing machine, a stove, a cooker, a microwave or a dishwasher, can also be provided in order to be controlled accordingly.

[0074] Figure 11 An embodiment is shown, in which a control system (40) is used for controlling a medical imaging system (500), for example an MRT, an X-ray or an ultrasound device. The sensor (30) can be given by an imaging sensor, for example. A display unit (10a) is controlled by the control system (40).

[0075] ​The sensor (30) is set up to determine an image of a patient, for example an X-ray image, an MRT image or an ultrasound image. At least a part of this image is transmitted as input image (x) to the machine learning system (60). The machine learning system (60) can for example be set up to classify different types of tissue to be identified on the input image (x), for example by means of semantic segmentation.

[0076] The manipulation signal (A) can then be selected such that the determined type of tissue is represented in color-highlighted fashion on the display unit (10a).

[0077] In other embodiments (not shown), the imaging system (500) can also be used for non-medical purposes, for example for determining material properties of a workpiece. For this purpose, the imaging system (500) can record an image of the workpiece. In this case, the machine learning system (60) can be set such that it accepts at least a part of the image as input image (x) and classifies it in terms of material properties of the workpiece. This can for example be done by means of semantic segmentation of the input image (x). The classification thus determined can for example be represented on the display device (10a) together with the input image.

[0078] The term "computer" includes any device for performing computable criteria which can be predefined. These computable criteria can exist in the form of software or can exist in the form of hardware or even in the form of a hybrid composed of software and hardware.

[0079] In general, the plural number can be understood as indexed, i.e. each element of the plural number is assigned a clear index, preferably by assigning consecutive integers to the elements contained in the plural number. Preferably, if the plural number comprises N elements, where N is the number of elements in the plural number, these elements are assigned integers from 1 to N.

Claims

1. A computer-implemented method (700) for determining a quality standard (g) for a dataset, wherein the method includes the following steps: a. Obtain the dataset (701), wherein the dataset includes a plurality of sensor data, and for each of the plurality of sensor data includes an environmental state (802), wherein the environmental state (802) corresponds to the sensor data and contains information about the state of the environment in which the sensor data is recorded; b. Determine (702) the envelope (800) of the environmental state (802) in the space of the environmental state; c. Determine the coverage of the envelope described in (703); d. Provide the coverage described in (704) as a quality standard (g).

2. The method (700) according to claim 1, wherein the envelope (800) is a convex envelope.

3. The method (700) according to claim 1 or 2, wherein the envelope (800) characterizes the desired coverage of the dimension.

4. The method (700) according to any one of claims 1 to 3, wherein the sensor data is image data of the sensor.

5. The method (700) according to claim 4, wherein the environmental state (802) characterizes the lighting characteristics of the environment and / or the characteristics of objects in the environment and / or the characteristics of the sensor and / or the radiation characteristics of the light source and / or the relative position of the light source with respect to the sensor.

6. The method (700) of any one of claims 1 to 5, wherein determining the coverage comprises the following steps: m. Define a region of predefined size (801), such as a supercube or a hypersphere, where the midpoint of the region corresponds to the environmental state (802). n. Determine the volume of the region (801) within the envelope (800); o. Provide the ratio of the determined volume to the volume of the envelope (800) as the coverage.

7. A method for determining quality criteria for a dataset, comprising the following steps: h. Record the dataset, wherein recording includes the following steps: • Record data from multiple sensors using sensors; • Record the environmental state for each of the plurality of sensor data; • Provide sensor data and environmental conditions from the dataset; i. Determine the quality standard based on any one of claims 1 to 6 and the dataset.

8. A computer-implemented method (900) for determining a first dataset, comprising the following steps: j. Determine the quality criteria for the (901) second dataset according to any one of claims 1 to 6; k. If the quality standard reaches or exceeds a predefined threshold, then: • Provide (902) the second dataset as the first dataset; l. Otherwise: • Identify at least one region within the envelope (903), wherein filling the region results in an improvement in the quality standard; • Enrich the dataset (904) with at least one additional data, wherein the environmental state corresponding to the data falls within the region; • Repeat step (905) and either step k or l using the rich dataset.

9. The method of claim 8, wherein in the enrichment step, the at least one data is the output of a sensor, the data is provided to the method, and a corresponding environmental state is provided for the data.

10. The method according to any one of claims 8 or 9, wherein additionally or alternatively, in the enrichment step, data and the corresponding environmental state are synthesized, and the dataset is expanded with the synthesized data and the corresponding environmental state.

11. A computer-implemented method for training a machine learning system (60), comprising the following steps: m. Determine the dataset according to any one of claims 8 to 10; n. Training the machine learning system (60), wherein the dataset is used as the training dataset for the machine learning system.

12. A training device (140) configured to perform the method according to any one of claims 8 to 10.

13. A computer program configured to perform the method according to any one of claims 1 to 8 when executed by a processor (45, 145).

14. A machine-readable storage medium (46, 146) having a computer program as claimed in claim 10 stored thereon.

Images