Method and arrangement for testing an image recognition device
The method addresses the inadequacy of existing validation techniques by simulating and optimizing for realistic physical interference in image recognition devices, ensuring robustness and reliability in safety-critical applications.
Patent Information
- Application Number
- PCT/EP2024/060525
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-15
- Filing Date
- 2024-04-18
- Publication Date
- 2024-11-21
AI Technical Summary
Current methods for testing complex machine learning models, such as deep neural networks, in safety-critical applications like autonomous driving are inadequate as they fail to comprehensively validate the models' performance under various real-world interference scenarios, leading to a risk of overlooking critical failures.
A method involving a perturbation device that simulates realistic physical interference in camera images, using a parameterized parameter vector to modify test images, and an optimization process to maximize deviations in image recognition outputs, ensuring robustness testing across various interference types and scenarios.
This approach effectively reduces the risk of overlooking critical test cases by simulating and optimizing for real-world interference, providing a more comprehensive validation of image recognition devices, thereby enhancing their reliability in safety-critical applications.
Smart Images

Figure EP2024060525_21112024_PF_FP_ABST
Abstract
Description
[0001] Description
[0002] Method and arrangement for testing an image recognition device
[0003] Data-driven machine learning methods are increasingly being used to control complex machines, such as robots, autonomous or semi-autonomous transport vehicles, or manufacturing plants. Artificial neural networks or other machine learning models are trained to recognize current operating situations, predict future operating situations, and / or generate situation-appropriate control actions.
[0004] One specific application is, for example, the automated detection of objects, e.g. persons or obstacles, in camera images taken during autonomous or semi-autonomous driving of trains, motor vehicles or other means of transport.
[0005] Before a complex machine learning model, such as a deep neural network, can be used in safety-critical applications such as autonomous driving, it is usually necessary to verify or ensure its correct functioning. In many cases, however, the internal model structures of such machine learning models are no longer interpretable by humans. Furthermore, mathematical methods that can formally prove the correct functioning of machine learning models are only available for simple models.
[0006] In practice, methods for empirical testing of machine learning models are therefore preferred. Methods for combinatorial testing are known, in which, for example, combinations of two given operating situations are tested. If, for example, a road condition is described by a first set { dry, wet, icy, ...} and a person's pose by a second set { standing, kneeling, lying down, ...}, all combinations of two elements of the first and second sets can be tested accordingly. One such method is known, for example, from the publication "A Combinatorial Approach to Testing Deep Neural Network-based Autonomous Driving Systems" by J. Chandrasekaran, Y. Lei, R. Kacker and D. Richard Kuhn, in IEEE International Conference on Software Testing, Verification and Validation, 2021.However, in more complex scenarios with higher-dimensional state spaces, the problem arises that the number of tested cases is much smaller than the number of possible cases. This increases the risk of overlooking cases in which a tested machine learning model fails.
[0007] It is also known that, to test the robustness of an image recognition system, input images can be manipulated according to predefined rules and the resulting image recognition response evaluated. However, this also presents the problem that, in practice, many cases can arise that are not covered by the predefined rules.
[0008] It is an object of the present invention to provide a method and an arrangement for testing an image recognition device which allow a better and / or more efficient validation of the image recognition device.
[0009] This object is achieved by a method having the features of patent claim 1, by an arrangement having the features of patent claim 10, by a computer program product having the features of patent claim 11 and by a computer-readable storage medium having the features of patent claim 12.
[0010] To test an image recognition device, a perturbation device for modifying images is provided, wherein the modifications to be carried out simulate physical interference when recording camera images, and wherein a parameter space of the modifications to be carried out is parameterized by a parameter vector. Furthermore, a large number of digital test images are read in, each of which is assigned a first item of image information to be recognized. In addition, a large number of test parameter vectors are generated, and the test images are modified by the perturbation device depending on a respective test parameter vector. The modified test images are fed into the image recognition device, which derives a second item of image information from each of them.For a respective test image and a respective test parameter vector, a deviation between the respective first image information and the respective second image information is then determined and assigned to the respective test parameter vector. Furthermore, the test parameter vectors are optimized by an optimization device such that the assigned deviations are maximized, at least on average. Maximizing is also understood, in particular, to mean systematically approaching a maximum. Depending on the maximized deviations, a test result is finally output to the image recognition device.
[0011] To carry out the method according to the invention, a corresponding arrangement, a computer program product and a computer-readable, preferably non-volatile storage medium are provided.
[0012] The method according to the invention and the arrangement according to the invention can be carried out or implemented, for example, by means of one or more computers, processors, application-specific integrated circuits (AS IC), digital signal processors (DSP) and / or so-called "field programmable gate arrays" (FPGA). Furthermore, the method according to the invention can be carried out at least partially in a cloud and / or in an edge computing environment.
[0013] A particular advantage of the invention is that real physical interference can be taken into account in a uniform manner when testing image recognition devices. Since such physical interference on camera images is largely known from practice, the invention can effectively reduce the risk of ignoring practice-relevant test cases in many cases.
[0014] Advantageous embodiments and further developments of the invention are specified in the dependent claims.
[0015] According to an advantageous embodiment of the invention, physical interference can be simulated in the form of optical distortion, perspective transformation, rotation, image blur, motion-related blur, weather-related impairment, a change in contrast and / or a change in brightness of camera images, soiling of a camera lens and / or image sensor noise. The physical interference can be caused in particular by rain, fog, snow, daily or annual brightness fluctuations, by drops, snowflakes, dirt or scratches on a camera lens, windscreen or window pane or by optical distortions of a windscreen. Such interference occurs frequently in practice and can be represented by known image generators or graphics suites, in particular by so-calledCompositing programs can simulate this in a sufficiently realistic manner. The aforementioned physical interference influences can cover a wide range of interference influences that occur in practice when testing image recognition systems.
[0016] Furthermore, the parameter vector can be used to parameterize a type or amplitude of a particular physical disturbance and / or an application sequence of multiple physical disturbances. The parameter vector can, in particular, comprise discrete and / or quasi-continuous values.
[0017] According to a further advantageous embodiment of the invention, a gradient-free optimization method can be used to maximize the deviations, in particular a genetic method, an evolutionary method, and / or particle swarm optimization. In this way, the deviations can be maximized even in cases where a gradient based on the parameter vector cannot be calculated or can only be calculated with difficulty. Furthermore, the image recognition device can be tested as a black box. In the case of learning-based image recognition devices, in particular, no access to their learning parameters is required.
[0018] According to a further advantageous embodiment of the invention, when maximizing the deviations, a test-image-spanning maximization of deviations accumulated across the test images can be carried out. Depending on the maximized accumulated deviations, a test-image-spanning test result can then be output. The deviations can be accumulated by forming a possibly weighted sum or an average of the deviations determined for all test images for a respective test parameter vector and assigning it to this test parameter vector. In this way, a global, image-spanning robustness of the image recognition device can be tested.
[0019] Furthermore, when maximizing deviations, a specific maximization of test-image-specific deviations can be performed for each test image. Depending on the maximized test-image-specific deviations, a test-image-specific test result can then be output. In this way, a separate optimization can be performed for each test image, specifically searching for test-image-specific perturbations that could potentially falsify image recognition. In particular, test images can be identified for which the image recognition device may or probably fail.
[0020] In addition, the test images can be divided into groups using cluster analysis according to a predetermined similarity measure. Such a similarity measure is often also referred to as a proximity measure in the context of cluster analysis. When maximizing the deviations, a maximization of the deviations accumulated across a group can then be carried out that is specific to each group. Depending on the maximized deviations accumulated across this group, a group-specific test result can finally be output. In this way, a separate optimization can be carried out for each group, specifically looking for group-specific perturbations that could potentially falsify image recognition. In particular, image groups can be identified for which the image recognition system may or probably fail.
[0021] According to a further advantageous embodiment of the invention, the image recognition device can derive an uncertainty of the respective second image information from the fed-in modified test images in addition to the respective second image information. The determination of a deviation between a respective first image information and a respective second image information can then be carried out depending on their uncertainty. In particular, the deviation can be increased by the uncertainty. Preferably, a respective deviation can be formed by adding a respective uncertainty, possibly weighted, to a distance between a respective first image information and a respective second image information. Such a distance can be determined, for example, as a Euclidean distance between vectors representing a respective image information.The uncertainty can be represented in particular by a spread, a variance, an error specification, an error quantile or a probability distribution of the respective second image information.
[0022] According to a further advantageous embodiment of the invention, real physical disturbances on camera images of real situations can be quantitatively recorded. The parameter space can then be restricted insofar as the modifications parameterized in this way are limited in quantitative dependence on the real physical disturbances. In particular, the parameter space can be restricted to modifications that quantitatively do not essentially exceed the recorded real physical disturbances. By restricting the parameter space to realistic modifications, it is possible in many cases to efficiently prevent the optimization from leading to image modifications that cannot be trivially assessed or recognized, e.g. image modifications with zero contrast.
[0023] An embodiment of the invention is explained in more detail below with reference to the drawing.
[0024] The figure illustrates an arrangement for testing an image recognition device.
[0025] The figure schematically shows a TST arrangement for testing a given image recognition device IREC. The latter can be designed, in particular, for the automated detection and / or localization of objects, persons, or other patterns in camera images. The image recognition device IREC can then be used, for example, to control a motor vehicle, a rail vehicle, an aircraft, another means of transport, an autonomous or semi-autonomous means of transport, a robot, a production facility, or an accident prevention system.
[0026] For the present exemplary embodiment, it is assumed that the image recognition device IREC is learning-based and comprises an artificial neural network NN. The neural network NN was trained in advance using a machine learning method to output image information about an object, a person and / or an image structure to be recognized on the basis of an input digital camera image. Such image information can in particular comprise an indication as to whether an object, a person or a predetermined structure is recognized, which object, person or structure is recognized, which property the object, person or structure has and / or where the object, person or structure is located.Furthermore, the image information can also include information about a recognized feature, a recognized property, or a recognized attribute of the input camera image. A variety of efficient learning methods, particularly supervised learning methods, are available for conducting such training. It should be noted at this point that the invention can also be used to test image recognition devices that are not learning-based.
[0027] Before the IREC image recognition device is used in a safety-critical application, the TST test setup is to be used to check whether the IREC image recognition device functions reliably with real camera images. This means that the IREC image recognition device can reliably recognize the image information to be recognized in typical application scenarios. It should be taken into account that real camera images are typically subject to various physical interference influences. The latter can be caused, for example, by rain, fog, snow, drops, dirt or scratches on a camera lens, windshield or window pane, or by optical distortions or image blur.The image recognition device IREC is designed to correctly recognize a given piece of image information under all or at least a sufficiently large number of real-world interference influences, or reliably indicate non-recognition if necessary. To simulate realistic physical interference influences on camera images, the test setup TST features a parameterizable perturbation device PER, which modifies the input images accordingly. A variety of well-known image processing programs, e.g., in the Blender graphics suite, are available for such image modifications.
[0028] The image modifications to be performed are each parameterized by a parameter vector PV, which specifies the application sequence of the physical perturbations, as well as their respective type and amplitude. The perturbation device PER is controlled accordingly by the respective parameter vector PV. In practice, the parameter vector PV can, for example, have 30 or more dimensions.
[0029] A parameter space PR of admissible parameter vectors PV is chosen or restricted in advance such that the parameter vectors PV of the parameter space PR induce only realistic image modifications. That is, image modifications that typically also occur in real-life situations.
[0030] Many image processing programs for simulating physical disturbances already have realistic amplitude ranges preconfigured for these disturbances. Such realistic amplitude ranges can be determined, for example, by comparing camera images of real situations subject to real physical disturbances with undisturbed images, thus quantitatively recording the respective physical disturbance. The amplitude ranges for realistic parameter vectors PV can then be limited such that the resulting image modifications do not substantially exceed the quantitative extent of the recorded physical disturbances.
[0031] In the present embodiment, the perturbation device PER is supplied with disturbance amplitude data PA, which specify a realistic amplitude range of a respective physical disturbance. The parameter space PR usable for image modifications by the perturbation device PER is thus restricted to realistic image modifications.
[0032] To test the image recognition device IREC, the test setup TST accesses a database DB in which a large number of digital camera images II, 12, ... of real situations are stored. Each of the camera images II or 12, ... is assigned a piece of image information LI or L2, ..., which should be correctly recognized by the image recognition device IREC as intended. Such image information to be recognized is often also referred to as a label in the context of machine learning. The camera images II, 12, ... are therefore labeled in this sense. A piece of image information LI or L2, ... can preferably be represented as a feature vector. The latter can comprise discrete vector elements that, for example, classify the assigned image, and / or quasi-continuous vector elements that, for example, indicate a position of an object recognized therein.
[0033] The camera images II, 12, ... are read as test images from the database DB by the test setup TST and fed into the perturbation device PER. In practice, several thousand or many thousands of test images can be processed by the test setup TST. For each test image II or 12, ..., the respective associated first image information LI or L2, ... is also read by the test setup TST.
[0034] The perturbation device PER modifies the test images II, I2, ... depending on an input test parameter vector PV from the parameter space PR. A modified test image Mil or MI2, ... is thus generated from each test image II or I2, ... As long as this test parameter vector PV originates from the permissible parameter space PR, the resulting image modification realistically simulates real physical interference.
[0035] At the beginning of the procedure, an initial test parameter vector PV can be randomly selected from the parameter space PR. Later in the procedure, the test parameter vectors PV are generated and optimized by an optimization device OPT of the test setup TST.
[0036] The modified test images Mil, MI2, ... are transmitted from the perturbation device PER to the image recognition device IREC. The latter derives a respective second image information item RI or R2, ... from each modified test image Mil or MI2, ... as intended.
[0037] The respective second image information RI or R2, ... is compared with the first image information LI or L2, ... assigned to the respective test image II or I2, .... In this case, a respective deviation D between the respective first image information LI or L2, ... and the respective second image information RI or R2, ... is determined and assigned to the test parameter vector PV parameterizing the image modification. A respective deviation D can in particular be determined as a Euclidean distance between feature vectors representing a first and a second piece of image information. Alternatively or additionally, a respective deviation D can also be calculated using a so-called intersection-over-union method. The deviations D can obviously be understood as a measure of a recognition error of the image recognition device IREC.
[0038] If the image recognition device IREC outputs not only a respective second image information RI or R2, ... but also their respective uncertainty, e.g. in the form of an error interval, this uncertainty can be taken into account when determining a respective deviation D.
[0039] The determined deviations D are fed into the optimization unit ORT and evaluated there. Depending on the results, the optimization unit OPT executes a preferably gradient-free optimization method, e.g., particle swarm optimization, a genetic optimization method, or an evolutionary optimization method, to optimize the test parameter vectors PV. The test parameter vectors PV are optimized such that the associated deviations D are maximized, at least on average.
[0040] For this purpose, the optimization device OPT generates a plurality of new test parameter vectors PV from the permissible parameter space PR depending on the deviations D and feeds these to the perturbation device PER. The generated test parameter vectors PV in turn cause the perturbation device PER to generate a plurality of differently modified test images Mil, MI2, .... From the modified test images Mil, MI2, ..., a second piece of image information RI or R2, ... is derived by the image recognition device IREC and compared with a respective first piece of image information LI or L2, ... in order to determine a respective deviation D, as described above. The determined deviations D are then fed back to the optimization device OPT.
[0041] The optimization process generates new test parameter vectors PV preferentially in those regions of the permissible parameter space PR whose application increases the deviations D and thus the recognition error of the image recognition device IREC. In this way, the optimization is driven specifically into regions of the parameter space PR where there is a high probability of falsifying the image recognition device IREC.
[0042] According to one variant of the optimization, the deviations D determined for a respective test parameter vector PV can be accumulated across the test images II, II, ... to form a cross-test-image deviation that is assigned to this test parameter vector PV. The accumulation can be carried out, in particular, by summation or averaging. Accordingly, the cross-test-image deviation can then be maximized, at least on average, within the framework of the optimization process. The cross-test-image deviation can be understood as a global recognition error of the image recognition device IREC.
[0043] Alternatively or additionally, a separate optimization can be carried out for each of the test images II, 12, ..., in which test image-specific parameter vectors PV are specifically sought which may falsify the image recognition. In particular, test images can be identified for which the image recognition device IREC is likely to fail. According to a further alternative or additional embodiment of the method, the test images II, 12, ... can be divided into groups of similar images by cluster analysis according to a predetermined similarity measure before they are fed to the perturbation device PER. For each of these groups, the deviations D determined can then be accumulated across the relevant group to form a group-specific deviation. The accumulation can be carried out in particular by summation or averaging.Accordingly, a separate group-specific optimization can be performed for each group by maximizing the respective group-specific deviation, at least on average. In this way, image groups can be identified for which the image recognition engine IREC is likely to fail.
[0044] After optimization, the optimization device OPT outputs a test result TR for the image recognition device IREC based on the maximized deviations D. According to the above-mentioned design variants, a test-image-spanning, a test-image-specific, and / or a group-specific test result TR can be determined and output.
[0045] In particular, a maximum deviation and / or a test image with a maximum deviation can be output as the test result TR. The maximum deviation can be determined specifically for each test image, across all test images, or for a specific group. The maximum deviation can clearly be interpreted as a measure of the recognition error of the image recognition device IREC.
[0046] In addition, the maximum deviation can be compared with a predefined threshold. If the maximum deviation does not exceed the threshold, the IREC image recognition device is validated for a safety-critical application; otherwise, it is not. As a result of such validation, the IREC image recognition device can be used, for example, to control a motor vehicle, a rail vehicle, an aircraft, another means of transport, an autonomous or semi-autonomous means of transport, a robot, a production facility, or an accident avoidance system.
[0047] Since potential physical interference with camera images is largely known in advance and can be simulated with sufficient quantitative accuracy using program libraries, the invention can effectively reduce the risk of ignoring practical test cases in many cases. In particular, even larger physical interference can be tested, provided they are realistic.
Claims
Patent claims 1. A computer-implemented method for testing an image recognition device (IREC), wherein a) a perturbation device (PER) is provided for modifying images (II, I2, ...), wherein the modifications to be made simulate physical interference when recording camera images, and wherein a parameter space (PR) of the modifications to be made is parameterized by a parameter vector (PV), b) a plurality of digital test images (II, I2, ...) are read in, each of which is assigned a first item of image information (LI, L2, ...) to be recognized, c) a plurality of test parameter vectors (PV) are generated, d) the test images (II, I2, ...) are modified by the perturbation device (PER) depending on a respective test parameter vector (PV), e) the modified test images (MI1, MI2, ...) are fed into the image recognition device (IREC), which each generates a second item of image information (RI, R2, ...), f) for a respective test image (II, I2, ...) and a respective test parameter vector (PV), a deviation (D) between the respective first image information (LI, L2, ...) and the respective second image information (RI, R2, ...) is determined and assigned to the respective test parameter vector (PV), g) the test parameter vectors (PV) are optimized by an optimization device (OPT) such that the assigned deviations (D) are maximized at least on average, and h) depending on the maximized deviations (D), a test result (TR) is output for the image recognition device (IREC).
2. Method according to claim 1, characterized in that an optical distortion, a perspective transformation, a rotation, an image blur, a motion-related blur, a weather-related impairment, a change in contrast and / or a change in brightness of camera images, a contamination of a camera lens and / or an image sensor noise are simulated as physical interference influences.
3. Method according to one of the preceding claims, characterized in that a type or an amplitude of a respective physical disturbance influence and / or an application sequence of several physical disturbance influences is parameterized by the parameter vector (PV).
4. Method according to one of the preceding claims, characterized in that a gradient-free optimization method is used to maximize the deviations (D).
5. Method according to one of the preceding claims, characterized in that in the maximization of the deviations (D), a cross-test image maximization of deviations (D) accumulated over the test images (II, I2, ...) is carried out, and that a cross-test image test result (TR) is output depending on the maximized accumulated deviations.
6. Method according to one of the preceding claims, characterized in that in the maximization of the deviations (D) a specific maximization of test image-specific deviations (D) is carried out for a respective test image (II, II, ...), and that a test image-specific test result (TR) is output depending on the maximized test image-specific deviations.
7. Method according to one of the preceding claims, characterized in that the test images are divided into groups by a cluster analysis according to a predetermined similarity measure, that in the maximization of the deviations (D) a maximization of deviations (D) accumulated over this group that is specific to a respective group is carried out, and that a group-specific test result (TR) is output depending on the maximized deviations accumulated over this group.
8. Method according to one of the preceding claims, characterized in that the image recognition device (IREC) derives an uncertainty of the respective second image information (RI, R2, ...) from the fed-in modified test images (Mil, MI2, ...) in addition to the respective second image information (RI, R2, ...), and in that the determination of a deviation (D) between a respective first image information (LI, L2, ...) and a respective second image information (RI, R2, ...) takes place depending on their uncertainty.
9. Method according to one of the preceding claims, characterized in that real physical disturbances on camera images of real situations are recorded quantitatively, and that the parameter space (PR) is limited insofar as the modifications parameterized thereby in quantitative Dependence on the real physical interference influences.
10. Arrangement for testing an image recognition device (IREC), with means for carrying out the steps of a method according to one of the preceding claims.
11. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out a method according to one of claims 1 to 9.
12. A computer-readable storage medium comprising a computer program product according to claim 11.
