Computer-implemented method for generating a reliability indication for computer vision

Abstract

Computer-implemented method (100) for generating a reliability indication for computer vision, comprising: - obtaining (102) visual data comprising an input image or sequence of images representing an observed scene, wherein the visual data is characterizable by a first set of visual parameters (10); - analyzing (104) the observed scene comprised in the visual data using a computer vision reliability model (45) sensitive to a second set of visual parameters, wherein the second set of visual parameters comprises a subset of the first set of visual parameters, wherein the second set of visual parameters is obtained from the first set of visual parameters according to a sensitivity analysis applied to a plurality of parameters of the first set of visual parameters, wherein the sensitivity analysis is performed during an offline training phase of the computer vision reliability model (45); - generating (106) reliability indication data for the observed scene using the analysis of the observed scene; and - outputting (108) the reliability indication data for the computer vision model.

Description

Technical Field

[0001] This specification relates to computer-implemented methods and associated apparatus for generating reliability indication data for computer vision models, computer-implemented methods for training computer vision reliability models, and associated computer program units, computer-readable media, and autonomous systems. Background Technology

[0002] Computer vision focuses on how computers can automatically gain a high-level understanding from digital images or videos. Computer vision systems are finding increasing applications in the automotive and robotic vehicle fields. Computer vision can process input from any interaction between at least one detector and its environment. This environment can be perceived by at least one detector as a scene or a series of scenes.

[0003] Specifically, the interaction can be generated by at least one electromagnetic source, which may or may not be part of the environment. Detectors capable of capturing such electromagnetic interactions can be, for example, cameras, multi-camera systems, radar, or lidar systems.

[0004] In automotive computer vision systems, while system computer vision is critical to safety, it often has to handle open contexts. Therefore, it is important to provide safeguards when applying computer vision models / functions. Summary of the Invention

[0005] According to the first aspect, a computer-implemented method for generating reliability indicator data for computer vision models is provided, comprising:

[0006] - Obtain visual data including an input image or image sequence representing the observed scene, wherein the visual data is characterizable by a first set of visual parameters;

[0007] - Use a computer vision reliability model that is sensitive to a second set of visual parameters to analyze the observed scene included in the visual data.

[0008] The second visual parameter set includes a subset of the first visual parameter set, wherein the second visual parameter set is obtained from the first visual parameter set based on a sensitivity analysis applied to a plurality of parameters in the first visual parameter set, wherein the sensitivity analysis is performed during a previous training phase of the computer vision reliability model.

[0009] - Use analysis of the observed scenario to generate reliability indicator data for the observed scenario; and

[0010] - Output reliability indicator data for the computer vision model.

[0011] The method according to the first aspect advantageously provides an online safety (or reliability) monitor capable of independently monitoring a visual scene based on a visual parameter space. The online safety monitor identifies when an autonomous system using the online safety monitor observes a scene under conditions described by visual parameters that imply unreliable execution by the computer vision model. In other words, prior global sensitivity analysis of the computer vision model performed during the training of the online safety monitor can have determined that, for a given visual input dataset, the computer vision model classifies or predicts visual elements within the visual input data with high variance, thereby indicating the unreliability of the computer vision model when observing a scene described by such visual parameters.

[0012] Testing computer vision models or statistically evaluating their performance is challenging because of the large input space. Theoretically, the input space consists of all possible images defined by the possible combinations of pixel values ​​given an input resolution. In practice, image datasets include real (captured by a physical camera) or synthetic (e.g., obtained using 3D rendering, image augmentation, or image synthesis) images.

[0013] Therefore, this specification discusses an automated system that can use image input from an autonomous or semi-autonomous system, such as a vehicle or robot, to detect when the image processing subsystem of the autonomous or semi-autonomous system may be operating in an unsafe mode.

[0014] A practical example is that when the sun is parameterized to be at an angle to a direction that is essentially behind the vehicle, a computer vision model parameterized by including the sun angle can accurately identify the content of road signs, enabling a good understanding of forward-facing road signs. In this case, the visual elements of the scene can be predicted with low variance, thus indicating the reliability of the computer vision model.

[0015] Alternatively, the sun can be parameterized to be angled directly toward the vehicle, causing forward-facing road signs to be blurred due to glare. In this case, the visual elements of the scene can be characterized as having high variance, indicating the unreliability of the computer vision model under conditions of significant forward glare. Those skilled in the art will appreciate that many different combinations of visual parameters can lead to high or low variance in the results of computer vision models, and the foregoing is one example.

[0016] Typically, different sets of visual parameters (defining world models or ontology) can be defined for testing or statistically evaluating computer vision models, and their implementations or precise interpretations can vary. This specification provides a methodological technique for implementing online reliability decision-making based on empirical results.

[0017] Due to the aforementioned size of the parameter space, it is difficult to fully validate the entire parameter space. According to the first aspect, given a set of visual parameters and computer vision capabilities as input, a sorted list of visual parameters can be provided. By selecting a sublist of visual parameters from the sorted list, a reduced input model (ontology) is defined.

[0018] Based on the first aspect, an online monitoring method for the performance of a computer vision model is proposed to monitor the safety or reliability of an autonomous system during operation. The safety or reliability status of the computer vision model is analyzed based on sensitivity analysis. A computational system (e.g., a deep neural network) is trained to detect visual conditions that cause the computer vision model to perform with increased or high variance. For example, the computational system may identify distribution shifts. Such conditions are monitored during the operation of the computer vision model. If the computer vision model operates under conditions where the global sensitivity analysis shows low performance, the technique signals a low confidence level or warning to the subsystems using the computer vision model.

[0019] To expand on the examples in the preceding paragraphs, if the reliability or safety monitor discussed herein were applied to the problem of verifying the detection of speed limits on road signs, then if the sun is positioned in the sky behind the vehicle, the reliability or safety monitor would signal that the given speed limit has been detected with a degree of certainty higher than a first threshold (such as 90%). Alternatively, if the sun is positioned in the sky directly in front of the vehicle, the reliability or safety monitor would signal that the given speed limit has been detected with a degree of certainty lower than a second threshold (such as 10%).

[0020] According to the second aspect, a computer implementation method for training a computer vision reliability model is provided, comprising:

[0021] - Sample a set of visual parameters from the visual parameter specification;

[0022] - Obtain a visual data itemset and provide a ground truth data itemset corresponding to the visual data itemset based on the sampled visual parameter set, wherein the visual data itemset and the ground truth data itemset form a training dataset;

[0023] - Iteratively train a first machine learning model to analyze at least one visual data item from a visual data item set and output a prediction of the mapping of the at least one visual data item to a subset of the visual parameter set used to generate the visual data item.

[0024] - Iteratively train a second machine learning model to predict reliability indicator data of the mappings made by the first machine learning model, wherein the reliability indicator data is obtained by comparing the predictions of the mappings from the first machine learning model with corresponding benchmark ground truth data items from the training dataset.

[0025] In one embodiment, the computer vision model of the first aspect is trained according to the method of the second aspect.

[0026] According to a third aspect, a data processing apparatus configured to generate reliability indicator data for a computer vision model is provided, comprising an input interface, a processor, a memory, and an output interface. The input interface is configured to acquire visual data comprising an input image or image sequence representing an observed scene, wherein the visual data is characterizable by a first set of visual parameters. The processor is configured to analyze the observed scene included in the visual data using a computer vision reliability model sensitive to a second set of visual parameters. The second set of visual parameters includes a subset of the first set of visual parameters, wherein the second set of visual parameters is obtained from the first set of visual parameters based on a sensitivity analysis applied to multiple parameters in the first set of visual parameters, wherein the sensitivity analysis is performed during a prior training phase of the computer vision reliability model. The processor is configured to generate reliability indicator data for the observed scene using the analysis of the observed scene. The output interface is configured to output the reliability indicator data for the computer vision model.

[0027] According to a fourth aspect, a computer program including machine-readable instructions is provided, which, when executed by a processor, is capable of implementing a computer-implemented method according to the first or second aspect.

[0028] According to a fifth aspect, a computer-readable medium is provided that includes at least one of the computer programs according to a fourth aspect.

[0029] According to a sixth aspect, an autonomous system is provided, comprising: a sensor configured to provide visual data including an input image or image sequence representing an observed scene; and a data processing device configured to generate reliability indication data of a computer vision model according to a second aspect. The autonomous system optionally further includes a motion control subsystem, and the autonomous system is optionally configured to generate or modify motion commands provided to the motion control subsystem based on the reliability indication data obtained using the data processing device.

[0030] Dependent embodiments of the aforementioned aspects are given in the dependent claims and explained in the description which the reader should now refer to.

[0031] Computer vision focuses on how computers can automatically gain high-level understanding from digital images or videos. In particular, computer vision can be applied in automotive engineering to detect road signs, instructions displayed on them, or obstacles around a vehicle. Obstacles can be static or dynamic objects that can interfere with the vehicle's intended driving maneuvers. Following the same line of thought, to avoid getting too close to obstacles, a key application in automotive engineering is detecting the free space (e.g., distance to the nearest obstacle or infinite distance) in the vehicle's intended direction of travel, thus determining where (and how fast) the vehicle can go.

[0032] To achieve this, one or more of the following can be computed for autonomous systems: object detection, semantic segmentation, 3D depth information, and navigation commands. Another common term used in computer vision is computer perception. In fact, computer vision can process input from any interaction between at least one detector and its environment. This environment can be perceived by at least one detector as a scene or a series of scenes. Specifically, the interaction can be generated by at least one electromagnetic source (e.g., the sun), which may or may not be part of the environment. Detectors capable of capturing such electromagnetic interactions can be, for example, cameras, multi-camera systems, radar or lidar systems, or infrared. Examples of non-electromagnetic interactions can be sound waves captured by at least one microphone to generate a sound map including sound levels across multiple solid angles, or sound waves captured by an ultrasonic sensor.

[0033] Computer vision is an important form of sensing in autonomous or semi-autonomous driving. In the following description, the term "autonomous driving" refers to fully autonomous driving, and also to semi-autonomous driving in which the driver retains ultimate control and responsibility for the vehicle. Applications of computer vision in the context of autonomous driving and robotics include the detection, tracking, and prediction of, for example, drivable and non-drivable surfaces and road lanes, moving objects such as vehicles and pedestrians, road signs and traffic lights, and potential road hazards.

[0034] Computer vision must handle open contexts. It is difficult to experimentally model all possible visual scenes. Machine learning—a technique that automatically creates generalizations from input data—can be applied to computer vision. The required generalizations can be complex, requiring consideration of contextual relationships within the image.

[0035] For example, a detected road sign indicating a speed limit is relevant in the context of it being directly above the lane the vehicle is traveling in, but it may have less direct contextual relevance if it is not above the lane the vehicle is traveling in.

[0036] Deep learning-based methods for computer vision have achieved improved performance results on a wide range of benchmarks across various fields. In fact, some deep learning network architectures have implemented concepts such as attention, confidence, and inference on images. With the increasing industrial applications of complex deep neural networks (DNNs), the need for verification and validation (V&V) of computer vision models is growing, especially in partially or fully automated systems where the responsibility for the interaction between the machine and the environment is unsupervised. Emerging safety standards for autonomous driving, such as the standard "Safety for Intended Functionality" (SOTIF), may contribute to the safety of CV functions.

[0037] One or more visual parameters define the visual state of a scene because they contain information about what is being observed in the scene and / or represent the boundary conditions used to capture and / or generate the scene being observed.

[0038] Visual parameters can be, for example: camera properties (e.g., spatial and temporal sampling, distortion, aberrations, color depth, saturation, noise, etc.), lidar or radar properties (e.g., surface absorptivity or reflectivity, etc.), lighting conditions in the scene (light bounce, reflection, light sources, fog and light scattering, overall illumination, etc.), materials and textures, objects and their positioning, size and rotation, geometry (of objects and environment), parameters defining the environment, environmental characteristics such as line-of-sight, precipitation characteristics, radiance (which is suspected to strongly interact with the detection process and may show a strong correlation with performance), image characteristics / statistics (such as contrast, saturation, noise, etc.), domain-specific descriptions of the scene and situation (e.g., cars and objects at an intersection), etc. Many more parameters are conceivable.

[0039] These parameters can be viewed as ontology, taxonomy, dimension, or linguistic entities. They can define a constrained view of the world or an input model. Given the assignment / selection of visual parameters, a specific set of images can be captured or rendered, or visual parameters can be used to describe images in an existing dataset. The advantage of using an ontology or input model is that, for testing, the expected test coverage target can be defined, such as using t-wise coverage, test termination criteria can be defined, and for statistical analysis, the distribution of these parameters can be defined.

[0040] Images, videos, and other visual data, along with other commonly annotated sensor data (GPS data, radiometric data, local weather characteristics), can be obtained in various ways. Real images or videos can be captured by image-capturing devices such as camera systems. Real images may already exist in a database, and a subset of images can be manually or automatically selected given visual parameters and / or other sensor data. Visual parameters and / or other sensor data can also be used to define the required experiment. Another approach is to synthesize images given visual parameters and / or other sensor data. Images can be synthesized using image augmentation techniques, deep learning networks (e.g., generative adversarial networks (GANs), variational autoencoders (VAEs)), and 3D rendering techniques. Tools for 3D rendering in driving simulation contexts include, for example, the CARLA tool (Koltun, 2017, available at www.arXiv.org:1711.03938).

[0041] The visual dataset of the observed scene is a set of items representing images or videos, the latter being image sequences such as JPEG or GIF images.

[0042] A computer vision model is a function (i.e., a mapping) parameterized by model parameters, which are learned during training using machine learning techniques based on a training dataset. A computer vision model is configured to map at least visual data items, or a portion or subset thereof, to predicted data items. One or more visual parameters define the visual state because they contain information about the content of the observed scene and / or represent the boundary conditions used to capture and / or generate the observed scene. The latent representation of a computer vision model is the output of intermediate layers (i.e., hidden layers) or a portion thereof in the computer vision model.

[0043] A ground truth data item corresponding to a visual data item is the classification and / or regression result that a computer vision model aims to output. In other words, when input along with visual data items that illustrate a predictable scene or scene elements, the ground truth data represents the correct answer from the computer vision model. The term "image" can refer to a subset of images, such as segments of road signs or obstacles. The correct answer may also include / be the probability of a classification result.

[0044] This specification proposes an online safety monitor or reliability monitor configured to monitor, for example, an extended computer vision model implemented in a deep neural network configured to integrate validation results into the design of the computer vision model. This specification provides means for identifying and prioritizing key visual parameters whose presence in an input image can be an indicator of unreliability, for example, using classification or regression results from a computer vision model employing offline global sensitivity analysis, which is then used to train the neural network providing the online safety monitor. The term "offline" means that the safety monitor is not used to provide real-time monitoring of vehicle safety during operation. Attached Figure Description

[0045] Figure 1 The advanced development process of an online reliability monitor is illustrated schematically.

[0046] Figure 2 The method implemented by a computer according to the first aspect is illustrated schematically.

[0047] Figure 3 The computer-implemented training method according to the second aspect is illustrated schematically.

[0048] Figure 4 An example of an autonomous system is illustrated, which includes a security monitor configured to calculate reliability indicator data.

[0049] Figure 5 A general flowchart based on an example is illustrated schematically.

[0050] Figure 6 The diagram illustrates the training of a machine learning model configured to be used as a reliability monitor.

[0051] Figure 7 An example of global sensitivity analysis of visual parameters used for computer vision functions is illustrated schematically.

[0052] Figure 8 An example of global sensitivity analysis of visual parameters is illustrated schematically.

[0053] Figure 9 The apparatus according to the third aspect is illustrated schematically.

[0054] Figure 10 The diagram schematically illustrates the autonomous system according to the sixth aspect. Detailed Implementation

[0055] Computer vision models identify elements within a scene in an image or video. For example, in automotive applications, an image sensor of a camera, such as one displaying road traffic signs within its field of view, can identify road traffic signs as elements of the scene, and further, can identify the speed displayed by the road traffic signs as sub-elements of the scene. Other visual parameters, such as the sun's direction relative to the vehicle, more general weather conditions, the vehicle's speed relative to the road traffic signs, and so on, can influence how computer-implemented computer vision functions understand road traffic signs as scene elements.

[0056] The visual parameter space affecting the performance of computer vision models is typically very large and cannot be fully verified a priori or "offline". Therefore, an "online" reliability monitor for computer vision models is discussed in this specification. For example, a reliability monitor observes a given image or sequence of images forming a scene and reports the reliability of the predicted scene content to downstream functions.

[0057] Figure 1 This diagram illustrates the development and validation process of a computer vision model. The illustrated model is used as the "V-model" in computer function development.

[0058] Unlike traditional approaches where development / design and validation / verification are separate tasks, according to the "V-model," development and validation / verification can be intertwined because, in this example, results from validation are fed back into the design of the computer vision model. Multiple visual parameters 10 are used to generate images and a ground truth (GT) set 42. The computer vision model 16 is tested 17, and then a (global) sensitivity analysis 19 is applied to identify the most critical visual parameters 10, i.e., those parameters that have the greatest impact on the performance 17 of the computer vision model. Specifically, the computer vision model 16 is analyzed 19 by comparing performance scores (such as variance performance scores) against multiple input images within the visual parameter space. The results of the sensitivity analysis 19 can be used when training 47 another computer vision model 45 to implement a security (or reliability) runtime monitor. For example, a particular computer vision model 16 might provide element-wise predictions caused by visual parameters with high variance (in other words, unreliable) above the ground truth. The security runtime monitor 45 is trained to identify similar visual parameters associated with high variance above the ground truth. In this way, the safety status of the autonomous system 46, which incorporates the computer vision model 16, can be accurately tracked during operation.

[0059] The safety runtime monitor 45 may be part of the autonomous system 46, 400, which may be, for example, a self-driving vehicle, a semi-autonomous vehicle, an autonomous or semi-autonomous robot, an autonomous or semi-autonomous drone, and the like, and may be integrated into the autonomous system 46, 400 or into the computer vision model 16 itself. Autonomous system 400.

[0060] It is difficult to test the computer vision model 16 on all possible combinations of visual parameters, and therefore the security monitor 45 extends the validation of the computer vision model throughout its entire lifecycle, providing warnings to the relevant system during use (when the computer vision model is in use or "online"). Users of the computer vision model 16 or the autonomous system 46 can respond to such warnings. Optionally, the security monitor is a computer vision model with a trained deep neural network 47. However, the security monitor also applies additional information based on global sensitivity analysis 19 and classification of the input based on test results 17.

[0061] Figure 2 The method implemented by a computer according to the first aspect is illustrated schematically.

[0062] According to a first aspect, a computer-implemented method 100 for generating reliability indicator data for a computer vision model is provided, comprising:

[0063] - Obtain visual data 102 including an input image or image sequence representing the observed scene, wherein the visual data is characterized by a first set of visual parameters;

[0064] - A computer vision reliability model sensitive to a second set of visual parameters was used to analyze the observed scene included in the visual data (104).

[0065] The second visual parameter set includes a subset of the first visual parameter set, wherein the second visual parameter set is obtained from the first visual parameter set based on a sensitivity analysis applied to a plurality of parameters in the first visual parameter set, wherein the sensitivity analysis is performed during a previous training phase of the computer vision reliability model.

[0066] - Reliability indication data for 106 observed scenarios were generated using analysis of the observed scenarios; and

[0067] - Output reliability indication data for 108 computer vision models.

[0068] One or more trained models, such as first and second deep neural networks 47a, 47b, are used to perform analysis 104 of the observed scene. The training of the deep neural networks is discussed in conjunction with at least the second aspect of the method, and... Figure 6 The diagram in the middle is shown.

[0069] Figure 4 An example of generating reliability indication data within an autonomous system 46 is illustrated schematically. The computer-implemented method 100 can be applied by, for example, a security (or reliability) runtime monitor 45 provided within a data processing device 300 included within the autonomous system 46.

[0070] The security runtime monitor 45 is configured to receive input images or image sequences from, for example, a camera, radar, or lidar. The security runtime monitor 45 includes multiple trained models capable of predicting the performance or uncertainty of the computer vision model 16. The security runtime monitor 45 is configured to output a prediction confidence 60, or security, for the computer vision model 16. The prediction confidence 60 is an example of reliability indicator data for the computer vision model. For example, the prediction confidence 60 could be a continuous variable representing the probability that the computer vision result is trustworthy. Alternatively, the prediction confidence 60 could be a binary result indicating a hard decision on whether to trust the computer vision result. The prediction confidence 60, or reliability indicator data, could be a conditional result, conditioned on a subset of visual parameters. For example, conditioned on visual parameters that limit the time of day to during daytime hours, an image prediction of a dark-colored vehicle could be more reliable.

[0071] While not essential, according to one embodiment, the security runtime monitor 45 can operate "online" in parallel with a computer vision model 16 configured to receive the same image input as the security runtime monitor 45. The computer vision model 16 is configured to generate computer vision predictions 61 (including, for example, object recognition results, segmentation, pose estimation, and the like). Optionally, the prediction confidence 60 generated by the computer vision model 16 from the security runtime monitor 45 can be combined with the computer vision model predictions 61 to provide predictions with a measure of uncertainty (or reliability).

[0072] The subsystems of autonomous system 46 may use one or both of the reliability indication data and predictions from a computer vision model combined with the reliability indication data. As illustrated, one option is for the autonomous system's motion planning subsystem 63 to use the reliability indication data as the basis for motion planning decisions. For example, if computer vision model 16 identifies a parking space pattern with reliability indication data indicating a high degree of certainty, the optional motion planning subsystem 63 may provide a motion command to the vehicle's motion subsystem 64 to move the autonomous system into the parking space. However, if computer vision model 16 identifies a parking space pattern with reliability indication data indicating a low degree of certainty, the optional motion planning subsystem 63 may provide a motion command to the vehicle's motion subsystem 64 to move the autonomous system beyond the unreliably identified parking space.

[0073] Advantageously, the computer-implemented method 100 according to the first aspect provides an online reliability or safety monitor capable of characterizing the reliability of the computer vision model when observing a given scene. Optionally, the reliability indication data can be combined with the output of the computer vision model 16 to provide confidence information 60—in other words, the probability that the scene observed by the input sensors at a given instant in time is reliable. Optionally, the confidence information 60 can be used by the motion planner 63 or the autonomous system control system 64 for controlling wheel direction or speed.

[0074] At the core of the computer vision model 16 is, for example, a deep neural network consisting of several neural network layers. However, other model topologies known to those skilled in the art can also be implemented according to this technique. The layers compute latent representations as higher-level representations of the input image. As an example, this specification proposes, optionally, extending existing DNN architectures with latent variables representing visual parameters that can influence the performance of the computer vision model, based on a (global) sensitivity analysis aimed at determining the relevance, importance, or criticality of visual parameters. In doing so, observations from validation are directly integrated into the computer vision model.

[0075] Typically, different sets of visual parameters (defining a world model or ontology) can be defined for testing or statistically estimating the computer vision model 16, and their implementation or precise interpretation can vary. This approach technique implements decisions based on empirical results 19 rather than merely expert opinion, and its implementation involves the concretization 42 of abstract parameters 10. Experts can still provide visual parameters as candidates 10.

[0076] Box 1 illustrates a visual parameter specification that can be used as a "world model." For example, when training a computer vision model, images can be synthetically generated within the visual parameter specification of Box 1. Alternatively, for example, real-world images classified according to the visual parameter specification of Box 1 can be selected. Alternatively, the visual parameter specification of Box 1 can form an experimental specification for obtaining additional real-world or synthetic images.

[0077] Images obtained that meet specific values ​​within the visual parameter specifications (e.g., in box 1) may result in poor performance (high variance) of the computer vision model. Therefore, when such values ​​are encountered during the operation of the computer vision model, it is desirable for the computer vision model's reliability or safety monitor to alert downstream processes.

[0078]

[0079] Box 1 – An example of the visual parameter specification in “Python”.

[0080] The visual dataset of the observed scene is an itemset representing images or videos, the latter being a sequence of images. Each visual data item can be a digital tensor, where videos have additional dimensions for the frame series. The benchmark ground truth data item corresponding to a visual data item is, for example, the classification and / or regression result that a computer vision model should ideally output. For example, if a visual data item is partially parameterized based on the presence of a wet road surface, and the presence or absence of a wet road surface is the expected output of the computer model to be trained, then the benchmark ground truth would return a description of that item in the associated visual data item as an image including the wet road.

[0081] Each ground truth data item can be another digital tensor, or, in a simpler case, a binary result vector. The computer vision model is a function (i.e., a mapping) parameterized by model parameters, which can be learned at training time using machine learning techniques based on a training dataset. The computer vision model is configured to at least map visual data items to predicted data items. Visual data items can be arranged (e.g., through embedding or resampling) such that their input to the computer vision model is explicitly defined. As an example, an image can be embedded into a video using a frame. One or more visual parameters define the visual state, as they contain information about the content of the observed scene and / or represent the boundary conditions used to capture and / or generate the observed scene. The latent representation of the computer vision model is the output of intermediate layers (i.e., hidden layers) or a portion thereof in the computer vision model.

[0082] Figure 5 The diagram schematically illustrates the overall flowchart based on an example, which adapts to... Figure 1Similar reference numerals are used in the accompanying drawings. The process occurring within dashed box 8 is typically performed "offline" or as part of the preparation or training process of security (or reliability) monitor 45. The process occurring within dashed box 7 is typically performed "online" as part of the execution of security monitor 45. Security monitor 45 monitors an "online" version of computer vision function 16 that is the same type or substantially similar to the computer vision function 16 used in the "offline" process.

[0083] At step 10, a "world model" is provided, comprising multiple visual parameters 1...n and representing the range of values ​​for image acquisition and sampling, based on the visual parameter specification language defined on the Operational Design Domain (ODD), where "Box 1" above is an example. At step 11, multiple samples of the visual parameters included in the "world model" are obtained, for example, using combined sampling. At step 42, multiple images or image sequences conforming to the samples from the "world model" of step 11 are generated. At step 42, multiple images or image sequences are also generated using corresponding benchmark ground values ​​so that the accuracy of the prediction, regression, or classification results can subsequently be verified.

[0084] As an example, an initial set of visual parameters and the values ​​or ranges of values ​​for the visual parameters in a given scene can be defined (e.g., by an expert). A simple scene would have a first parameter defining various sun altitudes relative to the direction of travel of the vehicle itself, although a much wider range of visual parameters is possible, as will be discussed later.

[0085] Sampling procedure 11 generates a distribution set of values ​​for visual parameter 10. Optionally, the parameter space is randomly sampled according to a Gaussian distribution. Optionally, the visual parameters are oversampled in regions where the performance inflection point of the CV model is suspected to be defined. Optionally, the visual parameters are undersampled in regions where the predictable performance of the CV model is suspected to be defined.

[0086] The next task is to acquire images according to the visual parameter specification. A synthetic image generator, physical capture settings, and / or database selection 42 can be implemented, allowing images and corresponding benchmark truth terms to be generated, captured, or selected based on samples 11 of visual parameters 10. For example, synthetic images are generated using the CARLA generator (discussed, for example, at https: / / carla.org). In the case of synthetic generation, the benchmark truth can be viewed as the sampled values ​​acting on the visual parameter space that generate the given synthetic image.

[0087] The physical capture settings allow experiments to be performed to obtain multiple test visual data sets within a specified parameter space. Alternatively, a database containing archives of historical visual data that have been appropriately labeled can be selected.

[0088] In practical applications, at step 42, images or image sequences can be selected from a labeled database, generated using a synthetic image or image sequence generator such as the "CARLA" generator discussed elsewhere in the specification. Alternatively, the images or image sequences can be actively captured (experimentally obtained) based on sampled visual parameters.

[0089] A computer vision model 16, with the same architecture and training as the expected "online" computer vision model, is applied to the multiple images generated in step 42. The computer vision model 16 may optionally be executed in a real autonomous system 16. The output of testing step 17 is a series of performance scores for each image or image sequence characterizing the accuracy of the computer vision model 16.

[0090] Global Sensitivity Analysis 19 (to be referenced later) Figure 8 (Discussed in more detail) is configured to compare the series of performance scores obtained in step 17 with the original visual parameter definitions sampled from step 11, for example, optionally using actual visual parameter information 22. Optionally, sensitivity analysis 19 may include some segmentation parameter ranges of visual parameter specification 10, or clustering regions of visual parameter specification 10. Sensitivity analysis 19 may include ranking visual parameter specification 10 to emphasize important visual parameters and de-emphasize unimportant visual parameters. Important visual parameters may, for example, be those that lead to high variance in the performance of the computer vision model.

[0091] In one embodiment, for each item in the image dataset, a performance score can be calculated based on a comparison between the predictions of one or more elements within the observed scene and the corresponding ground truth data item. The performance score can include one or any combination of the following: confusion matrix, precision, recall, F1 score, intersection-over-union ratio, mean-average, and optionally, the performance score of each of at least one visual data item from the training dataset can be considered during training. The performance score can be used in (global) sensitivity analysis; for example, the sensitivity of a parameter can be ranked based on the variance of the performance score when each visual parameter is varied.

[0092] Furthermore, the visual dataset of the observed scene may include one or more of the following: video sequences, independent image sequences, multi-camera video sequences, radar image sequences, LiDAR image sequences, depth map sequences, or infrared image sequences. Alternatively, visual data items may be, for example, sound maps with noise levels derived from a solid angle grid.

[0093] In one embodiment, visual parameters may include one or any combination of the following lists:

[0094] - One or more parameters describing the configuration of the image capture arrangement (optionally, an image or video capture device), wherein visual data may be obtained or synthesized in spatial and / or temporal sampling, distortion aberration, color depth, saturation, noise, and absorption rate;

[0095] - One or more lighting conditions, light bounce, reflection, surface reflectivity, light source, fog and light scattering, and overall lighting in the scene of the image / video; and / or

[0096] - One or more features of the scene in the image / video, optionally, one or more objects and / or their positioning, size, rotation, geometry, material, texture;

[0097] - One or more parameters of the environment of the image / video capture device, or one or more parameters of the simulated capture device for the synthetic image generator, optionally including environmental characteristics, line-of-sight, precipitation characteristics, and radiant intensity; and / or

[0098] - Image characteristics, optionally, contrast, saturation, noise;

[0099] - A scene-specific description of one or more images / videos, optionally, one or more car or road users, or one or more objects at an intersection.

[0100] In one embodiment, the computer vision model 16 can be configured to output at least one classification label and / or at least one regression value for at least one element in a scene included in at least one visual data item. The classification label may, for example, refer to object detection, particularly events like "obstacle in front of vehicle / no obstacle ahead" or free space detection, i.e., an area in which the vehicle can travel. The regression value may, for example, be a speed suggestion in response to road conditions, traffic signs, weather conditions, etc. As an example, a combination of at least one classification label and at least one regression value would output both speed limit detection and speed suggestion. When the computer vision model 16 is applied (feedforward), such output is correlated with the prediction. During training, such output of the computer vision model 16 is correlated with the ground truth (GT) data in the sense that the predictions (from the feedforward) on the training dataset should be at least as statistically close as possible to the (true) ground truth data item.

[0101] As will be described in detail below, the security (or reliability) runtime monitor 45 includes multiple machine learning models (such as deep neural networks) that are trained using the results of sensitivity analysis 19, the original generated images 42, and a series of performance scores 17 based on the performance of the offline computer vision model 16.

[0102] According to the embodiments, the following is further provided:

[0103] - Visual data is processed using an online computer vision model16 configured to perform classification or regression on the visual data, thereby characterizing the elements of the observed scene; and

[0104] - Generate predictions for the observed scene, where reliability indicator data characterizes the reliability of the predictions for the observed scene.

[0105] Therefore, the reliability indicator data enables downstream systems of the online computer vision model 16 to obtain an indication of the salience of the predictions for the observed scene.

[0106] According to one embodiment, the following is further provided:

[0107] - Transmit reliability indication data from the online computer vision model to the motion control system of the autonomous system 400; and optionally...

[0108] - Based on the reliability indication data, one or more motion commands are issued to the autonomous system via the motion control system.

[0109] For example, the one or more motion commands may include steering demand signals, speed, indicator light control, braking control, and gear control from the autonomous system. Alternatively, the one or more motion commands may include more advanced definitions, such as, for example, route planning across a map, robot actuator motion planning, or autonomous drone routes.

[0110] According to one embodiment, the subset of the visual parameter set is obtained based on an automatic evaluation of the sensitivity of the visual parameters sampled from the visual parameter set by an offline computer vision model (16), wherein high sensitivity indicates a high variance between the predictive performance and the expected performance of the offline computer vision model.

[0111] Therefore, a large number of potential image or image sequence scenes can be modeled a priori using offline computer vision models, which are used to investigate the sensitivity of offline computer vision models to changes in images or image sequences described by a subset of visual parameters in a "world model".

[0112] According to one embodiment, the offline computer vision model includes the same or similar type of network and / or parameterization as the online computer vision model.

[0113] According to one embodiment, using a computer vision reliability model to analyze an observed scene included in visual data further includes:

[0114] - Using the first trained machine learning model 47a, visual data is mapped to a second set of visual parameters obtained using sensitivity analysis of the first and / or second sets of visual parameters obtained during the offline training phase of the computer vision reliability model 45.

[0115] Therefore, input image or image sequence data can be associated with a reduced set of visual parameters from a "world model." A complete "world model" can include tens of thousands or even millions of parameters related to a description of the visual scene to which the computer vision model is applied. However, only a subset of the "world model" can be used to determine whether a given prediction obtained using the computer vision model is a reliable prediction.

[0116] According to one embodiment, using a computer vision reliability model to analyze an observed scene included in visual data further includes:

[0117] - Reliability indication data for mapping the second set of visual parameters to the mapping made by the first machine learning model using a second trained machine learning model 47b.

[0118] For example, if the first trained machine learning model 47a identifies a subset of visual parameters representing the sun angle directly in front of the windshield of the vehicle at a low azimuth angle, then the second trained machine learning model 47b can indicate that, under these conditions, the predicted road traffic signs can only be identified with a moderate degree of confidence.

[0119] Figure 3 The computer-implemented training method according to the second aspect is illustrated schematically.

[0120] According to the second aspect, a computer-implemented method 200 for training a computer vision reliability model is provided, comprising:

[0121] - Sample a 202-visual-parameter set from the visual parameter specification;

[0122] - Obtain 204 visual data itemsets and provide a benchmark ground truth data itemset corresponding to the visual data itemsets based on the sampled visual parameter set, wherein the visual data itemsets and the benchmark ground truth data itemsets form a training dataset;

[0123] - Iteratively train a first machine learning model 206 to analyze at least one visual data item from a visual data item set and output a prediction of the mapping of said at least one visual data item to a subset of the visual parameter set used to generate said visual data item; and

[0124] - Iteratively train 208 second machine learning models to predict reliability indicator data of the predictions of the mappings made by the first machine learning model, wherein the reliability indicator data is obtained by comparing the predictions of the mappings from the first machine learning model with corresponding benchmark ground truth data items from the training dataset.

[0125] Figure 6 A more detailed schematic illustration shows the training of a machine learning model configured to serve as a reliability monitor. Specifically, similar system blocks are illustrated using similar reference numerals. The security (or reliability) monitor 45 includes a first machine learning model 47a, which is trained to predict visual parameters given an input image. Optionally, the first machine learning model 47a is configured to predict the dominant or most important visual parameters of the world model present in the given input image. Optionally, the first machine learning model 47a is executed as an artificial “bottleneck” or parameter reduction stage. Optionally, the first machine learning model 47a reduces a potentially large number of visual parameters represented by the “world model” 10 by at least 50%, 60%, 70%, 80%, 90%, 95%, or 99%.

[0126] Optionally, the results of the global sensitivity analysis 19 are used to perform parameter reduction. In other words, the “world model” can be considered as a first set of visual parameters, and a second set of visual parameters can be considered as a subset of the first set of visual parameters, which causes performance variance in the range of at least 50%, 60%, 70%, 80%, 90%, 95%, or 99% percentiles.

[0127]

[0128] Box 2 – Example of a reduced visual parameter specification in Python.

[0129] Box 2 illustrates an example output of training 47a—a data structure comprising a list of two visual parameters from the original “world model”, which relate to the sun’s orientation relative to the self-vehicle and have a significant impact in a given situation.

[0130] The iterative training 206 of the first machine learning model 47a is therefore performed, for example, by inputting a large number of images 42 into the first machine learning model 47a and inputting the corresponding values ​​of the sample 11 of the "world model" to iteratively train the first machine learning model 47a to identify which type of image maps to a given subset of important visual parameters.

[0131] The security or reliability monitor 45 further includes a second machine learning model 47b, optionally including a second deep neural network. The function of the second machine learning model 47b is to predict the performance of a computer vision model 16 of the same type that the security monitor 45 is intended to monitor "online". Therefore, the iterative training 208 of the second machine learning model 47b obtains multiple visual parameter samples 11, and corresponding image or image sequence test results 17 applied to the corresponding image or image sequence 42 of the offline computer vision model 16, which is of the same type as the online computer vision model that the security monitor 45 is intended to monitor. Thus, the second machine learning model 47b learns how to predict the "online" performance of the computer vision function when certain combinations of visual parameters are observed from the results of the "offline" tests.

[0132]

[0133] Box 3 – Example of the output of the second machine learning model, “Python”.

[0134] Box 3 illustrates an example output of training 47b—a data structure comprising a list of two visual parameters from the first machine learning model 47a, ranked according to their uncertainty.

[0135] The result of the training process 47 of the security monitor 45 is a first machine learning model 47a, which is capable of acquiring input images or image sequences and outputting narrowed visual parameters from the "world model" 10 present in the acquired input images or image sequences. A second machine learning model 47b receives the definition of the narrowed visual parameters from the first machine learning model 47a and uses the definition to predict the uncertainty of the computer vision model 16 when viewing the same image or image sequence input to the first machine learning model 47a. Therefore, the functionality of the security monitor or reliability monitor 45 can be trained into a composite machine learning model 45, which optionally uses a deep neural network for representation.

[0136] Optionally, a subset of the visual parameters 10 for which the first machine learning model 47a and the second machine learning model 47b are trained is selected based on sensitivity analysis 19.

[0137] According to one embodiment, when the first machine learning model 47a is trained iteratively, a subset of the visual parameter set for generating visual data items is obtained using sensitivity analysis of the visual parameter set from the visual parameter specification and corresponding prediction reliability indication data predicted by the second machine learning model 47b.

[0138] Figure 7An example of global sensitivity analysis of visual parameters used for computer vision functions is illustrated schematically.

[0139] More specifically, the training method includes a first step of acquiring an initial set of visual parameters 10 and defining the values ​​or ranges of said parameters (e.g., defined by experts). Secondly, a synthetic image generator, dataset, or physical capture setup is implemented, allowing suitable images to be generated 42 or captured based on the visual parameters 10. Thirdly, offline computer vision functionality is provided, and optionally, an offline autonomous system 46 using the computer vision functionality is provided.

[0140] In one embodiment, generation step 42 outputs a combination 22 of actual visual parameter values ​​for the generated / selected image, which may include image characteristics and statistics calculated after image generation / capture, and may deviate from the expected sample 11 of the "world model" of visual parameters.

[0141] Image data 42 is used to test computer vision model 16 17 (optionally as part of autonomous system 46). For each image, performance scores such as confusion matrix, precision, recall, F1 score, crossover ratio, and mean average performance are estimated.

[0142] Given the performance results (scores) of each image on the selected performance metric from test step 17, a global sensitivity analysis 19 is applied to parameter 10. This analysis calculates the variance of the performance score for each visual parameter (10) and creates a ranking. The range of values ​​for the visual parameter may optionally be divided into sub-ranges 20, and said sub-ranges may optionally be considered as a new dimension 21 (a new visual parameter).

[0143] Global sensitivity analysis 19 outputs a ranking / sort of visual parameters (optionally for each sub-interval) based on the variance of the performance score. Optionally, clusters of conditions are also created, for example, if parameter 1 = "The camera is looking at the sun" and parameter 2 = "The road is wet", then the performance of computer vision function 16 may be low (i.e., critical), and parameters 1 and 2 are correlated (high ranking).

[0144] Composite models, such as deep neural networks 47, are trained to predict the confidence / safety of CV function 16 as follows:

[0145] First, a first model (such as a deep neural network) 47a is trained to map input images (a distribution, set, or sequence of images) 42 to a subset of raw visual parameters 10. The subset of visual parameters is selected based on a priority ranking from global sensitivity analysis 19.

[0146] The second model (such as a deep neural network) 47b is trained to map visual parameters to test results 17, and thus to the network's expected performance.

[0147] The output of training 47 is a runtime security monitor 45, which maps the input image (or image sequence, etc.) to the uncertainty / confidence / security prediction of the CV function of that image. Figure 3 The runtime security monitor 45 includes a network 47a that maps the input image to visual parameters and a network 47b that maps the visual parameters to the output of 45.

[0148] Advantageously, a safety runtime monitor 45 is provided for computer vision models 16 of the same or similar type. It predicts the uncertainty or confidence level of the computer vision model. High uncertainty or low confidence indicates situations where the computer vision model should not be trusted by downstream vehicle systems such as route planning software or motion control software.

[0149] Generally, sensitivity analysis (or more narrowly, global sensitivity analysis) can be viewed as a numerical quantification of how the uncertainty in the output of a model or system can be partitioned and distributed to different sources of uncertainty in its input. This quantification can be referred to as sensitivity or robustness. In the context of this specification, a model can be viewed, for example, as a result of visual parameters (or visual parameter coordinates) X based on visual data items it has captured / generated / selected. i The output performance score (or performance score coordinates) Y, i = 1, ..., n, is derived from the prediction and the baseline truth value. j Mapping of j = 1, ..., m

[0150] Φ: X→Y.

[0151] Variance-based sensitivity analysis—sometimes called the Sobol method or Sobol index—is a special kind of (global) sensitivity analysis. For this purpose, the samples of both the input and output of the aforementioned mapping Φ can be interpreted probabilistically. In fact, as an example, a (multivariate) empirical distribution of the input samples can be generated. Similarly, for the output samples, a (multivariate) empirical distribution can be computed. The variance of the input and / or output (i.e., the performance score) can thus be calculated. Variance-based sensitivity analysis is able to decompose the variance of the output into scores that can be attributed to the input coordinates or the set of input coordinates. For example, in the case of two visual parameters (i.e., n=2), one might find that 50% of the variance of the performance score is caused by the variance in the first visual parameter (X1), 20% by the variance in the second visual parameter (X2), and 30% by the interaction between the first and second visual parameters. For n>2, more than two visual parameters interact. Note that if such interactions prove significant, combinations of two or more visual parameters can be enhanced to create new visual dimensions and / or linguistic entities. Variance-based sensitivity analysis is an example of global sensitivity analysis.

[0152] Therefore, when applied in the context of this specification, a key outcome of variance-based sensitivity analysis is the variance of the performance score for each visual parameter. The greater the variance of the performance score for a given visual parameter, the more its performance score varies. This indicates that the computer vision model is more unpredictable based on the settings of that visual parameter. Unpredictability during the training of the computer vision model 16 may be undesirable, and therefore, visual parameters leading to high variance can be de-emphasized or removed during training.

[0153] In the context of this specification, the model can be viewed, for example, as a mapping from visual parameters that have been based on captured / generated / selected visual data items to output performance scores based on true and predicted benchmark ground truth items. An important result of sensitivity analysis can be the variance of the performance score for each visual parameter. The larger the variance of the performance score for a given visual parameter, the more the performance score of that visual parameter varies. This indicates that the computer vision model is more unpredictable based on the settings of that visual parameter.

[0154] Figure 8 An example of global sensitivity analysis of visual parameters is illustrated schematically.

[0155] As an example, nested loops are executed for each visual parameter 31, for each value of the current visual parameter 32, and for each visual data item, and for the current value of the current visual parameter, the corresponding baseline truth value item 33 is captured, generated, and selected, and the prediction made by 16 is obtained by, for example, applying the second method (according to the second aspect). In each such step, a performance score 17 can be calculated based on the current baseline truth value item and the current prediction. Thus, the mapping from visual parameters to performance scores can be defined, for example, in terms of a lookup table. It is possible, and often meaningful, to classify, group, or cluster the visual parameters, for example, in terms of subranges or combinations or conditions among various values / subranges of the visual parameters. Figure 8 In this context, the variance of performance scores (i.e., performance variance) can be calculated based on arithmetic operations, such as the minimum, maximum, or average performance scores within a class, group, or cluster.

[0156] Alternatively, global sensitivity analysis can be performed using the global sensitivity analysis tool 37. As an example, rankings of performance scores for visual parameters or their classes, groups, or clusters, and / or rankings of the variance of those performance scores, can be generated and visualized. This allows the relevance of visual parameters to be determined, particularly without considering biases in the human perceptual system. Adjustments to the visual parameters, i.e., adjustments to the operational design domain (ODD), can also be generated using quantitative criteria.

[0157] According to a third aspect, a data processing apparatus 300 configured to generate reliability indicator data for a computer vision model is provided, including an input interface 310, a processor 320, a memory 330, and an output interface 340. The input interface 310 is configured to acquire visual data, the visual data including an input image or image sequence representing an observed scene, wherein the visual data is characterizable by a first set of visual parameters. The processor 320 is configured to analyze the observed scene included in the visual data using a computer vision reliability model sensitive to a second set of visual parameters. The second set of visual parameters includes a subset of the first set of visual parameters, wherein the second set of visual parameters is obtained from the first set of visual parameters based on a sensitivity analysis applied to a plurality of parameters in the first set of visual parameters, wherein the sensitivity analysis is performed during a prior training phase of the computer vision reliability model. The processor 320 is configured to generate reliability indicator data for the observed scene using the analysis of the observed scene. The output interface 340 is configured to output the reliability indicator data for the computer vision model.

[0158] In one example, the data processing device 300 is an electronic control unit (ECU) of a vehicle, an embedded computer, or a personal computer. In one embodiment, the data processing device may be a server located remotely from the input 310 and / or output 340 interfaces or a cloud-based server. Processing occurring on a single physical processor is not essential. For example, it can be divided across multiple processor cores on the same processor or across multiple different processors. The processor may be a Hadoop™ cluster or provided on a commercial cloud processing service. A portion of the processing may be executed on unconventional processing hardware, such as a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), one or more graphics processors, a dedicated processor for machine learning, and the like.

[0159] The fourth aspect relates to a computer program comprising instructions that, when executed by a computer, cause the computer to perform a first method or a second method.

[0160] The fifth aspect relates to a computer-readable medium on which one or both of the said computer programs are stored.

[0161] The memory 330 of device 300 stores a computer program according to the fourth aspect, which, when executed by processor 320, causes processor 320 to perform functionality described by a computer-implemented method according to the first and / or second aspects. According to one example, the input interface 310 and / or output interface 340 is one of a USB interface, an Ethernet interface, a WLAN interface, or other suitable hardware capable of enabling input and output of data samples from device 300. In one example, device 330 further includes a volatile and / or non-volatile memory system 330 configured to receive input observations as input data from input interface 310. In one example, device 300 is as follows: Figure 10 The vehicle includes an embedded computer, which can connect to sensors 440a, 440b and a motion control subsystem 460 present in the vehicle. For example, the input interface 310 of device 300 can interface with an engine control unit (ECU) that provides speed, fuel consumption data, battery data, position data, and one or more such devices. Similarly, the output interface 340 of device 300 can interface with multiple brake actuators, throttle actuators, fuel-air mixture actuators, turbocharger controllers, battery management systems, automotive lighting systems, or entertainment systems, and one or more such devices.

[0162] Figure 10An example of an autonomous system according to the sixth aspect is schematically illustrated. According to the sixth aspect, an autonomous system 400 is provided, comprising: at least one sensor 440a, 440b configured to provide visual data including an input image or image sequence representing an observed scene; and a data processing device 450 according to the third aspect, configured to generate reliability indication data for a computer vision model.

[0163] The autonomous system 400 may optionally further include a motion control subsystem 460, and the autonomous system is configured to generate or modify motion commands provided to the motion control subsystem based on reliability indication data obtained using the data processing device 450.

[0164] Another aspect relates to a distributed data communication system comprising a remote data processing agent 410, a communication network 420 (e.g., USB, CAN or other peer-to-peer connections, broadband cellular networks such as 4G, 5G, 6G, etc.), and a terminal device 430, wherein the terminal device may optionally be included in a vehicle or robot. A server is configured to transmit to the terminal device via the communication network. As an example, the remote data processing agent 410 may include a server, a virtual machine, a cluster, or a distributed service.

[0165] In other words, according to the second aspect, the reliability monitor 47 can be trained at a remote facility and can be transmitted to the vehicle, such as an autonomous vehicle, semi-autonomous vehicle, car, or robot, via a communication network as a software update for the vehicle, automobile, or robot.

[0166] The examples provided in the accompanying drawings and those described in the foregoing written description are intended to provide an understanding of the principles of this specification. They are not intended to limit the scope of the appended claims. This specification describes variations and modifications to the illustrated examples. Only preferred examples are presented, and all changes, modifications, and further applications of these examples within the scope of this specification are intended to be protected.

Claims

1. A computer-implemented method (100) for generating reliability indicator data for a computer vision model, comprising: - Obtain (102) visual data including an input image or image sequence representing the observed scene, wherein the visual data is characterized by a first set of visual parameters (10); - The observed scene included in the visual data is analyzed (104) using a computer vision reliability model (45) sensitive to a second set of visual parameters. The analysis includes applying sensitivity analysis to a first set of visual parameters (10) to obtain a second set of visual parameters, the presence of which in the input image is an indicator of the unreliability of the computer vision model. The second set of visual parameters is a subset of the first set of visual parameters, wherein the sensitivity analysis is performed during the offline training phase of the computer vision reliability model (45); - Use the analysis of the observed scenario to generate (106) reliability indication data for the observed scenario; and - Output (108) Reliability indication data of the computer vision model.

2. The computer-implemented method (100) according to claim 1, further comprising: - Use an online computer vision model configured to perform classification or regression on visual data to process visual data, thereby characterizing the elements of the observed scene; and - Generate a prediction of the observed scene, wherein the reliability indicator data characterizes the reliability of the prediction of the observed scene.

3. The computer-implemented method (100) according to any one of claims 1 or 2, further comprising: - Transmit the reliability indication data of the online computer vision model to the motion control system of the autonomous system (400); And optionally - Based on the reliability indication data, the motion control system issues one or more motion commands to the autonomous system.

4. The computer-implemented method (100) according to any one of the preceding claims. The use of computer vision reliability models to analyze observed scenes included in visual data further includes: - Use the first trained machine learning model (47a) to map the visual data to a second set of visual parameters obtained using sensitivity analysis of the first and / or second sets of visual parameters obtained during the offline training phase of the computer vision reliability model (45).

5. The computer-implemented method (100) according to claim 4. The use of computer vision reliability models to analyze observed scenes included in visual data further includes: - Reliability indicator data of the prediction that the second set of visual parameters is mapped to the mapping made by the first machine learning model using the second trained machine learning model (47b).

6. The computer-implemented method (100) according to any one of the preceding claims. The visual data includes one or more of the following: video sequences, standalone image sequences, multi-camera video sequences, radar image sequences, lidar image sequences, depth map sequences, or infrared image sequences.

7. The computer-implemented method (100) according to any one of the preceding claims. The visual parameters mentioned include one or any combination of those selected from the following list: - One or more parameters describing the image capture arrangement, optionally the configuration of the image or video capture device, wherein visual data may be obtained or synthesized in spatial and / or temporal sampling, distortion aberrations, color depth, saturation, noise, absorptivity, surface reflectivity; - One or more lighting conditions, light bounces, reflections, light sources, fog and light scattering, and overall lighting in the scene of the image / video; and / or - One or more features of the scene in the image / video, optionally, one or more objects and / or their positioning, size, rotation, geometry, material, texture; - One or more parameters of the environment of the image / video capture device, or one or more parameters of the simulated capture device for the synthetic image generator, optionally including environmental characteristics, line-of-sight, precipitation characteristics, and radiation intensity; and / or - Image characteristics, optionally, contrast, saturation, noise; - A scene-specific description of one or more images / videos, optionally, one or more car or road users, or one or more objects at an intersection.

8. A data processing apparatus (300, 450) configured to generate reliability indicator data for a computer vision model, comprising: - Input interface (310); - Processor (320); -Memory (330); and - Output interface (340); The input interface (310) is configured to obtain visual data including an input image or image sequence representing the observed scene, wherein the visual data is characterized by a first set of visual parameters; The processor (320) is configured to analyze the observed scene included in the visual data using a computer vision reliability model (45) sensitive to a second set of visual parameters. This analysis includes applying sensitivity analysis to a first set of visual parameters to obtain a second set of visual parameters, the presence of which in the input image is an indicator of the unreliability of the computer vision model. The second set of visual parameters is a subset of the first set of visual parameters, wherein the sensitivity analysis is performed during the offline training phase of the computer vision reliability model (45); The processor (320) is configured to use analysis of the observed scene to generate reliability indication data for the observed scene; and The output interface (340) is configured to output reliability indication data of the computer vision model.

9. A computer-implemented method (200) for training a computer vision reliability model, comprising: - Sample the (202) visual parameter set from the visual parameter specification; - Obtain (204) visual data itemsets and provide a benchmark ground truth data itemsets corresponding to the visual data itemsets based on the sampled visual parameter set, wherein the visual data itemsets and the benchmark ground truth data itemsets form a training dataset; - Iteratively train (206) the first machine learning model (47a) to analyze at least one visual data item from the visual data item set and output a prediction of the mapping of the at least one visual data item to a subset of the visual parameter set used to generate the visual data item, the subset of the visual parameter set used to generate the visual data item being visual parameters whose presence in the input image is an indicator of the unreliability of the offline computer vision model. - Iteratively train (208) a second machine learning model (47b) to predict reliability indicator data of the predictions of the mappings made by the first machine learning model (47a), wherein the reliability indicator data is obtained by comparing the predictions of the mappings from the first machine learning model (47a) with corresponding benchmark ground truth data items from the training dataset.

10. The computer-implemented method (200) according to claim 9. When the first machine learning model (47a) is trained iteratively, a subset of the visual parameter set used to generate visual data items is obtained by using sensitivity analysis of the visual parameter set from the visual parameter specification and corresponding prediction reliability indicator data predicted by the second machine learning model (47b).

11. The computer-implemented method (100) according to claim 10. The subset of the visual parameter set is obtained based on an automatic assessment of the sensitivity of the offline computer vision model to visual parameters sampled from the visual parameter set, where high sensitivity indicates a high variance between the predictive performance and the expected performance of the offline computer vision model.

12. The computer-implemented method (100) according to claim 11. The offline computer vision model (16) includes networks and / or parameterizations of the same or the same type as those in the online computer vision model.

13. A computer program comprising machine-readable instructions that, when executed by a processor, are capable of performing (i) a computer-implemented method according to any one of claims 1 to 7, or (ii) a computer-implemented method according to claims 9 to 11.

14. A computer-readable medium comprising at least one of the computer programs according to claim 13.

15. An autonomous system (400), comprising: - Sensors (440a, 440b) are configured to provide visual data including an input image or image sequence representing the observed scene; - The data processing apparatus (450) according to claim 8 is configured to generate reliability indication data for a computer vision model; The autonomous system may optionally further include a motion control subsystem, and the autonomous system may optionally be configured to generate or modify motion commands provided to the motion control subsystem based on reliability indication data obtained using the data processing device.

Images