Computer-implemented method for providing insight into machine learning model

Abstract

The invention relates to a computer-implemented method for providing insight into a machine learning model. The method (100) comprises: obtaining (S102) sensor data relating to a scene of the surroundings of a vehicle equipped with an automated driving system; obtaining (S104) an internal representation of the sensor data for a machine learning model configured to perform tasks of the automatic driving system, the internal representation being formed by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model; and generating (S106) synthetic sensor data for subsequent comparison with the obtained sensor data, where the synthetic sensor data is generated by inputting the internal representation into a generative model trained to generate the synthetic sensor data based on the internal representation. The invention further relates to a computer-implemented method performed in a vehicle as well as a server and a vehicle.

Description

Technical Field

[0001] The present inventive concept relates to the field of autonomous vehicles. In particular, it relates to methods and apparatus for providing insights into machine learning models. Background Art

[0002] With the development of technology in recent years, image capture and processing technology has become widely used in different technical fields. In particular, vehicles produced today are usually equipped with some form of vision or perception system for realizing new functions. In addition, more and more modern vehicles have advanced driver assistance systems (ADAS) for improving vehicle safety and more general road safety. ADAS, which can be represented by adaptive cruise control, ACC, collision avoidance system, forward collision warning, lane support system, etc., is an electronic system that can help the driver of the vehicle. Today, research and development are being carried out in many technical fields associated with both the fields of ADAS and autonomous driving (AD). ADAS and AD can also be mentioned under the general term autonomous driving system (ADS) corresponding to all different automation levels defined, for example, by the SAE J3016 levels (0-5) of driving automation.

[0003] Some of the functionality of these systems can be implemented using simple rule-based techniques. However, in order to handle the complexity of real-world driving scenarios including changing road conditions, unpredictability of human or non-human behavior, and rapidly changing environments, machine learning models can be used in some cases, for example, to improve the capabilities or performance of ADS. Machine learning models such as deep neural networks are large and complex nonlinear functions that allow them to learn to recognize complex patterns in, for example, sensor data to build an understanding of the world. As an example, an ADS typically includes a perception system configured to interpret sensor data related to the driving of the vehicle to determine, for example, obstacles, lanes, relevant signs, appropriate navigation paths, etc. These functions can be performed, for example, by a machine learning model such as an object detection or classification model.

[0004] However, the black-box nature of these models poses a challenge to understanding how machine learning models work and what the cause of potential errors or inaccuracies that appear in the model outputs may be. For example, if a perception system incorrectly perceives its surroundings, it may be difficult to determine why and how to improve the system. Therefore, in the event of an error, determining its cause is challenging due to this lack of transparency.

[0005] By being able to determine the cause of such errors, any potential problems can be resolved more quickly, and any systems utilizing such models can thereby be further improved. Furthermore, providing a better understanding of these models can provide for increased productivity, reduced amounts of required training data, and enabled the acquisition of more accurate training data. Therefore, there is a need for providing insight into these models to facilitate a deeper understanding of the inner workings of machine learning models to aid in the development of new and improved solutions therefor. Summary of the invention

[0006] The technology disclosed herein seeks to mitigate, alleviate or eliminate one or more of the above-mentioned deficiencies and shortcomings in the prior art to address various problems related to the analysis of machine learning model performance.

[0007] In certain aspects, the currently disclosed technology provides insights into model performance by enabling visualization of the inner workings of machine learning models to aid in the analysis. This new and improved method of providing insights into model performance can be utilized in the further development of these models. More specifically, the currently disclosed technology builds on the technology of generative AI to address the above challenges. Generative AI such as generative adversarial networks or diffusion models have been used to generate synthetic data for data augmentation or generating additional training data. However, as the inventors recognize, similar techniques can be used to reconstruct the input data of a model in the way that the model sees the input data of the model. This facilitates comparison between the original input data and how the model represents the input data internally to determine potential errors or omissions caused by the machine learning model. Various aspects and embodiments of the disclosed invention are defined below and in the accompanying independent and dependent claims.

[0008] According to a first aspect, a computer-implemented method executed in a server is provided. The method includes obtaining sensor data related to a scene of an environment surrounding a vehicle equipped with an autonomous driving system. The method further includes: for a machine learning model configured to perform a task of the autonomous driving system, obtaining an internal representation of the sensor data, wherein the internal representation is formed by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model. The method further includes: generating synthetic sensor data for subsequent comparison with the obtained sensor data, wherein the synthetic sensor data is generated by inputting the internal representation into a generative model trained to generate synthetic sensor data based on the internal representation.

[0009] According to a second aspect, a computer program product comprising instructions is provided, and when a computing device executes the program, the instructions cause the computing device to perform a method according to any embodiment of the first aspect. According to an alternative embodiment of the second aspect, a (non-transitory) computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a processing system, and the one or more programs include instructions for performing the method according to an embodiment of the first aspect. When applicable, the above features of the first aspect also apply to the second aspect. In order to avoid unnecessary repetition, reference is made to the above.

[0010] According to a third aspect, a server including a control circuit is provided. The control circuit is configured to include obtaining sensor data related to a scene of the surrounding environment of a vehicle equipped with an autonomous driving system. The control circuit is further configured to obtain an internal representation of the sensor data for a machine learning model configured to perform a task of the autonomous driving system, wherein the internal representation is formed by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model. The control circuit is further configured to generate synthetic sensor data for subsequent comparison with the obtained sensor data, wherein the synthetic sensor data is generated by inputting the internal representation into a generative model trained to generate synthetic sensor data based on the internal representation. When applicable, the above-mentioned features of the first aspect and the second aspect also apply to the third aspect. In order to avoid unnecessary repetition, reference is made to the above.

[0011] According to a fourth aspect, a computer-implemented method performed by a vehicle equipped with an autonomous driving system is provided. The method includes: in response to detecting a deviation behavior of the vehicle, obtaining sensor data related to a scene of the surrounding environment in which the deviation behavior is detected. The method further includes: for a machine learning model configured to perform a task of the autonomous driving system, determining an internal representation of the sensor data by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model. The method further includes: transmitting the sensor data and the internal representation of the sensor data to a server for subsequent generation of synthetic sensor data based on the internal representation. When applicable, the above-mentioned features of the previous aspects also apply to this fourth aspect. In order to avoid unnecessary repetition, reference is made to the above.

[0012] According to a fifth aspect, a computer program product comprising instructions is provided, and when a computing device executes the program, the instructions cause the computing device to perform a method according to any embodiment of the fourth aspect. According to an alternative embodiment of the fifth aspect, a (non-transitory) computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a processing system, and the one or more programs include instructions for performing the method according to an embodiment of the fourth aspect. When applicable, the above-mentioned features of the previous aspects also apply to this fifth aspect. In order to avoid unnecessary repetition, refer to the above.

[0013] According to a sixth aspect, a vehicle equipped with an autonomous driving system is provided. The vehicle includes one or more sensors. The vehicle further includes a control circuit. The control circuit is configured to obtain sensor data related to a scene of the surrounding environment in which the deviation behavior is detected in response to detecting a deviation behavior of the vehicle. The control circuit is further configured to determine an internal representation for a machine learning model configured to perform a task of the autonomous driving system by inputting the sensor data into the machine learning model and extracting the internal representation of the sensor data from the machine learning model. The control circuit is further configured to transmit the sensor data and the internal representation of the sensor data to a server for subsequent generation of synthetic sensor data based on the internal representation. When applicable, the above-mentioned features of other aspects also apply to this sixth aspect. In order to avoid unnecessary repetition, refer to the above.

[0014] As used herein, the term "non-transitory" is intended to describe computer-readable storage media (or "memory") that do not include propagating electromagnetic signals, but is not intended to otherwise limit the types of physical computer-readable storage devices encompassed by the term computer-readable media or memory. For example, the term "non-transitory computer-readable medium" or "tangible memory" is intended to encompass types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in a non-transitory form on a tangible computer-accessible storage medium may be further transmitted via a transmission medium or signals such as electrical, electromagnetic, or digital signals that may be transmitted via a communication medium such as a network and / or wireless link. Therefore, as used herein, the term "non-transitory" is a limitation on the medium itself (i.e., tangible, not a signal), not a limitation on the persistence of data storage (e.g., RAM vs. ROM).

[0015] The disclosed aspects and preferred embodiments can be appropriately combined with each other in any manner that is obvious to those skilled in the art, so that one or more features or embodiments disclosed in relation to one aspect can also be considered to be disclosed in relation to another aspect or an embodiment of the other aspect. In addition, any advantages mentioned in conjunction with one aspect also apply to other aspects when applicable.

[0016] As previously mentioned, the currently disclosed technology can be advantageous because it provides for understanding the working principles of machine learning models in the field of autonomous driving. This is achieved by providing insights into the black box (machine learning models are usually black boxes). The effect of this is that it can help improve the safety, effectiveness, and capability of autonomous driving systems. One possible effect is that it enables the determination of potential errors or omissions in the system so that these errors or omissions can be resolved.

[0017] The disclosed techniques can be used to analyze scenarios where it is expected or known that a machine learning model has gone awry. Thus, ways to improve the model can be determined in this way. This can be done by analyzing the differences between the acquired sensor data and the synthesized sensor data. The disclosed techniques can also be used reactively, for example to raise a flag that something may not be working as it should or to give an indication that something may not be working as it should. In other words, the methods can be used to determine that an error has occurred or is occurring. This can be done by determining the differences between the acquired sensor data and the synthesized sensor data.

[0018] The techniques disclosed herein may be further advantageous because they can provide for the collection of relevant training data for improving a machine learning model. More specifically, subsequent comparisons enabled by the generated synthetic sensor data can give an indication of what data needs to be collected to improve the model or what the model's weaknesses are. In other words, based on this data, it may be possible to understand how to fix any deficiencies in the model, either by collecting more training data or by adjusting the model architecture or algorithm.

[0019] Further embodiments are defined in the dependent claims. It should be emphasized that when used in this specification, the term "comprising" is used to specify the presence of stated features, integers, steps or components. It does not exclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0020] These and other features and advantages of the disclosed technology will be further elucidated below with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above-described aspects, features, and advantages of the disclosed technology will be more fully understood by referring to the following illustrative and non-limiting detailed description of example embodiments of the present disclosure when taken in conjunction with the accompanying drawings, in which:

[0022] Figure 1 is a schematic flow chart representation of a method performed in a server according to some embodiments.

[0023] Figure 2is a schematic flow chart representation of a method performed in a vehicle according to some embodiments.

[0024] Figure 3 is a schematic diagram of a server in accordance with some embodiments.

[0025] Figure 4 is a schematic diagram of a vehicle in accordance with some embodiments.

[0026] Figure 5A and Figure 5B Raw sensor data and synthetic sensor data are illustrated by way of example respectively.

[0027] Figure 5C and Figure 5D Raw sensor data and synthetic sensor data are respectively illustrated by another example.

[0028] Figure 6 is a schematic diagram of a distributed system according to some embodiments. DETAILED DESCRIPTION

[0029] The present disclosure will now be described in detail with reference to the accompanying drawings, in which some example embodiments of the disclosed technology are shown. However, the disclosed technology may be embodied in other forms and should not be construed as limited to the disclosed example embodiments. The disclosed example embodiments are provided to fully convey the scope of the disclosed technology to those skilled in the art. Those skilled in the art will understand that the steps, services, and functions described herein may be implemented using separate hardware circuits, using software that works in conjunction with a programmed microprocessor or general-purpose computer, using one or more application-specific integrated circuits (ASICs), using one or more field programmable gate arrays (FPGAs), and / or using one or more digital signal processors (DSPs).

[0030] It will also be understood that when the present disclosure is described in terms of methods, it may also be embodied in a device comprising one or more processors, one or more memories coupled to the one or more processors, wherein computer code is loaded to implement the methods. For example, in some embodiments, the one or more memories may store one or more computer programs that, when executed by the one or more processors, cause the device to perform the steps, services, and functions disclosed herein.

[0031] It should also be understood that the terms used in this article are only used for the purpose of describing specific embodiments, and are not used to limit. It should be noted that, as used in the specification and the appended claims, unless the context clearly stipulates otherwise, the articles "one", "an", "the" and "said" are intended to indicate the presence of one or more elements. Thus, for example, in some contexts, reference to "a unit" or "the unit" may refer to more than one unit, and so on. In addition, the words "include", "comprise" do not exclude other elements or steps. It should be emphasized that when used in this specification, the term "include" is used to specify the presence of stated features, wholes, steps or parts. It does not exclude the presence or addition of one or more other features, wholes, steps, parts or groups thereof. The term "and / or" should be interpreted as meaning that "both" and each is optional.

[0032] It will also be understood that although the terms first, second, etc. may be used to describe various elements or features in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the embodiment, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. Both the first element and the second element are elements, but they are not the same element.

[0033] As used herein, the phrase "one or more of a group of elements" (such as "one or more of A, B, and C" or "at least one of A, B, and C") should be interpreted as conjunction or disjunction. In other words, it can refer to all elements, one element, or a combination of two or more elements in a group of elements. For example, the phrase "one or more of A, B, and C" can be interpreted as A or B or C, A and B and C, A and B, B and C, or A and C.

[0034] Several functions of the vehicle's automated driving system (ADS) typically utilize machine learning models to some extent. Throughout this disclosure, reference is made to such machine learning models configured to perform ADS tasks. The wording "machine learning model" herein means any form of machine learning algorithm such as a deep learning model and a neural network. Typically, a machine learning model can be any neural network-based model that operates on sensor data of an autonomous vehicle. A machine learning model can be a perception model. The perception model herein refers to a computing system or algorithm designed to perceive an environment depicted in sensor data such as digital images, video frames, lidar data, radar data, ultrasonic data, or other types of visual data. In other words, a perception model can be designed to detect, locate, determine, and / or identify instances of specific objects within sensor data. Therefore, a perception model can be configured to perform perception tasks such as object detection, object classification, lane estimation, free space estimation, etc. of an ADS. More specifically, a machine learning model can be an object detection model, an object classification model, a lane estimation model, or a free space estimation model. The perception model can employ a combination of advanced techniques from computer vision, machine learning, and pattern recognition to analyze visual sensor data, and output, for example, a bounding box or region of interest around an object of interest present in an input image. The perception model can be further configured to classify what type of object is detected. The perception model can include different architectures, including but not limited to convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, and other existing or future alternatives.

[0035] An autonomous driving system typically includes a perception system configured to interpret sensory data related to the driving vehicle to determine, for example, obstacles, lanes, relevant signs, appropriate navigation paths, etc. This is typically achieved through a perception model as described above. Therefore, the perception system is critical to the safety and effectiveness of the entire ADS. Therefore, it is important that the perception model correctly perceives the surrounding environment, and if it does not correctly perceive the surrounding environment, it is important to know why and how to correct it. However, as with many machine learning models, it may be difficult to know the cause of such errors and how to improve the system. This makes the currently disclosed technology particularly advantageous for these types of applications.

[0036] The output of the perception model can be used for downstream tasks such as trajectory prediction, path planning, or emergency braking systems or used by downstream systems of the ADS. In some embodiments, the machine learning model is an end-to-end model that is configured (in contrast to the above) to perform both perception tasks and downstream tasks. For example, the machine learning model can perform trajectory prediction or path planning directly based on sensor data.

[0037] The deployment of machine learning models typically involves a training phase during which the model learns from labeled training data to achieve accurate predictions during a subsequent inference phase. The training data (and input data during inference) can be, for example, images or image sequences, lidar data (i.e., point clouds), radar data, etc. In addition, the training / input data can include a combination or fusion of one or more different data types. The training / input data can, for example, include both an image depicting a scene of the vehicle's surroundings and a corresponding lidar point cloud of the same scene.

[0038] In some embodiments, the machine learning model may be implemented in any manner known to be suitable to a person of ordinary skill in the art using publicly available suitable software development machine learning code elements (e.g., such as code elements available in Pytorch, TensorFlow, and Keras or any other suitable software development platform).

[0039] Throughout this disclosure, reference will be further made to generative models. The term "generative model" herein means a machine learning model, but with the task of generating data. In other words, as described above, the general aspects of the machine learning model also apply to the generative model. However, the architecture and training method of the generative model may be different from, for example, a perceptual model (or a machine learning model for which an internal representation is generated). In the present case, as will be further described below, the generative model is trained to generate synthetic sensor data based on specific inputs. The generative model may be a so-called generative adversarial network (GAN). In another example, the generative model may be a so-called diffusion model. These are all well-known models in the art whose principles and features are easily understood by technicians. It goes without saying that since other types of generative models are also applicable, the currently disclosed technology is not limited to these examples.

[0040] Figure 1 is a schematic flowchart representation of a computer-implemented method 100 according to some embodiments. The method 100 may be combined with, for example, Figure 3 The method 100 is performed in a server of the server 300 described in the embodiment of the present invention. More generally, the method 100 may be performed by any suitable computing device. Advantageously, the server is a device with more available computing resources than the vehicle equipped with the ADS. This may facilitate the deployment of a more computationally intensive generative model.

[0041] The method 100 can be a method for providing insight into a machine learning model or the performance of a machine learning model. In other words, the method 100 can be a method for analyzing or visualizing the causes of potential errors or omissions of a machine learning model. Thus, the method can help develop a machine learning model or explain the inner workings of a model. As previously mentioned, the machine learning model can be a perceptual model.

[0042] Below, the different steps of method 100 are described in more detail. Even though illustrated in a particular order, the steps of method 100 may be performed in any suitable order and multiple times. Figure 1 A specific order of method steps can be shown, but the order of steps can be different from the order depicted. In addition, two or more steps can be performed simultaneously or partially concurrently. For example, based on a specific implementation, the steps represented as S110 and S112 can be performed independently of the steps represented as S114 and S116. This variation will depend on the selected software and hardware systems and the designer's choice. All these variations are within the scope of the present invention. Similarly, standard programming techniques with rule-based logic and other logic can be used to complete software implementation to complete various steps. According to the present disclosure, further variations of method 100 will become apparent. The embodiments mentioned and described herein are given as examples only and should not be limitations of the present invention. Other solutions, uses, objectives and functions within the scope of the present invention claimed in the patent claims described below should be apparent to those skilled in the art.

[0043] The method 100 includes obtaining S102 sensor data related to a scene of the surrounding environment of a vehicle equipped with an autonomous driving system. Obtaining S102 sensor data may include receiving sensor data from a vehicle that has collected sensor data using one or more on-board sensors. The sensor data may include one or more of image data (e.g., one or more images or video sequences), lidar data (i.e., point clouds), radar data, and ultrasonic data. The sensor data may be obtained as raw sensor data. Alternatively, the sensor data may be obtained as processed or fused sensor data of two or more different types of sensor data. The sensor data may further include map data or geographic location data (e.g., GPS data) of the vehicle.

[0044] Throughout this disclosure, the term "obtain" should be interpreted broadly and includes receiving, retrieving, collecting, acquiring, etc., directly and / or indirectly between two entities configured to communicate with each other or further communicate with other external entities. However, in some embodiments, the term "obtain" will be interpreted as determining, deriving, forming, calculating, etc. Therefore, as used herein, "obtaining" may indicate receiving a parameter at a first entity / unit from a second entity / unit, or determining a parameter at a first entity / unit, for example, based on data received from another entity / unit.

[0045] The surroundings of the vehicle can be understood as the general area around the vehicle in which objects (e.g., other vehicles, landmarks, obstacles, etc.) can be detected and determined by vehicle sensors (radar sensors, lidar sensors, cameras, etc.) (i.e., within the sensor range of the vehicle). The scene can be understood as the area of ​​the surrounding environment depicted in the sensor data. In other words, the scene can correspond to the area of ​​the surrounding environment where its ADS task (e.g., perception task) is to be performed. In the sense that the sensor data reflects one or more attributes of the scene, the sensor data is related to the scene. In other words, the sensor data includes information related to the scene. The sensor data related to the scene can be any type of data related to the scene. For example, the sensor data may include one or more of image data, lidar data, radar data, and ultrasonic data. Image data should be understood in this article as one or more images depicting a scene in which an evasive maneuver is performed. Therefore, the image can depict any object present at the scene. One or more images can constitute an image sequence. In other words, each image of the image sequence can depict the scene at different time points and / or from different perspectives. Therefore, the image sequence can be understood as a frame sequence of a video of the scene. Alternatively, one or more images may constitute multiple images captured by at least two different cameras. Thus, the multiple images may depict the scene from different angles.

[0046] Method 100 may further include: for a machine learning model configured to perform a task of the autonomous driving system, obtaining S104 an internal representation of the sensor data. The internal representation should be interpreted as a representation that reflects how the machine learning model (sometimes referred to as a "model") represents its sensor data internally. In other words, the internal representation can be understood as a representation in the so-called latent space of the machine learning model. The latent space in the field of machine learning generally refers to a lower-dimensional space in which complex and high-dimensional data (i.e., input data) is converted and encoded into states of generally different dimensions. This state is a state suitable for the machine learning model to predict it. In the current scenario, the input data is sensor data. Therefore, the sensor data can be converted into a state suitable for the machine learning model (e.g., a perception model) to predict it. In some embodiments, the latent space may have the same dimension as the input data or a higher dimension than the input data. This representation is learned by the machine learning model during training and captures the essential features or patterns in the input data. The term "latent" means that these features or patterns are not directly observed or explicitly provided, but are inferred by the model. Therefore, the internal representation (or "latent space representation" or "embedding") can be a representation of the sensor data used by the machine learning model to perform its tasks. In some embodiments, the internal representation is a one-dimensional or multi-dimensional vector representation. The internal representation can be formed by inputting or feeding sensor data into a machine learning model and extracting the internal representation from the machine learning model. For example, the internal representation can be extracted from an internal state or layer of the machine learning model. The internal state of the model can be a state before the final prediction or output state. It should be understood that the internal representation can be extracted from different layers or depths inside the model. Therefore, it is possible to extract multiple internal representations of the same model. For sensor data, which (one or more) internal representations are extracted from the model may vary depending on the specific implementation. In one example, the machine learning model may be an end-to-end model configured to perform perception tasks and downstream tasks. In this case, the internal representation can be extracted from an intermediate layer or state of the model, for example, corresponding to the part of the model that performs the perception task.

[0047] The internal representation may be received from the vehicle. Thus, the internal representation may be determined by a machine learning model local to the vehicle. Alternatively, the internal representation may be obtained S104 by determining in a server executing method 100. More specifically, the server may access a copy of a machine learning model provided in the vehicle (referred to herein as a global or master model). By determining the internal representation in the server, less data needs to be transmitted, and computing resources in the vehicle can be saved.

[0048] The scene related to the obtained sensor data may be a scene of interest. The scene may be determined by the vehicle as a scene of interest. In this case, in response to the vehicle having determined the scene as a scene of interest, sensor data related to the scene may be obtained S102.

[0049] Alternatively, the server may determine the scene as a scene of interest based on the obtained sensor data. Obtaining S104 the internal representation may be performed in response to determining the scene as a scene of interest. For example, the server may receive sensor data from the vehicle, process the sensor data to determine whether the scene is a scene of interest, and then determine the internal representation of the sensor data in response to the scene being determined as a scene of interest.

[0050] The scene of interest may correspond to a scene of interest for the development of the machine learning model (i.e., a scene that may be relevant for further investigation). Typically, this may be determined based on sensor data or based on the output of the machine learning model. As an example, a scene of interest may be a scene in which the machine learning model outputs its incorrect prediction. In other words, a scene of interest may be a scene in which the machine learning model is wrong. In another example, a scene of interest may be a scene that is challenging to the machine learning model to some extent, i.e., a scene in which the model output has a low confidence. For example, the scene of interest may depict a scene of low contrast or lighting conditions, or depict a rare or previously unseen object or an object in front of a complex or cluttered background. In short, determining a scene as a scene of interest may be understood as determining a potential failure mode of the machine learning model. Therefore, method 100 may include: detecting errors (or potential challenging scenes) of the machine learning model; and obtaining an internal representation of the sensor data in response thereto. By doing so, the amount of data that needs to be collected, transmitted, and processed can be reduced. Compared to analyzing all collected data, it can further save time and computing resources.

[0051] In response to a triggering event, a scene can be determined (or identified) as a scene of interest. The triggering event can be intervention by an ADS or a driver that takes precedence over the ADS (e.g., activation of an automatic emergency braking system). More specifically, the triggering event can be that a deviation behavior of the vehicle has been detected. Therefore, the scene can be identified as a scene of interest based on the deviation behavior of the vehicle detected at the scene. In other words, the scene can be identified as a scene of interest in response to detecting the deviation behavior of the vehicle in conjunction with the scene related to the sensor data.

[0052] The deviation behavior may be one of activation of an emergency braking system, an evasive maneuver performed by a driver, an evasive maneuver performed by an automated driving system, a driving incident, and a notification from an occupant of the vehicle.

[0053] The term "evasive maneuver" as used in the context of the present disclosure refers to a movement action taken by a vehicle to avoid a collision, potential obstacle, obstruction, or dangerous situation, etc. In this case, the evasive maneuver being performed can be used as an indication that something unexpected has occurred that the machine learning model failed to correctly predict. As an example, in response to the machine learning model failing to detect an obstacle or erroneously detecting an obstacle that does not actually exist, an evasive maneuver may be performed by the driver to override the autonomous driving system. As another example, the autonomous driving system may perform an evasive maneuver in response to the sudden detection of an obstacle that should have been detected at an earlier stage. The same applies to the activation of the emergency braking system.

[0054] An evasive maneuver may be a sudden deviation from the current direction of travel. An evasive maneuver may include dynamic adjustment of the vehicle's speed and / or its trajectory (i.e., by increasing speed or braking). In some embodiments, the evasive maneuver is an evasive steering maneuver. In other words, the evasive maneuver may be achieved by adjusting the vehicle's trajectory.

[0055] An evasive maneuver may be a maneuver that deviates from the current lane of the vehicle. This may be detected, for example, by a lane detection model (based on machine learning). An evasive maneuver may be a maneuver that deviates from the expected trajectory of the vehicle. In other words, an evasive maneuver may be detected as a vehicle deviating from the expected trajectory of the vehicle. The expected trajectory herein refers to the predicted or planned path or movement that the vehicle is expected to follow in a given environment or driving scenario. The expected trajectory may be determined based on the vehicle's machine learning-based trajectory prediction. The expected trajectory may be formed based on various factors of the vehicle including but not limited to sensor data, map or location data, vehicle dynamics, planned routes, behavior predictions, and real-time decision algorithms. Sensor data may provide information about the vehicle's surroundings, i.e., how the vehicle perceives its surroundings. Sensor data may provide information about the location of other vehicles, pedestrians, obstacles, and road conditions, etc. It may further provide information about the direction of the lane or road in which the vehicle is traveling. Maps or location data may provide additional information about how the vehicle can be expected to navigate. Vehicle dynamics may provide information about how the vehicle is traveling in terms of speed, acceleration, etc. The planned route may be based on the vehicle's destination and provide information about what route and / or lane the vehicle is expected to travel along. Behavior predictions may be based, for example, on a driving scenario such as driving along a highway. In this case, the vehicle may be expected to travel at a certain speed and along a certain lane, such that any deviation from that lane and speed may indicate an evasive maneuver. Behavior predictions may also be related to the driver's driving characteristics. Real-time decision making refers to how the vehicle (driver or automated driving system) reacts to sudden changes in the driving scenario that may cause a change in the vehicle's expected trajectory, such as a planned lane change.

[0056] Notifications may be received from a passenger of the vehicle in response to discomfort or dissatisfaction experienced by the passenger with the performance of the automated driving system.

[0057] In some embodiments, a scene may be determined to be a scene of interest by processing the sensor data. As an example, the sensor data may be determined to be related to a scene that is outside the distribution of the machine learning model or depicts a rare or previously unseen object. This can be done, for example, by inputting the sensor data into another machine learning model that is trained to determine such situations. Thus, more challenging or tricky situations than analytical machine learning models can be detected in an efficient manner.

[0058] In some embodiments, the sensor data may be obtained as randomly sampled sensor data or as sensor data sampled at defined intervals S102. This may facilitate continuous monitoring of the performance of the machine learning model. If a difference is determined in a subsequent comparison between the raw sensor data and the synthetic sensor data of the randomly (or at defined intervals) sampled sensor data, this may be used as a signal that the machine learning model should be further evaluated.

[0059] The method 100 further comprises: generating S106 synthetic sensor data for subsequent comparison with the acquired sensor data. The synthetic sensor data is generated by inputting the internal representation into a generative model trained to generate synthetic sensor data based on the internal representation. Thus, the generated synthetic sensor data can be used for subsequent comparison with the original sensor data. As will be further described below, in combination with FIG. 5A to FIG. 5D , it is possible to compare the synthetic sensor data with the acquired sensor data to identify any potential differences between them. This enables gaining insight into the inner workings of the model.

[0060] Synthetic sensor data can be viewed as a visual representation of the internal representation of sensor data. In other words, synthetic sensor data can be viewed as a reconstruction of the original sensor data based on the internal representation of the machine learning model. Therefore, synthetic sensor data provides a visualization of how the machine learning model perceives the scene depicted in the sensor data. The wording "synthetic" as in "synthetic sensor data" means in this article that the sensor data is artificially or machine-generated by a generative model rather than collected by real-world sensors.

[0061] As described above, the generative model can be, for example, a generative adversarial network or a diffusion model. However, the generative model can be any kind of generative model that is trained to generate synthetic sensor data based on an internal representation. More specifically, the generative model can be trained based on a training data set that includes multiple internal representations of a machine learning model as input and corresponding sensor data as basic facts. In other words, the generative model can be trained to reconstruct sensor data from data in the form of an internal representation of the machine learning model. As described above, the internal representation used in the training of the generative model can be obtained in the same manner as the internal representation obtained for the obtained sensor data. The generative model can be trained on the internal representation of a specific machine learning model with which it is intended to be used. However, alternatively, the generative model can be trained using the internal representation of another machine learning model. More specifically, the process of training the generative model can be described as follows.

[0062] First, a machine learning model trained to perform the task of the ADS can be provided. For the expected task of the machine learning model, the machine learning model can be pre-trained or trained on a training data set. Then, the trained machine learning model can be frozen so that it no longer continues to learn from new data. Thus, during the rest of the process, the model will not change. In the next step, an instance of sensor data (e.g., a set of images) for training can be embedded in the latent space of the machine learning model. In other words, the internal representation of the model corresponding to the instance of sensor data can be determined. This can be done by feeding the sensor data into the machine learning model and extracting the internal representation from the state of the machine learning model before it generates a final result (e.g., a prediction). Thus, a training set of paired sensor data instances (e.g., images) and corresponding internal representations can be obtained. In the next step, the training set can be used to train a generative model to reconstruct the sensor data based on the internal representation. Therefore, the generative model learns to generate sensor data that looks similar to the sensor data of the training set with the internal representation as input. The instance of sensor data used to form a training set for training the generative model can include sensor data used in the training of the machine learning model. Alternatively or in combination, the training set may include sensor data not seen by the machine learning model during its training.

[0063] The method 100 may further include identifying one or more differences between the sensor data obtained and the generated synthetic sensor data based on the comparison of the obtained sensor data and the generated synthetic sensor data. The wording "difference" in this article refers to any difference between the obtained sensor data (i.e., the original sensor data) and the synthetic sensor data. The difference may be, for example, a missing, blurred, unclear or distorted object. A distorted object may be an object that is only partially visible or is depicted in a different way. In addition, a curved object may be two or more objects that are mistakenly depicted as one object. In other words, the difference may be that many objects are indistinguishable from each other. These differences may provide an indication of why the machine learning model may fail. It should be noted that, in general, there is always some information loss between the original sensor data and the synthetic sensor data, for example, because the internal representation is generally of lower dimensionality than the sensor data, and due to differences between the machine learning model and the generated model. This information loss may, for example, manifest itself as an overall blur over the entire synthetic sensor data. The difference used in this article refers to a difference between the original sensor data and the synthetic sensor data that is more significant than this overall information loss. However, what the difference is can be determined in a subsequent comparison between the original sensor data and the synthetic sensor data. The following will be combined with FIG. 5A to FIG. 5D Further describe the differences between raw sensor data and synthetic sensor data.

[0064] In some embodiments, method 100 includes outputting the synthesized data and the raw sensor data to a display. Method 100 may further include prompting a user to compare between the synthesized sensor data and the raw sensor data. Method 100 may further include receiving an indication of a determined difference between the raw sensor data and the synthesized sensor data.

[0065] The comparison between raw sensor data and synthetic sensor data enabled by the presently disclosed techniques and performed in any of the ways described herein can provide information about how to improve the machine learning model. Based on such a comparison, it can be concluded, for example, that the model must become better in poor lighting conditions, detect debris better, or distinguish vehicles in a traffic jam. Based on this information, targeted training data collection methods can be employed to promote the improvement of the model in any determined scenario.

[0066] In some embodiments, method 100 can be used to determine errors or potential errors made by a machine learning model based on a subsequent comparison between the sensor data and the synthetic sensor data. Thus, instead of performing the steps of the method in response to detecting a potential error, method 100 itself can be used to detect potential errors. For example, if one or more differences are determined, it can be concluded that the machine learning model may have missed an object in the scene. This information can be used, for example, to determine that the scene depicted in the sensor data is somehow outside the distribution on which the machine learning model was trained.

[0067] The method 100 may further include, in response to receiving an indication of a difference between the obtained sensor data and the synthesized sensor data, assigning S110 annotation data to the sensor data, and storing S112 the sensor data with the assigned annotation data for subsequent training of the machine learning model. The indication of the difference may be received from a user input, or determined by a server.

[0068] The method 100 may further include: in response to receiving an indication of a difference between the obtained sensor data and the synthetic sensor data, obtaining S114 additional sensor data related to the scene of the physical environment, and storing S116 the additional sensor data for subsequent training of the machine learning model. The additional sensor data may be sensor data related to different types of scenes compared to previously obtained sensor data. Alternatively or in combination, the additional sensor data may be sensor data related to the scene from a different angle than the previously obtained sensor data. The additional sensor data may be captured by the vehicle. Alternatively, the additional sensor data may be captured by another vehicle. The server may send a request for additional sensor data to another vehicle that may be about to pass the location of the scene. By collecting additional sensor data for the scene, the machine learning model can be further improved when processing scenes like the current scene.

[0069] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0070] In general, computer-accessible media may include any tangible or non-transitory storage media or memory media such as electronic, magnetic, or optical media, for example, a disk or CD / DVD-ROM connected to a computer system via a bus. As used herein, the terms "tangible" and "non-transitory" are intended to describe computer-readable storage media (or "memory") that do not include propagating electromagnetic signals, but are not intended to otherwise limit the types of physical computer-readable storage devices encompassed by the term computer-readable media or memory. For example, the term "non-transitory computer-readable media" or "tangible memory" is intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in a non-transitory form on a tangible computer-accessible storage medium may be further transmitted via a transmission medium or signals such as electrical, electromagnetic, or digital signals that may be transmitted via a communication medium such as a network and / or a wireless link.

[0071] Figure 2 is a schematic flowchart representation of a computer-implemented method 200 according to some embodiments. Figure 4 The method 200 is performed in a vehicle 400 equipped with an autonomous driving system. In other words, the method 200 can be performed by a control circuit of the autonomous driving system. The method 200 provides for the subsequent generation of synthetic data and a later comparison with the raw sensor data.

[0072] Below, the different steps of method 200 are described in more detail. Even though illustrated in a particular order, the steps of method 200 may be performed in any suitable order and multiple times. Figure 2 A specific order of method steps can be shown, but the order of steps may be different from the order depicted. In addition, two or more steps may be performed simultaneously or partially concurrently. This variation will depend on the selected software and hardware systems and the designer's choice. All of these variations are within the scope of the present invention. Similarly, software implementations can be accomplished using standard programming techniques with rule-based logic and other logic to accomplish various steps. Further variations of method 200 will become apparent from this disclosure. The embodiments mentioned and described herein are given as examples only and should not be limitations of the present invention. Other solutions, uses, objectives, and functions within the scope of the present invention claimed for protection by the patent claims described below should be apparent to those skilled in the art.

[0073] Method 200 includes: in response to detecting a deviation behavior of the vehicle, obtaining S202 sensor data related to the scene of the surrounding environment in which the deviation behavior is detected. The sensor data can be obtained by the vehicle's on-board sensors. For further details related to the vehicle's deviation behavior and how to determine it, refer to the above. It should be noted that sensor data can be continuously recorded (or collected) while the vehicle is driving. The deviation behavior can be detected, for example, based on sensor data related to the scene of the surrounding environment or based on other types of sensor data of the vehicle. Therefore, obtaining S202 sensor data in response to detecting the deviation behavior can be understood as storing or otherwise selecting specific sensor data corresponding to the deviation behavior from the continuously recorded sensor data.

[0074] The method 200 may further include: for a machine learning model configured to perform a task of the autonomous driving system, determining S104 an internal representation of the sensor data. The internal representation is determined S204 by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model.

[0075] The method 200 further comprises: transmitting S206 the sensor data and the internal representation of the sensor data to a server for subsequent generation of synthetic sensor data based on the internal representation. The server may be, for example, a server that combines Figure 3 Server 300 is described.

[0076] In some embodiments, the method 200 further includes generating synthetic sensor data for subsequent comparison with the acquired sensor data. The synthetic sensor data may be combined with the above Figure 1 In other words, a generative model may be provided in the vehicle to allow synthetic sensor data to be generated locally in the vehicle. Method 200 may then further include transmitting the synthetic sensor data to a server for subsequent comparison. This may be performed, for example, in conjunction with transmitting S206 the sensor data and the internal representation of the sensor data to the server. In some embodiments, the step denoted S206 may involve transmitting the sensor data and the synthetic sensor data to a server for subsequent comparison. Therefore, in this case, there is no need to transmit the internal representation to the server.

[0077] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0078] Figure 3 is a schematic diagram of a server 300 according to some embodiments. The server 300 may be configured to perform a combination Figure 1 Method 100 is described.

[0079] The server 300 described herein for the purposes of this patent application (which may also be referred to as a remote server, cloud server, central server, background server, fleet server, or back-end server) refers to a computer system or networked device configured to provide various computing services, data storage, processing capabilities, or resources to a client or user through a communication network. In the present case, the term "client" refers to a connected vehicle of a fleet (e.g., vehicle 400 described below). Therefore, the server 300 described herein may refer to a general-purpose computing device provided on the outside of the vehicle. Although the server 300 is illustrated herein as one device, the server 300 may be a distributed computing system formed by multiple different computing devices.

[0080] The server 300 includes a control circuit 302. The control circuit 302 may physically include one single circuit device. Alternatively, the control circuit 302 may be distributed over several circuit devices.

[0081] As in Figure 3 As shown in the example of , the server 300 may further include a transceiver 306 and a memory 308. The control circuit 302 is communicatively connected to the transceiver 306 and the memory 308. The control circuit 302 may include a data bus, and the control circuit 302 may communicate with the transceiver 306 and / or the memory 308 via the data bus.

[0082] The control circuit 302 may be configured to perform overall control of the functions and operations of the server 300. The control circuit 302 may include a processor 304 such as a central processing unit (CPU), a microcontroller, or a microprocessor. The processor 304 may be configured to execute program code stored in the memory 308 to perform the functions and operations of the server 300. The control circuit 302 is configured to perform the above-mentioned Figure 1 The steps of method 100 are described. The steps may be implemented in one or more functions stored in memory 308.

[0083] The transceiver 306 is configured to enable the server 300 to communicate with other entities such as vehicles or other servers. The transceiver 306 can both send data to the server 300 and receive data from the server 300.

[0084] The memory 308 may be a non-transitory computer-readable storage medium. The memory 308 may be one or more of a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, a random access memory (RAM), or other suitable devices. In a typical arrangement, the memory 308 may include a non-volatile memory for long-term data storage and a volatile memory used as a system memory for the server 300. The memory 308 may exchange data with the circuit 302 via a data bus. There may also be accompanying control lines and an address bus between the memory 308 and the circuit 302.

[0085] The functions and operations of the server 300 may be implemented in the form of an executable logic program (e.g., code lines, software program, etc.) stored on a non-transitory computer-readable recording medium (e.g., memory 308) of the server 300, and executed by the circuit 302 (e.g., using the processor 304). In other words, when it is stated that the circuit 302 is configured to perform a specific function, the processor 302 of the circuit 304 may be configured to execute a program code portion stored on the memory 308, wherein the stored program code portion corresponds to the specific function. In addition, the functions and operations of the circuit 302 may be independent software applications, or form part of a software application that performs additional tasks associated with the circuit 302. The functions and operations described may be considered as being such as those described above in conjunction with the above. Figure 1 The method 100 discussed, the corresponding device is configured to perform the method. In addition, although the functions and operations described can be implemented in software, such functions can also be implemented via dedicated hardware or firmware or some combination of one or more hardware, firmware and software. In the following, the functions and operations of the server 300 are described.

[0086] The control circuit 302 is configured to obtain sensor data related to the scene of the surrounding environment of the vehicle equipped with the autonomous driving system. This can be performed, for example, by executing a first obtaining function.

[0087] The control circuit 302 is further configured to obtain an internal representation of the sensor data for a machine learning model configured to perform the tasks of the autonomous driving system. As previously described, the internal representation can be formed by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model.

[0088] The control circuit 302 is further configured to generate synthetic sensor data for subsequent comparison with the acquired sensor data, wherein the synthetic sensor data is generated by inputting the internal representation into a generative model trained to generate synthetic sensor data based on the internal representation.

[0089] The control circuit 302 may be further configured to determine one or more differences between the acquired sensor data and the generated synthetic sensor data based on the comparison of the acquired sensor data and the generated synthetic sensor data.

[0090] The control circuit 302 may be further configured to, in response to receiving an indication of a difference between the acquired sensor data and the synthesized sensor data: assign annotation data to the sensor data, and store the sensor data with the assigned annotation data for subsequent training of the machine learning model.

[0091] The control circuit 302 may be further configured to, in response to receiving an indication of a difference between the obtained sensor data and the synthesized sensor data: obtain additional sensor data related to the scene of the physical environment, and store the additional sensor data for subsequent training of the machine learning model. As described above, the indication of the difference may be determined by a user, or automatically by a machine.

[0092] It should be noted that, as mentioned above, Figure 1 The principles, features, aspects and advantages of the described method 100 are also applicable to the server 300 as described herein. To avoid unnecessary repetitions, reference is made to the above.

[0093] Figure 4 4 is a schematic diagram of a vehicle 400 in accordance with some embodiments. Vehicle 400 is equipped with an automated driving system (ADS) 410. As used herein, a "vehicle" is any form of motorized transportation. For example, vehicle 400 may be any road vehicle such as a car (as illustrated herein), a motorcycle, a (freight) truck, a bus, a smart bike, etc.

[0094] Vehicle 400 includes a number of elements commonly found in autonomous or semi-autonomous vehicles. It will be appreciated that vehicle 400 can have Figure 4 Any combination of the various elements shown in FIG. In addition, the vehicle 400 may include Figure 4 400. Although various elements are shown herein as being located inside the vehicle 400, one or more of the elements can be located outside the vehicle 400. Furthermore, as will be readily appreciated by those skilled in the art, even though various elements are described herein in a particular arrangement, the various elements can be implemented in a different arrangement. It should be further noted that the various elements can be communicatively connected to each other in any suitable manner. Since the elements of the vehicle 400 can be implemented in several different ways, Figure 4 The vehicle 400 should be considered as an illustrative example only.

[0095] The vehicle 400 includes a control system 402. The control system 402 is configured to perform overall control of the functions and operations of the vehicle 400. The control system 402 includes a control circuit 404 and a memory 406. The control circuit 402 may physically include a single circuit device. Alternatively, the control circuit 402 may be distributed over several circuit devices. As an example, the control system 402 may share its control circuit 404 with other parts of the vehicle. The control circuit 402 may include one or more processors such as a central processing unit (CPU), a microcontroller, or a microprocessor. One or more processors may be configured to execute program codes stored in the memory 406 in order to perform functions and operations of the vehicle 400. The processor may be or include any number of hardware components for performing data or signal processing or for executing computer codes stored in the memory 406. In some embodiments, the control circuit 404 or some of its functions may be implemented on one or more so-called systems on chips (SoCs). As an example, the ADS 410 may be implemented on the SoC. Optionally, the memory 406 includes a high-speed random access memory such as DRAM, SRAM, DDRRAM or other random access solid-state storage devices; and, optionally, includes a non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solid-state storage devices. The memory 406 may include a database component, an object code component, a script component or any other type of information structure for supporting various actions of this specification.

[0096] In the illustrated example, the memory 406 further stores map data 408. The map data 408 may be used, for example, by the ADS 410 of the vehicle 400 in order to perform autonomous functions of the vehicle 400. The map data 408 may include high-definition (HD) map data. It is contemplated that even if the memory 408 is illustrated as an element separate from the ADS 410, it may be provided as an integrated element of the ADS 410. In other words, according to some embodiments, in the implementation of the present inventive concept, any distributed or local storage device may be utilized. Similarly, the control circuit 404 may be distributed, for example, so that one or more processors of the control circuit 404 are provided as an integrated element of the ADS 410 or any other system of the vehicle 400. In other words, according to exemplary embodiments, in the implementation of the present inventive concept, any distributed or local control circuit device may be utilized.

[0097] The vehicle 400 further includes a sensor system 420. The sensor system 420 is configured to acquire sensory data about the vehicle itself or its surroundings. The sensor system 420 may, for example, include a global navigation satellite system (GNSS) module 422 (e.g., GPS) configured to collect geographic location data of the vehicle 400. The sensor system 420 may further include one or more sensors 424. The one or more sensors 424 may be any type of on-board sensor such as a camera, a lidar and radar, an ultrasonic sensor, a gyroscope, an accelerometer, an odometer, etc. It should be understood that the sensor system 420 may also provide the possibility of acquiring sensory data directly or via a dedicated sensor control circuit in the vehicle 400.

[0098] The vehicle 400 further includes a communication system 426. The communication system 426 is configured to communicate with other vehicles (ie, via a vehicle-to-vehicle (V2V) communication protocol), remote servers (eg, as described above in conjunction with Figure 3 The communication system 426 may communicate with an external unit such as a cloud server, a database, or other external device, i.e., a vehicle-to-infrastructure (V2I) or vehicle-to-everything (V2X) communication protocol. The communication system 426 may communicate using one or more communication technologies. The communication system 426 may include one or more antennas. Cellular communication technology may be used for remote communications such as to a remote server or cloud computing system. In addition, if the cellular communication technology used has low latency, it may also be used for V2V communication, V2I communication, or V2X communication. Examples of cellular radio technologies are GSM, GPRS, EDGE, LTE, 5G, 5G NR, etc., and also include future cellular solutions. However, in some solutions, medium- and short-range communication technologies such as wireless local area networks (LANs) (e.g., solutions based on IEEE 802.11) may be used to communicate with other vehicles near the vehicle 400 or with local infrastructure elements. ETSI is developing cellular standards for vehicle communications, for example, 5G is considered a suitable solution due to low latency and efficient handling of high bandwidth and communication channels.

[0099] The communication system 426 may further provide the possibility to send the output to a remote location (e.g. a remote server, operator or control center) by means of one or more antennas. Furthermore, the communication system 426 may be further configured to allow the various elements of the vehicle 400 to communicate with each other. For example, the communication system may provide a local network setup such as CAN bus, I2C, Ethernet, fiber optics, etc. The local communication within the vehicle may also be of wireless type with protocols such as WiFi, LoRa, Zigbee, Bluetooth or similar medium / short range technologies.

[0100] The vehicle 400 further includes a steering system 420. The steering system 428 is configured to control the steering of the vehicle 400. The steering system 428 includes a steering module 430 configured to control the direction of the vehicle 400. The steering system 428 further includes a throttle module 432 configured to control the actuation of the throttle of the vehicle 400. The steering system 428 further includes a brake module 434 configured to control the actuation of the brakes of the vehicle 400. The various modules of the steering system 428 can receive manual inputs from the driver of the vehicle 400 (i.e., from the steering wheel, throttle pedal, and brake pedal, respectively). However, the steering system 428 can be communicatively connected to the ADS 410 of the vehicle to receive instructions on how the various modules should act. Therefore, the ADS 410 can control the steering of the vehicle 400.

[0101] As described above, the vehicle 400 includes an ADS 410. The ADS 410 may be part of the vehicle's control system 402. The ADS 410 is configured to perform functions and operations of autonomous functions of the vehicle 400. The ADS 410 may include a plurality of modules, wherein each module is responsible for a different function of the ADS 410.

[0102] The ADS 410 may include a positioning module 412 or positioning block / system. The positioning module 412 is configured to determine and / or monitor the geographic location and orientation of the vehicle 400 and may utilize data from a sensor system 422, such as data from a GNSS module 420. Alternatively or in combination, the positioning module 412 may utilize data from one or more sensors 424. Alternatively, the positioning system may be implemented as a real-time kinematic (RTK) GPS for improved accuracy.

[0103] The ADS 410 may further include a perception module 414 or perception block / system. The perception module 414 may refer to any known module and / or function, for example, included in one or more electronic control modules and / or nodes of the vehicle 400, adapted and / or configured to interpret perception data related to driving of the vehicle 400 to identify, for example, obstacles, lanes, relevant signs, appropriate navigation paths, etc. Thus, the perception module 414 may be adapted to rely on and obtain input from a plurality of data sources, such as vehicle imaging, image processing, computer vision, and / or in-vehicle networking, in combination with, for example, sensory data from the sensor system 420.

[0104] A machine learning model, such as an object detection model for detecting and / or classifying objects in the environment surrounding the vehicle 400 , may be part of the ADS 410 , or more specifically, part of the perception module 414 .

[0105] The vehicle 400 is configured to perform the above combined Figure 2The functions of the method 200 described above. These functions can be implemented in a separate computing device provided in the vehicle. The computing device may include a control circuit configured to perform the steps of the method 200. Alternatively, as will be readily appreciated by those skilled in the art, the functions may be distributed over one or more modules, systems or elements of the vehicle 400. For example, the control circuit 404 of the control system 402 may be configured to perform the steps of the method 200. More specifically, the control circuit 404 of the vehicle 400 is configured to obtain sensor data related to the scene of the surrounding environment in which the deviation behavior is detected in response to detecting the deviation behavior of the vehicle. The sensor data may be collected by one or more sensors 424 of the vehicle 400. The control circuit 404 is further configured to determine the internal representation of the sensor data by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model for a machine learning model configured to perform the tasks of the ADS. The machine learning model may, for example, be the machine learning model of the perception module 414 described above. However, the machine learning model may also be a model of different modules of the ADS. The control circuit 404 is further configured to transmit the sensor data and the internal representation of the sensor data to the server for subsequent generation of synthetic sensor data based on the internal representation. In addition, as previously described, the control circuit 404 can be further configured to generate synthetic sensor data by inputting the internal representation into the generative model. The server can be as described above in combination with Figure 3 Server 300 is described.

[0106] The positioning module 412 and / or the perception module 414 may be communicatively connected to the sensor system 420 to receive sensor data from the sensor system 320. The positioning module 412 and / or the perception module 414 may further send control instructions to the sensor system 420.

[0107] The ADS may further include a path planning module 416. The path planning module 416 is configured to determine a planned path for the vehicle 400 based on the perception and position of the vehicle determined by the perception module 414 and the positioning module 412, respectively. The planned path determined by the path planning module 416 may be sent to the manipulation system 428 for execution.

[0108] The ADS may further include a decision and control module 418. The decision and control module 418 is configured to execute control of the ADS 410 and make decisions for the ADS 410. For example, the decision and control module 418 may decide whether the planned path determined by the path planning module 416 should be executed. The decision and control module 418 may be further configured to detect any deviation of the vehicle, such as deviation from the planned path or expected trajectory of the path planning module 416. This includes both evasive maneuvers performed by the ADS 410 and by the vehicle driver.

[0109] It should be understood that parts of the described solution may be implemented in the vehicle 400, in a system disposed externally to the vehicle, or in a combination of the vehicle and the vehicle; for example, in a server that communicates with the vehicle, a so-called cloud solution. Different features and steps of the embodiments may be combined in other combinations than those described. Furthermore, the elements of the vehicle 400 (i.e., systems and modules) may be implemented in combinations different from those described herein. Furthermore, it should be noted that, as described above in conjunction with Figure 2 The principles, features, aspects and advantages of the described method 200 are also applicable to the vehicle 400 as described herein. To avoid unnecessary repetitions, reference is made to the above.

[0110] Figure 5A and Figure 5B Raw sensor data 500 and synthetic sensor data 500 ′ are respectively illustrated by way of example. Figure 5A and Figure 5B The purpose of is to illustrate the principles of the presently disclosed technology. In this example, both the raw sensor data 500 and the synthetic sensor data 500' are image data depicting a scene of the vehicle's surroundings. The raw sensor data 500 herein represents an image captured by a camera of a vehicle. The image depicts a plurality of objects in a scene that is typically present in the surroundings of a vehicle traveling along a road. In particular, the raw sensor data 500 depicts a plurality of road users (here, a plurality of cars 502a-g and a truck 504). The raw sensor data 500 further depicts some road infrastructure such as a first pole 506a and a second pole 506b, as well as a number of surrounding buildings. As shown in Figure 5A As can be seen in FIG. 5 , objects depicted in the raw sensor data 500 are clearly distinguishable from each other.

[0111] Now turn to Figure 5B . As previously described, the synthetic sensor data is generated by a generative (machine learning) model that is trained to generate synthetic sensor data based on an internal representation of the raw sensor data. Thus, the synthetic sensor data 500' can be viewed as a machine-generated reconstruction of the raw sensor data 500 in the way the machine learning model sees it. Thus, the synthetic sensor data is a visualization of what data the machine learning model uses to perform its task. If, for example, the machine learning model is an object detection model configured to perform an object detection task, then the synthetic sensor data represents a visualization of what data the object detection model uses to perform object detection.

[0112] like Figure 5BAs shown, the depicted objects are slightly more difficult to distinguish than those in the original data. This may be due to the internal representation being a low-dimensional representation of the data, and therefore will not perfectly reflect it. Additionally, the generative model may introduce some minor inaccuracies in the reconstruction of the sensor data. However, some objects (e.g., the vehicles represented as 502c', 502e', and 502g' and the pillar represented as 506') are still distinguishable in the synthetic image. Therefore, when comparing the original sensor data 500 and the synthetic sensor data 500', this can be used as an indication that the machine learning model is able to perceive these objects. However, as Figure 5B As shown by the dashed lines in , some objects may no longer be distinguishable / visible. For example, these objects may blend with the background or other nearby objects. Therefore, these objects are examples of the differences between the original sensor data 500 and the synthetic sensor data 500'. In this example, the vehicles represented by 502a', 502b' and 502f and the pillar represented by 506b' are no longer visible in the synthetic sensor data. Since they are missing in the synthetic data, it can be concluded that, for example, an object detection model may not be able to detect these objects. In addition, as shown in this article, the vehicles represented as 504' and 502d' are an example of how two objects can blend together into one object. In other words, a machine learning model that performs tasks based on sensor data may incorrectly treat these vehicles as one.

[0113] By way of another example, Figure 5C and Figure 5D The raw sensor data 500 and the synthesized sensor data 500' are illustrated respectively. In this case, the raw sensor data 500 is a real-world image and the synthesized sensor data 500' is a reconstruction of the real-world image based on its internal representation. For better understanding, some details are highlighted in the images to illustrate what differences can be determined when comparing the two images.

[0114] As a first example, the pillars in the area denoted as 508 in the raw sensor data are no longer visible in the corresponding area (denoted as 508') in the synthetic sensor data. In the second example, the area denoted as 510 in the raw sensor data clearly shows the van behind the larger truck. However, in the synthetic sensor data, the corresponding area (denoted as 510') appears to depict only one larger vehicle. Therefore, this is an example of two vehicles being mistakenly perceived as one vehicle. In the third example, in the corresponding areas 512' and 514 in the synthetic sensor data, the cars in the areas denoted as 512 and 514 are no longer visible. Finally, the areas denoted as 516 and 518 show examples of vehicles that are still visible in the synthetic data as seen in the areas denoted as 516' and 518'. Therefore, it can be expected that these vehicles are correctly perceived by the machine learning model when performing its task.

[0115] Figure 6 The distributed system 600 according to some embodiments is illustrated by way of example. The system 600 should be viewed as a non-limiting example of implementing aspects disclosed herein of the inventive concept. For example, the system 600 is configured to perform the above-described Figure 1 The system 600 may be further configured to perform the above-described method 100 in conjunction with Figure 2 Therefore, unless otherwise stated, any features or principles described above are also applicable to the system 600 described herein, and vice versa.

[0116] System 600 includes a server 602 (or a remote server, a cloud server, a central server, a backend server, a fleet server, or a backend server), hereinafter referred to as a remote server 602 or simply referred to as server 602. Server 600 may be a combination of the above Figure 3 Therefore, the server 602 can be configured to perform the above combined Figure 1 Described method 100. As shown, server 602 may be provided in the cloud, ie, as a cloud-implemented server.

[0117] The system 600 further includes one or more vehicles 604a-c, also referred to as a fleet. The one or more vehicles 604a-c may be a combination of the above Figure 4 Thus, one or more vehicles 604a-c may be configured to perform a combined Figure 2 Method 200 is described.

[0118] One or more vehicles 604a-c are communicatively connected to a remote server 602 to transmit and / or receive data 606 between the vehicle and the server. One or more vehicles 604a-c may be further communicatively connected to each other. Data 606 may be any type of data such as communication signals or sensor data. Communication may be performed by any suitable wireless communication protocol. The wireless communication protocol may be, for example, a remote communication protocol such as a cellular communication technology (e.g., GSM, GPRS, EDGE, LTE, 5G, 5G NR, etc.) or a short-range to medium-range communication protocol such as a solution based on a wireless local area network (LAN) (e.g., IEEE 802.11). The server 602 includes a suitable memory and a control circuit such as one or more processors or processing circuits and one or more other components such as a data interface and a transceiver. The server 602 may further include a software module or other component so that the control circuit can be configured to execute machine-readable instructions loaded from the memory to implement the steps of the method to be executed.

[0119] As an example, Figure 6 The fleet illustrated in FIG. 6 includes three vehicles, a first vehicle, a second vehicle, and a third vehicle 604a-c. However, the system 600 may include any number of vehicles 604a-c. In the following, the system 600 will be described primarily with reference to the first vehicle 604a. It should be understood that these principles apply to any vehicle in the fleet.

[0120] In the following, an example of how the system 600 performs techniques according to some embodiments will be described. For further details on the different steps, refer to the above Figure 1 and Figure 2 to avoid inappropriate duplication.

[0121] In response to the vehicle detecting the deviation behavior, the vehicle 604a may acquire sensor data related to the scene of the surrounding environment of the vehicle 604a corresponding to the deviation behavior (i.e., corresponding to step S202). Alternatively or in combination, the sensor data may be obtained in response to detecting an error or failure of the machine learning configured to perform the task of the ADS of the vehicle 604a. It should be understood that the vehicle 604a may continuously collect sensor data while the vehicle is traveling. Then, in response to detecting the deviation behavior (or the error of the machine learning model), the sensor data depicting the relevant scene may be stored for further processing.

[0122] In some embodiments, the vehicle 604a may determine the internal representation of the sensor data of the machine learning model (i.e., corresponding to step S204) by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model. The vehicle 604a may then transmit the obtained sensor data and the determined internal representation to the server 602. Thus, the server 602 obtains the sensor data and the internal representation of the sensor data. In some alternative embodiments, the vehicle 604a transmits the obtained sensor data to the server 602. After obtaining the sensor data, the server 602 may determine the internal representation by inputting the sensor data into the machine learning model and extracting the internal representation of the sensor data from the machine learning model. In this case, the machine learning model may be a global version (also referred to as a master model) of the machine learning model. The global version of the machine learning model refers to the version of the machine learning model managed by the server. The same model may be deployed in vehicles in a fleet as a local version of the machine learning model.

[0123] After obtaining (i.e., corresponding to step S104) an internal representation of the sensor data (by receiving the internal representation from the vehicle 604a or determining the internal representation by the server 602), the server 602 may generate synthetic sensor data (i.e., corresponding to step S106). The synthetic sensor data can be formed by inputting the internal representation into a generative model (provided in the server). The generative model is trained to generate synthetic sensor data based on the internal representation. The generated synthetic sensor data can then be used for subsequent comparison with the obtained (raw) sensor data. The raw sensor data and the generated synthetic sensor data can be compared to determine any differences between them. The server can compare the raw sensor data and the synthetic sensor data by automatic processing means such as an image comparison algorithm. In some embodiments, the server may prompt the user to make a comparison. For example, the raw sensor data and the synthetic sensor data may be output to a display for user comparison. The server 602 may then receive the results of the comparison such as an indication of potential differences determined by the user. The results of the comparison can be used for continued development of the machine learning model. As an example, the sensor data may be annotated and stored for continued training of the machine learning model. In addition, a targeted approach can be used to collect additional training data for the same or similar scenarios for which it has been determined that its machine learning model must be improved. In this case, the server 602 can transmit a request to one or more vehicles in the fleet to collect sensor data related to certain scenarios.

[0124] The above-described process of system 600 should be understood as a non-limiting example of the presently disclosed technology to enhance understanding. Further variations are apparent from this disclosure and are readily recognized by those skilled in the art.

[0125] The present invention has been presented above with reference to specific embodiments. However, other embodiments than those described above are possible and within the scope of the present invention. Within the scope of the present invention, method steps different from the above method steps may be provided for performing the method by hardware or software. Therefore, according to an exemplary embodiment, a (non-transitory) computer-readable storage medium is provided, which stores one or more programs, the one or more programs being configured to be executed by one or more processors of a vehicle control system, the one or more programs including instructions for performing a method according to any of the above embodiments. Alternatively, according to another exemplary embodiment, a cloud computing system may be configured to perform any of the methods presented herein. The cloud computing system may include distributed cloud computing resources that jointly perform the methods presented herein under the control of one or more computer program products.

[0126] It should be noted that any reference signs do not limit the scope of the claims, that the present invention may be implemented at least in part by both hardware and software, and that the same hardware item may represent multiple "means" or "units".

Claims

1. A computer-implemented method (100) executed in a server, the method (100) comprising: obtaining ( S102 ) sensor data related to a scene of a surrounding environment of a vehicle equipped with an autonomous driving system; For a machine learning model configured to perform a task of the autonomous driving system, obtaining (S104) an internal representation of the sensor data, wherein the internal representation is formed by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model; and Synthetic sensor data is generated (S106) for subsequent comparison with the obtained sensor data, wherein the synthetic sensor data is generated by inputting the internal representation into a generative model trained to generate synthetic sensor data based on the internal representation.

2. The method (100) according to claim 1, wherein: The internal representation of the sensor data is obtained in response to identifying the scene as a scene of interest (S104).

3. The method (100) according to claim 1, wherein: The scene related to the obtained sensor data is a scene of interest.

4. The method (100) according to claim 2 or 3, wherein: The scene is identified as the scene of interest based on a deviation behavior of the vehicle detected at the scene.

5. The method (100) according to claim 4, wherein: The deviation behavior is one of activation of an emergency braking system, an evasive maneuver performed by a driver, an evasive maneuver performed by the automated driving system, a driving accident, and a notification from an occupant of the vehicle.

6. The method (100) of claim 1, further comprising: Based on the comparison of the acquired sensor data and the generated synthetic sensor data, one or more differences between the acquired sensor data and the generated synthetic sensor data are identified ( S108 ).

7. The method (100) of claim 1, wherein: The sensor data includes one or more of image data, lidar data, radar data and ultrasonic data.

8. The method (100) of claim 1, further comprising: In response to receiving an indication of a difference between the obtained sensor data and the synthesized sensor data, annotation data is assigned (S110) to the sensor data, and the sensor data with the assigned annotation data is stored (S112) for subsequent training of the machine learning model.

9. The method (100) of claim 1, further comprising: In response to receiving an indication of a difference between the obtained sensor data and the synthetic sensor data, additional sensor data related to the scene of the physical environment is obtained (S114) and the additional sensor data is stored (S116) for subsequent training of the machine learning model.

10. The method (100) of claim 1, wherein: The generative model is a generative adversarial network or a diffusion model.

11. A computer program product comprising instructions which, when executed by a computing device, cause the computing device to implement the method (100) according to claim 1.

12. A server (300) comprising a control circuit (302), wherein the control circuit (302) is configured to: obtaining sensor data related to a scene of a surrounding environment of a vehicle equipped with an automated driving system; For a machine learning model configured to perform a task of the autonomous driving system, an internal representation of the sensor data is obtained, wherein: The internal representation is formed by inputting the sensor data into the machine learning model and extracting the internal representation from the machine learning model; as well as Synthetic sensor data is generated for subsequent comparison with the acquired sensor data, wherein the synthetic sensor data is generated by inputting the internal representation into a generative model trained to generate synthetic sensor data based on the internal representation.

13. A computer-implemented method (200) performed by a vehicle equipped with an automated driving system, the method (200) comprising: In response to detecting a deviation behavior of the vehicle, obtaining (S202) sensor data related to a scene of a surrounding environment in which the deviation behavior is detected; For a machine learning model configured to perform the task of the autonomous driving system, determining (S204) the internal representation by inputting the sensor data into the machine learning model and extracting the internal representation of the sensor data from the machine learning model; as well as The sensor data and the internal representation of the sensor data are transmitted (S206) to a server for subsequent generation of synthetic sensor data based on the internal representation.

14. A computer program product comprising instructions which, when executed by a computing device, cause the computing device to implement the method (200) according to claim 13.

15. A vehicle (400) equipped with an automatic driving system (410), the vehicle (400) comprising: one or more sensors (424); and The control circuit (404) is configured to: In response to detecting a deviation behavior of the vehicle, obtaining sensor data related to a scene of a surrounding environment in which the deviation behavior was detected; for a machine learning model configured to perform the task of the autonomous driving system, determining the internal representation by inputting the sensor data into the machine learning model and extracting the internal representation of the sensor data from the machine learning model; as well as The sensor data and the internal representation of the sensor data are transmitted to a server for subsequent generation of synthetic sensor data based on the internal representation.

Images