Method for classifying traffic signs in image

Abstract

Relates to a method for classifying traffic signs in an image. A computer-implemented method includes obtaining an image captured by a camera of a vehicle, the image depicting at least a portion of an environment of the vehicle; identifying a region in the image corresponding to the traffic sign by processing the image via a first machine learning model configured to output a detection of the traffic sign in the input image; extracting a cut corresponding to the identified region from the image, where the cut has a native resolution based on a size of the identified region with respect to the obtained image; and determining classification data of the traffic signs by processing the cropping at the native resolution via a second machine learning model, where the second machine learning model is an attention-based neural network trained to process input images of the traffic signs at different resolutions and to generate corresponding classification data, where the classification data are classified by processing the cropping at the native resolution via the second machine learning model. The second machine learning model applies attention at the pixel level of the input image.

Description

A method for classifying traffic signs in images. Technical Field

[0001] The technology disclosed herein relates to the field of autonomous driving systems. Specifically, it relates to methods and apparatus for traffic sign recognition. Background Technology

[0002] Traffic sign recognition (TSR) systems are an integral part of advanced driver assistance systems (ADAS) and automated driving (AD) technologies. These systems are designed to automatically detect and interpret traffic signs in real time using cameras or other onboard sensors to provide drivers with information about speed limits and other traffic regulations, or to provide that information to automated driving systems as the basis for decisions and control of the vehicle's automated operation.

[0003] Early TSR systems used basic image processing techniques to detect the unique shapes and colors of signs. However, these systems may have had limitations in their ability to adapt to various environmental conditions such as changing lighting, weather, and blurred or worn signs. While these early systems were effective under standard conditions, they could sometimes lack performance in more complex driving environments. For example, they might have difficulty identifying faded, partially blurred, or unconventional signs. Furthermore, variations in sign design across different countries or regions present further challenges to these systems.

[0004] Recent advances in deep learning and artificial intelligence have improved the accuracy of TSR systems by enabling models to learn from large datasets of traffic signs and road environments. These systems typically use convolutional neural networks (CNNs), sometimes in conjunction with other machine learning techniques, to identify traffic signs very accurately, even under adverse conditions.

[0005] However, despite the promising nature of these approaches, there remains a need to improve the performance of TSR systems, such as in reducing false alarms and ensuring real-time performance and robustness across a wider range of driving environments. Such improvements could enhance the capabilities of autonomous driving systems, where accurate traffic sign interpretation is crucial for ensuring compliance with road regulations. Summary of the Invention

[0006] The techniques disclosed herein aim to mitigate, alleviate, or eliminate one or more of the aforementioned defects and disadvantages in the prior art to address various problems associated with traffic sign recognition (TSR) systems. More specifically, the inventors have recognized that the performance of CNN-based methods for TSR is limited by the fact that they require the input to have a certain size (i.e., the input image to have a certain resolution). In fact, each traffic sign captured by the camera will have a different size, depending on factors such as the distance to the camera at the capture point, or due to the fact that different types of traffic signs have different shapes and sizes. This means that in CNN-based methods, the image fed to the traffic sign classifier must be upsampled or downsampled. The purpose of the disclosed techniques is to address this problem by introducing an attention-based neural network approach into a two-stage traffic sign recognition pipeline. More specifically, a novel and improved manner of performing traffic sign recognition is configured to apply cross-attention directly to the image pixels, enabling it to work on images of different resolutions. Various aspects and embodiments of the disclosed techniques are defined in the following and appended independent and dependent claims.

[0007] According to a first aspect, a computer-implemented method for classifying traffic signs in an image is provided. The method includes: acquiring an image captured by a camera of a vehicle, the image depicting at least a portion of the vehicle's surrounding environment. The method further includes: identifying regions in the image corresponding to traffic signs by processing the image via a first machine learning model configured to output detections of traffic signs in the input image. The method further includes: extracting a crop from the image corresponding to the identified regions. The crop has a native resolution based on the size of the identified regions relative to the size of the acquired image. The method further includes: determining classification data for the traffic signs by processing the crop at the native resolution via a second machine learning model. The second machine learning model is an attention-based neural network trained to process input images of traffic signs at varying resolutions and generate corresponding classification data, wherein the second machine learning model applies attention at the pixel level of the input image. Similar advantages and preferred features exist for this aspect of the disclosed technology as for other aspects.

[0008] According to a second aspect, a computer program product including instructions is provided, which, when executed by a computing device, cause the computing device to perform the method according to any embodiment of the first aspect. According to an alternative embodiment of the second aspect, a (non-transitory) computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a processing system, the one or more programs including instructions for performing the method according to any embodiment of the first aspect. This aspect of the disclosed technology has similar advantages and preferred features to other aspects.

[0009] As used herein, the term "non-transitory" is intended to describe a computer-readable storage medium (or "memory") that does not include propagating electromagnetic signals, but is not intended to otherwise limit the type of physical computer-readable storage device included in the term computer-readable medium or memory. For example, the terms "non-transitory computer-readable medium" or "tangible memory" are intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in a non-transitory form on a tangible computer-accessible storage medium can be further transmitted via a transmission medium or a communication medium, such as an electrical, electromagnetic, or digital signal, that can be transmitted via a communication medium such as a network and / or a wireless link. Therefore, as used herein, the term "non-transitory" is a limitation on the medium itself (i.e., tangible, not a signal), rather than a limitation on the persistence of data storage (e.g., RAM vs. ROM).

[0010] According to a third aspect, a computing device is provided for classifying traffic signs in an image. The computing device includes control circuitry. The control circuitry is configured to acquire an image captured by a camera of a vehicle, the image depicting at least a portion of the vehicle's surrounding environment. The control circuitry is further configured to identify regions in the image corresponding to traffic signs by processing the image via a first machine learning model, the first machine learning model being configured to output the detection of traffic signs in the input image. The control circuitry is further configured to extract a cropping from the image corresponding to the identified regions, wherein the cropping has a native resolution based on the size of the identified regions relative to the size of the acquired image. The control circuitry is further configured to determine classification data for the traffic signs by processing the cropping at the native resolution via a second machine learning model, wherein the second machine learning model is an attention-based neural network trained to process input images of traffic signs at varying resolutions and generate corresponding classification data, wherein the second machine learning model applies attention at the pixel level of the input image. Similar advantages and preferred features exist for this aspect of the disclosed technology as for other aspects.

[0011] According to the fourth aspect, a vehicle is provided, which includes a camera and a computing device according to any embodiment of the third aspect. This aspect of the disclosed technology has similar advantages and preferred features as the other aspects.

[0012] The disclosed aspects and preferred embodiments may be appropriately combined with each other in any manner that is obvious to those skilled in the art, such that one or more features or embodiments disclosed with respect to one aspect may also be regarded as embodiments disclosed with respect to another aspect or another aspect.

[0013] The advantages of the publicly available technology stem at least in part from the two-stage TSR pipeline that incorporates an attention-based traffic sign classifier.

[0014] One advantage of some embodiments is that the traffic sign classification model can operate on data arrays of any size. In other words, the model can process images of any size, which has been found to be particularly useful for traffic sign recognition applications. In fact, eliminating the need for upsampling or downsampling can improve the performance of the TSR system. Otherwise, downsampling leads to information loss, and upsampling increases computational distortion and aspect ratio distortion.

[0015] Another advantage of some embodiments is that the traffic sign classification model can use information about the true aspect ratio of the cropped traffic sign to further consider this during the classification process. For example, information about how wide or multi-sided a traffic sign is can help distinguish between different traffic signs. This information is lost in CNN-based methods.

[0016] Another advantage of some embodiments is that additional inputs, such as text characters on a sign or other attributes, can be perfectly fed into the model. This can further extend the capabilities of the TSR system and improve the results.

[0017] Another advantage of some embodiments is that they are built on a two-state TSR pipeline, which is advantageous from a development perspective and is more adaptable to different geographic areas with different traffic sign sets.

[0018] Further embodiments are defined in the dependent claims. It should be emphasized that, when used in this specification, the term "comprising / including" is used to specify the presence of the stated feature, integral, step, or component. It does not exclude the presence or addition of one or more other features, integrals, steps, components, or groups thereof.

[0019] Referring to the embodiments described below, these and other features and advantages of the disclosed technology will be further illustrated below. Attached Figure Description

[0020] The foregoing aspects, features, and advantages of the disclosed technology will be more fully understood through the following illustrative and non-limiting detailed description of exemplary embodiments of the present disclosure, taken in conjunction with the accompanying drawings, in which:

[0021] Figure 1 is a schematic flowchart illustrating a method according to some embodiments;

[0022] Figure 2 is a schematic diagram of a computing device according to some embodiments;

[0023] Figure 3 is a schematic diagram of a vehicle according to some embodiments;

[0024] Figure 4A illustrates a two-stage traffic sign recognition pipeline according to some embodiments;

[0025] Figure 4B illustrates the architecture of a second machine learning model according to some embodiments; and

[0026] Figure 4C illustrates the architecture of a second machine learning model according to some embodiments, through another example. Detailed Implementation

[0027] This disclosure will now be described in detail with reference to the accompanying drawings, in which some exemplary embodiments of the disclosed technology are illustrated. However, the disclosed technology may be embodied in other forms and should not be construed as limited to the exemplary embodiments disclosed. The exemplary embodiments are provided to fully convey the scope of the disclosed technology to those skilled in the art. Those skilled in the art will understand that the steps, services, and functions described herein can be implemented using separate hardware circuitry, software that works in conjunction with a programmed microprocessor or general-purpose computer, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), and / or one or more digital signal processors (DSPs).

[0028] It will also be understood that when this disclosure is described in terms of method, it may also be embodied as a device including one or more processors and one or more memories coupled to the one or more processors, wherein computer code is loaded to implement the method. For example, in some embodiments, the one or more memories may store one or more computer programs that, when executed by the one or more processors, cause the device to perform the steps, services, and functions disclosed herein.

[0029] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. It should be noted that, as used in the specification and appended claims, unless the context clearly specifies otherwise, the articles “a,” “an,” “the,” and “the” are intended to indicate the presence of one or more elements. Thus, for example, in some contexts, references to “a unit” or “the unit” may refer to more than one unit, etc. Furthermore, the words “comprising” or “including” do not exclude other elements or steps. It should be emphasized that, when used in this specification, the term “comprising / including” is used to specify the presence of the stated feature, integral, step, or component. It does not exclude the presence or addition of one or more other features, integrals, steps, components, or groups thereof. The term “and / or” should be interpreted as meaning “both” and that each is optional.

[0030] It will also be understood that although the terms first, second, etc., may be used herein to describe various elements or features, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the embodiments, a first machine learning model may be referred to as a second machine learning model, and similarly, a second machine learning model may be referred to as a first machine learning model. Both the first machine learning model and the second machine learning model are machine learning models, but they are not the same machine learning models.

[0031] As used herein, the phrase “one or more of a set of elements” (such as “one or more of A, B, and C” or “at least one of A, B, and C”) should be interpreted as conjunction or disjunction logic. In other words, it can refer to all elements in a set of elements, one element, or a combination of two or more elements. For example, the phrase “one or more of A, B, and C” can be interpreted as A or B or C, A and B and C, A and B, B and C, or A and C.

[0032] Throughout this disclosure, reference is made to machine learning models (or simply "models"). Thus, it refers herein to any form of machine learning algorithm, such as a deep learning model or a neural network, capable of learning and adapting from input data and subsequently making predictions, decisions, or classifications based on new data.

[0033] Deploying a machine learning model typically involves learning from labeled or unlabeled training data to achieve accurate predictions during the subsequent inference phase—a training-phase model. Training data (and input data during inference) can be, for example, images or image sequences, LiDAR data (i.e., point clouds), radar data, or any other form of data. Furthermore, training / input data can include combinations or fusions of one or more different data types. Additionally or in combination, it can include combinations or fusions of two or more instances of the same data type, such as two or more images from different cameras.

[0034] In some embodiments, the machine learning model may be implemented using publicly available suitable software development machine learning code elements (e.g., code elements available in PyTorch, TensorFlow, and Keras, or any other suitable software development platform) in any suitable manner known to those skilled in the art.

[0035] Overview

[0036] As previously described, the disclosed technology relates to a two-stage traffic sign recognition system. In this two-stage approach, the recognition task is divided between two separate models. First, an object detection model (or more specifically, a traffic sign detection model) is used to detect traffic signs in an image. During inference, the object detection model identifies the location of traffic signs in the image without specifying their type. Then, identified patches can be cropped from the image and fed into the second stage. In the second stage, a second model is used to classify the traffic signs. More specifically, the cropped traffic signs are fed into a traffic sign classifier (or traffic sign classification model), which outputs classification data for each cropped sign. These two models can be trained separately. This means that the two parts can be used independently, for example, using the same object detection model while having different traffic sign classification models for different geographic regions (e.g., different countries). This simplifies the development process. Furthermore, if the separate traffic sign classification model is accessible, the corresponding data mining and model iteration, as well as data refinement, are easier. Furthermore, many optimizations and improvements in traffic sign classification are independent of improvements in object detection, so a separate pipeline allows for faster and more interpretable iterations of improvements for each of them. These are advantageous compared to single-stage methods that train a single neural network to identify the location of traffic signs in an image and classify what they are. This means that the detection and classification parts must be trained end-to-end. Therefore, if different traffic sign recognition systems are desired for different geographical regions (e.g., different countries with different types of traffic signs), the entire network needs to be retrained.

[0037] Typically, CNNs are used for object classification tasks in images. However, for TSR systems, this leads to the technical complexity of a two-stage approach. That is, the CNN must always operate at the same resolution. However, each traffic sign depicted in an image often has a different size, depending on factors such as the distance between the camera and the traffic sign, and the actual physical size of the traffic sign. This means that after the object detection step, but before being fed into the traffic sign classification model, some traffic signs must be downsampled and some must be upsampled. More specifically, large traffic signs or those close to vehicles will require significant downsampling, resulting in information loss and subsequent classification performance degradation. Conversely, small or distant traffic signs need to be upsampled, which can introduce problems such as aspect ratio distortion and require increased computation. To address this, a two-state scheme capable of operating at varying resolutions is proposed. This is built on an attention-based network, which makes it possible to operate at the native resolution of each traffic sign. Attention can be applied directly to the image pixels, i.e., without any further processing in between. By avoiding convolution (or similar) operations, it is possible to implement a model that is applicable to any resolution.

[0038] Example

[0039] Figure 1 is a schematic flowchart representation of a computer-implemented method 100 for classifying traffic signs in an image. In other words, it can be a method 100 for traffic sign recognition (TSR). As previously mentioned, this involves both traffic sign detection (or recognition) and traffic sign classification. Method 100 can be performed in a vehicle, such as vehicle 300 described below in conjunction with Figure 3 (i.e., by the vehicle's computing resources).

[0040] The different steps of method 100 are described in more detail below. Even though illustrated in a specific order, the steps of method 100 can be performed in any suitable order and multiple times. Therefore, although Figure 1 may show a specific order of method steps, the order of steps may differ from the depicted order. Furthermore, two or more steps can be performed concurrently or partially concurrently. For example, steps represented as S110, S112, and S114 can be performed independently of each other. This variation will depend on the chosen software and hardware system and the designer's choice. All these variations are within the scope of the invention. Similarly, standard programming techniques with rule-based logic and other logic can be used to accomplish the various steps. Further variations of method 100 will become apparent from this disclosure. The embodiments mentioned and described herein are given by way of example only and should not be limiting of the invention. Other solutions, uses, objectives, and functions within the scope of the invention claimed in the described patent claims will be apparent to those skilled in the art. It should be further understood that method 100 of Figure 1 includes some steps illustrated by solid lines (blocks) and some steps illustrated by dashed lines. The steps shown in solid lines are those included in the most extensive example embodiment of method 100. Steps included in dashed lines are examples of multiple optional steps that may form part of multiple alternative embodiments. It should be understood that the optional steps do not need to be performed in sequence. Furthermore, it should be understood that not all steps need to be performed. The example steps can be performed in any order and in any combination.

[0041] Method 100 includes acquiring, in S102, an image depicting at least a portion of the vehicle's surrounding environment. The image is captured by the vehicle's camera.

[0042] The vehicle's surroundings can be understood as the general area around the vehicle where objects (e.g., traffic signs or other vehicles, landmarks, obstacles, etc.) can be detected and identified by the vehicle's sensors (radar, lidar, cameras, etc.) (i.e., within the vehicle's sensor range). Therefore, an image can depict the world around the vehicle, including any potential traffic signs near the vehicle.

[0043] The term "obtain" is to be interpreted broadly herein and includes receiving, retrieving, collecting, acquiring, etc., directly and / or indirectly between two entities configured to communicate with each other or further with other external entities. However, in some embodiments, the term "obtain" will be interpreted as determining, deriving, forming, calculating, etc. In this particular case, the image can be obtained in S102 through the process of capturing an image using the camera. Alternatively, the image can be obtained in S102 by retrieving it from intermediate storage space or the like, where the image has already been stored after it was captured.

[0044] Method 100 further includes: identifying regions (or patches) in an image corresponding to traffic signs in image S104 by processing the image via a first machine learning model configured to output the detection of traffic signs in the input image. In other words, the image can be fed into the first machine learning model, which is an object detection model (or more specifically, a traffic sign detection model). The first machine learning model can then output the regions corresponding to the detected traffic signs. The regions can be represented by the regions corresponding to the traffic signs within a reference frame of the image. The regions can be represented, for example, by the bounding boxes around the traffic signs. As those skilled in the art will recognize, any suitable traffic sign detection model can be used.

[0045] It should be noted that method 100 is not limited to recognizing only one traffic sign in an image. In some cases, more than one traffic sign may be depicted in the same image. Each recognized traffic sign can then be detected and processed individually through the steps of method 100 as described below.

[0046] Method 100 further includes: extracting a cropping segment from the image corresponding to the identified region (S106). In other words, image pixels belonging to the identified region can be extracted for further processing. Therefore, cropping can be understood as a sub-part of the original image.

[0047] Therefore, cropping has a native resolution based on the size of the identified region relative to the size of the acquired image. In other words, the cropping resolution depends on the size of the traffic sign, as depicted in the image. Thus, the cropping resolution can be considered as the number of pixels, i.e., the number of pixels in height × the number of pixels in width. More specifically, each identified region can have a different resolution (or size) depending on how much of the image the traffic sign occupies. For example, a large traffic sign or a traffic sign located at a relatively short distance from the camera (i.e., at the time of capture) will appear larger than a smaller traffic sign or a traffic sign located at a relatively large distance from the camera. Therefore, the native resolution should be understood as the resolution obtained during cropping when extracting from the image, i.e., without any processing such as upsampling or downsampling.

[0048] Method 100 further includes determining the classification data of traffic signs S108 by processing the cropping at native resolution via a second machine learning model. In other words, the cropping can be processed by a traffic sign classification model to determine the associated classification data. The classification data can indicate the type of traffic sign, such as a stop sign, a 50 km / h speed limit sign, a yield sign, etc. The classification data can further indicate the confidence level of the predicted type of the traffic sign.

[0049] The second machine learning model is an attention-based neural network. In other words, the second machine learning model employs attention techniques on the input. An attention-based neural network, as used herein, refers to any neural network that employs attention operations such as cross-attention or self-attention. The second machine learning model may, for example, have a Transformer-based architecture. More specifically, the second machine learning model may include at least one cross-attention module and at least one self-attention module. In some embodiments, the second machine learning model may include multiple interleaved cross-attention modules (or blocks) and self-attention modules (or blocks). Further details regarding the second machine learning model are described below in conjunction with Figures 4B and 4C.

[0050] Furthermore, the second machine learning model is trained to process input images of traffic signs at varying resolutions and generate corresponding classification data. The second machine learning model can be trained using a training dataset comprising multiple images of various resolutions. Each image can have associated annotation data. The annotation data can, for example, be the real-world labels of different traffic signs included in the training dataset, for supervised learning of the second machine learning model. Therefore, the second machine learning model can be trained on images at at least two different resolutions. This can provide a model with better generality across different resolutions.

[0051] Furthermore, the second machine learning model applies attention at the pixel level of the input image. Applying attention at the pixel level in this paper means that attention is applied directly to the cropped pixel values ​​of the input. In other words, the second machine learning model can directly cross-attention the cropped pixels to embed them into a latent array for further processing. In some embodiments, the cropping is flattened into an input data array and then fed into the second machine learning model. Cross-attention is then applied to the input data array. This will be described in more detail below.

[0052] Processing the cropping using a second machine learning model can include the following sub-steps. First, (i) the cropping is flattened into a digital input data array. For example, for an image cropping of size (H, W, 3), it can be flattened into a data array of size (H*W, 3), where 3 represents the three color channels in the case of an RGB image. Second, (ii) a latent array with a set of initial values ​​is obtained. The latent array can be viewed as a set of a predefined number of high-dimensional vectors that can be used to make predictions after the latent array has learned useful information about the input. The initial values ​​of the latent array can be randomly assigned. Third, (iii) an updated latent array is generated by updating the latent array through multiple iterations of alternating processing of the input data array and the latent array via a cross-attention module and a self-attention module. In other words, the latent array is processed together with the input data array by alternately applying cross-attention and self-attention. The third step can be viewed as a process that allows the latent array to iteratively extract and learn useful information from the input. More specifically, this can be accomplished by applying cross-attention (e.g., a standard operation of Transformer networks) to the latent array and the digital input data array via a cross-attention module. This can be viewed as the latent array looking at the data array and extracting any information that might be useful for solving the final task. Then, the self-attention module applies self-attention (another standard Transformer operation) to the latent array. This can be viewed as the vectors of the latent array sharing what they have learned from the input data array. The process of applying cross-attention and self-attention can be repeated iteratively multiple times. Thus, the latent array is allowed to iteratively extract information from the input data array, e.g., until the latent array contains all relevant information from the input. The number of iterations can be a fixed number. Alternatively, the number of iterations can be set based on the convergence criterion being met. The updated latent array, obtained after multiple iterations, can then be used in the next step. Fourth, (iv) Based on the updated latent array, predict the classification data for traffic signs. Predicting the classification data can be accomplished by processing the updated latent array via a prediction module provided at the end of the second machine learning model.

[0053] The process described above can operate on data arrays of any size, making it possible to handle image cropping of varying sizes (i.e., different resolutions). This is because the cross-attention operation applies a transformation to each data array entry individually, rather than applying a convolutional filter with specific padding that depends on the size of the input image. In other words, after the cropping is transformed into a data array, cross-attention is applied directly to the cropped image pixels.

[0054] In some embodiments, method 100 further includes: determining vehicle control data S110 based on determined classification data. For example, as part of a decision and control module, the vehicle control data may be determined by an automated driving system (ADS). The vehicle control data may be, for example, a stop signal, a forward signal, a signal for adapting to the vehicle's speed, etc.

[0055] Method 100 may further include transmitting vehicle control data S112 to the vehicle's control system. Or more specifically, the vehicle control data may be transmitted to the vehicle's steering system. Thus, the vehicle control data can be transmitted for the execution of driving maneuvers of the vehicle.

[0056] In some embodiments, method 100 further includes displaying the classification data (S114) on a display device by presenting the classification data as a graphical representation on the vehicle's display device. The classification data can be displayed, for example, to assist the driver in operating the vehicle. As an example, the current speed limit can be displayed to the driver in case they forget the current speed limit or miss that the speed limit has changed.

[0057] Executable instructions for performing these functions may optionally be included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0058] Generally, computer-accessible media can include any tangible or non-transitory storage medium or memory medium, such as electronic, magnetic, or optical media, for example, a disk or CD / DVD-ROM connected to a computer system via a bus. As used herein, the terms “tangible” and “non-transitory” are intended to describe computer-readable storage media (or “memory”) that do not include propagating electromagnetic signals, but are not intended to otherwise limit the types of physical computer-readable storage devices included in the term computer-readable media or memory. For example, the terms “non-transitory computer-readable medium” or “tangible memory” are intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in a non-transitory form on a tangible computer-accessible storage medium can be further transmitted via a transmission medium or a communication medium, such as an electrical, electromagnetic, or digital signal, that can be transmitted via a communication medium such as a network and / or a wireless link.

[0059] Figure 2 is a schematic diagram of a computing device 200 according to some embodiments of the disclosed technology. The computing device 200 can be configured to perform the method 100 described in conjunction with Figure 1. Therefore, the computing device 200 can be a computing device 200 for classifying traffic signs in an image.

[0060] As described herein, computing device 200 refers to a computer system or any device or general-purpose computing system configured to perform various functions. Although computing device 200 is illustrated herein as a single device, computing device 200 can be a distributed computing system composed of multiple different devices.

[0061] The computing device 200 includes a control circuit 202. The control circuit 202 may physically comprise a single circuit device. Alternatively, the control circuit 202 may be distributed across several circuit devices.

[0062] As shown in the example of Figure 2, the computing device 200 may further include a transceiver 206 and a memory 208. Control circuitry 202 is communicatively connected to the transceiver 206 and the memory 208. Control circuitry 202 may include a data bus, and control circuitry 202 may communicate with transceiver 206 and / or memory 208 via the data bus.

[0063] Control circuitry 202 can be configured to perform overall control of the functions and operations of computing device 200. Control circuitry 202 may include processor 204, such as a central processing unit (CPU), microcontroller, or microprocessor. Processor 204 can be configured to execute program code stored in memory 208 to perform the functions and operations of computing device 200. Control circuitry 202 is configured to perform the steps of method 100 as described above in conjunction with FIG. 1. These steps may be implemented using one or more functions stored in memory 208.

[0064] Transceiver 206 is configured to enable computing device 200 to communicate with other entities such as other devices. Transceiver 206 can send data to computing device 200 and receive data from computing device 200. Computing device 200 may, for example, be part of a vehicle. Transceiver 206 can then allow computing device 200 to communicate with other systems in the vehicle, or with external entities such as other vehicles or remote servers.

[0065] Memory 208 may be a non-transitory computer-readable storage medium. Memory 208 may be one or more of a buffer, flash memory, hard disk drive, removable media, volatile memory, non-volatile memory, random access memory (RAM), or other suitable devices. In a typical arrangement, memory 208 may include non-volatile memory for long-term data storage and volatile memory serving as system memory for computing device 200. Memory 208 may exchange data with circuit 202 via a data bus. Additional control lines and address buses may also exist between memory 208 and circuit 202.

[0066] The functions and operations of computing device 200 can be implemented in the form of executable logic routines (e.g., lines of code, software programs, etc.) stored on a non-transitory computer-readable recording medium (e.g., memory 208) of computing device 200, and executed by circuit 202 (e.g., using processor 204). In other words, when circuit 202 is stated to be configured to perform a specific function, processor 204 of circuit 202 can be configured to execute a portion of program code stored on memory 208, wherein the stored portion of program code corresponds to a specific function. Furthermore, the functions and operations of circuit 202 can be a standalone software application, or part of a software application that performs additional tasks associated with circuit 202. The described functions and operations can be considered as methods such as method 100 discussed above in conjunction with FIG. 1, in which the corresponding apparatus is configured to perform. In addition, although the described functions and operations can be implemented in software, such functions can also be performed via dedicated hardware or firmware, or some combination of one or more hardware, firmware, and software. Hereinafter, the functions and operations of computing device 200 are described.

[0067] Control circuit 202 is configured to acquire an image captured by the vehicle's camera, the image depicting at least a portion of the vehicle's surrounding environment. This can be performed, for example, by executing acquisition function 210.

[0068] The control circuit 202 is further configured to identify regions in an image corresponding to traffic signs by processing the image via a first machine learning model configured to output the detection of traffic signs in the input image. This can be performed, for example, by executing the recognition function 212.

[0069] The control circuit 202 is further configured to extract a crop from the image corresponding to the identified region. The crop has a native resolution based on the size of the identified region relative to the size of the acquired image. This can be performed, for example, by executing the extraction function 214.

[0070] The control circuit 202 is further configured to determine the classification data of traffic signs by processing the cropped image at its native resolution via a second machine learning model. The second machine learning model is an attention-based neural network trained to process input images of traffic signs at different resolutions and generate corresponding classification data, wherein the second machine learning model applies attention at the pixel level of the input image. This can be performed, for example, by executing a first determination function 216.

[0071] The control circuit 202 can be further configured to determine vehicle control data based on determined classification data. This can be performed, for example, by executing a second determination function 218.

[0072] The control circuit 202 can be further configured to transmit vehicle control data to the vehicle's control system. This can be done, for example, by performing the transmission function 220.

[0073] The control circuit 202 can be further configured to display the classification data on the display device by presenting the classification data as a graphical representation on the vehicle's display device.

[0074] This can be done, for example, by executing display function 222.

[0075] It should be noted that the principles, features, aspects, and advantages of method 100 as described above in conjunction with Figure 1 also apply to computing device 200 as described herein. To avoid unnecessary repetition, refer to the above. Therefore, the control circuitry can be configured to perform any of the steps described as part of method 100.

[0076] Figure 3 is a schematic diagram of a vehicle 300 according to some embodiments. The vehicle 300 may be equipped with an automated driving system (ADS) 310. As used herein, "vehicle" means any form of motorized transportation. For example, the vehicle 300 may be any road vehicle such as a car (as illustrated herein), a motorcycle, a (freight) truck, a bus, a smart bicycle, etc. The vehicle 300 may be equipped with a computing device 200 as described above. This enables the vehicle 300 to perform the disclosed techniques.

[0077] In the current context, an Automated Driving System (ADS) refers to a complex combination of hardware and software components designed to control and operate a vehicle without direct human intervention. ADS technology aims to automate various aspects of driving, such as steering, acceleration, deceleration, and monitoring of the surrounding environment. The primary goal of ADS is to improve the safety, efficiency, and convenience of transportation. Classified by standards like SAE J3016, depending on their level of automation, ADS can range from basic driver assistance systems to highly advanced automated driving systems. These systems utilize various sensors, cameras, radar, lidar, and powerful computer algorithms to perceive the environment and make driving decisions. The specific capabilities and features / functions of ADS can vary greatly, from systems providing limited assistance to those capable of independently handling complex driving tasks under specific conditions.

[0078] While they don't necessarily offer complete autonomy, Advanced Driver Assistance Systems (ADAS) are technologies that assist the driver during driving. ADAS functions typically act as building blocks for ADS. Examples include adaptive cruise control, lane keeping assist, automatic emergency braking, and parking assist. They improve safety and convenience but usually require some degree of human supervision and intervention. Autonomous Driving (AD), on the other hand, is a technology designed to control and navigate a vehicle without human supervision. Accordingly, it can be said that the difference between ADAS and AD lies in the level of autonomy and control. ADAS systems are designed to assist and support the driver, while ADS aims for complete control of the vehicle without the need for continuous human supervision. Accordingly, AD aims for a higher level of autonomy (e.g., Levels 4 and 5 according to SAE International Standards) where the vehicle can operate independently in most or all driving scenarios without human intervention. As mentioned earlier, the term "ADS" is used herein as a general term encompassing both ADAS and AD. In the current context, ADS functions or ADS features can be understood as specific functions or features of the entire ADS stack, such as highway navigation features, traffic jam navigation features, route planning features, and so on.

[0079] Vehicle 300 includes several components common in autonomous or semi-autonomous vehicles. It will be understood that vehicle 300 is capable of having any combination of the various components shown in FIG. 3. Furthermore, vehicle 300 may include additional components besides those shown in FIG. 3. Although the various components are shown herein as being located inside vehicle 300, one or more components may be located outside vehicle 300. Moreover, as will be readily understood by those skilled in the art, even though the various components are described herein in a particular arrangement, the various components may be implemented in different arrangements. It should be further noted that the various components may be communicatively connected to each other in any suitable manner. Because the components of vehicle 300 can be implemented in several different ways, vehicle 300 of FIG. 3 should be considered merely as an illustrative example.

[0080] Vehicle 300 includes a control system 302. The control system 302 is configured to implement overall control of the functions and operations of vehicle 300. The control system 302 includes control circuitry 304 and memory 306. Control circuitry 302 may physically comprise a single circuit device. Alternatively, control circuitry 302 may be distributed across several circuit devices. As an example, control system 302 may share its control circuitry 304 with other parts of the vehicle. Control circuitry 302 may include one or more processors, such as a central processing unit (CPU), microcontroller, or microprocessor. One or more processors may be configured to execute program code stored in memory 306 to perform the functions and operations of vehicle 300. The processor may be or include any number of hardware components for performing data or signal processing or for executing computer code stored in memory 306. In some embodiments, control circuitry 304 or some of its functions may be implemented on one or more so-called system-on-a-chip (SoC). As an example, ADS 310 may be implemented on an SoC. Optionally, memory 306 includes high-speed random access memory such as DRAM, SRAM, DDRRAM, or other random access solid-state storage devices; and optionally, it includes non-volatile memory such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 306 may include database components, object code components, script components, or any other type of information structure for supporting various operations of this specification.

[0081] In the illustrated example, memory 306 further stores map data 308. Map data 308 can be used, for example, by the ADS 310 of vehicle 300 to perform autonomous functions of vehicle 300. Map data 308 may include high-definition (HD) map data and / or standard-definition (SD) map data. It is contemplated that even though memory 308 is illustrated as a separate element from ADS 310, it can also be provided as an integrated element of ADS 310. In other words, according to some embodiments, any distributed or local memory device can be utilized in implementations of the inventive concept. Similarly, control circuitry 304 can be distributed, for example, such that one or more processors of control circuitry 304 are provided as integrated elements of ADS 310 or any other system of vehicle 300. In other words, according to exemplary embodiments, any distributed or local control circuitry device can be utilized in implementations of the disclosed technology.

[0082] Vehicle 300 further includes a sensor system 320. Sensor system 320 is configured to acquire sensing data about the vehicle itself or its surrounding environment. Sensor system 320 may, for example, include a Global Navigation Satellite System (GNSS) module 322 (e.g., GPS) configured to collect geographic location data of vehicle 300. Sensor system 320 may further include one or more sensors 324. The one or more sensors 324 may be any type of onboard sensor such as a camera, lidar and radar, ultrasonic sensors, gyroscope, accelerometer, odometer, etc. It should be understood that sensor system 320 may also provide the possibility of acquiring sensing data directly or via dedicated sensor control circuitry in vehicle 300. In the context of the disclosed art, vehicle 300 includes at least one camera for capturing images in which traffic signs can be detected and classified.

[0083] Vehicle 300 further includes a communication system 326. Communication system 326 is configured to communicate with external units such as other vehicles (i.e., via vehicle-to-vehicle (V2V) communication protocols), remote servers (e.g., cloud servers), databases, or other external devices (i.e., vehicle-to-infrastructure (V2I) or vehicle-to-everything (V2X) communication protocols). Communication system 326 can communicate using one or more communication technologies. Communication system 326 may include one or more antennas. Cellular communication technologies can be used for remote communication, such as to remote servers or cloud computing systems. Additionally, if the cellular communication technology used has low latency, it can also be used for V2V communication, V2I communication, or V2X communication. Examples of cellular radio technologies are GSM, GPRS, EDGE, LTE, 5G, 5G NR, etc., and future cellular solutions are also included. However, in some solutions, short-to-medium range communication technologies such as wireless local area networks (LANs) (e.g., solutions based on IEEE 802.11) can be used to communicate with other vehicles near vehicle 300 or with local infrastructure components. ETSI is developing cellular standards for vehicle communications, and 5G is considered a suitable solution, for example, due to its low latency and efficient handling of high bandwidth and communication channels.

[0084] The communication system 326 can further provide the possibility of transmitting output to a remote location (e.g., a remote server, operator, or control center) via one or more antennas. Furthermore, the communication system 326 can be further configured to allow various components of the vehicle 300 to communicate with each other. As an example, the communication system can provide a local network setup such as CAN bus, I2C, Ethernet, fiber optics, etc. Local communication within the vehicle can also be a wireless type with protocols such as WiFi, LoRa, Zigbee, Bluetooth, or similar medium / short-range technologies.

[0085] Vehicle 300 further includes a control system 320. The control system 328 is configured to control the handling of vehicle 300. The control system 328 includes a steering module 330 configured to control the direction of vehicle 300. The control system 328 further includes a throttle module 332 configured to control actuation of the throttle of vehicle 300. The control system 328 further includes a braking module 334 configured to control actuation of the brakes of vehicle 300. The various modules of the steering system 328 can receive manual input from the driver of vehicle 300 (i.e., from the steering wheel, accelerator pedal, and brake pedal, respectively). However, the control system 328 can communicatively connect to the vehicle's ADS 310 to receive instructions on how the various modules should operate. Therefore, ADS 310 can control the handling of vehicle 300.

[0086] As described above, vehicle 300 includes ADS 310. ADS 310 may be part of the vehicle's control system 302. ADS 310 is configured to implement the autonomous functions and operations of vehicle 300. ADS 310 may include multiple modules, each responsible for a different function of ADS 310.

[0087] ADS 310 may include a positioning module 312 or a positioning block / system. The positioning module 312 is configured to determine and / or monitor the geographic location and orientation of the vehicle 300, and may utilize data from the sensor system 322, such as data from the GNSS module 322. Alternatively or in combination, the positioning module 312 may utilize data from one or more sensors 324. Alternatively, the positioning system may be implemented as a real-time motion (RTK) GPS. The device 200 described above may be provided, for example, as part of the positioning module 312. Therefore, the vehicle 300 is configured to perform the steps of method 100 described above.

[0088] ADS 310 may further include a perception module 314 or a perception block / system. Perception module 314 may refer to any known module and / or function included in one or more electronic control modules and / or nodes of vehicle 300, adapted and / or configured to interpret driving-related sensor data of vehicle 300 to identify, for example, obstacles, lanes, relevant signs, appropriate navigation paths, etc. Therefore, perception module 314 may be adapted to rely on and obtain inputs from multiple data sources, such as automotive imaging, image processing, computer vision, and / or in-vehicle networking, in conjunction with sensor data from sensor system 320, for example. As mentioned above, a production model may be provided as part of ADS 310, or more specifically, as part of perception module 314. Therefore, perception module 314 may include a TSR system for performing the techniques described above in conjunction with Figures 1 and 2.

[0089] The positioning module 312 and / or the sensing module 314 can be communicatively connected to the sensor system 320 to receive sensor data from the sensor system 320. The positioning module 312 and / or the sensing module 314 can further send control commands to the sensor system 320.

[0090] The ADS may further include a path planning module 316. The path planning module 316 is configured to determine a planned path for the vehicle 300 based on the vehicle's perception and position, as determined by the perception module 314 and the positioning module 312, respectively. The planned path determined by the path planning module 316 can be sent to the control system 328 for execution. As an example, the determined current position of the vehicle on the navigation map can be transmitted to the path planning module 316.

[0091] The ADS may further include a decision and control module 318. The decision and control module 318 is configured to perform control of the ADS 310 and make decisions for the ADS 310. For example, the decision and control module 318 may decide whether the planned path determined by the path planning module 316 should be executed. The decision and control module 318 may be further configured to detect any deviation behavior of the vehicle, such as deviating from the planned path or expected trajectory of the path planning module 316. This includes both maneuver avoidance performed by the ADS 310 and the vehicle driver.

[0092] It should be understood that portions of the described solution can be implemented in vehicle 300, in a system located outside the vehicle, or in a combination of inside and outside the vehicle; for example, in a server communicating with the vehicle, i.e., a so-called cloud solution. Different features and principles of the embodiments can be combined in combinations other than those described. Furthermore, the elements of vehicle 300 (i.e., systems and modules) can be implemented in combinations different from those described herein.

[0093] Figure 4A illustrates, by way of example, a two-stage traffic sign recognition pipeline 400 according to some embodiments. Figures 4B and 4C show more detailed views of a second machine learning model 408. The two-stage traffic sign recognition pipeline 400 can be viewed as an illustration of the steps of method 100 as described above in conjunction with Figure 1. For further details, refer to the above.

[0094] First, an image 402 depicting the vehicle's surroundings is acquired. In this example, the image depicts a road intersection and a first traffic sign 406a and a second traffic sign 406b. However, needless to say, image 402 can depict any number of traffic signs. It should be further noted that in this example, the first traffic sign 406a is positioned further away from the camera than the second traffic sign 406b, and therefore appears smaller in image 402.

[0095] Image 402 is then fed into a first machine learning model 404 to detect any traffic signs in image 402. Crops of the two traffic signs 406a and 406b can be obtained from the first machine learning model 404. As further shown in this example, the crop of the first traffic sign 406a is smaller than the crop of the second traffic sign 406b (i.e., has a lower native resolution) due to the different sizes in the original image 402.

[0096] Each clipping can then be fed into a second machine learning model 408. The second machine learning model 408 is configured to determine first classification data 410a and second classification data 410b for the corresponding traffic signs depicted in the clipping.

[0097] Turning now to Figure 4B, a more detailed view of the second machine learning model 408 is shown. The second machine learning model 408 may include at least one cross-attention module 408d and at least one self-attention module 408e. The second machine learning model 408 may further include a flattening module 408a and a prediction module 408f.

[0098] First, a cropped image of a traffic sign (in this document, the second traffic sign 406b is used as an example) can be fed to a flattening module 408a. The flattening module 408a is configured to flatten the cropped image into an input data array 408c. The input data array may include the numerical values ​​of the cropped pixels. Although not illustrated, the input data array may include, for example, multiple vectors of one vector for each color channel in an RGB image.

[0099] The flattening module 408b may further include positional encoding for each pixel in the input data array 408c. In other words, the input data array 408c may include positional information for each pixel in the input image (i.e., the cropped traffic sign). Positional information can be understood as any information indicating where each pixel is located in the image. Positional encoding can be implemented, for example, by converting each pixel into a 5D object with three dimensions corresponding to the pixel's RGB values ​​and the last two dimensions being the height and width of the pixel within the input image. By including positional encoding, the second machine learning model 408 is further provided with information about the aspect ratio of the input image, which can improve the prediction results.

[0100] In the first iteration (indicated by the arrow "i = start"), the latent array 408b with some initial values ​​is fed to the cross-attention module 408d along with the input data array 408c. The cross-attention module 408d then applies cross-attention between the latent array 408b and the input data array 408c, generating a partially updated latent array 408b'. The partially updated latent array 408b' is then fed to the self-attention module 408e. The self-attention module 408e applies self-attention to the partially updated latent array 408b' to generate an updated latent array 408''. Therefore, self-attention can be applied among the multiple vectors that constitute the latent array.

[0101] The process of applying the cross-attention module 408d and the self-attention module 408e is then repeated iteratively, here N iterations (indicated by the arrows "i = 1…N"), where the updated latent array 408b'' is fed back to the cross-attention module 408d along with the input data array. In other words, in each iteration, the input data array 408c is fed to the cross-attention module 408d along with the latest updated latent array 408''. The self-attention module 408e may simply take the latent array (as fed from the cross-attention module 408d) as input for each iteration.

[0102] After the iteration is complete, the resulting updated latent array 408b'' is fed into the prediction module (indicated by the arrow "i=end"). The prediction module 408f is configured to generate predictions of categorical data based on the updated latent array 408b''. The prediction module 408f may, for example, include a neural network trained for this purpose. Thus, in this example, the output of the prediction module 408f is the second classification data 410b of the second traffic sign 406b. As described above, the latent array 408b may include multiple vectors. These multiple vectors may be averaged into a single vector and fed into the prediction module 408f. The prediction module may then include a single linear layer with softmax. The output of this layer may be an output vector with a dimension equal to the number of categories (i.e., traffic sign types). Normalization may then be applied to the output vector to produce the final categorical data.

[0103] Although illustrated in a specific manner, the second machine learning model 408 can, of course, be implemented differently depending on the specific implementation. For example, the flattening module 408a and / or the prediction module 408f can be implemented as separate modules from the second machine learning model 408. Furthermore, the iterative application of the cross-attention and self-attention mechanisms is illustrated herein as a feedback loop using the same cross-attention and self-attention modules in each iteration. However, it is also possible to have multiple cross-attention and self-attention modules provided in an alternating series. In other words, the latent array and the input data array can be fed through a single chain of alternating cross-attention and self-attention modules. An example of such a model is shown in Figure 4C.

[0104] As shown in Figure 4C, the latent array 408b is fed by multiple cross-attention modules 408d, 408d' and multiple self-attention modules 408e, 408e'. It should be noted that any number of cross-attention modules and self-attention modules can be used (as shown by the three dots). It should also be noted that weights can be shared between repetitions; that is, different cross-attention modules can share weights among themselves, and different self-attention modules can share weights among themselves.

[0105] Then, the values ​​of the latent array are updated throughout this process (as shown by 408', 408'', 408''', and 408''''). Finally, the latent array 408'''' is fed into the prediction module 408f.

[0106] As further shown, the input data array 408c can be fed into each cross-attention module 408d, 408d'. The input data array 408c can be fed into the cross-attention module as a key ("K") and a value ("V"), while the latent array 408b is fed in as a query ("Q").

[0107] The disclosed technology has been presented above with reference to specific embodiments. However, other embodiments besides those described above are also possible and within the scope of the invention. Within the scope of the invention, method steps different from those described above, performed by hardware or software, can be provided. Therefore, according to an exemplary embodiment, a non-transitory computer-readable storage medium is provided storing one or more programs configured to be executed by one or more processors of a vehicle control system, the programs including instructions for performing the methods according to any of the above embodiments. Alternatively, according to another exemplary embodiment, a cloud computing system can be configured to perform any of the methods presented herein. The cloud computing system may include distributed cloud computing resources that collectively perform the methods presented herein under the control of one or more computer program products.

[0108] It should be noted that no reference numerals in the accompanying drawings limit the scope of the claims. The invention can be implemented, at least in part, by both hardware and software, and the same hardware item can refer to several “devices” or “units”.

Claims

1. A computer-implemented method (100) for classifying traffic signs in an image, the method (100) comprising: (S102) Obtain (S102) the image captured by the vehicle's camera, the image depicting at least a portion of the vehicle's surrounding environment; (S104) identify (S104) regions in the image corresponding to traffic signs by processing the image via a first machine learning model configured to output the detection of traffic signs in the input image; (S106) extract (S106) a cropping corresponding to the identified regions from the image, wherein the cropping has a native resolution based on the size of the identified regions relative to the size of the obtained image; and (S108) determine (S108) classification data of the traffic signs by processing the cropping at the native resolution via a second machine learning model, wherein the second machine learning model is an attention-based neural network trained to process input images of traffic signs at different resolutions and generate corresponding classification data, wherein the second machine learning model applies attention at the pixel level of the input image.

2. The method (100) according to claim 1, wherein, The second machine learning model has been trained using a training dataset comprising multiple images of various resolutions, wherein each of the multiple images has associated annotation data.

3. The method (100) according to claim 1, wherein, The second machine learning model has a Transformer-based architecture.

4. The method (100) according to claim 1, wherein, The second machine learning model includes at least one cross-attention module and at least one self-attention module.

5. The method (100) according to claim 1, wherein, Processing the cropping via the second machine learning model includes: flattening the cropping into a digital input data array; obtaining a latent array with a set of initial values; updating the latent array by iterating multiple times through alternating processing of the input data array and the latent array via the cross-attention module and the self-attention module, thereby generating an updated latent array; and predicting the classification data of the traffic sign based on the updated latent array.

6. The method (100) according to any one of claims 1 to 5, wherein, The first machine learning model is a traffic sign detection model, and the second machine learning model is a traffic sign classification model.

7. The method (100) according to claim 1, wherein, The classification data indicates the type of the traffic sign.

8. The method (100) according to claim 1, further comprising: Based on the determined classification data, vehicle control data is determined (S110).

9. The method (100) according to claim 8, further comprising: The vehicle control data is transmitted (S112) to the vehicle's control system.

10. The method (100) according to claim 1, further comprising: The classification data is displayed on the display device by presenting the classification data as a graphical representation on the display device of the vehicle (S114).

11. A computer-readable storage medium storing instructions that, when executed by a computing device, cause the computing device to perform the method (100) according to claim 1.

12. A computing device (200) for classifying traffic signs in an image, the computing device (200) including a control circuit (202) configured to: acquire an image captured by a camera of a vehicle, the image depicting at least a portion of the vehicle's surrounding environment; identify regions in the image corresponding to traffic signs by processing the image via a first machine learning model, the first machine learning model being configured to output the detection of traffic signs in the input image; and extract a crop from the image corresponding to the identified regions, wherein... The cropping has a native resolution based on the identified region relative to the size of the obtained image; Furthermore, the classification data of the traffic sign is determined by processing the cropping at the original resolution via a second machine learning model, wherein the second machine learning model is an attention-based neural network trained to process input images of traffic signs at different resolutions and generate corresponding classification data, wherein the second machine learning model applies attention at the pixel level of the input image.

13. The computing device (200) according to claim 12, wherein, The control circuit (202) is further configured to determine vehicle control data based on the determined classification data.

14. The computing device (200) according to claim 12, wherein, The control circuit (202) is further configured to display the classification data on the display device by presenting the classification data as a graphical representation on the display device of the vehicle.

15. A vehicle (300) comprising a camera and a computing device (200) according to any one of claims 12 to 14.

Images