Road classification method and apparatus

Abstract

The present disclosure relates to a road classification method and device, the method comprising: capturing at least one image frame of a current road; processing the image frame to identify a plurality of visual features within the image frame; matching the visual features with a predetermined number of predetermined candidate clustering clusters to determine a target clustering cluster to which the image frame belongs; selecting attribute features specified by the determined target clustering cluster from the plurality of visual features; processing the attribute features using a road classification model corresponding to the determined target clustering cluster to classify the current road.

Description

TECHNICAL FIELD

[0001] The present application relates to autonomous driving, and in particular to road recognition for autonomous driving. BACKGROUND

[0002] With the development of network communication technology and artificial intelligence, intelligent networking of vehicles can be realized, and the automation level of the autonomous driving system (ADS) of vehicles is increasingly high. As a result, autonomous driving has gradually become a reality. One important technology applied in autonomous driving is geo-fencing technology. Geo-fencing technology refers to a geographical area in which intelligent networked vehicles are electronically monitored in terms of entry, speed, and fuel usage, and communication with control centers such as intelligent vehicle and intelligent networked transportation systems is achieved. In geo-fencing technology, understanding the current road on which the vehicle is driving is an important factor that must be addressed, and the ADS needs to know whether the current road is a highway, an urban road, or a general rural road.

[0003] Currently, geo-fencing technology is mainly implemented using GPS signals or ETC signals received when passing through toll stations. Obviously, for vehicles driving on a highway, GPS signals become poor or are not available when passing through special environments such as tunnels and deserts, or ETC signals are not received when far away from toll stations, so the vehicle cannot know whether it is still on the highway. Therefore, GPS or ETC-based geo-fencing technology is greatly limited. SUMMARY

[0004] The present application proposes a technology for determining the type of a road based on direct visual features of the road with the aid of machine learning technology.

[0005] According to an aspect of the present application, a road classification method is provided, including: capturing at least one image frame of a current road; processing the image frame to identify a plurality of visual features within the image frame; matching the visual features with a predetermined number of predetermined candidate clustering clusters to determine a target clustering cluster to which the image frame belongs; selecting attribute features specified by the determined target clustering cluster from the plurality of visual features; and processing the attribute features using a road classification model corresponding to the determined target clustering cluster to classify the current road.

[0006] According to another aspect of the present application, there is provided a method for training a road classification model, comprising: obtaining a plurality of image frames, each of which comprises visual features of a road; processing each image frame to identify a plurality of visual features contained in the image frame; clustering the plurality of image frames based on the visual features to determine a cluster to which each image frame belongs, to form a plurality of clusters; selecting a plurality of visual features most relevant to the cluster from the visual features in the image frames under the same cluster as attribute features of the cluster; and using the plurality of attribute features in each image frame under the same cluster to form a training sample for training a road classification model matching the cluster.

[0007] According to another aspect of the present application, there is provided a computing system comprising a road classification module for classifying a road on which a vehicle is travelling, wherein the road classification module is configured to: process at least one image frame of the current road on which the vehicle is travelling to identify a plurality of visual features in the image frame; match the visual features with a predetermined number of candidate clusters determined in advance to determine a target cluster to which the image frame belongs; select attribute features specified by the determined target cluster from the plurality of visual features; and process the attribute features using a road classification model corresponding to the determined target cluster to classify the current road.

[0008] According to another aspect of the present application, there is provided a road classification apparatus comprising: a feature extraction module configured to process at least one image frame of a current road to identify a plurality of visual features in the image frame; a clustering module configured to match the visual features with a predetermined number of candidate clusters determined in advance to determine a target cluster to which the image frame belongs; a feature selection module configured to select attribute features specified by the determined target cluster from the plurality of visual features; and a classification module configured to process the attribute features using a road classification model corresponding to the determined target cluster to classify the current road.

[0009] According to the scheme of the present application, not only can roads that are obviously different, such as highways, urban streets and rural roads, be distinguished, but also roads that are very similar, such as highways and urban expressways, can be distinguished. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1A A flow chart of a road classification model training process according to an embodiment of the present application is shown;

[0011] Figure 1B A flow chart of a road classification model training process according to another embodiment of the present application is shown;

[0012] Figure 2 A schematic diagram of a road classification apparatus according to an embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram of a road classification device according to another embodiment of the application is shown;

[0014] Figure 4 A flowchart of a road classification method according to an example of the application is shown;

[0015] Figure 5 A flowchart of a road classification method according to another example of the application is shown. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only for the purpose of illustration and are not restrictive.

[0017] In road planning, different standards and traffic signs are usually designed for different roads. For example, for urban roads, signs such as street lamps, zebra crossings, no parking, no honking, etc. need to be set; while for expressways, signs such as service area, gas station identification, distance to the next exit or target kilometer, etc. are usually set. Conventionally, these features are learned by machine learning based on a neural network to determine the classification of the current road, i.e. whether it is an expressway, an urban expressway, a village road, or a suburban highway. However, in reality, due to the complexity of the road environment, the non-uniform or imperfect construction standards, some features or symbols appearing in the road environment do not match the road. For example, illegal trucks appear on urban expressways, and some signs specific to expressways appear on some non-highway roads, which will interfere with the identification of the road. In addition, a single neural network model is usually used to learn road features to achieve road classification in the past. However, due to the complexity of the road, this single general model usually filters out important features useful for the current road, so a single model cannot achieve accurate classification of the road. Therefore, the present application proposes a specific fusion scheme that can be specialized for multiple road classifications, which considers all visual features Vis_F of the current road and dynamically filters out important features from them, and uses a trained specialized road classification model to achieve classification of the road.

[0018] [Model training]

[0019] Figure 1AA flowchart of training a road classification model RCM according to an embodiment of the present application is shown. According to one example of the present application, the visual features Vis_F of a road can be processed using big data processing techniques (e.g. machine learning methods) to learn the correlation between the visual features Vis_F and the road classification, which can be represented by a road classification model RCM. It is noted that the road classification model RCM is not limited to a neural network model, but can be any big data machine learning model expressed in other algorithms or mathematical representations as long as the algorithm or mathematical representation reflects the correlation between the road classification and the visual features Vis_F of the road. Furthermore, according to the present application, different RCMs can be trained for different types of roads.

[0020] Firstly at step 101, image frames containing various visual features Vis_F of roads are collected for various types of roads, where each image frame is taken in real scene for one type of road. It is understood that each image frame can include multiple visual images reflecting the features of the road. For example, for a highway, the visual features Vis_F in the taken image frame can include trucks, toll stations, gas stations and other traffic signs specific to highways. For a city street, the visual features Vis_F in the taken image frame can include street lights, traffic lights, sidewalks and speed limit signs. However, it is apparent that there can be the same visual features such as speed limit signs for different roads. It is assumed that N image frames X1, X2, X3, X4,... XN are collected for different roads. N .

[0021] At step 103, each image frame X obtained at step 101 is processed to identify visual features Vis_F contained in each image frame. Due to the uncertainty of visual features such as traffic signs contained in the road, according to an embodiment of the present application, a predetermined number and type of visual feature Vis_F detection processing is performed on each frame image, where the number and type of visual features to be detected can cover typical traffic signs or facilities under existing traffic regulations and road construction standards. For example, for all image frames, W item visual feature detection of street lamps, sidewalks, trucks, traffic sign A, traffic sign B, traffic sign C…toll station is performed. Here, feature detection techniques known in the prior art can be used to identify visual features Vis_F that can be contained in the image. For example, for the first image frame X1, a street lamp detection model trained based on machine deep learning can be used to detect whether the image frame X1 contains a street lamp. As an example, the detection result can be represented by a confidence value, which represents the probability of detecting a street lamp in the road. In addition, the detection result can also contain the relative position of the street lamp in the image frame X1. In this way, the same image frame X1 can be subjected to corresponding visual feature Vis_F detection using a pre-trained sidewalk detection model, truck detection model, traffic sign A detection model, traffic sign B detection model, traffic sign C detection model…toll station detection model, so as to detect each visual feature Vis_F for the image frame X1, thereby generating data for the image frame X1, denoted as X1= (x1 (1) ,x1 (2) ,x1 (3) …x1 (W) ,y1), where x1 (i) represents the i-th visual feature Vis_F of the image frame X1, and y1 represents the road category corresponding to the image frame X1, for example, a highway.

[0022] In this way, by applying the street lamp detection model, sidewalk detection model, truck detection model, traffic sign A detection model, traffic sign B detection model, traffic sign C detection model…toll station detection model to the remaining N-1 image frames X2-X N , respectively, the visual feature data of each image frame is generated:

[0023] X2= (x2 (1) ,x2 (2) ,x2 (3) …x2 (W) ,y2),

[0024] …

[0025] X N = (x N (1) ,xN (2) x N (3) …x N (W) ,y N ).

[0026] After the sample data {X1, X2, … X N} is generated, in step 105, a clustering algorithm is used to perform clustering processing on these data samples to determine the data samples X that can be classified into the same cluster. According to the present application, since the number of road categories is known in advance, the number of clusters formed can be specified here. For example, y1, y2, … y N represent one of the highway, urban expressway, ordinary urban road, suburban road and rural road respectively. Therefore, the number of clusters can be specified as 5 here. The purpose of performing clustering on the data samples here is to divide the samples into different clusters according to the similarity between the samples, thereby determining which samples are more suitable for making the category prediction of the road corresponding to the cluster, and also eliminating those samples that are not suitable for prediction.

[0027] To perform sample clustering, first a sample visual feature matrix is constructed, which contains the visual features Vis_F of each road and the road category label, wherein the visual features Vis_F are represented by their confidence values. Table 1 below shows the matrix constructed using the data samples (as an example, only the confidence data of the street light in the image frames (X1, X2, X3, X4, … X N ) is given in the figure):

[0028] Table 1: Visual feature matrix

[0029]

[0030] From the matrix, it can be seen that each row represents an image frame, wherein the element values in the matrix are the confidence of the visual features detected in the image frame, and the last label column shows the road category corresponding to the image frame. At the same time, each column of the matrix represents the confidence of the appearance of a certain visual feature in each image frame. For example, for the street light element, the appearance probabilities in the 1st-N image frames are: 0.91, 0.09, 0.89, 0.89, …, 0.18 respectively.

[0031] According to the present invention, clustering is performed based on the confidence scores of visual feature elements in the sample matrix, as indicated by the dashed boxes in the figure. It is important to note that labels are not included during clustering, thus achieving unsupervised learning. Any clustering algorithm known in the art can be used, such as the k-means algorithm, the Mean-Shift algorithm, density-based noise spatial clustering (DBSCAN), and the EM (Expectation-Maximization) algorithm. As an example of using the k-means algorithm, since the number of road classifications is predetermined (e.g., 5), the cluster centers CP are ultimately determined based on the Euclidean distance between the data samples and the assumed center points, generating K (=5) clusters. For each determined cluster, the Euclidean distance D from the samples within that cluster to the center point CP is within a preset radius R. In this example, by clustering the confidence scores of the visual feature elements in Table 1, five clusters C1 to C5 are obtained. As shown in Table 2 below, three of these clusters, C1 to C3, are illustrated using three boxes: dotted line, dashed line, and solid line.

[0032] Table 2: Sample Clustering

[0033]

[0034] The cluster C1 shown in the dotted frame contains sample X1, X N Elements such as X2 and X3 are included in cluster C2 (shown by the dashed box), and X4 is included in cluster C3 (shown by the solid box). It's important to note that the labels of some data samples or even a single data sample may be inconsistent. For example, data X originally collected might belong to a national highway. m After clustering, some roads may be classified as rural roads, so they can be corrected or removed after clustering.

[0035] After determining clusters C1 to C5 in step 105, the algorithm information and clustering conditions applied to each cluster are stored. For example, taking K-means as an example, the K-means algorithm used for clustering, the centroids CP_1, CP_2, CP_3, CP_4, and CP_5 of each cluster C1 to C5, and the Euclidean distance thresholds R1, R2, R3, R4, and R5 used to determine the corresponding clusters are stored in storage units for use in subsequent road classification.

[0036] In the above embodiments, the K-means algorithm is used, and the number of clusters to be clustered is specified in advance, but the present application is not limited to this. In another embodiment of the present application, an algorithm that does not specify the number of clustering clusters can also be used, such as the Mean-shift algorithm. Compared with the K-means clustering algorithm, the Mean-shift algorithm does not need to select the number of clusters, because it is automatically found that the current data samples can be divided into several categories. Using this algorithm, samples deviating greatly can be filtered out, and only clusters of interest are selected.

[0037] In step 107, in each clustering cluster, M features most relevant to the cluster label, i.e., the road type, are selected from the visual features (x (1) ,x (2) ,x (3) …x (W) ) in the cluster, M≤W. It can be considered that these M features determine the category of the photographed road, and in this disclosure, these M visual features Vis_F are also referred to as “road attribute features Attr_F”. According to an example of the present application, the Spermann correlation coefficient algorithm can be used to select the top M features most relevant to each label. The Spearman correlation coefficient is commonly used in statistics to determine sample correlation, and is used to calculate the correlation between elements in two sets according to their ranks in the respective sets, so as to analyze the data. As an application example, the present application determines the M features in the following manner:

[0038] Step (1): Calculate the Spermann correlation coefficient ρ using the following formula:

[0039]

[0040] In the formula, x represents each of the W visual features Vis_F in each sample, and y represents the label value of the data sample, which can be, for example, a unique ID number assigned to each clustering cluster. Obviously, in the case where each clustering cluster corresponds to a road category in this example, the above ID number is also considered as a road number, or other feature values of the road can also be used as the label value of the sample. As can be seen from the formula, in this example, the Spermann correlation coefficient is used to determine the correlation between each visual feature x and the label y, which represents the contribution of each visual feature x to the label y.

[0041] Step (2): rank the visual features x according to their correlation value p with the label y, in this example, rank the visual features in descending order according to their contribution p, and select the top M visual features as the attribute features Attr_F which have important influence on the label y, i.e. the road classification. It is noted that the number M of the specified "road attribute features Attr_F" can be different for each cluster C, or can be the same, depending on the actual needs, for example, half of the number of visual features in each cluster can be selected as the attribute features. For the purpose of illustration, it is assumed that for the cluster C1, the first (1), (2), (5),... (j) visual features Vis_F (total M) are specified as the road attribute features Attr_F1, i.e. Attr_F1= (x (1) ,x (2) ,x (5) …x (j) ).

[0042] For other clusters C2~C5, the above method is also performed, so as to obtain the road attribute features Attr_F under each cluster, for example, for the cluster C2, the (1), (3), (8),... (k) visual features Vis_F are specified as the road attribute features, i.e. Attr_F2= (x (1) ,x (3) ,x (8) …x (k) ).

[0043] Subsequently, in step 109, based on the road attribute features Attr_F determined for each cluster, a training sample is formed for training a road classification model RCM adapted to the cluster. For example, for the cluster C1: containing samples (X1, X3, X6, X7, X 12 ), the training sample for the cluster C1 is constructed as follows:

[0044] X1= (Attr_F1, y1) = (x1 (1) ,x1 (2) ,x1 (5) …x1 (j) ,y1)

[0045] X3= (Attr_F3, y3) = (x3 (1) ,x3 (2) ,x3 (5) …x3 (j) ,y3)

[0046] X6= (Attr_F6, y6) = (x6 (1) ,x6 (2) ,x6 (5) …x6 (j) ,y6)

[0047] X7= (Attr_F7, y7) = (x7 (1) ,x7 (2) ,x7 (5) …x7 j) ,y7)

[0048] X 12 =(Attr_F 12 ,y 12 )=(x 12 (1) ,x 12 (2) ,x 12 (5) …x 12 (j) ,y 12 )

[0049] In the above sample, each label y1, y3, y6, y7, y 12 is the same, all refer to the same road category. Then the training sample (X1, X3, X6, X7, X 12 ) is used to train a neural network, and a road classification model RCM1 applicable to the clustering cluster C1 is obtained.

[0050] According to the embodiment of the present application, in step 103, when identifying the visual feature Vis_F in the image frame, not only the confidence U of the corresponding visual feature Vis_F is given, but also the relative position L of the visual feature Vis_F in the image frame X is given. For example, after determining the center line of the road contained in the image frame or the edge of the outermost road, the relative position L here can be the distance of each visual feature Vis_F relative to the center line or the edge. For example, for the identified green belt or building, the relative position L can be the distance from the outermost lane line of the rightmost lane. Therefore, when constructing the sample for training the road classification model RCM1, each attribute feature x (n) here not only contains the confidence, but also contains the relative position information, i.e. x (n) = f (U n ,L n ), wherein U n represents the confidence of detecting the visual feature Vis_Fx (n) , L n represents the relative position of the detected visual feature Vis_Fx (n) , and f represents a combination processing function of U n , L n , which is converted by the function f and provided to the neural network to train the road classification model RCM1. It is not difficult to understand that U nL n directly to the neural network.

[0051] In this way, for other cluster, the attribute feature Attr_F and label within each cluster are utilized to train the corresponding road classification model RCM2~RCM5. For example for cluster C2: suppose it contains samples (X2, X3, X6, X8, X 11 ). Then the training samples for cluster C2 are constructed as follows:

[0052] X2=(Attr_F2,y2)=(x2 (1) ,x2 (3) ,x2 (8) …x2 (k) ,y2)

[0053] X3=(Attr_F3,y3)=(x3 (1) ,x3 (3) ,x3 (8) …x3 (k) ,y3)

[0054] X6=(Attr_F6,y6)=(x6 (1) ,x6 (3) ,x6 (8) …x6 (k) ,y6)

[0055] X8=(Attr_F8,y8)=(x8 (1) ,x8 (3) ,x8 (8) …x8 k) ,y8)

[0056] X 11 =(Attr_F 11 ,y 11 )=(x 11 (1) ,x 11 (3) ,x 11 (8) …x 11 (k) ,y 11 )

[0057] As mentioned above, at this time, each element x in the training sample is a function of the confidence U and relative position L corresponding to the visual feature Vis_F.

[0058] Then the training samples (X2, X3, X6, X8, X 11) to train a neural network individually, and obtain a road classification model RCM2 applicable to the clustering cluster C2. In this way, road classification models RCM2-RCM5 applicable to the clustering clusters C1-C5 are generated. After the training of the road classification models RCM1-RCM5 is completed, they can be stored in the storage unit of the road classification device for road classification.

[0059] In the examples of the present application, the neural network used to learn the attribute features Attr_F of the road can be implemented by any neural network engine known in the art. For example, convolutional neural networks CNN, recurrent neural networks RNN can be used to extract the relevance features in these attribute features Attr_F of the road. As to how to design the structure of the neural network CNN, RNN and the units in each RNN, the person skilled in the art can determine it according to the actual situation. For example, in a more preferred embodiment, the RNN adopts a bidirectional recurrent neural network (BRNN) and each node can be implemented by different units such as long short-term memory neural network (LSTM) or gated recurrent neural network (GRU). In addition, in another embodiment of the present application, the neural network used to learn the attribute features Attr_F of the road can also be implemented by a cascade network of different types of neural networks.

[0060] In the above embodiments of the present application, K-means, Mean-shift, DBSCAN algorithm and other single algorithms are used to cluster the samples when generating the clustering clusters of the samples in step 105. However, in another embodiment of the present application, multiple clustering analyses can also be performed on the same sample library. In addition, when training the corresponding road classification model, multiple classification methods can also be used for training. Figure 1B A training method according to another embodiment of the present application is shown.

[0061] As shown in Figure 1B As in the previous embodiments, after obtaining the visual features Vis_F contained in each image frame in step 103, the image frames containing the visual features Vis_F are clustered in step 105. However, unlike the embodiment shown in Figure 1A In this embodiment, multiple clustering methods are used to cluster the image frames based on the visual features Vis_F obtained in step 103, as an example, Figure 1B K-means, DBSCAN and EM (Expectation Maximization) algorithms are used to perform clustering. Thus, in step 105, each clustering method outputs its own clustering result, which is assumed to still correspond to five road classifications, Figure 1BThe K-means, DBSCAN and EM algorithms produce five clusters respectively, where the clusters produced by the K-means algorithm are denoted as C_K1~C_K5, the clusters produced by the DBSCAN algorithm are denoted as C_DB1~C_DB5, and the clusters produced by the EM algorithm are denoted as C_EM1~C_EM5.

[0062] Similar to the previous embodiment, next in step 107, for each cluster produced by each clustering algorithm, the M attribute features most relevant to the cluster label, i.e. the road type, are selected from the visual features (x (1) ,x (2) ,x (3) …x (W) ) in this example, as shown in Table 2, the attribute features C_K1_Attr_F~C_K5_Attr_F are determined for the clusters C_K1~C_K5, the attribute features C_DB1_Attr_F~C_DB5_Attr_F are determined for the clusters C_DB1~C_DB5, and the attribute features C_EM1_Attr_F~C_EM5_Attr_F are determined for the clusters C_EM1~C_EM5. Figure 1B

[0063] Subsequently in step 109, the road classification models for the respective clusters are trained based on the attribute features determined in step 107 using various classification methods. As shown in the figure, four classification methods are shown as examples, namely the k- nearest neighbor algorithm (k-NN), the AdaBoost cascade classifier (AD), the support vector machine (SVM), and the naive Bayes (NB) algorithm. Of course, the classification algorithms that can be used by the present application are not limited to this, and other algorithms known in the art can also be used. In step 109, the k-NN, AD, SVM, and NB classification models are trained respectively for all the clusters produced in step 105 based on the respective attribute features determined in step 107 using the classification algorithms k-NN, AD, SVM, and NB. Table 3 below shows the road classification models RCM for each cluster.

[0064] Table 3: Road classification models RCM under multi-cluster-classification

[0065]

[0066] As shown in Table 3, for each cluster produced by each of the K-means, DBSCAN, and EM clustering algorithms, road classification models corresponding to the four classification algorithms are trained. For example, for the cluster C_K1 produced by the K-means algorithm, the classification models RCM K 11 ​, the classification model RCM is trained according to the AD algorithm K 12 , the classification model RCM is trained according to the SVM algorithm K 13 , the classification model RCM is trained according to the BM algorithm K 14 .

[0067]

Road classification

[0068] With the pre-trained road classification model and the corresponding cluster information, the road classification device according to the present application can be used to predict the classification of the current driving road.

[0069] Figure 2 A schematic diagram of a road classification device according to an embodiment of the present application is shown. As shown in the figure, the road classification device 200 comprises a storage unit 201, an image capturing unit 202, a feature recognition unit 203, a matching unit 204 and a classification unit 205. In the storage unit 201, the trained cluster information and the corresponding road classification model RCM are stored. As an example of a single clustering and classification method, the storage unit 201 stores, for example, the center points CP_1, CP_2, CP_3, CP_4, CP_5 of the clusters C1~C5 and the corresponding threshold radius R1~R5 information, and the road classification models RCM1~RCM5 corresponding to the clusters C1~C5. The road classification models RCM1~RCM5 here are trained according to the example combined with Figure 1A

[0070] The image capturing unit 202 is used to take the environmental image of the current driving road to obtain one or more image frames X containing a plurality of visual features Vis_F. The visual features Vis_F here can be all features related to the current driving road, such as traffic signs such as street lamps, traffic lights, speed limit signs, exits, gas stations, etc., and can also include auxiliary facilities such as sidewalks, isolation belts or green belts, and can even include features of the road itself such as lane lines, etc.

[0071] ​The feature recognition unit 203 processes the image frame X to identify visual features Vis_F contained in the image frame. According to an embodiment of the present application, a predetermined number W and type of visual feature Vis_F detection processing is performed on the image frame, where the number and type of visual features to be detected can cover typical traffic signs or facilities under existing traffic regulations and road construction standards. For example, W items of visual feature Vis_F detection such as street lamp, sidewalk, truck, traffic sign A, traffic sign B, traffic sign C, toll station, etc. are performed on the image frame. The existing feature detection technology can be used to identify the visual features Vis_F that can be contained in the image, such as the technology commonly used in the prior art to identify road features using a feature model trained by a convolutional neural network. For example, for the image frame X, a street lamp detection model trained based on machine deep learning can be used to detect whether the image frame X contains a street lamp, and the detection result can be expressed in confidence, which represents the probability of detecting a street lamp in the road. According to an embodiment of the present application, when detecting each visual feature Vis_F, the relative position information of each visual feature Vis_F contained in the image frame can be further determined.

[0072] In this way, the pre-trained sidewalk detection model, truck detection model, traffic sign A detection model, traffic sign B detection model, traffic sign C detection model, … toll station detection model, etc. can be used to perform corresponding visual feature Vis_F detection on the image frame X respectively. Thus, for the image frame X, the confidence data U and the relative position data L of the detected visual features Vis_F are obtained, denoted as X= (x (1) ,x (2) ,x (3) …x (W) ), where x (i) represents the i-th visual feature Vis_F of the image frame X.

[0073] The matching unit 204 matches the W visual features Vis_F of the image frame X with each of the predetermined number of cluster centers C1-C5 stored in the storage unit 201 to determine which of the cluster centers C1-C5 the image frame X belongs to. In the present example, where the cluster centers C1-C5 are trained based on the confidence of the visual features Vis_F using the K-means algorithm, the matching unit 204 uses the confidence of each visual feature Vis_F in the image frame X to sequentially calculate the relative distance D of X from the center point CP_1, CP_2, CP_3, CP_4, CP_5, and compare it with the Euclidean distance threshold R1, R2, R3, R4, R5 of each cluster center, and determine the cluster center for which the relative distance D is smaller than the corresponding threshold R as the cluster center to which the image frame X belongs. For example, assuming that the relative distance D2 of X from CP_2 is smaller than R2, it can be determined that the current image frame X belongs to the cluster center C2.

[0074] The classification unit 205 determines the road attribute features Attr_F2 of the cluster center C2 from the cluster center information stored in the storage unit 201 based on the cluster center C2 to which the image frame X belongs, and the road attribute features Attr_F2 are: (1) ,x (3) ,x (8) …x (k) , a total of M attribute features. Therefore, from the visual features (x (1) ,x (2) ,x (3) …x (W) ) of the current image frame X, the visual features (x (1) ,x (3) ,x (8) …x (k) ) are selected as the attribute features Attr_F2, which are provided to the road classification model RCM2 corresponding to the cluster center C2, and the road classification model RCM2 processes the attribute features Attr_F2 to generate a classification indication Road_ID of the corresponding road, thereby realizing the prediction of the current road. As mentioned above, the attribute features Attr_F2 provided to the road classification model RCM2 here not only include the confidence, but also include the relative position information, i.e., each element x in the attribute features includes the confidence and the relative position information. Thus, the classification unit 205 estimates the road classification by invoking the matching road classification model RCM2 to process the road attribute features Attr_F2 (x (1) ,x (3) ,x (8) …x (k) ).

[0075] It is to be noted that in the above embodiment, the road classification device only predicts for one image frame. In another embodiment of the present application, a series of image frames can also be processed, and the prediction results for each image frame in the series of image frames can be integrated to determine the current road type.

[0076] In another embodiment of the present application, the road classification device can also utilize the clustering information to perform road classification. Figure 1B The road classification device can utilize the multi-cluster-classification model trained to perform road classification, wherein the 15 clustering clusters and the corresponding 60 road classification models RCM shown in Table 3 are stored in the storage unit 201 of the road classification device. Of course, the clustering cluster information and the road classification model can also be stored remotely for the road classification device to call.

[0077] As described above, after the feature recognition unit 203 identifies the visual features Vis_F (i.e., X= (x (1) ,x (2) ,x (3) …x (W) ) contained in the image frame, the matching unit 204 matches the W visual features Vis_F of the image frame X with each of the predetermined number of clustering clusters C_K1~C_K5, C_DB1~C_DB5, C_EM1~C_EM5 stored in the storage unit 201 to determine which clustering cluster the image frame X belongs to under each type of clustering cluster. As an example, it is assumed here that through comparison, it is determined that under the K-means algorithm, the current image frame X belongs to C_K2, under the DBSCAN algorithm, the current image frame X belongs to C_DB3, and under the EM algorithm, the current image frame X belongs to C_EM5.

[0078] The classification unit 205 determines the road attribute features K_Attr_F2 of the clustering cluster C_K2 to which the image frame X belongs from the storage unit 201 based on the clustering cluster C_K2, and the road attribute features K_Attr_F2 are: x (1) ,x (3) ,x (8) …x (k) , a total of M attribute features. Therefore, the visual features (x (1) ,x (2) ,x (3) …x (W) ) are selected from the visual features Vis_F= (x (1) ,x (3) ,x (8) …x (k) ) of the current image frame X as attribute features K_Attr_F2, and provided to the road classification model RCM K 21, to achieve prediction of the current road. As mentioned above, the attribute features K_Attr_F of the road classification model RCM K 21 include not only the confidence but also the relative position information of the visual features x. It is assumed here that the road classification model RCM K 21 outputs the road classification indication as Road_ID1 = 'Expressway' after processing.

[0079] Similarly, the classification unit 205 provides the attribute features K_Attr_F2 to the road classification model RCM K 22 corresponding to the cluster C_K2 trained by the AD algorithm to achieve prediction of the current road. It is assumed here that the road classification model RCM K 22 outputs the road classification indication as Road_ID2 = 'Expressway' after processing. In addition, the classification unit 205 provides the attribute features K_Attr_F2 to the road classification model RCM K 23 trained by the SVM algorithm corresponding to the cluster C_K2 and outputs the road classification indication as Road_ID3 = 'Urban Expressway'; the classification unit 205 provides the attribute features K_Attr_F2 to the road classification model RCM K 24 trained by the NB algorithm corresponding to the cluster C_K2 and outputs the road classification indication as Road_ID4 = 'Urban Expressway'.

[0080] Similar to the above processing for the cluster C_K2, for the current image frame belonging to the cluster C_DB3 under the DBSCAN algorithm and the cluster C_EM5 under the EM algorithm, the above four classification models are also used to achieve prediction of the current road. Specifically, the classification unit 205 determines the road attribute features DB_Attr_F3 of the clustering cluster C_DB3 from the storage unit 201 based on the clustering cluster C_DB3 to which the image frame X belongs, for example: (1) (3) (7) (k) , a total of M attribute features. Therefore, from the visual features Vis_F = (x (1) (2) (3) (W) of the current image frame X, the visual features (x (1) (3) (7) (k) ​​​​​​​​​, as the attribute feature DB Attr F3, to the road classification model RCM corresponding to the cluster C DB3 trained by the K-NN algorithm DB 31 , as the attribute feature DB Attr F3, to the road classification model RCM corresponding to the cluster C DB3 trained by the K-NN algorithm DB 32 , as the attribute feature DB Attr F3, to the road classification model RCM corresponding to the cluster C DB3 trained by the K-NN algorithm DB 33 , as the attribute feature DB Attr F3, to the road classification model RCM corresponding to the cluster C DB3 trained by the K-NN algorithm DB 34 , as the attribute feature DB Attr F3, to the road classification model RCM corresponding to the cluster C DB3 trained by the K-NN algorithm

[0081] For the cluster C EM5, the classification unit 205 can determine the road attribute feature EM Attr F5 of the cluster C EM5 from the storage unit 201, for example, as: x (1) ,x (3) ,x (8) …x (p) , a total of M attribute features. Therefore, the corresponding visual features (x (1) ,x (2) ,x (3) …x (W) ) are selected from the visual features Vis F= (x (1) ,x (3) ,x (8) …x (p) ) of the current image frame X, as the attribute feature EM Attr F5, to the road classification model RCM corresponding to the cluster C EM5 trained by the K-NN algorithm EM 51 , as the attribute feature EM Attr F5, to the road classification model RCM corresponding to the cluster C EM5 trained by the K-NN algorithm EM 52 , as the attribute feature EM Attr F5, to the road classification model RCM corresponding to the cluster C EM5 trained by the K-NN algorithm EM 53 , as the attribute feature EM Attr F5, to the road classification model RCM corresponding to the cluster C EM5 trained by the K-NN algorithm EM 54 , as the attribute feature EM Attr F5, to the road classification model RCM corresponding to the cluster C EM5 trained by the K-NN algorithm 10= 'highway', Road_ID 11 = 'highway', Road_ID 12 = 'highway'.

[0082] Thus, after the prediction classification processing of different models under different clustering clusters, 12 road classification indication results (Road_ID1, Road_ID2, Road_ID3, Road_ID4, Road_ID5, Road_ID6, Road_ID7, Road_ID8, Road_ID9, Road_ID 10 , Road_ID 11 , Road_ID 12 ) are obtained. Thus, the classification unit 205 can analyze the above classification indication results to finally determine the classification of the current road. As an example, the classification unit 205 can vote on the above classification indication results, and the classification with the most votes is taken as the final classification result of the current road. Obviously, in this example, 'highway' has the most votes, Road_ID1 = Road_ID2 = Road_ID6 = Road_ID7 = Road_ID 10 = Road_ID 11 = Road_ID 12 = 'highway', so the classification unit 205 makes a final decision that the current road is a highway.

[0083] In another embodiment, the corresponding model prediction results Road_ID can also be weighted according to the matching degree of the current image frame X with the 15 clustering clusters. For example, if the matching unit determines that the matching degrees of the clustering clusters to which the current image frame belongs under the K-means, DNSCAN and EM algorithms are C_DB > C_K > C_EM in order during clustering processing, then the four prediction results Road_ID5, Road_ID6, Road_ID7 and Road_ID8 under the clustering cluster C_DB can be assigned a high weight value such as 1.2, while the four prediction results Road_ID9, Road_ID 10 , Road_ID 11 , Road_ID 12 under the clustering cluster C_EM can be assigned a low weight value such as 0.9, and the four prediction results under the clustering cluster C_K remain unchanged. Then the classification unit 205 votes on the weighted classification indication results to determine the final classification.

[0084] In the above embodiment, the road classification device is realized in the form of independent units. However, the present application is not limited thereto, and the road classification device can be realized in any hardware, software or combination thereof. Figure 3Another example of a road classification device is shown. As shown, the road classification device 300 comprises a processor 301 and a memory 302. In the memory 302, there are stored program modules for implementing the method steps of the present application, including a feature identification module 303, a matching module 304, and a classification module 305. In addition, the memory 302 also comprises an image storage area 306 for obtaining and storing the current road image or video X received from an external sensor, and a data storage area 307 for storing the cluster information and the corresponding road classification model RCM information obtained through training. Thus, the processor 301 can be configured to implement the method of the present application by executing the program modules in the memory 302.

[0085] Figure 4 A flowchart of a road classification method performed by a road classification device according to an example of the present application is shown. In step 401, an image sensor such as a camera is used to obtain an environmental image of the current driving road to obtain one or more image frames X containing a plurality of visual features Vis_F, and the image frames X are stored in a memory.

[0086] In step 402, the image frames X are processed to identify the visual features Vis_F contained in the image frames X. Here, the detection processing of the visual features Vis_F is performed on the image frames X using a streetlight detection model, a sidewalk detection model, a truck detection model, a traffic sign A detection model, a traffic sign B detection model, a traffic sign C detection model, a toll station detection model, etc. The detection results can be represented using the confidence of the detected visual features Vis_F. In addition, according to an embodiment of the present application, when detecting the visual features Vis_F, the relative positions of the visual features Vis_F contained in the image frames can also be further determined. In this way, the visual features Vis_F for the image frames X are generated, i.e. X = (x (1) ,x (2) ,x (3) …x (W) ).

[0087] Step 403: match the W visual features Vis_F of the image frame X with each of the predetermined number of cluster centers C1-C5 stored in the memory to determine which cluster center C1-C5 the image frame X belongs to. In this example, the road classification device calculates the relative distance D of X to the center point CP_1, CP_2, CP_3, CP_4, CP_5 of the five cluster centers in turn using the confidence of each visual feature Vis_F in X, and compares the relative distance D with the Euclidean distance threshold R1, R2, R3, R4, R5 of each cluster center to determine the cluster center whose relative distance D is smaller than the corresponding radius threshold R as the cluster center to which the image frame X belongs. In this example, it is assumed that the image frame X belongs to the cluster center C2.

[0088] Step 404: based on the cluster center C to which the current image frame X belongs determined in step 403, construct the input feature, i.e. the attribute feature Attr_F, for input to the road classification model RCM. As an example, based on the cluster center C2 to which the image frame X belongs, the road attribute feature Attr_F2 of the cluster center C2 can be determined as: x (1) ,x (3) ,x (8) …x (k) , a total of M attribute features. Therefore, the visual features (x (1) ,x (2) ,x (3) …x (W) ) in the visual features Vis_F (x (1) ,x (3) ,x (8) …x (k) ) of the current image frame X are selected as the attribute feature Attr_F2. As mentioned above, the attribute feature Attr_F2 provided to the road classification model RCM here not only contains the confidence but also contains the relative position information.

[0089] Step 405: call the road classification model RCM2 matched with the cluster center C2 to estimate the road classification by processing the road attribute feature Attr_F2 (x (1) ,x (3) ,x (8) …x (k) ).

[0090] Figure 5 A flowchart showing a road classification method performed by a road classification device according to another example of the present application is shown in Figure 1B The road classification can be implemented by the road classification device using the multi-cluster-classification model trained by the embodiment shown in Figure 5 As shown in Figure 4The steps 401-403 are the same as shown above, and will not be repeated here. However, it is noted that in step 503, the road classification device needs to match the current image frame X with the pre-stored 15 clustering clusters C_K1-C_K5, C_DB1-C_DB5, C_EM1-C_EM5 respectively, to determine the best matching clustering cluster under each type of clustering method. It is assumed that the matching results are C_K2, C_DB3, and C_EM5.

[0091] Step 504: Based on the clustering cluster to which the current image frame X belongs and its specified attribute features determined in step 503, the input for the road classification model RCM is constructed. As an example, based on the clustering cluster C_K2 to which the image frame X belongs, the road attribute features K_Attr_F2 of the clustering cluster C_K2 are determined. Thus, based on the K_Attr_F 2， The input for the classification model is constructed using the corresponding visual features in the current image frame X.

[0092] Step 505: The road classification model RCM K 21 that matches the clustering cluster C_K2 is invoked, and the road classification indication Road_ID1 is generated by processing the road attribute features K_Attr_F2. Then in step 506, it is determined whether all types of road classification model processing have been completed for the clustering cluster C_K2, and if not, the process returns to step 405 to continue invoking the next classification model, for example, the road classification model RCM K 22 trained based on the AD algorithm, the road classification model RCM K 23 trained based on the SVM algorithm, and the road classification model RCM K 24 trained based on the NB algorithm, to obtain the classification indication outputs Road_ID2, Road_ID3, Road_ID4.

[0093] After it is determined in step 506 that all types of road classification model processing have been completed for the current clustering cluster C_K2, the process proceeds to step 507 to determine whether road classification processing has been completed for all matching clustering clusters. If not, the process returns to step 504 to continue processing the next clustering cluster. In this example, after C_K2 is processed, the process returns to step 504 to determine the road attribute features DB_Attr_F3 of the clustering cluster C_DB3 based on the clustering cluster C_DB3 to which the image frame X belongs. Thus, based on the DB_Attr_F 3，The input of the classification model is constructed by the corresponding visual features in the current image frame X, and then the classification indication outputs Road_ID5, Road_ID6, Road_ID7, Road_ID8 for the clustering cluster C_DB3 are obtained by performing steps 505 and 506. In this way, the classification indication outputs Road_ID9, Road_ID 10 , Road_ID 11 , Road_ID 12 are generated for the clustering cluster C_EM5. In this way, the road classification processing for all clustering clusters is completed.

[0094] After determining that the road classification processing for all matching clustering clusters is completed in step 507, proceed to step 508, vote on the 12 classification indication candidate results (Road_ID1, Road_ID2, Road_ID3, Road_ID4, Road_ID5, Road_ID6, Road_ID7, Road_ID8, Road_ID9, Road_ID 10 , Road_ID 11 , Road_ID 12 ) generated, and select the classification result with the highest number of votes as the final classification result of the current road.

[0095] The above describes an embodiment of a road classification device and method, and the road classification device and method according to the present application can be used in an automatic driving system, and by processing image frames captured by a camera of a vehicle, the classification of the current road of the vehicle is realized.

[0096] In addition, although the embodiments of the present application are described and illustrated in detail through the accompanying drawings and preferred embodiments, the present application is not limited to these disclosed embodiments, and any modifications can be made by those skilled in the art on the basis of the above detailed disclosure, including the combination, replacement, addition and deletion of features, and these solutions should be considered to fall within the protection scope defined by the appended claims.

Claims

1. A method for road recognition, comprising: capturing an environment image of a current driving road by an image capturing unit to capture at least one image frame of the current driving road; processing the image frame to identify a plurality of visual features in the image frame and obtain a relative position of each visual feature in the image frame, wherein the visual features are features related to the current driving road and include traffic signs; matching the visual features with a predetermined number of candidate clustering clusters to determine a target clustering cluster to which the image frame belongs; selecting attribute features specified by the determined target clustering cluster from the plurality of visual features; processing the attribute features using a road classification model corresponding to the determined target clustering cluster to classify the current road, wherein the attribute features include the relative positions. 2.The method of claim 1, wherein the predetermined number of candidate clustering clusters includes a plurality of sets of clustering clusters trained using different clustering algorithms, each set of clustering clusters including a plurality of candidate clustering clusters generated using a same algorithm; wherein determining the target clustering cluster to which the image frame belongs includes, for each set of clustering clusters, determining a target clustering cluster in the set of clustering clusters to which the image frame belongs; wherein the road classification model corresponding to the determined target clustering cluster includes a plurality of road classification models trained using different machine learning methods for the target clustering cluster, wherein the processing the attribute features using the road classification model corresponding to the determined target clustering cluster to classify the current road further includes: processing the attribute features using the plurality of road classification models respectively to generate a plurality of candidate classification results, voting the plurality of candidate classification results to determine the classification of the current road.

3. The method of claim 2, wherein voting on the plurality of candidate classification results to determine a classification of the current road comprises: selecting a classification result indicating a same classification result in the plurality of candidate classification results as the classification of the current road.

4. The method of claim 2, wherein voting on the plurality of candidate classification results to determine a classification of the current road comprises: weighting candidate classification results generated by different road classification models and accumulating weighted candidate results indicating a same classification result, and selecting a candidate result with a largest weighted sum as the classification of the current road.

5. The method of any of claims 1-4, wherein processing the image frame to identify a plurality of visual features within the image frame comprises: detecting each of the plurality of visual features using a feature detection model trained for each of the plurality of visual features, wherein each of the plurality of visual features is represented by a confidence.

6. The method of any one of claims 1-4, wherein matching the visual feature to a predetermined predetermined number of candidate cluster clusters comprises: recomputing clustering conditions of the plurality of visual features with each of the predetermined number of candidate clustering clusters using clustering algorithms employed when the predetermined number of candidate clustering clusters is determined to determine the target clustering cluster to which the image frame belongs.

7. The method of claim 6, wherein, processing the attribute features using the road classification model corresponding to the determined target clustering cluster includes constructing an input of the road classification model based on the confidence and the relative position of the attribute features. 8.A method for training a road classification model, comprising: obtaining a plurality of image frames captured by an image capturing unit from an environment of a driving road, each image frame including visual features of the road, wherein the visual features are features related to the driving road and include traffic signs; processing each image frame to identify a plurality of visual features contained in the image frame and obtaining a relative position of each visual feature in the image frame; performing clustering on the plurality of image frames based on the visual features to determine a cluster to which each image frame belongs, to form a plurality of clusters; selecting a plurality of visual features most relevant to the cluster from the visual features in the image frames under the same cluster as attribute features of the cluster; forming a training sample using the attribute features in each image frame under the same cluster, wherein the attribute features include the relative positions, for training a road classification model matching the cluster.

9. The method of claim 8, wherein, performing clustering on the plurality of image frames to determine a cluster to which each image frame belongs includes processing the plurality of image frames using different clustering algorithms to generate a plurality of sets of clusters, each set of clusters including a plurality of clusters generated using the same clustering algorithm; wherein forming a training sample using the attribute features in each image frame under the same cluster for training a road classification model matching the cluster includes learning attribute features of image frames under each set of clusters using different machine learning methods, thereby generating a plurality of road classification models for each cluster under each set of clusters; wherein a voting result of a plurality of candidate classification results generated by processing the attribute features using the plurality of road classification models respectively is taken as a classification of a current road.

10. The method of claim 9, wherein the voting result is a classification result indicating a same classification result most frequently in the plurality of candidate classification results.

11. The method of claim 9, wherein the voting comprises: weighting candidate classification results generated by different road classification models and accumulating weighted candidate results indicating a same classification result, and selecting a candidate result with a largest weighted sum as a classification of a current road.

12. The method of any one of claims 8-11, processing each frame of the image to identify a plurality of visual features contained within the frame of the image comprises: detecting each of the plurality of visual features using a feature detection model trained for each of the plurality of visual features, wherein each of the plurality of visual features is represented by a confidence score.

13. The method of any one of claims 8-11, wherein an input of the road classification model is constructed based on the confidence scores and the relative positions of the attribute features.

14. The method of claim 13, comprising: storing generated cluster-related information including: clustering algorithm information, a number and a type of the attribute features; storing a road classification model matching the cluster trained.

15. The method of any one of claims 8-11, wherein determining a plurality of attribute features most relevant to the cluster from the plurality of visual features is implemented using a Spearman rank correlation coefficient algorithm.

16. A computing system comprising a road classification module for identifying a road traveled, wherein the road classification module is configured to: processing at least one image frame of a current driving road to identify a plurality of visual features in the image frame and obtain a relative position of each visual feature in the image frame, wherein the visual features are features related to the current driving road and include traffic signs, and the at least one image frame is captured by capturing an environment image of the current driving road by an image capturing unit; matching the visual features with a predetermined number of candidate clustering clusters to determine a target clustering cluster to which the image frame belongs; selecting an attribute feature specified by the determined target clustering cluster from the plurality of visual features; processing the attribute feature by using a road classification model corresponding to the determined target clustering cluster to classify the current road, wherein the attribute feature includes the relative position.

17. The computing system of claim 16, wherein the predetermined number of candidate clustering clusters includes a plurality of sets of clustering clusters trained by using different clustering algorithms, and each set of clustering clusters includes a plurality of candidate clustering clusters generated by using a same algorithm; wherein the road classification module is further configured to determine, for each set of clustering clusters, a target clustering cluster in the set of clustering clusters to which the image frame belongs; wherein the road classification model corresponding to the determined target clustering cluster includes a plurality of road classification models trained for the target clustering cluster by using different machine learning methods, wherein the road classification module is further configured to: process the attribute feature by using the plurality of road classification models to generate a plurality of candidate classification results, respectively, vote the plurality of candidate classification results to determine the classification of the current road.

18. The computing system of claim 16 or 17, wherein the road classification module is configured to detect each of the plurality of visual features by using a feature detection model trained for each of the plurality of visual features, wherein each of the plurality of visual features is represented by a confidence score.

19. The computing system of claim 16 or 17, wherein the road classification module is configured to re-compute a clustering condition of each of the plurality of visual features with each of the predetermined number of candidate clustering clusters to determine the target clustering cluster to which the image frame belongs, by using a clustering algorithm employed when the predetermined number of candidate clustering clusters is determined.

20. The computing system of claim 19, wherein the road classification module is configured to: construct an input of the road classification model based on the confidence score and the relative position of the attribute feature.

21. A road classification apparatus, comprising: a feature extraction module configured to process at least one image frame of a current driving road to identify a plurality of visual features in the image frame and obtain a relative position of each visual feature in the image frame, wherein the visual features are features related to the current driving road and include traffic signs, and the at least one image frame is captured by capturing an environment image of the current driving road by an image capturing unit; a clustering module configured to match the visual features with a predetermined number of candidate clustering clusters to determine a target clustering cluster to which the image frame belongs; a feature selection module configured to select an attribute feature specified by the determined target clustering cluster from the plurality of visual features; and a road classification module configured to process the attribute feature by using a road classification model corresponding to the determined target clustering cluster to classify the current road, wherein the attribute feature includes the relative position. a feature selection module configured to select attribute features specified by the determined target cluster of clusters from the plurality of visual features; a classification module configured to process the attribute features using a road classification model corresponding to the determined target cluster of clusters to classify the current road, wherein the attribute features include the relative position.

22. The apparatus of claim 21, wherein the predetermined number of candidate clusters of clusters includes a plurality of groups of clusters of clusters trained using different clustering algorithms, each group of clusters of clusters including a plurality of candidate clusters of clusters generated using a same algorithm; wherein the clustering module determines, for each group of clusters of clusters, a target cluster of clusters of the group of clusters of clusters to which the image frame belongs; wherein the road classification model corresponding to the determined target cluster of clusters includes a plurality of road classification models trained using different machine learning methods for the target cluster of clusters, wherein the classification module is further configured to: process the attribute features using the plurality of road classification models to generate a plurality of candidate classification results, respectively, vote the plurality of candidate classification results to determine the classification of the current road.

23. The apparatus of claim 22, wherein voting on the plurality of candidate classification results to determine a classification of the current road comprises: select a classification result from the plurality of candidate classification results that indicates a same classification result as a majority of the plurality of candidate classification results as the classification of the current road.

24. The apparatus of claim 22, wherein voting on the plurality of candidate classification results to determine a classification of the current road comprises: weight candidate classification results generated by different road classification models and accumulate weighted candidate results that indicate a same classification result, and select a candidate result with a largest weighted sum as the classification of the current road.

25. The apparatus of any one of claims 21-24, wherein the feature extraction module uses a feature detection model trained for each of the plurality of visual features to detect each of the plurality of visual features, respectively, wherein each of the plurality of visual features is represented by a confidence score.

26. The apparatus of any one of claims 21-24, wherein the clustering module re-computes a clustering condition of each of the plurality of visual features with each of the predetermined number of candidate clusters of clusters using a clustering algorithm employed in pre-determining the predetermined number of candidate clusters of clusters to determine the target cluster of clusters to which the image frame belongs.

27. The apparatus of claim 26, wherein, the classification module constructs an input of the road classification model based on the confidence score and the relative position of the attribute features.

Images