Vehicle control method, device, storage medium and vehicle

Abstract

The present disclosure relates to a vehicle control method, device, storage medium and vehicle, and belongs to the field of vehicle control. The method comprises: acquiring a road condition image in a vehicle driving process; inputting the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being obtained through semi-supervised training based on a category label weight in advance, the road condition detection result comprising position information and category information of each target object in the road condition image, and the category label weight being used to represent category similarity; and controlling the vehicle according to the road condition detection result. Training the road condition detection model based on the category label weight can not only reduce the probability of model overfitting, but also reduce the influence of false pseudo labels, improve the robustness of the trained road condition detection model, make the detection result output by the road condition detection model more accurate, and improve driving safety.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent networked vehicles, and particularly relates to a vehicle control method and device, a storage medium and a vehicle. BACKGROUND

[0002] Target detection is a key technology in the field of automatic driving, and directly affects the safety of personnel and vehicles. Target detection is also one of the most challenging problems in computer vision. Semi-supervised target detection realizes data expansion on the basis of traditional target detection, reduces the manual cost of data labeling, and reduces the application cost of target detection in various fields.

[0003] However, in the related art, most semi-supervised target detection currently uses hard labels for training during training, which is not conducive to reducing the continuous adverse effects of false pseudo labels on training, and is prone to overfitting to false pseudo labels, which may cause inaccurate target detection during vehicle driving in automatic driving, leading to accidents and danger. SUMMARY

[0004] To overcome the problems in the related art, the present disclosure provides a vehicle control method and device, a storage medium and a vehicle.

[0005] According to a first aspect of an embodiment of the present disclosure, a vehicle control method is provided, comprising:

[0006] obtaining a road condition image in a vehicle driving process;

[0007] inputting the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being obtained by pre-training based on a class label weight, the road condition detection result including position information and class information of each target object in the road condition image, and the class label weight being used to represent class similarity;

[0008] controlling the vehicle according to the road condition detection result.

[0009] Optionally, the data set of the road condition detection model includes a first data set and a second data set, the first data set including road condition image samples labeled with real label information, and the second data set including road condition image samples without real label information, and the training of the road condition detection model includes:

[0010] inputting the first data set into the road condition detection model for pre-training, and determining a first loss between a first predicted label output by the road condition detection model after pre-training and the real label information for the first data set;

[0011] determine the class label weight of the road condition detection model based on the class feature information output by the intermediate layer of the road condition detection model in the pre-training process;

[0012] generate a third data set and a pseudo label corresponding to the third data set according to the class label weight and the second data set;

[0013] re-train the road condition detection model according to the third data set and the pseudo label, and obtain the trained road condition detection model.

[0014] Optionally, the determining the class label weight of the road condition detection model based on the class feature information output by the intermediate layer of the road condition detection model in the pre-training process comprises:

[0015] calculating the inter-class distance between each class label according to the class feature information;

[0016] determining the class similarity information between each class according to the inter-class distance;

[0017] determining the class label weight according to the class similarity information.

[0018] Optionally, the generating a third data set and a pseudo label corresponding to the third data set according to the class label weight and the second data set comprises:

[0019] inputting the second data set into the trained road condition detection model to obtain a first pseudo label;

[0020] determining a second pseudo label based on the class label weight;

[0021] performing a data enhancement operation on the second data set and the second pseudo label to obtain an enhanced third data set and a third pseudo label, the third pseudo label being a pseudo label corresponding to the third data set.

[0022] Optionally, the inputting the second data set into the trained road condition detection model to obtain a first pseudo label comprises:

[0023] inputting the second data set into the trained road condition detection model to obtain a first prediction value corresponding to the second data set, the first prediction value comprising a position prediction value and a class prediction value, the class prediction value comprising a class probability that a target object belongs to each class;

[0024] performing a non-maximum suppression operation on the position prediction value to obtain a first target object with high position confidence;

[0025] According to the category probability of each category in the category prediction value corresponding to each first target object, a category confidence of each first target object is determined.

[0026] A second target object in each first target object is determined, in which the category confidence is greater than a preset confidence threshold.

[0027] According to the position information determined according to the position prediction value corresponding to each second target object, and the category information determined according to the category probability of each category in the corresponding category prediction value, the first pseudo label is obtained.

[0028] Optionally, the retraining of the road condition detection model according to the third data set and the pseudo label to obtain the trained road condition detection model comprises:

[0029] The third data set is input into the first road condition detection model to obtain a second prediction label.

[0030] According to the second prediction label and the third pseudo label, a second loss is calculated.

[0031] Based on the first loss and the second loss, the parameters of the road condition detection model are updated to obtain the trained road condition detection model.

[0032] Optionally, the updating of the parameters of the road condition detection model based on the first loss and the second loss comprises:

[0033] The first loss and the second loss are normalized to obtain a first normalized loss and a second normalized loss.

[0034] According to the sum of the first normalized loss and the second normalized loss, the parameters of the road condition detection model are updated.

[0035] Optionally, the data enhancement operation on the second data set and the second pseudo label to obtain the enhanced third data set and the third pseudo label comprises:

[0036] At least one of the data enhancement operations of color conversion, geometric transformation, scaling, box-level transformation and gray frame filling is performed on the images in the second data set; and

[0037] In the case where the pixel position of the image in the second data set changes, the position information in the second pseudo label corresponding to the image in the second data set is changed accordingly.

[0038] According to a second aspect of the embodiments of the present disclosure, a vehicle control device is provided, comprising:

[0039] an acquisition module configured to acquire a road condition image in a driving process of a vehicle;

[0040] a detection module configured to input the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being obtained through semi-supervised training based on a category label weight in advance, the road condition detection result including position information and category information of each target object in the road condition image, and the category label weight being used to represent category similarity;

[0041] a control module configured to control the vehicle according to the road condition detection result.

[0042] According to a third aspect of embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the steps of the vehicle control method provided in the first aspect of the present disclosure.

[0043] According to a fourth aspect of embodiments of the present disclosure, a vehicle is provided, which includes:

[0044] a processor;

[0045] a memory configured to store processor-executable instructions;

[0046] The processor is configured to:

[0047] acquire a road condition image in a driving process of a vehicle;

[0048] input the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being obtained through semi-supervised training based on a category label weight in advance, the road condition detection result including position information and category information of each target object in the road condition image, and the category label weight being used to represent category similarity;

[0049] control the vehicle according to the road condition detection result.

[0050] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects: the image in the driving process of the vehicle is detected by the road condition detection model obtained through semi-supervised training based on the category label weight, and the vehicle is controlled according to the detection result. Since the category label weight can represent category similarity, training the road condition detection model based on the category label weight can not only reduce the probability of model overfitting, but also reduce the influence of false pseudo labels, improve the robustness of the road condition detection model obtained through training, make the detection result output by the road condition detection model more accurate, and improve driving safety.

[0051] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] Figure 1 The figure is a flow chart showing a vehicle control method according to an exemplary embodiment.

[0054] Figure 2 The figure is a flowchart of a method for training a road condition detection model according to an exemplary embodiment.

[0055] Figure 3 is a block diagram of a vehicle control device according to an exemplary embodiment.

[0056] Figure 4 is a block diagram of a vehicle control device according to an exemplary embodiment.

[0057] Figure 5 is a block diagram of a vehicle control device according to an exemplary embodiment.

[0058] Figure 6 The figure is a functional block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0059] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0060] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0061] In order to enable those skilled in the art to better understand the improvements of the technical solutions provided by the present disclosure, the present disclosure first further introduces the relevant technologies.

[0062] Object detection is a key technology in autonomous driving, directly impacting the safety of both drivers and vehicles. It's also one of the most challenging problems in computer vision. Its goal is to determine whether objects of a specified category exist in an image and, if so, return the location and category of each object.

[0063] Convolutional neural networks have driven the continuous development and improvement of object detection algorithms. However, at the same time, the problem of lack of training data caused by high artificial labeling cost is increasingly valued. In most current semi-supervised object detection algorithms based on vision, the portion of data labeled by artificial labeling accounts for only 5%-30% of the total data. Semi-supervised learning allows algorithms to add massive unlabeled data to the existing data set and improve accuracy from the training of these data. Semi-supervised object detection realizes data expansion on the basis of traditional object detection, reduces the artificial cost of data labeling, and reduces the application cost of object detection in various fields.

[0064] STAC and most current semi-supervised object detection use hard labels for training during training, which is not conducive to reducing the continuous adverse effects of false pseudo labels on training and is prone to overfitting of false pseudo labels. Although label smoothing can effectively reduce the overfitting phenomenon, it uses the same weight for all classes, erases the similarity information between classes, and cannot reduce the influence of false pseudo labels. False pseudo labels will have a continuous impact on model training, ultimately affecting detection accuracy, and current semi-supervised object detection methods using soft labels mostly use soft labels when judging foreground and background, without reducing the harm of false pseudo labels of incorrect classification.

[0065] Further, in autonomous driving, it may cause inaccurate target detection during vehicle driving, leading to accidents and danger.

[0066] To solve the problems in the related art, the present disclosure provides a vehicle control method, device, storage medium and vehicle.

[0067] Figure 1 is a flowchart of a vehicle control method according to an exemplary embodiment, which is applied to a vehicle, such as an electronic control unit, a driving computer, etc. configured on the vehicle, an electronic device with information processing capability, as shown in Figure 1 The method includes the steps of:

[0068] S101, acquiring a road condition image in the driving process of the vehicle.

[0069] The road condition image can be acquired by a camera configured on the vehicle and can include images in front and / or rear of the vehicle.

[0070] S102, inputting the road condition image into a road condition detection model to obtain a road condition detection result.

[0071] The road condition detection model is obtained through semi-supervised training based on category label weights, and the road condition detection result includes position information and category information of each target object in the road condition image, and the category label weights are used to represent category similarity.

[0072] For example, the position information can be represented in the form of a bounding box in the road condition image, and the category information can be represented by text, such as "pedestrian", "vehicle", "speed bump", and the like.

[0073] In a possible implementation, the road condition image and the corresponding detection result can also be displayed on a display screen on the vehicle.

[0074] The road condition detection model can be trained by other electronic devices such as a server and then sent to the vehicle, or can be trained by a vehicle computer, and the present disclosure does not limit this.

[0075] Those skilled in the art should know that semi-supervised training refers to a model training method combining supervised learning and unsupervised learning, and semi-supervised training uses a large amount of unlabeled data and labeled data at the same time to train the model.

[0076] Specifically, the category label weights can be obtained based on category feature information extracted from an intermediate layer of the road condition detection model during supervised training of the road condition detection model, for example, calculating the inter-class distance between each category label according to the category feature information; determining the category similarity information between each category according to the inter-class distance; and determining the category label weights according to the category similarity information. It can be understood that the above intermediate layer can be the penultimate layer of the network, and the output of this layer can be the feature vector of the corresponding training set output by the network based on the training set.

[0077] The inter-class distance represents low category similarity, and the inter-class distance represents high category similarity. The category label weights can be represented in the form of a category label weight matrix.

[0078] S103, controlling the vehicle according to the road condition detection result.

[0079] For example, if the road condition detection result indicates that a speed bump appears in front of the road, the vehicle can be controlled to slow down. Alternatively, if the detection result indicates that a pedestrian appears in front of the vehicle, the vehicle can be controlled to slow down and sound a horn at the same time.

[0080] In the embodiments of the present disclosure, the image in the vehicle driving process is detected by the road condition detection model pre-trained based on the class label weight, and the vehicle driving is controlled according to the detection result. Since the class label weight can represent the class similarity, training the road condition detection model based on the class label weight can not only reduce the probability of model overfitting, but also reduce the influence of false pseudo labels, improve the robustness of the trained road condition detection model, and make the detection result output by the road condition detection model more accurate, thereby improving the driving safety.

[0081] In some optional embodiments, the data set of the road condition detection model includes a first data set and a second data set, the first data set includes road condition image samples labeled with real label information, and the second data set includes road condition image samples without real label information. The training of the road condition detection model includes:

[0082] The first data set is input into the road condition detection model for pre-training, and a first loss between the first predicted label output by the road condition detection model for the first data set after pre-training is completed and the real label information is determined.

[0083] Based on the class feature information output by the intermediate layer of the road condition detection model in the pre-training process, the class label weight of the road condition detection model is determined.

[0084] According to the class label weight and the second data set, a third data set and a pseudo label corresponding to the third data set are generated.

[0085] According to the third data set and the pseudo label, the road condition detection model is trained again to obtain the trained road condition detection model.

[0086] The number of image samples in the first data set can be much smaller than the number of images in the second image set, and the road condition detection model can be a FastRCNN model.

[0087] The pre-training completion can be determined by calculating whether the road condition detection model converges, or can be determined according to the training time length. For example, if the training step is greater than a preset threshold, it can be determined that the pre-training is completed, and the pre-trained road condition detection model is obtained. At this time, the pre-trained road condition detection model can be regarded as a teacher model. The second data set without label is input into the teacher model for prediction, and the corresponding pseudo label output by the teacher model can be obtained. The pseudo label includes the position information and the class information of each target object in the corresponding data set.

[0088] With the above scheme, the teacher model is first pre-trained based on the dataset with labels to make the teacher model have the ability of target detection and type discrimination, and then the class label weight is obtained based on the class feature information to make the dataset without label information have pseudo labels to further train the teacher model to obtain the final road condition detection model, so as to ensure that the road condition detection model can not only reduce the probability of model overfitting, but also reduce the influence of false pseudo labels.

[0089] In some possible implementation manners, the determining, by the road condition detection model based on the intermediate layer, of the class label weight of the road condition detection model based on the class feature information output by the pre-training process comprises:

[0090] calculating an inter-class distance between each class label according to the class feature information;

[0091] determining class similarity information between each class according to the inter-class distance;

[0092] determining the class label weight according to the class similarity information.

[0093] Wherein, the inter-class distance, the class similarity information and the class label weight are introduced above, and will not be described here again. With the present scheme, the class label weight can be accurately obtained by calculating the inter-class distance to obtain the class similarity information.

[0094] Optionally, the generating, by the road condition detection model, of the third dataset and the pseudo label corresponding to the third dataset according to the class label weight and the second dataset comprises:

[0095] inputting the second dataset into the trained road condition detection model to obtain a first pseudo label;

[0096] determining a second pseudo label based on the class label weight;

[0097] performing a data enhancement operation on the second dataset and the second pseudo label to obtain an enhanced third dataset and a third pseudo label, the third pseudo label being the pseudo label corresponding to the third dataset.

[0098] It can be understood that the network predicts different categories of targets in the image, and the pseudo-label obtained after screening is actually the same. For example, the category prediction value of a certain target object A of a certain image in the second data set is {0.9, 0.09, 0.01}, the category prediction value of another target B is {0.9, 0.05, 0.05}, and the category prediction value of another target C is {0.5, 0.25, 0.25}. In the prediction value of target A, 0.9 represents the probability that the target is a car, 0.09 represents the probability that the target is a tricycle, and 0.01 represents the probability that the target is a bicycle. The same is true for targets B and C. After screening, the first pseudo-label of target A and target B represents that the target object is a "car", and target C is removed as an unreliable object because the maximum probability is 0.5, which is less than the preset threshold. Specifically, the screening of the prediction value will be further described in the following possible embodiments, and will not be described here.

[0099] For example, the first pseudo-label of a certain target object A of a certain image in the second data set represents that the object A is a "car". After processing the label based on the category label weight, the second pseudo-label obtained can represent that the probability of the object A being a "car" is 0.88, the probability of the object A being a "tricycle" is 0.09, and the probability of the object A being a "bicycle" is 0.03. That is, the category label weight can represent that the similarity between the car label and the tricycle label is higher than the similarity between the car label and the tricycle label. Specifically, the first pseudo-label and can be processed with the corresponding column in the category label weight matrix, such as addition, normalization, etc.

[0100] By using the scheme, after the teacher model outputs the pseudo-label, the pseudo-label can be processed to add category relevance to the pseudo-label, so that the pseudo-label can represent the association information between categories, so that the road condition detection model finally trained can make more accurate judgments on road conditions, and the robustness of the model is improved.

[0101] In some other optional examples, the inputting the second data set into the trained road condition detection model to obtain the first pseudo-label includes:

[0102] The second data set is input into the trained road condition detection model to obtain a first prediction value corresponding to the second data set, the first prediction value including a position prediction value and a category prediction value, the category prediction value including a category probability that a corresponding target object belongs to each category;

[0103] Performing a non-maximum suppression operation on the position prediction value to obtain a first target object with high position confidence;

[0104] Determining a category confidence corresponding to each of the first target objects based on the category probabilities of the category prediction values ​​corresponding to each of the first target objects belonging to each category;

[0105] Determining a second target object whose category confidence is greater than a preset confidence threshold among each of the first target objects;

[0106] The position information is determined according to the position prediction values ​​corresponding to each of the second target objects, and the category information is determined according to the category probabilities of belonging to each category in the corresponding category prediction values, to obtain the first pseudo label.

[0107] Among them, the preset confidence threshold can be determined according to the actual training situation of the model. For example, it can be calibrated to 0.7, 0.8, etc., and this disclosure does not limit this.

[0108] It is understandable that the road condition detection model will predict a large number of target box prediction results for the second data set, that is, the prediction of the target position. The use of non-maximum suppression can eliminate a large number of redundant boxes to find the best target bounding box, so as to reduce the calculation amount of the model and improve the calculation efficiency.

[0109] Furthermore, the category confidence can be determined based on the maximum probability category of the prediction results of the road condition detection model. For example, the category prediction value of a target object A is {0.9, 0.09, 0.01}, the category prediction value of another target B is {0.9, 0.05, 0.05}, and the category prediction value of another target C is {0.5, 0.25, 0.25}. Then, the category confidence corresponding to target A and target B is 0.9, and the category confidence corresponding to target C is 0.5. If the preset confidence threshold is 0.8, the confidence of target C is lower than the preset confidence threshold, and the pseudo label corresponding to target C can be removed.

[0110] This solution uses non-maximum suppression to eliminate a large number of redundant boxes, and then uses a preset execution threshold to select high-confidence category labels, thereby filtering out pseudo labels with high confidence output by the teacher model to avoid the continued impact of incorrect labels on model training.

[0111] In some optional embodiments, retraining the road condition detection model based on the third data set and the pseudo labels to obtain the trained road condition detection model includes:

[0112] Inputting the third data set into the road condition detection model to obtain a second predicted label;

[0113] Calculating a second loss based on the second predicted label and the third pseudo label;

[0114] update parameters of the road condition detection model based on the first loss and the second loss to obtain the trained road condition detection model.

[0115] The first loss and the second loss can be calculated based on cross-entropy.

[0116] According to the scheme, the third pseudo label obtained based on the category label weight is compared with the prediction value of the teacher model on the third data set, the second loss is calculated, and the teacher model is further trained based on the first loss obtained in the first stage and the second loss, so that the model considers the loss in the training stage of the teacher model and the loss based on the pseudo label training, and the model can have more accurate road condition detection capability, ensuring the safety of driving.

[0117] Optionally, the updating of the parameters of the road condition detection model based on the first loss and the second loss comprises:

[0118] The first loss and the second loss are normalized to obtain a first normalized loss and a second normalized loss;

[0119] The parameters of the road condition detection model are updated according to the sum of the first normalized loss and the second normalized loss.

[0120] Specifically, the normalization of the first loss can be calculated based on the number of images in the first data set, the category loss and the position loss (i.e. the regression frame loss), and the normalization of the second loss can be calculated based on the number of images in the third data set, the category loss and the position loss.

[0121] According to the scheme, by normalizing the first loss and the second loss respectively and then adding them, the difference between supervised and semi-supervised loss values can be effectively reduced without ignoring one side, and the overall training effect of the model is improved.

[0122] Optionally, the data augmentation operation on the second data set and the second pseudo label to obtain the enhanced third data set and the third pseudo label comprises:

[0123] At least one of the data augmentation operations of color conversion, geometric transformation, scaling, box-level transformation and gray frame filling is performed on the images in the second data set; and

[0124] In the case where the pixel position of the image in the second data set changes, the position information in the second pseudo label corresponding to the image in the second data set is changed accordingly.

[0125] It can be understood that if the second data set and the second pseudo label are directly input into the teacher model without processing, the output result of the teacher model may be the same as the second pseudo label, and thus the loss of the model cannot be accurately obtained.

[0126] By adopting the scheme, the image is subjected to data enhancement such as color conversion, geometric transformation, proportional scaling, box-level transformation or gray frame filling, and the position information of the corresponding pseudo label is subjected to corresponding transformation, and then the enhanced third data set and the corresponding pseudo label are obtained, and the road condition detection model is trained, which can effectively ensure the training effect of the road condition detection model and improve the generalization ability of the road condition detection model.

[0127] In order to enable those skilled in the art to better understand the technical solutions provided by the present disclosure, the present disclosure provides a flow chart of a training method of a road condition detection model as shown in Figure 2 As shown in Figure 2 The method comprises the steps of:

[0128] S201, acquiring a first data set and a second data set.

[0129] The first data set comprises road condition image samples labeled with real label information, and the second data set comprises road condition image samples without real label information.

[0130] S202, inputting the first data set into a first model for pre-training to obtain a teacher model.

[0131] The first model can be a FastRCNN model.

[0132] S203, determining the class label weight of the road condition detection model based on the class feature information output by the intermediate layer of the teacher model in the pre-training process.

[0133] The class similarity information is determined by the inter-class distance, and the similarity of classes with a long distance is low, and the similarity of classes with a short distance is high. The inter-class distance is obtained based on the class feature information of the penultimate layer of the network in the full-supervised training stage.

[0134] S204, determining the first loss between the first prediction result of the teacher model based on the first data set and the real label information.

[0135] S205, inputting the second data set into the teacher model to obtain a first pseudo label after screening the prediction result of the teacher model.

[0136] The screening comprises non-maximum suppression on the position information, and judging whether the maximum probability class of the class prediction value is greater than a preset threshold.

[0137] S206, determining a second pseudo label according to the first pseudo label and a category label weight.

[0138] S207, performing synchronous data enhancement processing on the second data set and the second pseudo label to obtain a third data set and a third pseudo label.

[0139] The data enhancement processing can include image-level color jittering, translation, rotation, clipping, small-amplitude translation, rotation, clipping, and the like.

[0140] S208, inputting the third data set into the teacher model to obtain a second prediction result.

[0141] S209, determining a second loss based on the second prediction result and the third pseudo label.

[0142] S210, updating parameters of the teacher model based on the first loss and the second loss to obtain a trained road condition detection model.

[0143] The above training method improves label smoothing, uses soft label weights based on category similarity information to replace the average distribution of soft label weights, and reduces the harm of false pseudo labels to training. While enhancing the robustness of the model, it reduces the impact of false pseudo labels on subsequent training. It can be understood that since the effect of the soft label is only calculated when calculating the loss in the second stage of training, the category label weight can be added to the pseudo label before calculating the loss.

[0144] For example, after step S207, the final pseudo label used for calculating the loss can be obtained according to the pseudo label after data enhancement and the category label weight.

[0145] It can be understood that the execution subject of the road condition detection model training method can be a vehicle terminal or a server. The server can send the trained road condition detection model to the vehicle terminal after training to enable the vehicle terminal to perform road condition detection according to the road condition detection model and control the vehicle according to the road condition detection result to ensure the driving safety of the vehicle.

[0146] Based on the same inventive concept, Figure 3 is a block diagram of a vehicle control device 30 according to an example embodiment, as shown in Figure 3 The device 30 includes:

[0147] The acquisition module 31 is configured to acquire a road condition image during vehicle driving.

[0148] a detection module 32 configured to input the traffic image into a traffic detection model to obtain a traffic detection result, wherein the traffic detection model is pre-trained through semi-supervised training based on category label weights, and the traffic detection result includes location information and category information of each target object in the traffic image, wherein the category label weights are used to represent category similarity;

[0149] The control module 33 is used to control the vehicle according to the road condition detection result.

[0150] Optionally, the data set of the road condition detection model includes a first data set and a second data set, the first data set includes road condition image samples annotated with real label information, and the second data set includes road condition image samples not annotated with real label information, and the device 30 further includes:

[0151] An input module, configured to input the first data set into a road condition detection model for pre-training, and determine a first loss between a first predicted label for the first data set output by the road condition detection model after pre-training and the true label information;

[0152] A weight determination module, configured to determine the category label weight of the road condition detection model based on the category feature information output by the intermediate layer of the road condition detection model during the pre-training process;

[0153] A generating module, configured to generate a third data set and pseudo labels corresponding to the third data set according to the category label weights and the second data set;

[0154] A training module is used to retrain the road condition detection model based on the third data set and the pseudo labels to obtain the trained road condition detection model.

[0155] Optionally, the weight determination module is specifically configured to:

[0156] Calculating the inter-class distances between class labels based on the class feature information;

[0157] Determining category similarity information between categories based on the inter-category distance;

[0158] The category label weight is determined according to the category similarity information.

[0159] Optionally, the generating module is specifically configured to:

[0160] Inputting the second data set into the trained road condition detection model to obtain a first pseudo label;

[0161] Determining a second pseudo label based on the category label weight;

[0162] perform a data augmentation operation on the second data set and the second pseudo label to obtain an augmented third data set and a third pseudo label, the third pseudo label being a pseudo label corresponding to the third data set.

[0163] Optionally, the generating module is further configured to:

[0164] input the second data set into the trained road condition detection model to obtain a first prediction value corresponding to the second data set, the first prediction value including a position prediction value and a category prediction value, the category prediction value including a category probability of each category to which a corresponding target object belongs;

[0165] perform a non-maximum suppression operation on the position prediction value to obtain a first target object with a high position confidence;

[0166] determine a category confidence of each first target object according to a category probability of each category in a category prediction value corresponding to the first target object;

[0167] determine a second target object in each first target object with a category confidence greater than a preset confidence threshold;

[0168] determine position information according to a position prediction value corresponding to each second target object and determine category information according to a category probability of each category in a corresponding category prediction value to obtain the first pseudo label.

[0169] Optionally, the training module is specifically configured to:

[0170] input the third data set into the first road condition detection model to obtain a second prediction label;

[0171] calculate a second loss according to the second prediction label and the third pseudo label;

[0172] update parameters of the road condition detection model based on the first loss and the second loss to obtain the trained road condition detection model.

[0173] Optionally, the training module is further configured to:

[0174] perform a normalization operation on the first loss and the second loss respectively to obtain a first normalized loss and a second normalized loss;

[0175] update the parameters of the road condition detection model according to a sum of the first normalized loss and the second normalized loss.

[0176] Optionally, the generating module is further configured to:

[0177] performing at least one of color conversion, geometric transformation, scaling, box-level transformation, and padding of a gray box on the images in the second dataset; and

[0178] In a case where a pixel position of the image in the second dataset is changed, the position information in the second pseudo label corresponding to the image in the second dataset is changed correspondingly.

[0179] As to the apparatus in the above embodiments, the specific manners in which the respective modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0180] The present disclosure also provides a computer readable storage medium, having stored thereon computer program instructions, which when executed by a processor, implement the steps of the vehicle control method provided by the present disclosure.

[0181] Figure 4 is a block diagram of a vehicle control apparatus 400 according to an exemplary embodiment. For example, the apparatus 400 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0182] Referring to Figure 4 , the apparatus 400 can include one or more of the following components: a processing component 402, a memory 404, a power supply component 406, a multimedia component 408, an audio component 410, an input / output (I / O) interface 412, a sensor component 414, and a communication component 416.

[0183] The processing component 402 usually controls overall operations of the apparatus 400, such as operations associated with displaying, making phone calls, data communications, camera operations, and recording operations. The processing component 402 can include one or more processors 420 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 402 can include one or more modules to facilitate interaction between the processing component 402 and other components. For example, the processing component 402 can include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0184] The memory 404 is configured to store various types of data to support the operation of the device 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phonebook data, messages, pictures, videos, and the like. The memory 404 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0185] The power component 406 provides power to the various components of the device 400. The power component 406 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 400.

[0186] The multimedia component 408 includes a screen providing an output interface between the device 400 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 400 is in an operation mode, such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0187] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC) configured to receive external audio signals when the device 400 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0188] The I / O interface 412 provides an interface between the processing component 402 and peripheral interface modules, which can be a keyboard, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0189] The sensor component 414 includes one or more sensors to provide the status of various aspects of the device 400. For example, the sensor component 414 can detect the open / closed status of the device 400, the relative positioning of components of the device 400, such as a display and a keypad of the device 400, changes in position of the device 400 or a component of the device 400, the presence or absence of user contact with the device 400, the orientation or acceleration / deceleration / g-force and the temperature of the device 400. The sensor component 414 can include proximity sensor(s) configured to detect the presence of objects in a proximity without any physical contact. The sensor component 414 can also include a light sensor(s) such as a CMOS or CCD image sensor for use in imaging applications. In some embodiments, the sensor component 414 can also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor or a temperature sensor.

[0190] The communication component 416 is configured to facilitate wired or wireless communication between the device 400 and another device. The device 400 can access a wireless network based on a corresponding communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-WideBand (UWB) technology, Bluetooth (BT) technology and other technologies.

[0191] In an exemplary embodiment, the device 400 can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic elements to perform the vehicle control method described above.

[0192] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 404 including instructions, is also provided, which can be executed by the processor 420 of the device 400 to perform the vehicle control method described above. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0193] In another exemplary embodiment, a computer program product is also provided, which contains a computer program executable by a programmable apparatus, the computer program having code segments for performing the vehicle control method described above when executed by the programmable apparatus.

[0194] Figure 5 is a block diagram of a vehicle control apparatus 500 according to an exemplary embodiment. For example, the apparatus 500 can be provided as a server. Referring to Figure 5 , the apparatus 500 includes a processing component 522, which further includes one or more processors, and a memory resource represented by a memory 532, for storing instructions executable by the processing component 522, such as an application program. The application program stored in the memory 532 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 522 is configured to execute the instructions to perform the vehicle control method described above.

[0195] The apparatus 500 can also include a power supply component 526 configured to perform power management of the apparatus 500, a wired or wireless network interface 550 configured to connect the apparatus 500 to a network, and an input / output (I / O) interface 558. The apparatus 500 can operate based on an operating system stored in the memory 532, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or the like.

[0196] Referring to Figure 6 , Figure 6 is a functional block diagram of a vehicle 600 provided by an embodiment of the present application.

[0197] Figure 6 is a functional block diagram of a vehicle 600 according to an exemplary embodiment. The vehicle 600 can be configured in a fully or partially autonomous driving mode. For example, the vehicle 600 can obtain environmental information around it through a perception system 620, and obtain an autonomous driving strategy based on the analysis of the surrounding environmental information to achieve full autonomous driving, or present the analysis result to the user to achieve partial autonomous driving.

[0198] The vehicle 600 can include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. Optionally, the vehicle 600 can include more or fewer subsystems, and each subsystem can include multiple components. Additionally, each subsystem and component of the vehicle 600 can be interconnected by wired or wireless means.

[0199] In some embodiments, the infotainment system 610 can include a communication system 611, an entertainment system 612, and a navigation system 613.

[0200] The communication system 611 can include a wireless communication system that can wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE. Or 5G cellular communication. The wireless communication system can communicate with a wireless local area network (WLAN) using WiFi. In some embodiments, the wireless communication system can communicate directly with a device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system can include one or more dedicated short range communications (DSRC) devices that can include public and / or private data communication between vehicles and / or roadside stations.

[0201] The entertainment system 612 can include a display device, a microphone, and a sound system, based on which a user can listen to the radio, play music in the vehicle, or connect a mobile phone to the vehicle and implement mobile phone projection on the display device. The display device can be touch-enabled, and the user can operate it by touching the screen.

[0202] In some cases, the user's voice signal can be obtained through the microphone, and some control of the vehicle 600 by the user, such as adjusting the temperature in the vehicle, can be implemented based on the analysis of the user's voice signal. In other cases, music can be played to the user through the sound system.

[0203] The navigation system 613 can include a map service provided by a map provider, thereby providing the vehicle 600 with navigation of the driving route. The navigation system 613 can be used in conjunction with the global positioning system 621 and the inertial measurement unit 622 of the vehicle. The map service provided by the map provider can be a two-dimensional map or a high-definition map.

[0204] The perception system 620 can include several types of sensors that sense information about the environment surrounding the vehicle 600. For example, the perception system 620 can include a global positioning system 621 (which can be a GPS system, a Beidou system, or other positioning system), an inertial measurement unit (IMU) 622, a lidar 623, a millimeter wave radar 624, an ultrasonic radar 625, and a camera 626. The perception system 620 can also include sensors that monitor internal systems of the vehicle 600 (e.g., in-cabin air quality monitors, fuel gauges, oil temperature gauges, etc.). Sensor data from one or more of these sensors can be used to detect objects and their respective characteristics (location, shape, orientation, velocity, etc.). Such detection and identification are key functions for the safe operation of the vehicle 600.

[0205] The global positioning system 621 is used to estimate the geographic position of the vehicle 600.

[0206] The inertial measurement unit 622 is used to sense changes in the pose of the vehicle 600 based on inertial acceleration. In some embodiments, the inertial measurement unit 622 can be a combination of an accelerometer and a gyroscope.

[0207] The lidar 623 uses laser light to sense objects in the environment in which the vehicle 600 is located. In some embodiments, the lidar 623 can include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0208] The millimeter wave radar 624 uses radio signals to sense objects within the surrounding environment of the vehicle 600. In some embodiments, in addition to sensing objects, the millimeter wave radar 624 can also be used to sense the speed and / or heading of the objects.

[0209] The ultrasonic radar 625 can use ultrasonic signals to sense objects around the vehicle 600.

[0210] The camera 626 is used to capture image information of the surrounding environment of the vehicle 600. The camera 626 can include monocular cameras, binocular cameras, structured light cameras, and panoramic cameras, etc., and the image information acquired by the camera 626 can include still images or video stream information.

[0211] The decision control system 630 includes a computing system 631 that makes analytical decisions based on the information acquired by the perception system 620, and the decision control system 630 also includes a vehicle controller 632 that controls the power system of the vehicle 600, as well as a steering system 633, a throttle 634, and a braking system 635 for controlling the vehicle 600;

[0212] The computing system 631 can operate to process and analyze various information acquired by the perception system 620 in order to identify targets, objects, and / or features in the environment surrounding the vehicle 600. Targets can include pedestrians or animals, and objects and / or features can include traffic signals, road boundaries, and obstacles. The computing system 631 can use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and / or the like. In some embodiments, the computing system 631 can be used to map the environment, track objects, estimate the velocity of objects, and / or the like. The computing system 631 can analyze the acquired information and derive a control strategy for the vehicle.

[0213] The vehicle controller 632 can be used to coordinate the control of the power battery and the engine 641 of the vehicle in order to improve the power performance of the vehicle 600.

[0214] The steering system 633 can be used to adjust the heading direction of the vehicle 600. For example, the steering system 633 can be a steering wheel system in one embodiment.

[0215] The throttle 634 can be used to control the operating speed of the engine 641 and, in turn, the speed of the vehicle 600.

[0216] The braking system 635 can be used to control the deceleration of the vehicle 600. The braking system 635 can use friction to slow the wheels 644. In some embodiments, the braking system 635 can convert the kinetic energy of the wheels 644 into electrical current. The braking system 635 can also take other forms to slow the wheels 644 and, in turn, control the speed of the vehicle 600.

[0217] The drive system 640 can include components that provide motive power for the vehicle 600. In one embodiment, the drive system 640 can include the engine 641, the energy source 642, the transmission system 643, and the wheels 644. The engine 641 can be an internal combustion engine, an electric motor, an air compression engine, or other types of engines in combination, such as a hybrid engine that includes a gasoline engine and an electric motor, a hybrid engine that includes an internal combustion engine and an air compression engine. The engine 641 converts the energy source 642 into mechanical energy.

[0218] Examples of the energy source 642 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source 642 can also provide energy for other systems of the vehicle 600.

[0219] The drivetrain 643 can transmit mechanical power from the engine 641 to the wheels 644. The drivetrain 643 can include a transmission, a differential, and drive shafts. In one embodiment, the drivetrain 643 can also include other devices such as a clutch. The drive shafts can include one or more shafts that can be coupled to one or more wheels 644.

[0220] Some or all of the functionality of the vehicle 600 is controlled by a computing platform 650. The computing platform 650 can include at least one processor 651 that can execute instructions 653 stored in a non-transitory computer readable medium such as a memory 652. In some embodiments, the computing platform 650 can also be a plurality of computing devices that control individual components or subsystems of the vehicle 600 in a distributed manner.

[0221] The processor 651 can be any conventional processor such as commercially available CPUs. Alternatively, the processor 651 can include a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof. Although the processor 651 is illustrated as a single processor, it should be understood that the processor 651 can include one or more processors in communication with each other or remote from each other. Figure 6 Although the processor, memory, and other elements of the computer are functionally illustrated as being within the same block, it should be understood that the processor, computer, or memory can actually include a plurality of processors, computers, or memories that can or can not be stored within the same physical housing. For example, the memory can be a hard drive or other storage medium located in a housing different from that of the computer. Accordingly, references to the processor or computer will be understood to include references to a collection of processors or computers or memories that can or can not operate in parallel. Rather than using a single processor to perform the steps described herein, some components such as the steering assembly and the deceleration assembly can each have their own processor that only performs calculations related to the functionality specific to the component.

[0222] In the embodiments of the present application, the processor 651 can acquire a road condition image in a vehicle driving process; input the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being obtained through semi-supervised training based on a category label weight, the road condition detection result including position information and category information of each target object in the road condition image, and the category label weight being used to represent category similarity; and control the vehicle 600 according to the road condition detection result.

[0223] In various aspects described herein, the processor 651 can be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0224] In some embodiments, the memory 652 can include instructions 653 (e.g., program logic) that can be executed by the processor 651 to perform various functions of the vehicle 600. The memory 652 can also include additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the infotainment system 610, the perception system 620, the decision control system 630, the drive system 640.

[0225] In addition to the instructions 653, the memory 652 can store data, such as road maps, route information, the vehicle's location, direction, speed, and other such vehicle data, as well as other information. Such information can be used by the vehicle 600 and the computing platform 650 during operation of the vehicle 600 in autonomous, semi-autonomous, and / or manual modes.

[0226] The computing platform 650 can control the functions of the vehicle 600 based on inputs received from various subsystems, such as the drive system 640, the perception system 620, and the decision control system 630. For example, the computing platform 650 can utilize inputs from the decision control system 630 to control the steering system 633 to avoid an obstacle detected by the perception system 620. In some embodiments, the computing platform 650 can be operable to provide control over many aspects of the vehicle 600 and its subsystems.

[0227] Optionally, one or more of the components described above can be installed separately from or in association with the vehicle 600. For example, the memory 652 can exist partially or entirely separately from the vehicle 600. The components described above can be communicatively coupled together in a wired and / or wireless manner.

[0228] Optionally, the above components are just an example, in actual applications, components in each module described above can be added or deleted according to actual needs, Figure 6 It should not be understood as a limitation to the embodiments of the present disclosure.

[0229] An autonomous vehicle traveling on a roadway, such as the vehicle 600 above, can identify objects within its surroundings to determine an adjustment to a current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently and, based on respective characteristics of the object, such as its current speed, acceleration, spacing from the vehicle, etc., can be used to determine a speed at which the autonomous vehicle is to adjust.

[0230] Optionally, the vehicle 600 or a perception and computing device associated with the vehicle 600 (e.g., the computing system 631, the computing platform 650) can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of the others, so all of the identified objects can also be considered together to predict the behavior of a single identified object. The vehicle 600 can adjust its speed based on the predicted behavior of the identified object. In other words, the autonomous vehicle can determine what steady state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the object. Other factors can also be considered in determining the speed of the vehicle 600 during this process, such as the lateral position of the vehicle 600 in the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0231] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 600 to cause the autonomous vehicle to follow a given trajectory and / or maintain a safe lateral and longitudinal distance from objects in the vicinity of the autonomous vehicle (e.g., a car in the adjacent lane on the road).

[0232] The vehicle 600 described above can be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawnmower, an amusement park vehicle, a construction device, a trolley, a golf cart, a train, etc., and the embodiments of the present disclosure are not particularly limited.

[0233] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the present disclosure. It is intended that the present disclosure cover any and all variations of the present disclosure including combinations of features falling within the general scope of the disclosure. The specification and examples given are intended as illustrative only and not restrictive of the true scope and spirit of the present disclosure, which is set forth in the following claims.

[0234] It should be understood that the present disclosure is not limited to the precise structures herein described and illustrated and that various modifications and changes can be made without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A vehicle control method characterized by, The method comprises: acquiring a road condition image in a vehicle driving process; inputting the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being obtained through pre-training based on a class label weight, the road condition detection result comprising position information and class information of each target object in the road condition image, and the class label weight being used to represent class similarity; controlling the vehicle according to the road condition detection result; a data set of the road condition detection model comprises a first data set and a second data set, the first data set comprising road condition image samples labeled with real label information, and the second data set comprising road condition image samples without real label information, and the training of the road condition detection model comprising: inputting the first data set into the road condition detection model for pre-training, and determining a first loss between first predicted labels of the first data set output by the road condition detection model after pre-training and the real label information; determining a class label weight of the road condition detection model based on class feature information output by an intermediate layer of the road condition detection model in the pre-training process; generating a third data set and pseudo labels corresponding to the third data set according to the class label weight and the second data set; re-training the road condition detection model according to the third data set and the pseudo labels to obtain a trained road condition detection model.

2. The method of claim 1, wherein, The determination of the class label weight of the road condition detection model based on the class feature information output by the intermediate layer of the road condition detection model in the pre-training process comprises: calculating an inter-class distance between each class label according to the class feature information; determining class similarity information between each class according to the inter-class distance; determining the class label weight according to the class similarity information.

3. The method of claim 1, wherein, The generation of the third data set and the pseudo labels corresponding to the third data set according to the class label weight and the second data set comprises: inputting the second data set into the trained road condition detection model to obtain first pseudo labels; determining second pseudo labels based on the class label weight; performing a data enhancement operation on the second data set and the second pseudo labels to obtain an enhanced third data set and third pseudo labels, the third pseudo labels being pseudo labels corresponding to the third data set.

4. The method of claim 3, wherein, The inputting of the second data set into the trained road condition detection model to obtain first pseudo labels comprises: inputting the second data set into the trained road condition detection model to obtain first prediction values corresponding to the second data set, the first prediction values comprising position prediction values and class prediction values, and the class prediction values comprising class probabilities of each class to which a corresponding target object belongs; performing a non-maximum suppression operation on the position prediction values to obtain first target objects with high position confidence; determining class confidence of each first target object according to the class probability of each class in the class prediction value corresponding to each first target object; determining second target objects in each first target object with class confidence greater than a preset confidence threshold. Determine position information according to the position prediction value corresponding to each second target object, and determine category information according to the category probability belonging to each category in the corresponding category prediction value, to obtain the first pseudo label.

5. The method of claim 3, wherein, The retraining of the road condition detection model according to the third data set and the pseudo label includes: inputting the third data set into the road condition detection model to obtain a second prediction label; calculating a second loss according to the second prediction label and the third pseudo label; updating parameters of the road condition detection model based on the first loss and the second loss to obtain the trained road condition detection model.

6. The method of claim 5, wherein, The updating of the parameters of the road condition detection model based on the first loss and the second loss includes: performing normalization operations on the first loss and the second loss respectively to obtain a first normalized loss and a second normalized loss; updating the parameters of the road condition detection model according to the sum of the first normalized loss and the second normalized loss.

7. The method according to any one of claims 3-6, characterized in that, The data enhancement operation on the second data set and the second pseudo label to obtain an enhanced third data set and a third pseudo label includes: performing at least one of the data enhancement operations of color conversion, geometric transformation, scaling, box-level transformation and gray frame filling on the images in the second data set; and in the case that the pixel position of the image in the second data set changes, changing the position information in the second pseudo label corresponding to the image in the second data set accordingly.

8. A vehicle control device characterized by comprising: including: an acquisition module configured to acquire a road condition image in a vehicle driving process; a detection module configured to input the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being obtained by pre-training based on a category label weight, and the road condition detection result including position information and category information of each target object in the road condition image, the category label weight being used to represent category similarity; a control module configured to control the vehicle according to the road condition detection result. The data set of the road condition detection model includes a first data set and a second data set, the first data set includes road condition image samples labeled with real label information, and the second data set includes road condition image samples without real label information, and the vehicle control device further includes: an input module configured to input the first data set into a road condition detection model for pre-training, and determine a first loss between a first prediction label output by the road condition detection model for the first data set and the real label information after pre-training is completed; a weight determination module configured to determine a category label weight of the road condition detection model based on category feature information output by an intermediate layer of the road condition detection model during the pre-training process; a generation module configured to generate a third data set and a pseudo label corresponding to the third data set according to the category label weight and the second data set; The training module is configured to retrain the road condition detection model according to the third data set and the pseudo label, and obtain the trained road condition detection model.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The program instructions, when executed by a processor, implement the steps of the method of any one of claims 1-7.

10. A vehicle characterized by comprising: The vehicle comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: obtain a road condition image during vehicle driving; input the road condition image into a road condition detection model to obtain a road condition detection result, the road condition detection model being pre-trained based on a class label weight, the road condition detection result comprising position information and class information of each target object in the road condition image, the class label weight being used to represent class similarity; control the vehicle according to the road condition detection result; a data set of the road condition detection model comprises a first data set and a second data set, the first data set comprising road condition image samples labeled with real label information, the second data set comprising road condition image samples without real label information, and training of the road condition detection model comprising: inputting the first data set into the road condition detection model for pre-training, and determining a first loss between first predicted labels output by the road condition detection model for the first data set after pre-training and the real label information; determining a class label weight of the road condition detection model based on class feature information output by an intermediate layer of the road condition detection model during the pre-training process; generating a third data set and a pseudo label corresponding to the third data set according to the class label weight and the second data set; retraining the road condition detection model according to the third data set and the pseudo label, and obtaining the trained road condition detection model.

Images