A training method and device of a car-machine user interface element detection model
Patent Information
- Application Number
- CN202611015595.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-08
- Publication Date
- 2026-08-18
AI Technical Summary
由于车机界面与非车机界面的用户界面元素外观存在明显的差异（如尺寸、图标样式等），而现有利用非车机用户界面元素数据集训练的检测模型提取的特征无法表征车机用户界面元素，所以导致车机界面图像的用户界面元素检测效果很差
[0039] The present invention provides a training method and apparatus for a vehicle-mounted infotainment (VMI) element detection model, which involves: acquiring VMI images; adding corresponding annotation information to each VMI element in the VMI image; the annotation information including at least the position information of the corresponding VMI element; cropping the VMI image into multiple image blocks to obtain a target dataset composed of the image blocks; training a non-VMI element detection model based on transfer learning and the target dataset to obtain a VMI element detection model; the non-VMI element detection model being pre-trained using a non-VMI image dataset. In this scheme, transfer learning and a small number of samples are used to further train the pre-obtained non-VMI element detection model to obtain a VMI element detection model, thereby achieving the goal of training a VMI element detection model suitable for VMI images using a small VMI element dataset.
Smart Images
Figure CN122597950A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine learning technology, and specifically to a training method and apparatus for a vehicle infotainment user interface element detection model. Background Technology
[0002] The task of in-vehicle infotainment (IVA) user interface element detection is to locate the positions of all IVA elements within the IVA interface, which has significant application requirements in automated testing of in-vehicle systems. Traditional image processing methods use manually extracted features, such as contour edges, shape, and color, to detect IVA elements. These manually extracted features are quite simple and can only be used to detect simple IVA elements; for complex IVA elements, the false negative and false positive rates are high. Therefore, current research focuses more on deep learning-based IVA element detection methods. Deep learning-based IVA element detection algorithms can accurately extract more abstract features of IVA elements and do not produce strong feature responses in background areas of complex patterns.
[0003] However, existing deep learning-based user interface element detection algorithms are all trained using non-vehicle infotainment system (VMS) user interface element datasets, such as mobile phone VMS user interface element datasets. The trained detection models are mainly used to detect VMS user interface elements in mobile phone interfaces. Since there are significant differences in the appearance of VMS user interface elements between VMS and non-VMS interfaces (such as size and icon style), the features extracted by existing detection models trained using non-VMS user interface element datasets cannot represent VMS user interface elements, resulting in poor detection performance for VMS user interface elements in VMS images. Furthermore, existing non-VMS user interface element detection models are trained using a large number of non-VMS interface image samples from these datasets. Similarly, if a VMS user interface element detection model is specifically trained using a VMS user interface element dataset, it would also require a dataset containing a large number of VMS interface image samples. Therefore, collecting such a dataset would be time-consuming, leading to a long training time.
[0004] In summary, how to train a vehicle infotainment user interface element detection model suitable for vehicle infotainment interface images using a small dataset of vehicle infotainment user interface elements is an urgent problem to be solved. Summary of the Invention
[0005] In view of this, embodiments of the present invention provide a training method and apparatus for a vehicle infotainment user interface element detection model, so as to achieve the purpose of training a vehicle infotainment user interface element detection model suitable for vehicle infotainment interface images using a small amount of vehicle infotainment user interface element dataset.
[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:
[0007] The first aspect of this invention discloses a training method for a vehicle infotainment user interface element detection model, the method comprising:
[0008] Capture images of the vehicle's infotainment system interface;
[0009] Add corresponding annotation information to each vehicle user interface element in the vehicle interface image; the annotation information includes at least: the position information of the corresponding vehicle user interface element;
[0010] The vehicle interface image is cropped into multiple image blocks to obtain a target dataset composed of each image block;
[0011] A non-vehicle infotainment user interface element detection model is trained based on transfer learning technology and the target dataset to obtain a vehicle infotainment user interface element detection model; the non-vehicle infotainment user interface element detection model is pre-trained using a non-vehicle infotainment interface image dataset.
[0012] Preferably, cropping the vehicle interface image into multiple image patches to obtain a target dataset composed of each of the image patches includes:
[0013] Based on a preset image size and a preset overlap size, the vehicle interface image is cropped to obtain multiple image blocks whose image size is equal to the preset image size and whose overlap area with adjacent image blocks is equal to the preset overlap size.
[0014] Preferably, after cropping the vehicle interface image into multiple image blocks, the method further includes:
[0015] For each of the vehicle infotainment user interface elements, based on the position information in the corresponding annotation information, it is determined whether the vehicle infotainment user interface element is completely located in at least one of the image blocks;
[0016] If so, the position information is transformed to obtain target position information; the target position information is used to indicate the position of the vehicle user interface element in the image block;
[0017] If not, then delete the annotation information corresponding to the vehicle user interface element from the image block containing the annotation information corresponding to the vehicle user interface element.
[0018] Preferably, the training process of the non-vehicle user interface element detection model includes:
[0019] Obtain an open-source dataset of non-vehicle infotainment interface images;
[0020] Construct a non-vehicle infotainment user interface element detection model to be trained;
[0021] Using the stochastic gradient descent algorithm and the non-vehicle interface image dataset, the non-vehicle interface element detection model to be trained is iteratively trained until a trained non-vehicle interface element detection model is obtained.
[0022] Preferably, the step of training a non-vehicle infotainment user interface element detection model based on transfer learning technology and the target dataset to obtain a vehicle infotainment user interface element detection model includes:
[0023] The pre-training weights of the non-vehicle user interface element detection model are fixed so that the pre-training weights remain unchanged during the training process;
[0024] Using the target dataset and the stochastic gradient descent algorithm, the detection head of the non-vehicle user interface element detection model is iteratively trained until the vehicle user interface element detection model is obtained.
[0025] A second aspect of this invention discloses a training apparatus for a vehicle infotainment user interface element detection model, the apparatus comprising:
[0026] The acquisition unit is used to acquire images from the vehicle's infotainment interface.
[0027] The annotation unit is used to add corresponding annotation information to each vehicle user interface element in the vehicle interface image; the annotation information includes at least: the position information of the corresponding vehicle user interface element;
[0028] The cropping unit is used to crop the vehicle interface image into multiple image blocks to obtain a target dataset composed of each image block;
[0029] The transfer learning unit is used to train a non-vehicle infotainment user interface element detection model based on transfer learning technology and the target dataset to obtain a vehicle infotainment user interface element detection model; the non-vehicle infotainment user interface element detection model is pre-trained using a non-vehicle infotainment interface image dataset.
[0030] Preferably, the cutting unit is specifically used for:
[0031] Based on a preset image size and a preset overlap size, the vehicle interface image is cropped to obtain multiple image blocks whose image size is equal to the preset image size and whose overlap area with adjacent image blocks is equal to the preset overlap size.
[0032] Preferably, the device further includes:
[0033] The transformation unit is configured to, after cropping the vehicle interface image into multiple image blocks, determine, for each vehicle interface element, whether the vehicle interface element is completely located in at least one image block based on the position information in the corresponding annotation information; if yes, transform the position information to obtain target position information; the target position information is used to indicate the position of the vehicle interface element in the image block; if no, delete the annotation information corresponding to the vehicle interface element from the image block containing the annotation information corresponding to the vehicle interface element.
[0034] Preferably, the device further includes:
[0035] The training unit is used to acquire an open-source non-vehicle interface image dataset; construct a non-vehicle interface element detection model to be trained; and iteratively train the non-vehicle interface element detection model to be trained using the stochastic gradient descent algorithm and the non-vehicle interface image dataset until a trained non-vehicle interface element detection model is obtained.
[0036] Preferably, the transfer learning unit is specifically used for:
[0037] The pre-training weights of the non-vehicle user interface element detection model are fixed so that the pre-training weights remain unchanged during the training process;
[0038] Using the target dataset and the stochastic gradient descent algorithm, the detection head of the non-vehicle user interface element detection model is iteratively trained until the vehicle user interface element detection model is obtained.
[0039] The present invention provides a training method and apparatus for a vehicle-mounted infotainment (VMI) element detection model, which involves: acquiring VMI images; adding corresponding annotation information to each VMI element in the VMI image; the annotation information including at least the position information of the corresponding VMI element; cropping the VMI image into multiple image blocks to obtain a target dataset composed of the image blocks; training a non-VMI element detection model based on transfer learning and the target dataset to obtain a VMI element detection model; the non-VMI element detection model being pre-trained using a non-VMI image dataset. In this scheme, transfer learning and a small number of samples are used to further train the pre-obtained non-VMI element detection model to obtain a VMI element detection model, thereby achieving the goal of training a VMI element detection model suitable for VMI images using a small VMI element dataset. Attached Figure Description
[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0041] Figure 1 This is a flowchart of a training method for a vehicle infotainment user interface element detection model disclosed in an embodiment of the present invention;
[0042] Figure 2 This is a comparison diagram of the detection effect of vehicle infotainment user interface elements disclosed in an embodiment of the present invention;
[0043] Figure 3 This is a structural diagram of a training device for a vehicle infotainment user interface element detection model disclosed in an embodiment of the present invention. Detailed Implementation
[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0045] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0046] As the background technology shows, how to train a vehicle infotainment user interface element detection model suitable for vehicle infotainment interface images using a small dataset of vehicle infotainment user interface elements is an urgent problem to be solved.
[0047] Therefore, this invention discloses a training method and apparatus for a vehicle-mounted infotainment user interface element detection model. In this scheme, transfer learning technology and a small number of samples are used to further train a pre-obtained non-vehicle-mounted infotainment user interface element detection model to obtain a vehicle-mounted infotainment user interface element detection model, thereby achieving the goal of training a vehicle-mounted infotainment user interface element detection model suitable for vehicle-mounted interface images using a small number of vehicle-mounted infotainment user interface element datasets.
[0048] like Figure 1 The diagram shown is a flowchart of a training method for a vehicle infotainment user interface element detection model disclosed in an embodiment of the present invention, including the following steps:
[0049] Step S101: Acquire images of the vehicle's infotainment system interface.
[0050] In step S101, the system connects to the vehicle infotainment system via an interface, and captures and saves an image of the entire screen area of the vehicle infotainment system as the vehicle infotainment interface image using a screenshot command. The vehicle infotainment interface image should contain a comprehensive range of vehicle UI (User Interface) elements, and the number of different categories of vehicle UI elements should be balanced.
[0051] Step S102: Add corresponding annotation information to each vehicle user interface element in the vehicle interface image.
[0052] The annotation information includes at least the location information of the corresponding vehicle infotainment user interface element, and may also include the category information of the vehicle infotainment user interface element, such as ICON.
[0053] For example, the location information can be in the form of [x,y,width,height]. Assuming the top left corner of the vehicle interface image is at coordinates 0,0, x is the x-axis coordinate of the top left corner of a vehicle interface element, y is the y-axis coordinate of the top left corner of a vehicle interface element, width is the width of a vehicle interface element, and height is the height of a vehicle interface element.
[0054] In step S102, each vehicle user interface element in the vehicle interface image is identified according to the predefined vehicle user interface elements to be detected, and corresponding annotation information is added.
[0055] It should be noted that, since the application scenario of the vehicle infotainment user interface element detection model trained in this application is: the vehicle infotainment user interface element detection model automatically acquires clickable and swipeable vehicle infotainment user interface elements in an automated test scenario, in this invention, vehicle infotainment user interface elements in the vehicle interface image that change after being clicked or swiped are defined as vehicle infotainment user interface elements to be detected. All vehicle infotainment user interface elements to be detected are labeled as a category ICON. That is to say, the category information of vehicle infotainment user interface elements in each labeling information added in the embodiments of this invention is the same.
[0056] In other embodiments, multiple categories of in-vehicle user interface elements can be defined according to specific application scenarios. Accordingly, each annotation will contain category information corresponding to that category of in-vehicle user interface element.
[0057] Step S103: Crop the vehicle interface image into multiple image blocks to obtain the target dataset composed of each image block.
[0058] In the specific implementation of step S103, the vehicle interface image is cropped based on the preset image size and the preset overlap size to obtain multiple image blocks with an image size equal to the preset image size and an overlap area between adjacent image blocks equal to the preset overlap size.
[0059] It should be noted that the purpose of the vehicle interface image is to subsequently train the non-vehicle interface element detection model. However, since the size of the vehicle interface image (e.g., 2880x1800) is usually large compared to the input image size (e.g., 640x640) set in the non-vehicle interface element detection model, and since the vehicle interface image contains many small-sized vehicle interface elements, directly downsampling the uncropped vehicle interface image to 640x640 and inputting it into the non-vehicle interface element detection model for training would result in significant loss of information about the small-sized vehicle interface elements. Therefore, cropping is necessary, and the cropped image patches are used as the target dataset for subsequent training of the non-vehicle interface element detection model.
[0060] It is understandable that the image block size should be equal to the input image size set in the non-vehicle user interface element detection model. In other words, the preset image size should be equal to the input image size set in the non-vehicle user interface element detection model.
[0061] The purpose of cropping according to the preset overlap size is to ensure that the vehicle's user interface elements at the cutting boundary do not lose information.
[0062] Generally, a 25% overlap (i.e., the preset overlap size) is maintained at the cropping boundary. This means that each 640×640 image block overlaps with its adjacent image blocks by 25% in both the horizontal and vertical directions. In other words, there is a 640×(1-25%)=480 pixel overlap between adjacent image blocks.
[0063] In one embodiment, after cropping the vehicle interface image into multiple image blocks, the method further includes:
[0064] For each vehicle infotainment user interface element, based on the position information in the corresponding annotation information, it is determined whether the vehicle infotainment user interface element is completely located in at least one image block; if so, the position information is transformed to obtain the target position information; the target position information is used to indicate the position of the vehicle infotainment user interface element in the image block; if not, the annotation information corresponding to the vehicle infotainment user interface element is deleted from the image block containing the annotation information corresponding to the vehicle infotainment user interface element.
[0065] It should be noted that the vehicle interface element may be located on the cropping boundary, which means that multiple image patches contain a part of the vehicle interface element. In this case, the annotation information corresponding to the vehicle interface element will be discarded. The reason is that the vehicle interface element is only partially in the cropped image patch. When the image patch containing the partial vehicle interface element is used to train the non-vehicle interface element detection model, it will be difficult to extract the features of the vehicle interface element.
[0066] It should also be noted that if the entire in-vehicle user interface element is located in an image block, the position information in the annotation information corresponding to this in-vehicle user interface element needs to be transformed. The reason is that the original position information represents the position of the in-vehicle user interface element in the in-vehicle interface image, which cannot reflect the position of the in-vehicle user interface element in the image block, so it needs to be transformed.
[0067] In the specific implementation, the method for determining whether a certain vehicle infotainment user interface element is completely located within at least one image block is as follows:
[0068] For each image block, if x>=x_offset, y>=y_offset, x+width<=x_offset+640, and y+height<=y_offset+640 are all true, then the vehicle infotainment user interface element is determined to be completely located within the image block.
[0069] Where x_offset and y_offset represent the coordinates of the top-left corner of the image block in the original vehicle infotainment interface image, and x, y, width, and height represent the horizontal and vertical coordinates, width, and height of the top-left corner of the vehicle infotainment user interface element in the original vehicle infotainment interface image, respectively.
[0070] Furthermore, after confirming that the vehicle infotainment user interface element is completely located within the image block, the method for transforming the position information of the vehicle infotainment user interface element to obtain the target position information is as follows:
[0071] Based on the position information of the vehicle's user interface element, x_trans=x–x_offset, y_trans=y–y_offset, width_trans=width, and height_trans=height, the target position information is obtained.
[0072] Among them, x_trans, y_trans, width_trans and height_trans constitute the target position information, where x_trans, y_trans, width_trans and height_trans represent the horizontal coordinate, vertical coordinate, width and height of the upper left corner of the vehicle's user interface element after transformation, respectively.
[0073] Step S104: Train a non-vehicle user interface element detection model based on transfer learning technology and the target dataset to obtain a vehicle user interface element detection model.
[0074] The non-vehicle infotainment user interface element detection model was pre-trained using a non-vehicle infotainment interface image dataset, as detailed below:
[0075] Obtain an open-source non-vehicle interface image dataset; construct a non-vehicle interface element detection model to be trained; use the stochastic gradient descent algorithm and the non-vehicle interface image dataset to iteratively train the non-vehicle interface element detection model to be trained until a fully trained non-vehicle interface element detection model is obtained.
[0076] For example, an open-source non-vehicle interface image dataset could be the Rico dataset (which contains a large number of mobile user interface elements), and correspondingly, a non-vehicle interface element detection model could be a mobile user interface element detection model.
[0077] Based on the definition of mobile user interface elements in the Rico dataset, which includes 27 categories of mobile user interface elements, since there is no need to focus on the specific category information of the vehicle-mounted user interface elements in this application, the category information in the annotation information of the Rico dataset is processed, and the category information of all 27 categories of mobile user interface elements is changed to the ICON category. At the same time, the image size in the Rico dataset is cropped to 640x640 to obtain the processed Rico dataset.
[0078] Furthermore, the vehicle user interface elements detected in this application belong to only one category, namely the ICON category. Therefore, a mobile user interface element detection model to be trained is constructed (using existing object detection networks, such as YOLOv8, RetinaNet, SSD, etc.), and the number of categories output by the detection head of the mobile user interface element detection model to be trained is set to 1, and the input image size of the mobile user interface element detection model to be trained is set to 640x640.
[0079] Finally, based on the SGD (Stochastic Gradient Descent) algorithm and the processed Rico dataset, the mobile user interface element detection model is trained until the loss function no longer decreases within a certain number of iterations or meets the preset number of iterations, at which point training is complete. In this invention, the number of iterations is generally 60-80.
[0080] It's important to note that the principle of transfer learning is as follows: In practical applications, the amount of image data collected for vehicle infotainment system (VMS) element detection tasks is very small. Retraining the VMS element detection model using a limited number of VMS element images will lead to overfitting. To obtain a VMS element detection model using only a small number of VMS element images, it's necessary to first train a non-VMS element detection model. For example, using a dataset of mobile phone user interface elements similar to VMS elements, we can train a mobile phone user interface element detection model, allowing the detection network to learn the common features of both VMS and mobile phone user interface elements.
[0081] Because there are differences in features between vehicle infotainment system (VMS) user interface elements and mobile phone user interface elements, and in order to retain common features, reduce training parameters, and avoid overfitting, it is necessary to fix the pre-training weights of the mobile phone user interface element detection model so that the detection head of the mobile phone user interface element detection model is trained only using the target dataset, and finally the vehicle infotainment system (VMS) user interface element detection model is obtained.
[0082] Here, pre-trained weights refer to the parameters in the backbone network of the detection network responsible for extracting basic image features. The detection head refers to the part of the detection network responsible for outputting the location and category of user interface elements, thus adapting the detection network to the task of detecting user interface elements in the vehicle system.
[0083] In the specific implementation of step S104, the pre-training weights of the non-vehicle user interface element detection model are fixed so that the pre-training weights remain unchanged during the training process; the detection head of the non-vehicle user interface element detection model is iteratively trained using the target dataset and the stochastic gradient descent algorithm until the vehicle user interface element detection model is obtained.
[0084] Taking the mobile user interface element detection model as an example, the size of the input image and the number of categories output by the detection head of the mobile user interface element detection model are kept unchanged, and the pre-training weights of the mobile user interface element detection model are fixed, and only the weights of the detection head of the mobile user interface element detection model are trained.
[0085] The mobile phone user interface element detection model is trained using the SGD stochastic gradient descent algorithm and the target dataset (containing multiple image patches, each containing vehicle user interface elements). Training is completed when the loss function no longer decreases within a certain number of iterations (generally 60-80 iterations) or meets the preset number of iterations.
[0086] like Figure 2 The image shown is a comparison chart of the detection effects of vehicle infotainment user interface elements disclosed in an embodiment of the present invention.
[0087] In one embodiment, a trained vehicle infotainment user interface element detection model is used to detect vehicle infotainment user interface elements in a vehicle infotainment interface image. Specifically, the vehicle infotainment interface image for which vehicle infotainment user interface element detection is required is input into the trained vehicle infotainment user interface element detection model, and the detection results of the vehicle infotainment user interface elements are output.
[0088] For example, the detection results output by the vehicle infotainment user interface element detection model are as follows: Figure 2 As shown on the right, Figure 2 The left side shows the detection results of the traditional algorithm. Figure 2 Traditional algorithms for detecting in-vehicle infotainment system (IVS) user interface elements often produce many false detection boxes due to the complexity of the patterns. However, the method in this invention can correctly extract the IVS user interface elements without generating unnecessary false detection boxes.
[0089] Based on the training method for a vehicle infotainment user interface element detection model disclosed in the above embodiments of the present invention, this solution improves the network model's ability to detect small-sized vehicle infotainment user interface elements by cropping large-sized original vehicle infotainment interface images into small-sized image patches that conform to the network input; it extracts more abstract deep features of vehicle infotainment user interface elements through deep learning, enabling the extraction of more effective feature representations in complex vehicle infotainment user interface elements and complex background pattern regions; and it enables the training of a vehicle infotainment user interface element detection model on a dataset containing a small amount of labeled vehicle infotainment user interface elements, thereby achieving the goal of training a vehicle infotainment user interface element detection model suitable for vehicle infotainment interface images using a small dataset of vehicle infotainment user interface elements.
[0090] Based on the above-described embodiment of the present invention, a training method for a vehicle infotainment user interface element detection model is disclosed, such as... Figure 3 The diagram shown is a structural diagram of a training device for a vehicle infotainment user interface element detection model disclosed in an embodiment of the present invention, including: a data acquisition unit 301, a labeling unit 302, a cropping unit 303, and a transfer learning unit 304.
[0091] Acquisition unit 301 is used to acquire images of the vehicle infotainment interface;
[0092] The annotation unit 302 is used to add corresponding annotation information to each vehicle user interface element in the vehicle interface image; the annotation information includes at least: the position information of the corresponding vehicle user interface element;
[0093] The cropping unit 303 is used to crop the vehicle interface image into multiple image blocks to obtain a target dataset composed of each image block;
[0094] The transfer learning unit 304 is used to train a non-vehicle infotainment user interface element detection model based on transfer learning technology and the target dataset to obtain a vehicle infotainment user interface element detection model; the non-vehicle infotainment user interface element detection model is pre-trained using a non-vehicle infotainment interface image dataset.
[0095] In one embodiment, the cutting unit 303 is specifically used for:
[0096] Based on a preset image size and a preset overlap size, the vehicle interface image is cropped to obtain multiple image blocks whose image size is equal to the preset image size and whose overlap area with adjacent image blocks is equal to the preset overlap size.
[0097] In one embodiment, the device further includes:
[0098] The transformation unit is configured to, after cropping the vehicle interface image into multiple image blocks, determine, for each vehicle interface element, whether the vehicle interface element is completely located in at least one image block based on the position information in the corresponding annotation information; if yes, transform the position information to obtain target position information; the target position information is used to indicate the position of the vehicle interface element in the image block; if no, delete the annotation information corresponding to the vehicle interface element from the image block containing the annotation information corresponding to the vehicle interface element.
[0099] In one embodiment, the device further includes:
[0100] The training unit is used to acquire an open-source non-vehicle interface image dataset; construct a non-vehicle interface element detection model to be trained; and iteratively train the non-vehicle interface element detection model to be trained using the stochastic gradient descent algorithm and the non-vehicle interface image dataset until a trained non-vehicle interface element detection model is obtained.
[0101] In one embodiment, the transfer learning unit 304 is specifically used for:
[0102] The pre-training weights of the non-vehicle user interface element detection model are fixed so that the pre-training weights remain unchanged during the training process;
[0103] Using the target dataset and the stochastic gradient descent algorithm, the detection head of the non-vehicle user interface element detection model is iteratively trained until the vehicle user interface element detection model is obtained.
[0104] Based on the training device for a vehicle infotainment user interface element detection model disclosed in the above embodiments of the present invention, this solution improves the network model's ability to detect small vehicle infotainment user interface elements by cropping large-size original vehicle infotainment interface images into small-size image patches that conform to network input; it extracts more abstract deep features of vehicle infotainment user interface elements through deep learning methods, enabling the extraction of more effective feature representations in complex vehicle infotainment user interface elements and complex background pattern regions; and it enables the training of a vehicle infotainment user interface element detection model on a dataset containing a small amount of labeled vehicle infotainment user interface elements, thereby achieving the goal of training a vehicle infotainment user interface element detection model suitable for vehicle infotainment interface images using a small dataset of vehicle infotainment user interface elements.
[0105] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0106] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0107] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A training method for a vehicle infotainment user interface element detection model, characterized in that, The method includes: Capture images of the vehicle's infotainment system interface; Add corresponding annotation information to each vehicle user interface element in the vehicle interface image; the annotation information includes at least: the position information of the corresponding vehicle user interface element; The vehicle interface image is cropped into multiple image blocks to obtain a target dataset composed of each image block; A non-vehicle infotainment user interface element detection model is trained based on transfer learning technology and the target dataset to obtain a vehicle infotainment user interface element detection model; the non-vehicle infotainment user interface element detection model is pre-trained using a non-vehicle infotainment interface image dataset.
2. The method according to claim 1, characterized in that, The step of cropping the vehicle interface image into multiple image blocks to obtain a target dataset composed of each image block includes: Based on a preset image size and a preset overlap size, the vehicle interface image is cropped to obtain multiple image blocks whose image size is equal to the preset image size and whose overlap area with adjacent image blocks is equal to the preset overlap size.
3. The method according to claim 1, characterized in that, After cropping the vehicle interface image into multiple image blocks, the method further includes: For each of the vehicle infotainment user interface elements, based on the position information in the corresponding annotation information, it is determined whether the vehicle infotainment user interface element is completely located in at least one of the image blocks; If so, the position information is transformed to obtain target position information; the target position information is used to indicate the position of the vehicle user interface element in the image block; If not, then delete the annotation information corresponding to the vehicle user interface element from the image block containing the annotation information corresponding to the vehicle user interface element.
4. The method according to claim 1, characterized in that, The training process of the non-vehicle user interface element detection model includes: Obtain an open-source dataset of non-vehicle infotainment interface images; Construct a non-vehicle infotainment user interface element detection model to be trained; Using the stochastic gradient descent algorithm and the non-vehicle interface image dataset, the non-vehicle interface element detection model to be trained is iteratively trained until a trained non-vehicle interface element detection model is obtained.
5. The method according to any one of claims 1 to 4, characterized in that, The method of training a non-vehicle infotainment user interface element detection model based on transfer learning technology and the target dataset to obtain a vehicle infotainment user interface element detection model includes: The pre-training weights of the non-vehicle user interface element detection model are fixed so that the pre-training weights remain unchanged during the training process; Using the target dataset and the stochastic gradient descent algorithm, the detection head of the non-vehicle user interface element detection model is iteratively trained until the vehicle user interface element detection model is obtained.
6. A training device for a vehicle infotainment user interface element detection model, characterized in that, The device includes: The acquisition unit is used to acquire images from the vehicle's infotainment interface. The annotation unit is used to add corresponding annotation information to each vehicle user interface element in the vehicle interface image; the annotation information includes at least: the position information of the corresponding vehicle user interface element; The cropping unit is used to crop the vehicle interface image into multiple image blocks to obtain a target dataset composed of each image block; The transfer learning unit is used to train a non-vehicle infotainment user interface element detection model based on transfer learning technology and the target dataset to obtain a vehicle infotainment user interface element detection model; the non-vehicle infotainment user interface element detection model is pre-trained using a non-vehicle infotainment interface image dataset.
7. The apparatus according to claim 6, characterized in that, The cutting unit is specifically used for: Based on a preset image size and a preset overlap size, the vehicle interface image is cropped to obtain multiple image blocks whose image size is equal to the preset image size and whose overlap area with adjacent image blocks is equal to the preset overlap size.
8. The apparatus according to claim 6, characterized in that, The device further includes: The transformation unit is configured to, after cropping the vehicle interface image into multiple image blocks, determine, for each vehicle interface element, whether the vehicle interface element is completely located in at least one image block based on the position information in the corresponding annotation information; if yes, transform the position information to obtain target position information; the target position information is used to indicate the position of the vehicle interface element in the image block; if no, delete the annotation information corresponding to the vehicle interface element from the image block containing the annotation information corresponding to the vehicle interface element.
9. The apparatus according to claim 6, characterized in that, The device further includes: The training unit is used to acquire an open-source non-vehicle interface image dataset; construct a non-vehicle interface element detection model to be trained; and iteratively train the non-vehicle interface element detection model to be trained using the stochastic gradient descent algorithm and the non-vehicle interface image dataset until a trained non-vehicle interface element detection model is obtained.
10. The apparatus according to any one of claims 6 to 9, characterized in that, The transfer learning unit is specifically used for: The pre-training weights of the non-vehicle user interface element detection model are fixed so that the pre-training weights remain unchanged during the training process; Using the target dataset and the stochastic gradient descent algorithm, the detection head of the non-vehicle user interface element detection model is iteratively trained until the vehicle user interface element detection model is obtained.