Hyperparameter search method and hyperparameter search apparatus

Abstract

Provided are a hyperparameter search method and a hyperparameter search device. The hyperparameter search method comprises: obtaining a hyperparameter set for a predetermined task feature dimension, each task feature dimension in the predetermined task feature dimension comprising a predetermined plurality of feature values, the hyperparameter set comprising a hyperparameter corresponding to each task feature combination in a plurality of task feature combinations, the plurality of task feature combinations being obtained by one-to-one combination of each feature value under each different task feature dimension, and the hyperparameter corresponding to each task feature combination being an optimal hyperparameter pre-searched in a first hyperparameter search range for each task feature combination; searching, from the hyperparameter set, a hyperparameter corresponding to a predetermined K task feature combinations according to a first task feature combination of a to-be-processed task, the first task feature combination being a combination of feature values of each task feature dimension in the predetermined task feature dimension of the to-be-processed task; and searching a hyperparameter corresponding to the to-be-processed task based on the searched hyperparameter.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a hyperparameter search method and a hyperparameter search apparatus. Background Technology

[0002] In the field of AutoML, hyperparameter search is a crucial process for machine learning models, as the settings of hyperparameters can significantly impact the model's performance. Common hyperparameter search methods include grid search, random search, and automated hyperparameter search (such as annealing, Bayesian search, and Gaussian process search). These methods typically require dozens to hundreds of trials to find an optimal combination of hyperparameters. However, when AutoML products are delivered to users, resources are limited, and the time required for a single hyperparameter search is substantial, making the waiting time unacceptable to users.

[0003] Furthermore, hyperparameter search requires splitting the dataset into multi-fold training and validation sets. Each attempt trains the model on the training set of each fold and tests its performance on the validation set. The average performance across the multi-fold validation set is used as the performance metric for this hyperparameter search. A common method for splitting datasets into multi-fold sets is random splitting. While effective in most cases, random splitting can lead to inaccurate search results due to imbalanced sample distributions, such as in image segmentation and object detection tasks where some categories have a smaller proportion. If the splitting is done randomly and evenly at the sample granularity (image granularity), the data distribution of the validation set can easily differ significantly from the overall distribution, resulting in inaccurate search results. Summary of the Invention

[0004] This disclosure proposes a hyperparameter search method and a hyperparameter search apparatus, which can at least partially solve the above-mentioned problems.

[0005] According to one aspect of this disclosure, a hyperparameter search method executed by a computer device is provided, characterized by comprising: acquiring a hyperparameter set for a predetermined task feature dimension, wherein each task feature dimension in the predetermined task feature dimension includes a predetermined plurality of feature values, the hyperparameter set includes hyperparameters corresponding to each task feature combination in a plurality of task feature combinations, the plurality of task feature combinations being obtained by combining each feature value under each different task feature dimension, and the hyperparameters corresponding to each task feature combination being the optimal hyperparameters pre-searched for each task feature combination within a first hyperparameter search range; searching for hyperparameters corresponding to a predetermined K task feature combinations from the hyperparameter set based on a first task feature combination of the task to be processed, wherein the first task feature combination is a combination of feature values ​​of each task feature dimension in the predetermined task feature dimension of the task to be processed, and K is an integer greater than 0; and searching for hyperparameters corresponding to the task to be processed based on the found hyperparameters.

[0006] Optionally, the step of searching for the hyperparameters corresponding to the K task feature combinations from the hyperparameter set based on the first task feature combination of the task to be processed may include: searching for the K task feature combinations that are closest to the first task feature combination from the hyperparameter set; and obtaining the hyperparameters corresponding to the K task feature combinations from the hyperparameter set.

[0007] Optionally, the step of finding the K task feature combinations that are closest to the first task feature combination from the hyperparameter set may include: selecting K task feature combinations from the hyperparameter set based on the degree of closeness between the feature value of each task feature dimension in the first task feature combination and the feature value of the corresponding task feature dimension in the hyperparameter set, wherein at least one task feature dimension in the K task feature combinations has the highest degree of closeness to the feature value of the corresponding task feature dimension in the first task feature combination.

[0008] Optionally, when K is 1, the step of searching for the hyperparameters corresponding to the task to be processed based on the found hyperparameters may include: directly using the found hyperparameters as the hyperparameters corresponding to the task to be processed.

[0009] Optionally, when K is greater than 1, the step of searching for the hyperparameters corresponding to the task to be processed based on the found hyperparameters may include: interpolating the found hyperparameters to obtain a second hyperparameter search range; and searching for the optimal hyperparameter within the second hyperparameter search range as the hyperparameters corresponding to the task to be processed.

[0010] Optionally, if the dataset used by the task to be processed is an image dataset, the optimal hyperparameters can be searched in the following way: based on the distribution of each object category in at least one object category in the image dataset, the image dataset is split into N subsets, where N is an integer greater than 1; based on the N subsets, the optimal hyperparameters are searched.

[0011] Optionally, considering the distribution of each object category in the image dataset and splitting the image dataset into N sub-datasets may include: randomly and evenly distributing image samples in the image dataset that do not include objects to the N sub-datasets; and distributing image samples in the image dataset that include objects to the N sub-datasets based on a first distribution ratio and a second distribution ratio for each object category in the image samples in the image dataset, wherein the first distribution ratio is the distribution ratio of each object category in the image dataset in the image dataset, and the second distribution ratio is the distribution ratio of each object category in each sub-dataset in that sub-dataset.

[0012] Optionally, the step of allocating image samples containing objects in the image dataset to the N sub-datasets based on the first and second distribution ratios of each object category in the image samples containing objects in the image dataset may include: allocating image samples containing objects in the image dataset to the N sub-datasets based on the difference between the second distribution ratio of each object category in each sub-dataset and the first distribution ratio of the corresponding object category, such that the difference between the second distribution ratio of each object category in each sub-dataset and the first distribution ratio of the corresponding object category is minimized.

[0013] Optionally, the step of allocating image samples containing objects in the image dataset to the N subsets based on the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category may include: calculating the first distribution ratio R of the image dataset. j , j is the object category number; randomly select N image samples containing objects from the image samples containing objects in the image dataset, and randomly and evenly distribute them into the N subsets; repeatedly perform the following first operation until all image samples containing objects in the image dataset have been distributed: calculate the second distribution ratio R of each subset. i,j And calculate the second distribution proportion R for each subset. i,j The first distribution ratio R of the corresponding object category j The difference r i,j Where i is the subset number, i,j =R j -Ri,j From the remaining image samples containing objects in the image dataset, randomly select N image samples; repeatedly perform the following second operation until the N image samples are assigned to the N sub-data: count the category set of all object categories in the image samples that have not yet been assigned in the N image samples; from the category set of each sub-data set in the sub-data set that has not yet been assigned to image samples in the current second operation, find the gap r. i,j The largest first category number and the largest first subset number; the image sample with the largest distribution of the object category corresponding to the first category number among the unassigned image samples is assigned to the subset corresponding to the first subset number.

[0014] Optionally, when the image dataset is a dataset for image semantic segmentation, the object is a foreground object, the first distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in the image dataset to the total number of pixels of all image samples in the image dataset, and the second distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in each sub-dataset to the total number of pixels of all image samples in that sub-dataset.

[0015] Optionally, when the image dataset is a dataset for image object detection, the object is a target object, the first distribution ratio is the proportion of the number of target objects of each target object category in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target category in each sub-dataset to the total number of target objects in that sub-dataset.

[0016] Optionally, the first distribution ratio is the proportion of the number of target objects of each target object category at each scale range in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target object category at each scale range in each sub-dataset to the total number of target objects in that sub-dataset.

[0017] Optionally, the predetermined task feature dimension may include at least one of the following: dataset size, number of object categories, sample distribution, and machine learning model size.

[0018] Optionally, the hyperparameters may include at least one of batch size, learning rate, optimizer, learning rate adjustment strategy, and regularization coefficient applied to the optimizer.

[0019] According to another aspect of this disclosure, a hyperparameter search apparatus is provided, characterized in that it comprises: a hyperparameter set acquisition unit configured to acquire a hyperparameter set for a predetermined task feature dimension, wherein each task feature dimension in the predetermined task feature dimension includes a predetermined plurality of feature values, the hyperparameter set includes hyperparameters corresponding to each task feature combination in a plurality of task feature combinations, the plurality of task feature combinations being obtained by combining each feature value under each different task feature dimension, and the hyperparameters corresponding to each task feature combination being the optimal hyperparameters pre-searched for each task feature combination within a first hyperparameter search range; a search unit configured to search for hyperparameters corresponding to a predetermined K task feature combinations from the hyperparameter set based on a first task feature combination of the task to be processed, wherein the first task feature combination is a combination of feature values ​​of each task feature dimension in the predetermined task feature dimension of the task to be processed, and K is an integer greater than 0; and a search unit configured to search for hyperparameters corresponding to the task to be processed based on the found hyperparameters.

[0020] Optionally, the search unit can be configured to: search for K task feature combinations that are closest to the first task feature combination from the hyperparameter set; and obtain the hyperparameters corresponding to the K task feature combinations from the hyperparameter set.

[0021] Optionally, the search unit can be configured to: select K task feature combinations from the hyperparameter set based on the proximity of the feature value of each task feature dimension in the first task feature combination to the feature value of the corresponding task feature dimension in the hyperparameter set, wherein the feature value of at least one task feature dimension in the K task feature combinations is closest to the feature value of the corresponding task feature dimension in the first task feature combination.

[0022] Optionally, when K is 1, the search unit can be configured to directly use the found hyperparameters as the hyperparameters corresponding to the task to be processed.

[0023] Optionally, when K is greater than 1, the search unit can be configured to: interpolate the found hyperparameters to obtain a second hyperparameter search range; and search for the optimal hyperparameter within the second hyperparameter search range as the hyperparameter corresponding to the task to be processed.

[0024] Optionally, the hyperparameter search device may further include: a splitting unit; when the dataset used by the task to be processed is an image dataset, the splitting unit may be configured to: split the image dataset into N sub-datasets based on the distribution of each object category in at least one object category in the image dataset, wherein N is an integer greater than 1; the search unit may be configured to: search for optimal hyperparameters based on the N sub-datasets.

[0025] Optionally, the splitting unit can be configured to: randomly and evenly distribute image samples in the image dataset that do not include objects to the N sub-datasets; and distribute image samples in the image dataset that include objects to the N sub-datasets based on a first distribution ratio and a second distribution ratio for each object category in the image samples in the image dataset, wherein the first distribution ratio is the distribution ratio of each object category in the image dataset in the image dataset, and the second distribution ratio is the distribution ratio of each object category in each sub-dataset in that sub-dataset.

[0026] Optionally, the splitting unit can be configured to: based on the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category, distribute image samples including objects in the image dataset to the N subsets, such that the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category is minimized.

[0027] Optionally, the splitting unit can be configured to: statistically analyze a first distribution ratio R of the image dataset. j , j is the object category number; randomly select N image samples containing objects from the image samples containing objects in the image dataset, and randomly and evenly distribute them into the N subsets; repeatedly perform the following first operation until all image samples containing objects in the image dataset have been distributed: calculate the second distribution ratio R of each subset. i,j And calculate the second distribution proportion R for each subset. i,j The first distribution ratio R of the corresponding object category j The difference r i,j Where i is the subset number, i,j =R j -R i,jFrom the remaining image samples containing objects in the image dataset, randomly select N image samples; repeatedly perform the following second operation until the N image samples are assigned to the N sub-data: count the category set of all object categories in the image samples that have not yet been assigned in the N image samples; from the category set of each sub-data set in the sub-data set that has not yet been assigned to image samples in the current second operation, find the gap r. i,j The largest first category number and the largest first subset number; the image sample with the largest distribution of the object category corresponding to the first category number among the unassigned image samples is assigned to the subset corresponding to the first subset number.

[0028] Optionally, when the image dataset is a dataset for image semantic segmentation, the object is a foreground object, the first distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in the image dataset to the total number of pixels of all image samples in the image dataset, and the second distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in each sub-dataset to the total number of pixels of all image samples in that sub-dataset.

[0029] Optionally, when the image dataset is a dataset for image object detection, the object is a target object, the first distribution ratio is the proportion of the number of target objects of each target object category in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target category in each sub-dataset to the total number of target objects in that sub-dataset.

[0030] Optionally, the first distribution ratio is the proportion of the number of target objects of each target object category at each scale range in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target object category at each scale range in each sub-dataset to the total number of target objects in that sub-dataset.

[0031] Optionally, the predetermined task feature dimension may include at least one of the following: dataset size, number of object categories, sample distribution, and machine learning model size.

[0032] Optionally, the hyperparameters may include at least one of batch size, learning rate, optimizer, learning rate adjustment strategy, and regularization coefficient applied to the optimizer.

[0033] According to another aspect of this disclosure, a computer-readable storage medium for storing instructions is provided, wherein when the instructions are executed by at least one computing device, the at least one computing device causes the at least one computing device to perform a hyperparameter search method according to this disclosure.

[0034] According to another aspect of this disclosure, a system is provided that includes at least one computing device and at least one storage device for storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to perform a hyperparameter search method according to this disclosure.

[0035] According to the hyperparameter search method and hyperparameter search device disclosed herein, hyperparameter search can be performed in advance on the task feature dimensions of the training model as diverse as possible, summarizing the approximate value range of the optimal hyperparameters corresponding to all task feature combinations, and obtaining a hyperparameter set. Then, for the task to be processed, the hyperparameters corresponding to the task feature combinations that are close to the task to be processed in the hyperparameter set can be searched. Based on the relevant range of the found hyperparameters, a small-range search can be performed. Under the premise of greatly shortening the hyperparameter search time and resource consumption, a relatively good hyperparameter combination can be obtained.

[0036] According to the hyperparameter search method and hyperparameter search apparatus disclosed herein, performing effective data splitting on image datasets, such as those used for image semantic segmentation or image object detection tasks, can ensure that the sample distribution of each category in each split dataset is basically close to the overall sample distribution, so that the result of hyperparameter search is not affected by the dataset splitting. Attached Figure Description

[0037] These and / or other aspects and advantages of this disclosure will become clearer and more readily understood from the following description of the embodiments, taken in conjunction with the accompanying drawings, wherein:

[0038] Figure 1 This illustrates a hyperparameter search method performed by a computer device according to exemplary embodiments of the present disclosure.

[0039] Figure 2 This is a flowchart illustrating a dataset splitting method according to an exemplary embodiment of the present disclosure.

[0040] Figure 3 This is a flowchart illustrating the allocation of image samples including objects according to an exemplary embodiment of the present disclosure.

[0041] Figure 4 This is a flowchart illustrating an algorithm for splitting an image dataset for image semantic segmentation according to an exemplary embodiment of the present disclosure.

[0042] Figure 5This is a block diagram illustrating a hyperparameter search apparatus according to an exemplary embodiment of the present disclosure.

[0043] Figure 6 This is a block diagram illustrating a dataset splitting apparatus according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0044] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of embodiments of the present disclosure as defined by the claims and their equivalents. Various specific details are included to aid understanding, but these details are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted.

[0045] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0046] Common hyperparameter search methods include grid search, random search, and automatic hyperparameter search (such as annealing, Bayesian search, and Gaussian process search). Grid search requires empirically setting the range of each hyperparameter, then iterates through all permutations and combinations of the input parameters, using cross-validation to return the evaluation metric scores for all parameter combinations. The disadvantages of this method are the high number of searches, high resource consumption, and the exponential increase in search time with the number of hyperparameters. Random search is similar to grid search, but instead of exploring all possible combinations, it selects a specific number of random combinations of a random value for each hyperparameter. Random search can significantly reduce the number of attempts, thus reducing the search time, but it still requires dozens or even hundreds of attempts. Automatic hyperparameter search typically involves dozens of random attempts across the entire search space, and then, based on the hyperparameter values ​​from each previous attempt and the model's performance on the validation set, uses a specific algorithm to determine the hyperparameter values ​​for the next attempt. The hyperparameter search methods described above all require at least dozens to hundreds of attempts to find a relatively optimal combination of hyperparameters. However, when AutoML products are presented to users, users have limited resources, and the time required to perform a single hyperparameter search is enormous, which users find difficult to tolerate.

[0047] Furthermore, hyperparameter search requires splitting the dataset into multi-fold training and validation sets. Each attempt trains the model (i.e., a machine learning model, such as an AutoML product, hereinafter referred to as the model) on the training set of each fold, and tests the model's performance on the validation set. The average performance on the multi-fold validation set is used as the performance metric for this hyperparameter search. A common method for multi-fold dataset splitting is random splitting, for example, randomly dividing the dataset into 10 equal parts, each containing the same number of samples, denoted as d1, d2, ..., d... 10 ,(d2∪d3∪…∪d 10 d1) is the first fold's split between the validation and training sets (the first part represents the training set, and the second part represents the validation set), (d1∪d3∪d4∪…∪d 10 d2) constitutes the second split, and so on. Random splitting is effective in most cases, but when the sample distribution is unbalanced, such as in image segmentation and image object detection tasks, where some categories have a smaller proportion, if the split is randomly averaged at the sample granularity (image granularity), it is easy to cause a large difference between the data distribution of the validation set and the overall distribution, resulting in inaccurate search results for hyperparameter search due to the large difference in the distribution of the validation set.

[0048] To address the aforementioned technical issues, this disclosure proposes a hyperparameter search method. Based on grid search, the range of each hyperparameter is set empirically. Then, a grid search is performed across a wide variety of task feature dimensions in the training model (e.g., datasets of different sizes, models of different sizes, number of object categories in the dataset, sample distribution, etc.) to summarize the approximate range of optimal hyperparameter values ​​for all task feature combinations (e.g., an empirical table can be created). In AutoML products, the task feature combination closest to the task features in the empirical table is found based on the user's training task. A small-scale search is then performed within the optimal hyperparameter value range corresponding to the found task feature combination. This significantly reduces the time and resource consumption of the hyperparameter search while obtaining a relatively good hyperparameter combination. Furthermore, according to the hyperparameter search method of this disclosure, a greedy dataset splitting algorithm can be used to effectively split image datasets (typically containing multiple different types and quantities of objects) used for tasks such as image semantic segmentation or image object detection. This ensures that the sample distribution of each category in each split dataset is basically close to the overall sample distribution, preventing the hyperparameter search results from being affected by the dataset splitting. The following will refer to Figures 1 to 6 The hyperparameter search method and apparatus according to this disclosure will be described in detail.

[0049] Figure 1This illustrates a hyperparameter search method performed by a computer device according to exemplary embodiments of the present disclosure.

[0050] In step 101, a set of hyperparameters for a predetermined task feature dimension can be obtained. Here, "task" can refer to a model training task, and "task feature dimension" can refer to the task feature dimension used for the model training task, such as, but not limited to, at least one of the following: dataset size, number of object categories, sample distribution, and machine learning model size. The dataset size can refer to the size of the number of samples in the dataset used for model training; the number of object categories can refer to the number of object categories included in the samples of the dataset (e.g., an image dataset); the sample distribution can refer to whether the sample distribution in the dataset is balanced; and the machine learning model size can refer to the size of the model trained for the model training task.

[0051] Each task feature dimension may include multiple predetermined feature values. In the example above, the feature value for dataset size may be the number of samples in the dataset, for example, but not limited to, it may include 5 feature values, namely 1000, 3000, 5000, 10000, and 30000. The feature value for the number of object categories may be the number of object categories included in the samples in the dataset, for example, but not limited to, it may include 5 feature values, namely 2, 5, 10, 30, and 50. The feature value for sample distribution may be an indicator of whether the sample distribution in the dataset is balanced, for example, but not limited to, it may include 2 feature values, namely yes and no. The feature value for machine learning model size may be an indicator representing the size of the machine learning model, for example, but not limited to, it may include 2 feature values, namely large model (e.g., Resnest101+ASPP+OCR) and small model (e.g., EfficientNet-B1+FPN). Of course, the model size is not limited to large and small models, and models with different parameter sizes can also be selected.

[0052] The hyperparameter set can be pre-constructed based on predetermined task feature dimensions. Specifically, each feature value under each different task feature dimension can be combined to obtain multiple task feature combinations. The optimal hyperparameters for each task feature combination can be pre-searched within a predetermined hyperparameter search range (e.g., through grid search) and used as the hyperparameters corresponding to that task feature combination. The hyperparameters corresponding to all task feature combinations are then constructed into a hyperparameter set. For example, in the example above, the feature values ​​of dataset size, number of object categories, sample distribution, and machine learning model size can be iterated to obtain 100 task feature combinations. For example, task feature combination 1 (dataset size "1000", number of object categories "2", sample distribution "yes", machine learning model size "large-scale"), task feature combination 2 (dataset size "3000", number of object categories "2", sample distribution "yes", machine learning model size "large-scale"), task feature combination 3 (dataset size "5000", number of object categories "2", sample distribution "yes", machine learning model size "large-scale"), and so on. Furthermore, for situations where no matching dataset can be found for certain combinations of task features—for example, a dataset with 30,000 images and 2 object categories—a dataset satisfying these conditions can be obtained by downsampling images or categories from other datasets. Of course, the choice of task feature dimensions when constructing the hyperparameter set can be expanded or reduced as needed, and the range of feature values ​​for each dimension can also be expanded or reduced as required.

[0053] The hyperparameters to be searched may include at least one hyperparameter required for model training; in the case of multiple hyperparameters, it may be referred to as a hyperparameter combination. Hyperparameters may include, but are not limited to, at least one of batch size, learning rate, optimizer, learning rate adjustment strategy (lr_scheduler), and regularization coefficient applied to the optimizer (weight_decay). The hyperparameters to be searched can be pre-selected, and the search range for each selected hyperparameter can be set. Within the predetermined hyperparameter search range comprised of the search ranges of each hyperparameter, the optimal hyperparameters corresponding to each task feature combination are searched. Of course, the selection of hyperparameters and the search range of each hyperparameter can be adaptively adjusted based on needs, experience, etc. For example, in the example above, batch size and learning rate can be selected as the hyperparameters to be searched. This is because, with a fixed model, batch size and learning rate have a significant impact on the model's performance, and datasets of the same size, number of classes, and sample distribution have similar optimal values ​​on models of the same size. When hyperparameters include batch size and learning rate, the search range for batch size and learning rate can be preset. For example, the search range for batch size can include [3, 6, 8, 12, 16, 20, 24, 32]. The search range for learning rate is different for different batch sizes, as shown in Table 1 below:

[0054] Table 1

[0055]

[0056] Therefore, all possible combinations of batch size and learning rate in Table 1 can constitute a predetermined hyperparameter search range, for example, (3, 0.0001), (3, 0.0003), (3, 0.0005), ..., (32, 0.07), (32, 0.08), (32, 0.1). For each of the above 100 task feature combinations, the model can be trained on each training set of, for example, 3-fold cross-validation using all possible combinations of batch size and learning rate. The model performance is then tested on the corresponding validation set. The average of the results from the three validation sets is the effect of that combination of learning rate and batch size on the corresponding model for that task feature combination. The optimal combination of learning rate and batch size is selected as the optimal combination of learning rate and batch size for that task feature combination on the corresponding model.

[0057] When all task feature combinations have corresponding hyperparameter combinations found, each task feature combination and its corresponding hyperparameter combination can be recorded to obtain a hyperparameter set for a predetermined task feature dimension. For example, but not limited to, an empirical table can be created to record the hyperparameter set for a predetermined task feature dimension. This table can record each task feature combination and its corresponding hyperparameter combination. For example, in the above example, a table recording each task feature combination and its corresponding hyperparameter combination can be obtained, containing 100 records, each corresponding to a task feature combination and its corresponding hyperparameter combination, as shown in Table 2 below:

[0058] Table 2

[0059]

[0060]

[0061] In step 102, based on the first task feature combination of the task to be processed, the hyperparameters corresponding to a predetermined K task feature combinations can be searched from the hyperparameter set. Here, the first task feature combination is a combination of feature values ​​from each task feature dimension in the predetermined task feature dimensions of the task to be processed, and K is an integer greater than 0. In other words, for a new model training task (i.e., the task to be processed), the search space can be adaptively adjusted according to resource availability. For example, based on the first task feature combination of the task to be processed, the K task feature combinations closest to the first task feature combination of the task to be processed can be found from the hyperparameter set (e.g., an empirical table recording the hyperparameter set), and the hyperparameters (or hyperparameter combinations) corresponding to these K task feature combinations can be obtained from the hyperparameter set.

[0062] According to the exemplary embodiments of this disclosure, K task feature combinations can be selected from the hyperparameter set based on the proximity of the feature values ​​of each task feature dimension in the first task feature combination to the feature values ​​of the corresponding task feature dimensions in the hyperparameter set. The feature values ​​of at least one task feature dimension in the K task feature combinations have the highest proximity to the feature values ​​of the corresponding task feature dimensions in the first task feature combination. For example, among multiple task feature dimensions, the proximity of the feature values ​​related to dataset size can be given priority in selecting the K task feature combinations. Of course, the closest task feature combination can be found according to any feasible rule. For example, in the above example, the first task feature combination of the task to be processed includes dataset size (1000), number of object categories (3), sample distribution (balanced), and model size (small). The task feature combination Task set 2 that is closest to the first task feature combination can be found from Table 2 above, and the hyperparameter combination (3, 0.0007) corresponding to Task set 2 can be obtained. For example, in the example above, the first task feature combination of the task to be processed includes the dataset size (2000), the number of object categories (2), the sample distribution (balanced), and the model size (small). The two task feature combinations that are closest to the first task feature combination, Task set 2 and Task set 6, can be found in Table 2 above. The hyperparameter combination (3, 0.0007) corresponding to the task feature combination Task set 2 and the hyperparameter combination (6, 0.0005) corresponding to the task feature combination Task set 6 can be obtained.

[0063] In step 103, the hyperparameters corresponding to the task to be processed can be searched based on the found hyperparameters.

[0064] According to an exemplary embodiment of this disclosure, when K is 1, the found hyperparameters can be directly used as the hyperparameters corresponding to the task to be processed. When K is greater than 1, the found hyperparameters can be interpolated to obtain a second hyperparameter search range; the optimal hyperparameter is searched within the second hyperparameter search range and used as the hyperparameters corresponding to the task to be processed. For example, in the above example, when the first task feature combination of the task to be processed includes dataset size (1000), number of object categories (3), sample distribution (balanced), and model size (small), the hyperparameter combination (3, 0.0007) corresponding to the closest task feature combination Task set 2 can be found from Table 2 above, and the hyperparameter combination (3, 0.0007) can be used as the hyperparameters corresponding to the task to be processed. For example, in the example above, when the first task feature combination of the task to be processed includes the dataset size (2000), the number of object categories (2), the sample distribution (balanced), and the model size (small), the two closest task feature combinations, Task set 2 and Task set 6, corresponding to the hyperparameter combinations (3, 0.0007) and (6, 0.0005), can be found from Table 2 above. Interpolation can be performed on the two hyperparameter combinations (3, 0.0007) and (6, 0.0005) to obtain more hyperparameter combinations, such as (3, 0.0007), (6, 0.0005), (3, 0.0006), (6, 0.0006), (4, 0.0007), (4, 0.0005), and (4, 0.0006), forming the second hyperparameter search range. Within this second hyperparameter search range, the optimal hyperparameter is searched using, for example, a grid search, to obtain the hyperparameters corresponding to the task to be processed. Of course, this disclosure does not limit the rules of interpolation, and hyperparameters can be interpolated according to any feasible interpolation method.

[0065] Using the methods described above, whether the optimal hyperparameters for the task are obtained directly through a search, or by finding the optimal range of hyperparameter values ​​and then performing a smaller-scale hyperparameter search based on that range, a relatively good combination of hyperparameters can be obtained while significantly reducing the time and resource consumption of the hyperparameter search. Of course, the hyperparameter search method disclosed herein can be applied to any AutoML product, and the selection of the dataset, the range of models, and the range of hyperparameters can be adjusted according to the type of task and experience.

[0066] Furthermore, during the hyperparameter search process, the dataset needs to be split, and the optimal hyperparameters are searched based on the multiple subset datasets obtained from the split. That is, the dataset can be split into multi-fold training and validation sets. In each attempt of the hyperparameter search, the model is trained on the training set of each fold, and the model's performance is tested on the validation set. The average performance on the multi-fold validation set is used as the performance metric for this hyperparameter search. This disclosure proposes a dataset splitting method that can effectively split image datasets (typically containing a large number of objects of different types) for tasks such as image semantic segmentation or image object detection. Specifically, the distribution of each object category in the image dataset can be considered, and the image dataset can be split into multiple subset datasets. The general idea is to first randomly distribute image samples without objects from the image dataset into multiple subsets. Then, image samples with objects are evenly distributed into multiple subsets based on the distribution of each object category. More specifically, the distribution can be based on the difference between the distribution of each object category in each subset and its distribution in the entire dataset, ensuring that the sample distribution of each object category in each subset is approximately close to the overall sample distribution. Of course, this dataset splitting method can be applied not only to hyperparameter search scenarios but also to any scenario requiring the splitting of the full dataset into multiple subsets, ensuring that the data distribution of the subsets approximates the distribution of the full dataset, thus preventing the hyperparameter search results from being affected by the dataset splitting.

[0067] According to an exemplary embodiment of this disclosure, when the dataset used for the task to be processed is an image dataset, the image dataset can be split into N subsets based on the distribution of each object category in at least one object category in the image dataset, where N is an integer greater than 1; and the optimal hyperparameter is searched based on the N subsets. For example, in the hyperparameter search method according to this disclosure described above, when constructing the hyperparameter set, the optimal hyperparameter is pre-searched within a first hyperparameter search range for each task feature combination. Furthermore, when searching for the hyperparameter corresponding to the task to be processed, the optimal hyperparameter is searched within a second hyperparameter search range for the task to be processed. Both of these hyperparameter search processes require splitting the dataset; therefore, the dataset splitting method according to this disclosure can be applied to both. The following will refer to... Figure 2 This section will describe in detail the dataset splitting method based on this disclosure.

[0068] Figure 2 This is a flowchart illustrating a dataset splitting method according to an exemplary embodiment of the present disclosure. Figure 2The dataset splitting method shown can be applied to any scenario where the full dataset needs to be split into multiple sub-datasets, so that the data distribution of the sub-datasets can be close to the distribution of the full dataset.

[0069] Reference Figure 2 In step 201, an image dataset can be obtained, wherein the image dataset includes multiple image samples. Here, the image dataset can be a training dataset used for model training. For example, but not limited to, the image dataset can be a dataset used to train a model for image semantic segmentation, or the image dataset can be a dataset used to train a model for image object detection.

[0070] In step 202, at least one object category in the image dataset can be determined. For example, in an image semantic segmentation algorithm, the object can be a foreground object; therefore, the foreground object categories included in the image samples in the image dataset can be determined. For example, the foreground object categories can be set to include people, vehicles, animals, buildings, etc. As another example, in an image object detection algorithm, the object can be a target object to be detected; therefore, the target object categories included in the image samples in the image dataset can be determined. For example, the target object categories can be set to include cats, dogs, people, etc.

[0071] In step 203, the image dataset can be split into N subsets based on the distribution of each object category in at least one object category in the image dataset, where N is an integer greater than 1.

[0072] According to an exemplary embodiment of the present disclosure, image samples in an image dataset that do not include objects can be randomly and evenly distributed into N sub-datasets, and image samples in an image dataset that include objects can be distributed into N sub-datasets based on a first distribution ratio and a second distribution ratio for each object category in the image samples in the image dataset, wherein the first distribution ratio is the distribution ratio of each object category in the image dataset in the image dataset, and the second distribution ratio is the distribution ratio of each object category in each sub-dataset in the sub-dataset.

[0073] According to an exemplary embodiment of this disclosure, image samples including objects in an image dataset can be allocated to the N subsets based on the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category, thereby minimizing the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category. In other words, through the above allocation method, the sample distribution of each object category in each split subset is substantially close to the overall sample distribution. The following will refer to... Figure 3 The algorithm for allocating image samples containing objects is described in detail.

[0074] Figure 3 This is a flowchart illustrating the allocation of image samples including objects according to an exemplary embodiment of the present disclosure.

[0075] Reference Figure 3 In step 301, the first distribution ratio R of the image dataset can be statistically analyzed. j j is the object category number. For example, if the number of object categories is m, then j = 1, 2, ..., m.

[0076] In step 302, N image samples containing objects can be randomly selected from the image samples containing objects in the image dataset, and then randomly and evenly distributed among N subsets. That is, each subset can be randomly assigned to one image sample from the N randomly selected image samples.

[0077] In step 303, the second distribution proportion R of each subset can be calculated. i,j And calculate the second distribution proportion R for each subset. i,j The first distribution ratio R of the corresponding object category j The difference r i,j r i,j =R j -R i,j , where i is the subset number, therefore i = 1, 2, ..., N.

[0078] In step 304, N image samples are randomly selected from the remaining image samples containing objects in the image dataset.

[0079] In step 305, the set of categories C of all object categories is counted among the N randomly selected image samples that have not yet been assigned.

[0080] In step 306, find the gap r from the category set C of each of the N subsets that have not yet been assigned to image samples. i,j The largest first category number c and the largest subset number s. For example, if the category set C includes categories 1, 2, 4, 6, m, and there are currently 1, 2, ..., n subsets of data that have not yet been assigned to image samples, then we can look up {r}. i,j}(i∈{1,2,…,n},j∈C=1,2,4,6,m) The first category number c and the first subset number s corresponding to the maximum value are, i.e.,

[0081] In step 307, the image sample with the largest distribution of the object category corresponding to the first category code c among the unassigned image samples can be assigned to the subset corresponding to the first subset number s. For example, the unassigned image samples include a first image sample and a second image sample. Both the first and second image samples include object categories corresponding to category codes c, and the distribution of the object category corresponding to category code c in the first image sample is larger than that in the second image sample. In this case, the first image sample can be assigned to the subset corresponding to the first subset number s. For example, in the case of image samples used for image semantic segmentation, the largest distribution may refer to the largest total number of pixels of that object category. Similarly, in the case of image samples used for image object detection, the largest distribution may refer to the largest number of objects of that object category.

[0082] For the N image samples randomly selected in step 304, steps 305 to 306 can be repeated until the N image samples randomly selected in step 304 have been allocated. That is, each of the N subsets can be allocated to one of the N image samples according to steps 305 to 307. Of course, if only one image sample remains unallocated, corresponding to only one subset of the dataset not yet allocated to an image sample, steps 305 to 307 can be skipped, and the image sample can be directly allocated to that subset.

[0083] For the remaining image samples containing objects in the image dataset, steps 303 to 307 can be repeated until all image samples containing objects in the image dataset have been allocated.

[0084] According to an exemplary embodiment of this disclosure, when the image dataset is a dataset for image semantic segmentation, and the object is a foreground object, the first distribution ratio may be the ratio of the total number of pixels of the foreground object of each foreground object category in the image dataset to the total number of pixels of all image samples in the image dataset, and the second distribution ratio may be the ratio of the total number of pixels of the foreground object of each foreground object category in each subset of the dataset to the total number of pixels of all image samples in that subset of the dataset. In this case, an image sample excluding the object may represent an image sample containing only the background and excluding the foreground object.

[0085] According to an exemplary embodiment of this disclosure, when the image dataset is a dataset for image object detection, the object is a target object. A first distribution ratio can be the proportion of the number of target objects of each target object category in the image dataset to the total number of target objects in the image dataset. A second distribution ratio can be the proportion of the number of target objects of each target object category in each sub-dataset to the total number of target objects in that sub-dataset. In this case, image samples excluding objects can represent image samples excluding target objects. Furthermore, in this case, the granular statistical proportion and segmentation operation can be performed more finely, using different scale ranges for each target object category. For example, the first distribution ratio can be the proportion of the number of target objects of each target object category at each scale range in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio can be the proportion of the number of target objects of each target object category at each scale range in each sub-dataset to the total number of target objects in that sub-dataset.

[0086] The following section will use an image semantic segmentation algorithm as an example to describe the algorithmic process of splitting an image dataset. Figure 4 This is a flowchart illustrating an algorithm for splitting an image dataset for image semantic segmentation according to an exemplary embodiment of the present disclosure. The algorithm input is an image semantic segmentation dataset D containing *a* image samples. The algorithm output is N subsets {d1, d2, d3, ..., dn}. N}, each subset d i It contains D / N image samples, and the intersection of any two distinct subsets is empty, i.e. The union of all subsets is the entire dataset D, i.e., d1∪d2∪…∪d N =D.

[0087] Reference Figure 4 In step 401, N subsets of the dataset can be initialized. That is, the N subsets can be set to an empty set.

[0088] In step 402, the proportion of the total number of pixels for each foreground object category in the image dataset to the total number of pixels in the entire dataset can be calculated, i.e., the first distribution proportion, denoted as R1, R2, ..., R m , where m represents the number of categories of foreground objects.

[0089] In step 403, the subset S of the pure background image sample set is... b The data is randomly and evenly divided into N subsets.

[0090] In step 404, from the remaining subset of image samples containing foreground objects In this process, N image samples are randomly selected, and then these N image samples are randomly and evenly distributed into N subsets, that is, each subset is randomly assigned to one of the N image samples.

[0091] In step 405, the proportion R of the total number of pixels for each foreground object category in each subset of the dataset to the total number of pixels in all images of that subset is calculated. i,j (i is the subset number, j is the foreground object category number), and calculate R for each subset. i,j The proportion R of the corresponding category j in the image dataset j The difference r i,j That is, r i,j =R j -R i,j .

[0092] In step 406, from subset S r N image samples are randomly selected from the data.

[0093] In step 407, the set of categories C of all foreground object categories is counted among the N image samples that have not yet been assigned.

[0094] In step 408, find the gap r from the category set of each of the N subsets that have not yet been assigned to image samples. i,j The category number c and the subset number s corresponding to the maximum value.

[0095] In step 409, the image sample ε with the largest total number of pixels belonging to category c can be found from the N unassigned image samples, and image sample ε is assigned to the subset s. That is, d s =d s ∪{ε}.

[0096] For subset S r For N randomly selected image samples, repeat steps 407 to 409 until a subset S is obtained. r N image samples randomly selected from the dataset are assigned to N subsets.

[0097] Additionally, it is possible to target subset S r For the remaining image samples, repeat steps 405 to 409 until subset S is reached. r The image samples in the dataset have been allocated, that is,

[0098] According to an exemplary embodiment of this disclosure, the dataset splitting method of this disclosure is applied to a 4-class semantic segmentation dataset, splitting the dataset into 10 subsets. The proportion of pixels in each class in each subset is approximately the same as the proportion of pixels in each class in the overall dataset. Tables 3 and 4 below show the distribution obtained by randomly performing the dataset splitting method of this disclosure twice, where class_1, class_2, class_3, and class_4 represent the four foreground classes, total represents the pixel distribution of each class in the overall dataset, and split_0,…,split_9 represents the pixel distribution of each class in the 10 subsets.

[0099] Table 3

[0100]

[0101] Table 4

[0102]

[0103] Figure 4 The algorithmic flow for splitting an image dataset for image semantic segmentation is shown, but the same algorithmic flow can also be applied to an image dataset for image object detection. The difference is that, when applied to an image dataset for image object detection, the proportion R counted in step 402 will be... j Replace the image samples with the proportion of the number of target objects in each target object category in the image dataset to the total number of target objects in the image dataset. Replace the pure background image samples in step 403 with image samples that do not contain target objects, and change the proportion R calculated in step 405. i,j Replace the proportion of target objects of each target category in each subset of the dataset with the proportion of all target objects in that subset, and replace the image sample ε with the largest total number of pixels including category c found in step 409 with the image sample ε with the largest number of target objects including category c. Furthermore, more finely, the proportion and segmentation operation can be performed at a granular level based on different scale ranges for each target category. Therefore, the proportion R calculated in step 402 can be... j Replace the image samples with the proportion of the number of target objects in each scale range of each target object category in the image dataset to the total number of target objects in the image dataset. Replace the pure background image samples in step 403 with image samples that do not contain target objects, and change the proportion R counted in step 405. i,jReplace the proportion of the number of target objects in each scale range of each target category in each subset of the dataset to the total number of target objects in that subset, and replace the image sample ε with the largest total number of pixels including category c found in step 409 with the image sample ε with the largest number of target objects including category c.

[0104] Figure 5 This is a block diagram illustrating a hyperparameter search apparatus according to an exemplary embodiment of the present disclosure.

[0105] Reference Figure 5 The hyperparameter search apparatus 500 according to an exemplary embodiment of the present disclosure may include a hyperparameter set acquisition unit 501, a search unit 502, and a search unit 503.

[0106] The hyperparameter set acquisition unit 501 can acquire a hyperparameter set for a predetermined task feature dimension. Here, "task" can refer to a model training task, and "task feature dimension" can refer to the task feature dimension used for the model training task, such as, but not limited to, at least one of the following: dataset size, number of object categories, sample distribution, and machine learning model size. The dataset size can refer to the size of the number of samples in the dataset used for model training; the number of object categories can refer to the number of object categories included in the samples of the dataset (e.g., an image dataset); the sample distribution can refer to whether the sample distribution in the dataset is balanced; and the machine learning model size can refer to the size of the model trained for the model training task.

[0107] Each task feature dimension may include a number of predefined feature values. In the example above, the feature value for dataset size could be the number of samples in the dataset. The feature value for the number of object categories could be the number of object categories included in the samples in the dataset. The feature value for sample distribution could be an indicator of whether the sample distribution in the dataset is balanced. The feature value for machine learning model size could be an indicator representing the size of the machine learning model.

[0108] The hyperparameter set can be pre-constructed based on predetermined task feature dimensions. Specifically, each feature value under each different task feature dimension can be combined to obtain multiple task feature combinations. For each task feature combination, the optimal hyperparameters, pre-searched within a predetermined hyperparameter search range (e.g., through grid search), can be used as the hyperparameters corresponding to that task feature combination. The hyperparameters corresponding to all task feature combinations are then constructed into a hyperparameter set. Furthermore, for cases where some task feature combinations cannot find matching datasets, datasets satisfying the combination conditions can be obtained by downsampling images or categories from other datasets. Of course, the choice of task feature dimensions when constructing the hyperparameter set can be expanded or reduced as needed, and the range of feature values ​​for each dimension can also be expanded or reduced as needed.

[0109] The hyperparameters to be searched may include at least one hyperparameter required for model training; in the case of multiple hyperparameters, it may be referred to as a hyperparameter combination. Hyperparameters may include, but are not limited to, at least one of the following: batch size, learning rate, optimizer, learning rate adjustment strategy (lr_scheduler), and regularization coefficient applied to the optimizer (weight_decay). The hyperparameters to be searched can be pre-selected, and the search range for each selected hyperparameter can be set. Within the predetermined hyperparameter search range comprised of the search ranges of each hyperparameter, the optimal hyperparameters corresponding to each task feature combination are searched. Of course, the selection of hyperparameters and the search range of each hyperparameter can be adaptively adjusted based on needs, experience, etc.

[0110] When corresponding hyperparameter combinations are found for all task feature combinations, each task feature combination and its corresponding hyperparameter combination can be recorded to obtain a hyperparameter set for the predetermined task feature dimension. For example, but not limited to, an empirical table can be created to record the hyperparameter set for the predetermined task feature dimension, in which each task feature combination and its corresponding hyperparameter combination can be recorded.

[0111] The search unit 502 can search for the hyperparameters corresponding to a predetermined K task feature combinations from the hyperparameter set based on the first task feature combination of the task to be processed. Here, the first task feature combination is a combination of feature values ​​for each task feature dimension in the predetermined task feature dimensions of the task to be processed, and K is an integer greater than 0. In other words, for a new model training task (i.e., the task to be processed), the search space can be adaptively adjusted according to resource availability. For example, the search unit 502 can find the K task feature combinations that are closest to the first task feature combination of the task to be processed from the hyperparameter set (e.g., an empirical table recording the hyperparameter set) based on the first task feature combination of the task to be processed, and obtain the hyperparameters (or hyperparameter combinations) corresponding to these K task feature combinations from the hyperparameter set.

[0112] According to the exemplary embodiments of this disclosure, the search unit 502 can select K task feature combinations from the hyperparameter set based on the proximity of the feature values ​​of each task feature dimension in the first task feature combination to the feature values ​​of the corresponding task feature dimensions in the hyperparameter set, wherein the feature value of at least one task feature dimension in the K task feature combinations has the highest proximity to the feature value of the corresponding task feature dimension in the first task feature combination. For example, among multiple task feature dimensions, the search unit 502 can prioritize the proximity of feature values ​​based on the dataset size to select K task feature combinations. Of course, the closest task feature combination can be found according to any feasible rule.

[0113] The search unit 503 can search for the hyperparameters corresponding to the task to be processed based on the K hyperparameters found.

[0114] According to an exemplary embodiment of this disclosure, when K is 1, the search unit 503 can directly use the found hyperparameters as the hyperparameters corresponding to the task to be processed. When K is greater than 1, the search unit 503 can interpolate the found K hyperparameters to obtain a second hyperparameter search range; and search for the optimal hyperparameter within the second hyperparameter search range as the hyperparameter corresponding to the task to be processed.

[0115] Furthermore, during the hyperparameter search process, the dataset needs to be split, and the optimal hyperparameters are searched based on the multiple subset datasets obtained from the split. That is, the dataset can be split into multi-fold training and validation sets. In each attempt of the hyperparameter search, the model is trained on the training set of each fold, and the model's performance is tested on the validation set. The average performance on the multi-fold validation set is used as the performance metric for this hyperparameter search. This disclosure proposes a dataset splitting method that can perform effective data splitting for image datasets (typically containing a large number of objects of different types) used for tasks such as image semantic segmentation or image object detection. Specifically, the image dataset can be split into multiple subset datasets based on the distribution of each object category within at least one object category in the image dataset. The general idea is to first randomly distribute image samples without objects from the image dataset into multiple subsets. Then, image samples with objects are evenly distributed into multiple subsets based on the distribution of each object category. More specifically, the distribution can be based on the difference between the distribution of each object category in each subset and its distribution in the entire dataset, ensuring that the sample distribution of each object category in each subset is approximately close to the overall sample distribution. Of course, this dataset splitting method can be applied not only to hyperparameter search scenarios but also to any scenario requiring the splitting of the full dataset into multiple subsets, ensuring that the data distribution of the subsets approximates the distribution of the full dataset, thus preventing the hyperparameter search results from being affected by the dataset splitting.

[0116] According to an exemplary embodiment of this disclosure, the hyperparameter search apparatus 500 may further include a splitting unit (not shown). When the dataset used by the task to be processed is an image dataset, the splitting unit may split the image dataset into N subsets based on the distribution of each object category of at least one object category in the image dataset, where N is an integer greater than 1; and search for optimal hyperparameters based on the N subsets. For example, in the hyperparameter search apparatus according to this disclosure described above, when constructing the hyperparameter set, the optimal hyperparameter is pre-searched within a first hyperparameter search range for each task feature combination. Furthermore, when searching for hyperparameters corresponding to the task to be processed, the optimal hyperparameter is searched within a second hyperparameter search range for the task to be processed. In both of these hyperparameter search processes, the splitting unit is required to split the dataset; therefore, the dataset splitting method described above according to this disclosure can be applied. The following will refer to… Figure 6 To specifically describe the dataset splitting apparatus according to this disclosure.

[0117] Figure 6 This is a block diagram illustrating a dataset splitting apparatus according to an exemplary embodiment of the present disclosure. Figure 6 The dataset splitting device shown can be applied to any scenario that requires splitting the full dataset into multiple sub-datasets, so that the data distribution of the sub-datasets can be close to the distribution of the full dataset.

[0118] Reference Figure 6 The dataset splitting apparatus 600 according to an exemplary embodiment of the present disclosure may include a dataset acquisition unit 601, a determination unit 602, and a splitting unit 603. Here, the splitting unit 603 in the dataset splitting apparatus 600 performs the same function as the splitting unit (not shown) in the hyperparameter search apparatus 500.

[0119] The dataset acquisition unit 601 can acquire an image dataset, which includes multiple image samples. Here, the image dataset can be a training dataset used for model training. For example, but not limited to, the image dataset can be a dataset used to train a model for image semantic segmentation, or it can be a dataset used to train a model for image object detection.

[0120] The determining unit 602 can determine at least one object category in the image dataset. For example, in an image semantic segmentation algorithm, the object can be a foreground object; therefore, the determining unit 602 can determine the foreground object categories included in the image samples in the image dataset. For example, the foreground object categories can be set to include people, vehicles, animals, buildings, etc. As another example, in an image object detection algorithm, the object can be a target object to be detected; therefore, the determining unit 602 can determine the target object categories included in the image samples in the image dataset. For example, the target object categories can be set to include cats, dogs, people, etc.

[0121] The splitting unit 603 can split the image dataset into N subsets based on the distribution of each object category in at least one object category in the image dataset, where N is an integer greater than 1.

[0122] According to an exemplary embodiment of the present disclosure, the splitting unit 603 can randomly and evenly distribute image samples in the image dataset that do not include objects into N sub-datasets, and distribute image samples in the image dataset that include objects into N sub-datasets based on a first distribution ratio and a second distribution ratio of each object category in the image samples in the image dataset, wherein the first distribution ratio is the distribution ratio of each object category in the image dataset in the image dataset, and the second distribution ratio is the distribution ratio of each object category in each sub-dataset in the sub-dataset.

[0123] According to an exemplary embodiment of this disclosure, the splitting unit 603 can allocate image samples including objects in the image dataset to the N subsets based on the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category, thereby minimizing the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category. In other words, through the above allocation method, the sample distribution of each object category in each split subset is substantially close to the overall sample distribution. According to an exemplary embodiment of this disclosure, the splitting unit 603 can, as shown in the example... Figure 3 The process shown is used to allocate image samples, including objects.

[0124] In step 301, the splitting unit 603 can statistically analyze the first distribution ratio R of the image dataset. j j is the object category number. For example, if the number of object categories is m, then j = 1, 2, ..., m.

[0125] In step 302, the splitting unit 603 may randomly select N image samples containing objects from the image samples containing objects in the image dataset, and randomly and evenly distribute them into N subsets. That is, each subset may be randomly assigned to one image sample from the N randomly selected image samples.

[0126] In step 303, the splitting unit 603 can calculate the second distribution ratio R of each subset of data. i,j And calculate the second distribution proportion R for each subset. i,j The first distribution ratio R of the corresponding object category j The difference r i,j r i,j =R j -R i,j, where i is the subset number, therefore i = 1, 2, ..., N.

[0127] In step 304, the splitting unit 603 randomly selects N image samples from the remaining image samples containing objects in the image dataset.

[0128] In step 305, the splitting unit 603 counts the number of image samples that have not yet been assigned among the N randomly selected image samples, including the category set C of all object categories;

[0129] In step 306, splitting unit 603 searches for gaps r from the category set C of each of the N subsets that have not yet been assigned to image samples. i,j The largest first category number c and the largest subset number s. For example, if the category set C includes categories 1, 2, 4, 6, m, and there are currently 1, 2, ..., n subsets of data that have not yet been assigned to image samples, then we can look up {r}. i,j The first category number c and the first subset number s corresponding to the maximum value of}(i∈{1,2,…,n},j∈C={1,2,4,6,m}) are, i.e.,

[0130] In step 307, the splitting unit 603 can allocate the image sample with the largest distribution of the object category corresponding to the first category number c among the unallocated image samples to the subset corresponding to the first subset number s. For example, the unallocated image samples include a first image sample and a second image sample. Both the first and second image samples include the object category corresponding to category code c, and the distribution of the object category corresponding to category code c in the first image sample is larger than the distribution of the object category corresponding to category code c in the second image sample. In this case, the splitting unit 603 can allocate the first image sample to the subset corresponding to the first subset number s. For example, in the case of image samples used for image semantic segmentation, the largest distribution may refer to the largest total number of pixels of that object category. As another example, in the case of image samples used for image object detection, the largest distribution may refer to the largest number of objects of that object category.

[0131] For the N image samples randomly selected in step 304, the splitting unit 603 may repeat steps 305 to 306 until the N image samples randomly selected in step 304 have been allocated. That is, the splitting unit 603 may allocate each of the N subsets to one of the N image samples according to steps 305 to 307. Of course, if only one image sample remains unallocated, corresponding to only one subset of the dataset remaining unallocated, the splitting unit 603 may skip steps 305 to 307 and directly allocate the image sample to that subset.

[0132] For the remaining image samples containing objects in the image dataset, the splitting unit 603 may repeat steps 303 to 307 until all image samples containing objects in the image dataset have been allocated.

[0133] According to an exemplary embodiment of this disclosure, when the image dataset is a dataset for image semantic segmentation, and the object is a foreground object, the first distribution ratio may be the ratio of the total number of pixels of the foreground object of each foreground object category in the image dataset to the total number of pixels of all image samples in the image dataset, and the second distribution ratio may be the ratio of the total number of pixels of the foreground object of each foreground object category in each subset of the dataset to the total number of pixels of all image samples in that subset of the dataset. In this case, an image sample excluding the object may represent an image sample containing only the background and excluding the foreground object.

[0134] According to an exemplary embodiment of this disclosure, when the image dataset is a dataset for image object detection, the object is a target object. A first distribution ratio can be the proportion of the number of target objects of each target object category in the image dataset to the total number of target objects in the image dataset. A second distribution ratio can be the proportion of the number of target objects of each target object category in each sub-dataset to the total number of target objects in that sub-dataset. In this case, image samples excluding objects can represent image samples excluding target objects. Furthermore, in this case, the granular statistical proportion and segmentation operation can be performed more finely, using different scale ranges for each target object category. For example, the first distribution ratio can be the proportion of the number of target objects of each target object category at each scale range in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio can be the proportion of the number of target objects of each target object category at each scale range in each sub-dataset to the total number of target objects in that sub-dataset.

[0135] The above has been referred to Figures 1 to 6 A hyperparameter search method and apparatus thereof, as well as a dataset splitting method and apparatus thereof, are described according to exemplary embodiments of the present disclosure.

[0136] Figure 5 The hyperparameter search device shown and Figure 6 The devices in the illustrated dataset splitting apparatus can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, each device may correspond to a dedicated integrated circuit, pure software code, or a module combining software and hardware. Furthermore, one or more functions implemented by each device may also be uniformly executed by components in a physical entity device (e.g., a processor, client, or server).

[0137] In addition, refer to Figure 1 The described hyperparameter search method and reference Figures 2 to 4 The described dataset splitting method can be implemented by a program (or instructions) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device causes the at least one computing device to perform a hyperparameter search method or a dataset splitting method according to the present disclosure.

[0138] The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, and servers. It should be noted that the computer program can also be used to perform additional steps beyond those described above, or to perform more specific processing while performing the above steps. The details of these additional steps and further processing are already described in the reference... Figures 1 to 4 The relevant methods were mentioned in the description of the process, so they will not be repeated here to avoid repetition.

[0139] It should be noted that each unit in the hyperparameter search apparatus and dataset splitting apparatus according to the exemplary embodiments of this disclosure can rely entirely on the operation of a computer program to realize its corresponding function. That is, the functional architecture of the computer program in each unit corresponds to each step, so that the entire system is called through a special software package (e.g., a lib library) to realize the corresponding function.

[0140] on the other hand, Figure 5 and Figure 6 The various units within can also be implemented using hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segment used to perform the corresponding operation can be stored in a computer-readable medium such as a storage medium, enabling the processor to perform the corresponding operation by reading and running the corresponding program code or code segment.

[0141] For example, exemplary embodiments of this disclosure can also be implemented as a computing device including a storage component and a processor, wherein the storage component stores a set of computer-executable instructions, and when the set of computer-executable instructions is executed by the processor, a hyperparameter search method or a dataset splitting method according to exemplary embodiments of this disclosure is executed.

[0142] Specifically, the computing device can be deployed on a server or client, or on node devices in a distributed network environment. Furthermore, the computing device can be a PC, tablet, personal digital assistant, smartphone, web application, or other device capable of executing the aforementioned set of instructions.

[0143] Here, the computing device is not necessarily a single computing device, but can be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The computing device can also be part of an integrated control system or system manager, or can be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0144] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0145] Some operations described in the hyperparameter search method or dataset splitting method according to exemplary embodiments of this disclosure can be implemented in software, some operations can be implemented in hardware, and some operations can be implemented in a combination of software and hardware.

[0146] The processor can execute instructions or code stored in one of the storage components, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transport protocol.

[0147] Storage components can be integrated with the processor, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, storage components can include separate devices, such as external disk drives, storage arrays, or other storage devices that can be used by any database system. Storage components and the processor can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the storage component.

[0148] In addition, the computing device may include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be interconnected via a bus and / or network.

[0149] The hyperparameter search method or dataset splitting method according to exemplary embodiments of this disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logic device or operate according to non-precise boundaries.

[0150] Therefore, refer to Figure 1 The described hyperparameter search method or reference Figures 2 to 4 The dataset splitting method can be implemented by a system comprising at least one computing device and at least one storage device containing storage instructions.

[0151] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing a hyperparameter search method or a dataset splitting method according to an exemplary embodiment of the present disclosure, and a storage device stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by at least one computing device, the hyperparameter search method or dataset splitting method according to the present disclosure is executed.

[0152] The foregoing has described various exemplary embodiments of this disclosure. It should be understood that the foregoing description is exemplary only and not exhaustive, and this disclosure is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A hyperparameter search method executed by a computer device, characterized in that, include: Obtain a set of hyperparameters for a predetermined task feature dimension, wherein each task feature dimension includes a predetermined number of feature values, the set of hyperparameters includes hyperparameters corresponding to each task feature combination among a number of task feature combinations, the number of task feature combinations are obtained by combining each feature value under each different task feature dimension, and the hyperparameters corresponding to each task feature combination are the optimal hyperparameters pre-searched within the first hyperparameter search range for each task feature combination. Based on the first task feature combination of the task to be processed, find the hyperparameters corresponding to the predetermined K task feature combinations from the hyperparameter set, wherein the first task feature combination is a combination of the feature values ​​of each task feature dimension in the predetermined task feature dimension of the task to be processed, and K is an integer greater than 0. Based on the found hyperparameters, search for the hyperparameters corresponding to the task to be processed; In the case where the dataset used by the task to be processed is an image dataset, the optimal hyperparameters can be searched in the following manner: Based on the distribution of each object category in at least one object category in the image dataset, the image dataset is split into N sub-datasets, where N is an integer greater than 1; Based on the N subsets of data, search for the optimal hyperparameters; Specifically, the step of splitting the image dataset into N sub-datasets based on the distribution of each object category within at least one object category in the image dataset includes: The image samples in the image dataset that do not include objects are randomly and evenly distributed among the N sub-datasets; Based on the first and second distribution ratios of each object category in the image samples containing objects in the image dataset, the image samples containing objects in the image dataset are distributed into the N sub-datasets, wherein the first distribution ratio is the distribution ratio of each object category in the image dataset in the image dataset, and the second distribution ratio is the distribution ratio of each object category in each sub-dataset in the sub-dataset.

2. The hyperparameter search method as described in claim 1, characterized in that, The step of searching for hyperparameters corresponding to K combinations of task features from the hyperparameter set based on the first combination of task features to be processed includes: Find the K task feature combinations that are closest to the first task feature combination from the hyperparameter set; Obtain the hyperparameters corresponding to the K combinations of task features from the hyperparameter set.

3. The hyperparameter search method as described in claim 2, characterized in that, The step of finding the K task feature combinations that are closest to the first task feature combination from the hyperparameter set includes: Based on the proximity of the feature value of each task feature dimension in the first task feature combination to the feature value of the corresponding task feature dimension in the hyperparameter set, K task feature combinations are selected from the hyperparameter set, wherein the feature value of at least one task feature dimension in the K task feature combinations is closest to the feature value of the corresponding task feature dimension in the first task feature combination.

4. The hyperparameter search method as described in claim 1, characterized in that, When K is 1, the step of searching for the hyperparameters corresponding to the task to be processed based on the found hyperparameters includes: The found hyperparameters are directly used as the hyperparameters corresponding to the task to be processed.

5. The hyperparameter search method as described in claim 1, characterized in that, When K is greater than 1, the step of searching for the hyperparameters corresponding to the task to be processed based on the found hyperparameters includes: The found hyperparameters are interpolated to obtain the second hyperparameter search range; The optimal hyperparameter is searched within the second hyperparameter search range and used as the hyperparameter corresponding to the task to be processed.

6. The parameter search method as described in claim 1, characterized in that, The process of allocating image samples containing objects in the image dataset to the N sub-datasets based on the first and second distribution ratios of each object category in the image samples containing objects in the image dataset includes: Based on the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category, the image samples including objects in the image dataset are distributed to the N subsets, so that the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category is minimized.

7. The parameter search method as described in claim 6, characterized in that, The method of allocating image samples containing objects in the image dataset to the N subsets based on the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category includes: Statistical analysis of the first distribution ratio of the image dataset , It is the object category number; N image samples containing objects are randomly selected from the image samples containing objects in the image dataset, and randomly and evenly distributed among the N subsets; The following first operation is performed repeatedly until all image samples containing objects in the image dataset have been allocated: Calculate the proportion of the second distribution for each subset of data. And calculate the second distribution proportion for each subset. The first distribution ratio of the corresponding object category The gap ,in, It is the sub-dataset number. ; From the remaining image samples containing objects in the image dataset, N image samples are randomly selected; The second operation is performed repeatedly until the N image samples are assigned to the N sub-data: The set of all object categories is included in the N image samples that have not yet been assigned. The gap is found from the category set of each of the N sub-data sets that have not yet been assigned to image samples in the current second operation. The largest first category number and the largest first subset number; The image sample with the largest distribution of the object category corresponding to the first category number among the unassigned image samples is assigned to the subset corresponding to the first subset number.

8. The parameter search method as described in claim 1, characterized in that, In the case where the image dataset is a dataset for image semantic segmentation, the object is a foreground object. The first distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in the image dataset to the total number of pixels of all image samples in the image dataset. The second distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in each sub-dataset to the total number of pixels of all image samples in that sub-dataset.

9. The parameter search method as described in claim 1, characterized in that, In the case where the image dataset is a dataset for image object detection, the object is a target object, the first distribution ratio is the proportion of the number of target objects of each target object category in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target category in each sub-dataset to the total number of target objects in that sub-dataset.

10. The parameter search method as described in claim 1, characterized in that, In the case where the image dataset is a dataset for image object detection, the object is a target object, the first distribution ratio is the proportion of the number of target objects of each target object category at each scale range in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target object category at each scale range in each sub-dataset to the total number of target objects in that sub-dataset.

11. The parameter search method as described in any one of claims 1 to 5, characterized in that, The predetermined task feature dimensions include at least one of the following: dataset size, number of object categories, sample distribution, and machine learning model size.

12. The parameter search method as described in any one of claims 1 to 5, characterized in that, The hyperparameters include at least one of batch size, learning rate, optimizer, learning rate adjustment strategy, and regularization coefficient applied to the optimizer.

13. A hyperparameter search device, characterized in that, include: The hyperparameter set acquisition unit is configured to acquire a hyperparameter set for a predetermined task feature dimension, wherein each task feature dimension in the predetermined task feature dimension includes a predetermined number of feature values, the hyperparameter set includes hyperparameters corresponding to each task feature combination in a number of task feature combinations, the number of task feature combinations are obtained by combining each feature value under each different task feature dimension, and the hyperparameters corresponding to each task feature combination are the optimal hyperparameters pre-searched for each task feature combination in a first hyperparameter search range; The lookup unit is configured to look up hyperparameters corresponding to a predetermined K task feature combinations from the hyperparameter set based on a first task feature combination of the task to be processed, wherein the first task feature combination is a combination of feature values ​​of each task feature dimension in the predetermined task feature dimension of the task to be processed, and K is an integer greater than 0. The search unit is configured to search for the hyperparameters corresponding to the task to be processed based on the found hyperparameters; This also includes: splitting units; When the dataset used by the task to be processed is an image dataset, the splitting unit is configured to split the image dataset into N sub-datasets based on the distribution of each object category in at least one object category in the image dataset, where N is an integer greater than 1; The search unit is configured to search for the optimal hyperparameters based on the N subsets of data. The splitting unit is configured as follows: The image samples in the image dataset that do not include objects are randomly and evenly distributed among the N sub-datasets; Based on the first and second distribution ratios of each object category in the image samples containing objects in the image dataset, the image samples containing objects in the image dataset are distributed into the N sub-datasets, wherein the first distribution ratio is the distribution ratio of each object category in the image dataset in the image dataset, and the second distribution ratio is the distribution ratio of each object category in each sub-dataset in the sub-dataset.

14. The hyperparameter search device as described in claim 13, characterized in that, The search unit is configured as follows: Find the K task feature combinations that are closest to the first task feature combination from the hyperparameter set; Obtain the hyperparameters corresponding to the K combinations of task features from the hyperparameter set.

15. The hyperparameter search device as described in claim 14, characterized in that, The search unit is configured as follows: Based on the proximity of the feature value of each task feature dimension in the first task feature combination to the feature value of the corresponding task feature dimension in the hyperparameter set, K task feature combinations are selected from the hyperparameter set, wherein the feature value of at least one task feature dimension in the K task feature combinations is closest to the feature value of the corresponding task feature dimension in the first task feature combination.

16. The hyperparameter search device as described in claim 13, characterized in that, When K is 1, the search unit is configured as follows: The found hyperparameters are directly used as the hyperparameters corresponding to the task to be processed.

17. The hyperparameter search device as described in claim 13, characterized in that, When K is greater than 1, the search unit is configured as follows: The found hyperparameters are interpolated to obtain the second hyperparameter search range; The optimal hyperparameter is searched within the second hyperparameter search range and used as the hyperparameter corresponding to the task to be processed.

18. The hyperparameter search device as described in claim 13, characterized in that, The split unit is configured as follows: Based on the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category, the image samples including objects in the image dataset are distributed to the N subsets, so that the difference between the second distribution ratio of each object category in each subset and the first distribution ratio of the corresponding object category is minimized.

19. The hyperparameter search device as described in claim 18, characterized in that, The split unit is configured as follows: Statistical analysis of the first distribution ratio of the image dataset , It is the object category number; N image samples containing objects are randomly selected from the image samples containing objects in the image dataset, and randomly and evenly distributed among the N subsets; The following first operation is performed repeatedly until all image samples containing objects in the image dataset have been allocated: Calculate the proportion of the second distribution for each subset of data. And calculate the second distribution proportion for each subset. The first distribution ratio of the corresponding object category The gap ,in, It is the sub-dataset number. ; From the remaining image samples containing objects in the image dataset, N image samples are randomly selected; The second operation is performed repeatedly until the N image samples are assigned to the N sub-data: The set of all object categories is included in the N image samples that have not yet been assigned. The gap is found from the category set of each of the N sub-data sets that have not yet been assigned to image samples in the current second operation. The largest first category number and the largest first subset number; The image sample with the largest distribution of the object category corresponding to the first category number among the unassigned image samples is assigned to the subset corresponding to the first subset number.

20. The hyperparameter search device as described in claim 13, characterized in that, In the case where the image dataset is a dataset for image semantic segmentation, the object is a foreground object. The first distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in the image dataset to the total number of pixels of all image samples in the image dataset. The second distribution ratio is the ratio of the total number of pixels of the foreground object of each foreground object category in each sub-dataset to the total number of pixels of all image samples in that sub-dataset.

21. The hyperparameter search device as described in claim 13, characterized in that, In the case where the image dataset is a dataset for image object detection, the object is a target object, the first distribution ratio is the proportion of the number of target objects of each target object category in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target category in each sub-dataset to the total number of target objects in that sub-dataset.

22. The hyperparameter search device as described in claim 13, characterized in that, In the case where the image dataset is a dataset for image object detection, the object is a target object, the first distribution ratio is the proportion of the number of target objects of each target object category at each scale range in the image dataset to the total number of target objects in the image dataset, and the second distribution ratio is the proportion of the number of target objects of each target object category at each scale range in each sub-dataset to the total number of target objects in that sub-dataset.

23. The hyperparameter search apparatus as described in any one of claims 13 to 17, characterized in that, The predetermined task feature dimensions include at least one of the following: dataset size, number of object categories, sample distribution, and machine learning model size.

24. The hyperparameter search apparatus as described in any one of claims 13 to 17, characterized in that, The hyperparameters include at least one of batch size, learning rate, optimizer, learning rate adjustment strategy, and regularization coefficient applied to the optimizer.

25. A computer-readable storage medium for storing instructions, wherein, When the instruction is executed by at least one computing device, it causes the at least one computing device to perform the hyperparameter search method as described in any one of claims 1 to 12.

26. A system comprising at least one computing device and at least one storage device for storing instructions, wherein, When the instruction is executed by the at least one computing device, it causes the at least one computing device to perform the hyperparameter search method as described in any one of claims 1 to 12.

Images