Data processing method, processing circuit, and program

Abstract

To automatically divide a dataset into a training dataset and a test set.SOLUTION: A data processing method includes: generating, for a first dataset, a plurality of first high-order features associated with for each task; obtaining information on the first high-order features to construct a library in units of the task, using a processing circuit; extracting, for a second dataset, a plurality of second high-order features associated with tasks of the second dataset; selecting, based on the information in the library, grouping feature for dividing data from the second high-order features, using the processing circuit; and dividing, based on the grouping feature, the second dataset into a training dataset and a test set, using the processing circuit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to a data processing method, a processing circuit, and a program. [Background technology]

[0002] The performance of neural network models used in deep learning is closely related to not only the structure of the model, but also the division of the data set. These models usually need to be trained using a large number of clinical samples, and when training the model, it is expected that the data is sufficiently large and can cover the data range of the model's actual application scenario. If the data set can be accurately divided into a training set and a test set, it can accelerate the training of the model and improve the performance of the model, which is often the key to determining whether the model is good or bad.

[0003] In conventional techniques, a data set is generally divided into a training set and a test set by random or manual splitting. However, random splitting cannot guarantee that different features of the data can be appropriately assigned to the training set and the test set, so the performance of the trained model is often not ideal. In addition, manual splitting relies on the expert's past experience and understanding of the data set to determine the accuracy of the split. In addition, a large amount of human and time resources are required to perform the split. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Chinese Patent No. 108319938 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to automatically divide a dataset into a training set and a test set. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0006] A data processing method according to an embodiment generates, for a first dataset, a plurality of first high-dimensional features associated with the task for each task, obtains information about the plurality of first high-dimensional features, and constructs a library with the task as a unit using a processing circuit; extracts, for a second dataset, a plurality of second high-dimensional features associated with the tasks of the second dataset, selects, by the processing circuit, grouping features for dividing data from the plurality of second high-dimensional features based on the information in the library, and divides the second dataset into a training set and a test set based on the grouping features using the processing circuit. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a flowchart showing a process for automatically dividing a data set in the first embodiment. [Figure 2A] FIG. 2A is a flow chart showing step 200 in detail. [Figure 2B] FIG. 2B is a diagram illustrating synthesis of multi-dimensional feature vectors into high-dimensional features. [Diagram 3] FIG. 3 is an example of a data fingerprint. [Figure 4] FIG. 4 is a flow chart showing step 300 in detail. [Diagram 5] FIG. 5 is an example showing a method for selecting grouping features. [Figure 6]Figure 6 is a flowchart of splitting the test set and the data set using a multiple random grouping scheme. [Figure 7] FIG. 7 is a diagram showing a comparison of model performance between the random grouping of the conventional technique and the automatic segmentation of the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment will be described with reference to the drawings. The embodiment described below does not limit the present invention. In the following description, components having substantially the same functions and configurations are denoted by the same reference numerals, and repeated description will be given only when necessary.

[0009] In the embodiments of the present application, the terms "first", "second", "third", etc. are used to distinguish different elements nominally, but do not indicate the spatial arrangement or temporal order of these elements, and these elements are not limited by these terms. The term "and / or" includes any one and all combinations of the associated listed term or terms. The terms "including", "including", "having", etc. refer to the presence of stated features, elements, or components, but do not exclude the presence or addition of one or more other features, elements, or components.

[0010] The terms "first high-dimensional features" and "second high-dimensional features" correspond to the high-dimensional features obtained during feature extraction of the reference dataset during construction of the a priori knowledge library and the high-dimensional features obtained during feature extraction of the target dataset during automatic dataset segmentation, respectively. Note that the reference dataset and the target dataset are also referred to as the first dataset and the second dataset, respectively.

[0011] The term "first threshold" refers to a threshold for indicating the task similarity between an existing task in the a priori knowledge library and a task to which the target dataset belongs.

[0012] The term "second threshold" refers to a threshold value that indicates the distribution similarity between the distribution of a candidate feature in the a priori knowledge library and the distribution of the candidate feature in the target dataset.

[0013] The term "third threshold" refers to a threshold for indicating the distribution similarity between the distribution of a non-candidate feature in a dataset to be split and a normal distribution. The above "first threshold", "second threshold", and "third threshold" are predefined and stored by a user before automatic splitting of a dataset.

[0014] The term "reference dataset" is the dataset used when building the a priori knowledge library, and the term "target dataset" is the dataset that we wish to automatically split into training and test sets.

[0015] (First embodiment) A splitting method for automatically splitting a dataset according to an embodiment includes an a priori knowledge library construction step of generating a plurality of first high-dimensional features associated with each task for a reference dataset, acquiring a priori knowledge information on the plurality of first high-dimensional features, and constructing an a priori knowledge library on a task-by-task basis; a grouping feature selection step of extracting a plurality of second high-dimensional features associated with tasks of a dataset to be split for a dataset to be split, and selecting grouping features for splitting data from the plurality of second high-dimensional features based on the a priori knowledge information in the a priori knowledge library; and a splitting step of automatically splitting the dataset to be split into a training set and a test set based on the grouping features.

[0016] Fig. 1 is a flowchart for automatically dividing a dataset in the first embodiment. As shown in Fig. 1, in order to automatically divide a dataset, it is necessary to first complete the construction of a priori knowledge library before dividing the dataset.

[0017] In FIG. 1, first, in step S100, a reference data set in which each data item includes a truth value (GT) is obtained.

[0018] Next, in step S200, a plurality of first high-dimensional features of data are generated for the reference data set in units of tasks, and a priori knowledge information including distribution status and importance weights for the plurality of first high-dimensional features is obtained based on the generated plurality of first high-dimensional features, and an a priori knowledge library is constructed. Then, the construction of the a priori knowledge library is completed.

[0019] Next, in step S300, a data set to be split is acquired, and a plurality of second high-dimensional features of the data are generated for the acquired data set to be split. Then, based on a priori knowledge information on the plurality of first high-dimensional features acquired in step S200, grouping features for splitting the data are selected from the plurality of second high-dimensional features.

[0020] Next, in step S400, the target dataset is automatically divided into a training set and a test set based on the selected grouping feature, and then in step S500, training and testing are performed using the divided training set and test set, respectively. This completes the automatic division of the dataset.

[0021] Each step will be described in detail below with reference to FIGS. 2A to 6.

[0022] (Building a priori knowledge libraries) 2A is a flowchart showing the details of step S200 (constructing an a priori knowledge library). Step S200 further includes a step of extracting a data fingerprint (step S210), a step of generating a plurality of first high-dimensional features based on the data fingerprint (step S220), and a step of constructing an a priori knowledge library (step S230).

[0023] It is known that raw data in a dataset generally contains a large amount of information, and different data contains different information. Since information extracted from similar tasks often has similarities, features (data fingerprints) of each task are extracted from the dataset as a unit of tasks. In this embodiment, as shown in FIG. 3, based on a priori knowledge, the raw data in the dataset is divided according to four main dimensions: inside the target GT, between the target GT, the target GT and its surroundings, and non-image information, and different types of data fingerprints are included in each dimension. Taking the dimension between the target GT as an example, it is assumed that this dimension also contains different types of data fingerprints such as overlap, distance, and connectivity. As the amount of data expands and the types of tasks become more abundant, these data fingerprints are also continuously updated. Therefore, it is expected that high-density data features will be collected (step S210) by extracting data fingerprints in different dimensions and different types from the dataset.

[0024] For image information (inside the target GT, between target GTs, and the target GT and its surroundings), features can be extracted using convolutional neural networks (CNNs) and for non-image information such as text, it can be converted into features using embedding.

[0025] Furthermore, in this embodiment, features in each dimension are not used as they are, but high-dimensional features with higher information density are used.

[0026] As a method for generating high-dimensional features, for example, in each of m dimensions such as inside the target GT, between the target GT, the target GT and its surroundings, and non-image information, each data fingerprint is weighted, averaged, max-min, median, or stitched to generate a multidimensional feature vector X i (X i =[x i1 ,x i2 ,x i3 …xij ],i=1,2,3,…m,j is the dimension of the feature vector), and to improve the characterization ability for the dataset, we aggregate the dataset into a multidimensional feature vector set X={x1,x2,…,x m}, the feature vectors in X are fused using a method such as Cartesian product to generate the above multidimensional feature vector as a high-dimensional feature F (step S220).

[0027] The generated high-dimensional feature F corresponds to the first high-dimensional feature in the feature extraction of the reference dataset, and corresponds to the second high-dimensional feature in the feature extraction of the dataset to be split.

[0028] Although the method for generating high-dimensional features has been described above, in the following, for ease of understanding, the high-dimensional features F generated in the process of constructing the a priori knowledge library will be referred to as the first high-dimensional features, and the high-dimensional features F generated in the process of automatically dividing the data set to be divided will be referred to as the second high-dimensional features. However, in order to show their commonality, there are cases where the first high-dimensional features and the second high-dimensional features are not distinguished from each other and are simply referred to as high-dimensional features.

[0029] In this embodiment, the a priori knowledge library is constructed on a task-by-task basis, with each task having its own data set, and after generating multiple first high-dimensional features for each task, the distribution status of GTs within each first high-dimensional feature is statistically calculated.

[0030] Also, for different types of tasks, one or more features may have different importance for interpreting differences, and therefore may be assigned different weights. In constructing the a priori knowledge library, the importance weights of the first high-dimensional features may be predefined based on experience, or may be automatically obtained by other models.

[0031] As a result, when constructing the a priori knowledge library, a priori knowledge information such as the corresponding high-dimensional features (first high-dimensional features) according to different types of tasks, the distribution status of each high-dimensional feature (first high-dimensional feature), and the importance weights can be saved according to different types of tasks, as shown in Table 1 described below, and can be used later when comparing with the dataset to be divided.

[0032] [Table 1]

[0033] Table 1 is only an example, and the high-dimensional features between tasks may overlap with each other, and the number of high-dimensional features varies depending on the task.

[0034] (Selecting grouping features) FIG. 4 is a flow chart showing the details of the selection of grouping features, and is a flow chart showing details of step 300 described above.

[0035] 4, in the grouping feature selection step, first, a data set to be split is acquired, and in step S310, a plurality of second high-dimensional features of data are generated for the acquired data set to be split in the same manner as in generating the first high-dimensional feature, and a distribution state regarding the second high-dimensional feature is acquired for each of the second high-dimensional features. As described above, the second high-dimensional feature is obtained by aggregating feature information of data extracted from each dimension for each of a plurality of predetermined dimensions into a multidimensional feature vector, and fusing the feature vector in X by a method such as a Cartesian product, similar to the first high-dimensional feature.

[0036] Next, in step S320, the task similarity between the tasks of the data set to be split and each task in the a priori knowledge library is compared. For example, the task similarity between the tasks of the data set to be split and existing tasks in the a priori knowledge library is compared according to the subject, the organ to which the subject belongs, and the purpose of the task.

[0037] Specifically, when organs are divided into head and neck, chest, abdomen, lower limbs, etc. according to the organ position, similarity is considered high if they belong to the same organ, and similarity is considered low as the distance between organs increases.

[0038] When objects can be classified into blood vessels, organ segments or lobes, tumors, etc. according to the object of the task, if they belong to the same object, the similarity is considered to be high, and the larger the difference in the object's morphology, the lower the similarity.

[0039] Regarding the task objectives, when the objectives are divided into segmentation, positioning, detection, classification, etc. according to the task objectives, if they belong to the same task objective, they are considered to have a high similarity, and if they belong to different task objectives, they are considered to have a low similarity.

[0040] When it is desired to comprehensively consider the similarities in each of the above aspects, the above similarities can be weighted and added together to obtain a total task similarity. In this embodiment, when comparing the tasks in the data set to be divided with the existing tasks in the a priori knowledge library, the comparison is performed based on the total task similarity.

[0041] Next, in step S330, it is determined whether or not there is an existing task in the a priori knowledge library whose task similarity with the splitting target dataset is equal to or greater than a first threshold value. If it is determined that there is no existing task, the splitting target dataset is not similar to all the existing tasks previously stored, and the splitting target dataset is typical, so that the splitting target dataset needs to be updated to the a priori knowledge library.

[0042] In this case, the process proceeds to step S340. In step S340, all of the second high-dimensional features in the data set to be split are classified as non-candidate features F1 to F2. n Using the distribution status of the plurality of second high-dimensional features generated in step S310, a distribution similarity D between the distribution status of each second high-dimensional feature in the data set to be divided and a normal distribution is calculated by a method such as a chi-square test or KL divergence. i, (i=1,2,3...n) and calculate the similarity D i By normalizing as in equation (1), the importance weights W i get.

[0043]

number

[0044] Next, if the task similarity is less than the first threshold, at least some of the second high-dimensional features are selected from the plurality of second high-dimensional features in step S350. Here, the at least some of the second high-dimensional features may be all of the second high-dimensional features, or may be a second high-dimensional feature further selected from all of the second high-dimensional features. When it is desired to select a representative high-dimensional feature for a task at the time of updating, a second high-dimensional feature whose distribution similarity to the distribution situation when the distribution situation is a normal distribution is greater than a third threshold value determined in advance may be selected. That is, in the data processing method according to the embodiment, in the selection of grouping features, a plurality of second high-dimensional features associated with the tasks of the second data set are extracted for the second data set.

[0045] Next, in step S360, the tasks of the dataset to be split, at least a portion of the selected second high-dimensional features, distribution information corresponding to the at least a portion of the second high-dimensional features, and importance weights corresponding to the at least a portion of the second high-dimensional features are jointly updated into the a priori knowledge library.

[0046] If it is determined in step S330 that an existing task whose task similarity is equal to or greater than a first threshold value is present in the a priori knowledge library, the process proceeds to step S370. In step S370, one or more existing tasks having the highest task similarity are selected from the existing tasks that satisfy the first threshold value retrieved from the a priori knowledge library, and a priori knowledge information (distribution status and importance weight) regarding the first high-dimensional features associated with the one or more existing tasks is acquired. That is, in the data processing method according to the embodiment, an existing task having the highest task similarity with the second data set and a plurality of first high-dimensional features associated with the existing task are retrieved from the a priori knowledge library.

[0047] Next, in step S380, candidate features and non-candidate features are obtained based on the retrieved first high-dimensional features and the generated second high-dimensional features. As an example, in the data processing method according to the embodiment, among the second high-dimensional features, a second high-dimensional feature that overlaps with the retrieved first high-dimensional features is set as a candidate feature. As shown in FIG. 5, the first high-dimensional feature is a high-dimensional feature stored in the a priori knowledge library with a task as a unit, and the second high-dimensional feature is a high-dimensional feature with a task as a unit acquired from the data set to be divided. The second high-dimensional feature can be divided into candidate features and non-candidate features. The candidate feature is a part of the second high-dimensional features that overlaps with the retrieved first high-dimensional features. The non-candidate feature is a part of the second high-dimensional features that does not overlap with the retrieved first high-dimensional features. In this embodiment, the features for automatically dividing the dataset into a test set and a training set are called grouping features, and the grouping features are selected from the second high-dimensional features. However, the grouping features may be selected only from the candidate features, or may be selected from both the candidate features and the non-candidate features.

[0048] The two grouping feature selection methods are described below.

[0049] (Select from candidate features only) By the selection in steps S310 to S330 and S370, the existing tasks in the searched a priori knowledge library can be regarded as tasks with high task similarity to the data set to be divided. If the task similarity is high, there is a high possibility that the same features exist in different tasks.

[0050] As mentioned above, the candidate features are common features between the first high-dimensional feature and the second high-dimensional feature, i.e., the candidate features are common features between the existing task and the task of the target dataset. The features shared by similar tasks are considered to be key features for automatic segmentation of the target dataset. Using the key features as grouping features is expected to produce good segmentation results.

[0051] Therefore, when selecting grouping features from the candidate features, at least some of the candidate features are selected as grouping features as in step S391. Here, "at least some" may mean all of the multiple candidate features or may mean some of the multiple candidate features.

[0052] In one selection method, when selecting a grouping feature from the candidate features, for each of the multiple candidate features, the distribution similarity in the distribution situation between the candidate feature in the data set to be divided and the candidate feature in the existing task searched from the a priori knowledge library is compared, and the candidate feature whose distribution similarity is equal to or greater than a second predetermined threshold is selected as the grouping feature. The distribution similarity is obtained by a method such as a chi-square test or KL divergence.

[0053] In addition, the a priori knowledge library stores an importance weight that corresponds one-to-one to each high-dimensional feature. By referring to the importance weights stored in the a priori knowledge library, it is possible to know which features are important for the existing tasks. Since the task similarity between the existing tasks and the data to be split is high, it can be roughly inferred that features that are important for the existing tasks are also important for the tasks of the data set to be split.

[0054] Therefore, as another selection method, the top K candidate features with the highest importance weights are selected from the multiple candidate features based on the importance weights corresponding to each candidate feature stored in the a priori knowledge library, and for each of the top K (K is an integer) candidate features, the distribution similarity in the distribution situation between the candidate feature in the dataset to be split and the candidate feature in an existing task searched from the a priori knowledge library is compared, and one or more candidate features whose distribution similarity is equal to or greater than a predetermined second threshold can be set as grouping features.

[0055] (Select from candidate features and non-candidate features) In addition to selecting grouping features from the candidate features, grouping features can also be selected from non-candidate features. Since non-candidate features are not stored in the a priori knowledge library but are important for some tasks, grouping features are selected from the non-candidate features in addition to the candidate features, as in step S390. By selecting grouping features from each of the candidate features and non-candidate features, the accuracy of data division can be further improved.

[0056] As a method for selecting a grouping feature from non-candidate features, it is possible to select one or more non-candidate features as grouping features from the non-candidate features based on the distribution status of the non-candidate features. Specifically, by utilizing the distribution status corresponding to the non-candidate features among the plurality of second high-dimensional features generated in step S310, a distribution similarity D between the distribution status and a normal distribution is calculated from the plurality of non-candidate features by a method such as chi-square test or KL divergence. i, (i=1, 2, 3...n) is greater than a third predetermined threshold and one or more non-candidate features are selected as grouping features.

[0057] In addition, the distribution similarity D i After obtaining the similarity scores D for each non-candidate feature (i=1,2,3...n), i By normalizing it as in Equation 1 above, the importance weight W i This importance weight W i can then be used for multiple random splits.

[0058] In other words, when the grouping features are derived from candidate features, the importance weights corresponding to the grouping features are the importance weights corresponding to each of the candidate features stored in the a priori knowledge library, and when the grouping features are derived from non-candidate features, the importance weights corresponding to the grouping features may be, for example, importance weights normalized by the distribution similarity between the distribution situation of the non-candidate features in the second dataset (the dataset to be split) and a normal distribution.

[0059] In addition, in this embodiment, the dataset to be split may be automatically split into a training set and a test set based on grouping features from the candidate features and / or grouping features from the non-candidate features, or the dataset to be split may be split into a training set and a test set based on grouping features manually selected from the above grouping features (grouping features from candidate features and / or grouping features from non-candidate features).

[0060] The procedure for automatically splitting a data set to be split will be described below with reference to Fig. 6. Fig. 6 is a flowchart showing the details of step S400 in Fig. 1, and shows the flow of splitting a test set and a training set using a multiple-time random grouping method.

[0061] In the prior art, a random grouping method is used to divide the training set and the test set, but the distribution of features in each randomly grouped data set is not necessarily the same as the distribution of features in the entire data set, resulting in an unreasonable data distribution problem. Step S400 aims to select a grouping method in which the distribution of grouping features in each data set is closest to the distribution of grouping features in the entire data set.

[0062] In this embodiment, random grouping is performed multiple times, the score for each random grouping is calculated, and the random grouping with the highest score is selected. One of the two groups obtained in that random grouping is used as the training set, and the remaining group of the two groups obtained in that random grouping is used as the test set.

[0063] Specifically, first, in step S410, the data set to be divided is randomly divided into two groups (group A and group B) at a fixed ratio (for example, a ratio of 20%:80%).

[0064] Next, in step S420, for the grouping features selected in step S300, the distribution similarity in the distribution situation between the grouping features of each group (group A and group B) itself and the grouping features of the entire data set to be divided is compared for each grouping feature, and a score is assigned based on the distribution similarity.

[0065] In addition, the importance weight corresponding to each grouping feature can be used to know which grouping feature is more important for the current task, and the weight of the grouping feature can be increased to obtain a better segmentation result. If the grouping feature is selected from the candidate features, the corresponding importance weight is the importance weight stored in the a priori knowledge library, and if the grouping feature is selected from the non-candidate features, the corresponding importance weight is the one calculated using Equation 1 above.

[0066] Therefore, in step S430, the score obtained in step S420 is weighted according to the importance weight of each grouping feature, and a weighted score for the current grouping is calculated.

[0067] Next, in step S440, it is determined whether or not the procedure of steps S410 to S430 has been executed a predetermined number of times (for example, N times), and if it has not been executed N times, the process returns to step S410 and continues to execute steps S410 to S430. If it has been executed N times, the process proceeds to step S450.

[0068] In step S450, the weighted scores of each of the N random groupings are compared, and the grouping scheme with the highest score is selected. Group B in the grouping scheme is used as the training set, and group A in the grouping scheme is used as the test set.

[0069] As described above, in the data processing method according to the embodiment, in dividing a second dataset (dataset to be divided), the second dataset is randomly divided into two groups, and for each grouping feature, a distribution similarity between the distribution status of each of the two grouped groups of datasets and the distribution status of the entire dataset is calculated, and a weight is assigned to each of the calculated distribution similarities based on the importance weight corresponding to each grouping feature. After repeatedly calculating a grouping score for each group, the grouping scheme with the highest score is retained, and one group of the grouping scheme with the highest score is used as a training set, and the remaining group of the grouping scheme with the highest score is used as a test set.

[0070] Next, in step S500, training is performed using the training set and test set obtained in step S400.

[0071] Although the method of dividing the test set and the data set using the multiple random grouping method has been described above, the data set to be divided may be automatically divided into the training set and the test set using the Kennard-Stone algorithm based on the grouping features selected in step S300. The Kennard-Stone algorithm is a conventional technique, and therefore a detailed description thereof will be omitted.

[0072] By adopting the partitioning method for automatically partitioning a dataset according to the present embodiment, different high-dimensional features for different tasks and the distribution status and importance weights corresponding to each high-dimensional feature can be obtained. By comparing the dataset to be partitioned with an a priori knowledge library constructed in advance based on this information, grouping features that have a large impact on the grouping effect can be selected from the dataset to be partitioned. Then, by performing grouping based on the selected grouping features, the data distribution can be made more rational and the performance of the model can be further improved.

[0073] Compared to manual data grouping, this embodiment reduces or even avoids the dependency on the expert's level of understanding of the dataset attributes and task goals, saving significant human and time resources.

[0074] As shown in FIG. 7, compared with the conventional random grouping method, this embodiment can make the data distribution more reasonable and improve the model performance.

[0075] Furthermore, since the segmentation method for automatically segmenting a dataset according to the present embodiment does not require manual intervention, the technique can also be integrated into an online fully automated training system as a front-end module.

[0076] (Modification of the first embodiment) As described in the first embodiment, different types of tasks correspond to different high-dimensional features, and after generating the high-dimensional features, information such as distribution status and importance weights corresponding to the high-dimensional features can be obtained. By comparing the distribution status of the multiple high-dimensional features (first high-dimensional features) stored in the a priori knowledge library with the distribution status of the multiple high-dimensional features (second high-dimensional features) in the data set to be split, the difference in a certain high-dimensional feature between the data in the data set to be split and the data of the existing task can be found. Here, the certain high-dimensional feature is a high-dimensional feature shared by the multiple first high-dimensional features and the multiple second high-dimensional features, that is, a candidate feature that is an intersection of the first high-dimensional feature and the second high-dimensional feature shown in FIG. 5.

[0077] Furthermore, as described in the first embodiment, when selecting grouping features from candidate features, it is necessary to consider whether the similarity of the distribution states of the first high-dimensional feature and the second high-dimensional feature is greater than a predetermined threshold, and to select only candidate features whose similarity of distribution states is greater than the predetermined threshold as grouping features. However, if the difference in distribution states of all candidate features is less than the predetermined threshold, it is not possible to select grouping features from the candidate features. Also, due to an insufficient amount of samples, grouping features that should be extracted from the candidate features may not be correctly extracted.

[0078] The variant relates to a data supplementation strategy that can intelligently increase the amount of samples after finding candidate features with too large a discrepancy.

[0079] For example, when comparing the distribution status for each candidate feature, if the difference in distribution status between the candidate feature of the data set to be split and the candidate feature of the existing task retrieved from the a priori knowledge library is greater than a predetermined threshold, the candidate feature is not discarded as is, but the amount of samples for the candidate feature is first increased, and the distribution status of the candidate feature with the replenished sample amount is then compared between the data set to be split and the existing task. That is, in splitting the second data set (data set to be split), if the distribution similarity in the distribution status between the candidate feature in the second data set and the candidate feature in the existing task retrieved from the a priori knowledge library is less than a second threshold, the amount of samples for the candidate feature in the second data set is increased.

[0080] Therefore, according to this modification, failure to extract grouping features or inaccurate extraction due to an insufficient amount of samples can be eliminated.

[0081] Second Embodiment In the above modification, a measure to deal with the large difference (in distribution state) when the sample is easily replenished is described. However, there are also cases where the amount of sample is difficult to replenish.

[0082] In response to the above-mentioned situation, the second embodiment proposes a data augmentation measure in which, after automatic division of the data set to be divided is completed, data augmentation is performed on grouping features in the divided training set that are significantly different from the a priori knowledge library.

[0083] For example, after automatic division of the data set to be divided is completed, for a certain grouping feature, the difference in distribution between that grouping feature in the training set and the grouping feature in an existing task searched for from the a priori knowledge library is compared, and if the difference in distribution is large, for example exceeding a predetermined threshold, the number of samplings is increased for the grouping feature with a large distribution difference, and the number of samplings for that grouping feature in the divided training set is increased to expand the amount of data.

[0084] Here, what needs to be compared is the difference between the distribution of grouping features in the training set and the distribution of grouping features in existing tasks (high task similarity) retrieved from the a priori knowledge library, so the grouping features are actually candidate features that are common to the first high-dimensional feature and the second high-dimensional feature described above. Note that, although the training set is exemplified here, data expansion may be performed on grouping features in the test set that are significantly different from existing tasks in the a priori knowledge library.

[0085] According to the second embodiment, when it is difficult to add a sample amount, data expansion can be performed appropriately for grouping features with large differences in distribution state.

[0086] (Third embodiment) For tasks where automatic segmentation has been completed or where automatic segmentation is in progress, data sets may be added due to new targets or demands. The third embodiment relates to situations where new data needs to be added while automatic segmentation is in progress or after completion, and this third embodiment can be seamlessly applied to the first embodiment.

[0087] For example, if you want to add a new dataset while the automatic splitting of the dataset to be split is in progress or after completion, you can automatically split the newly added dataset into a training set and a test set according to the splitting method for automatically splitting a dataset related to the first embodiment, and add the newly split training set and test set to the existing training set and test set, respectively.

[0088] According to the third embodiment, when new data is to be added, whether during or after the task is completed, it is not necessary to repeatedly perform the entire automatic splitting process, but only split the new data into training and test sets according to the method of the first embodiment, and add them to the existing training and test sets, which facilitates data replenishment and improves the efficiency of the model.

[0089] (Other embodiments) In the above embodiment, the method for automatically dividing a data set is described using high-dimensional features as an example in order to obtain more comprehensive information. However, it should be understood that the present application also includes automatically dividing a data set using low-dimensional features.

[0090] In the above embodiment, a method for automatically dividing a dataset has been described. However, it should be understood that it is sufficient to realize each function in the method for automatically dividing a dataset, and that these functions may be realized by hardware or a mixture of hardware and software.

[0091] For example, when a division method for automatically dividing a data set is realized by a processor, which is a processing circuit, each processing function of the processor is stored in a storage circuit in the form of a program executable by a computer. The processor realizes the function corresponding to each program by reading each program from the storage circuit and executing it. Also, instead of storing the program in the storage circuit, the program may be configured to be directly incorporated into the circuit of the processor. In this case, the processor realizes the function by reading and executing the program incorporated in the circuit. Also, the processor of the above-mentioned embodiment is not limited to being configured as a single circuit, and may be configured as one processor by combining multiple independent circuits to realize its function.

[0092] The term "processor" used in the above description refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)).

[0093] The program executed by the processor may be provided by being pre-installed in a ROM (Read-Only Memory), a storage circuit, etc. Also, the program may be provided by being recorded in a computer-readable storage medium such as a CD (Compact Disc)-ROM, a FD (Flexible Disk), a CD-R (Recordable), or a DVD (Digital Versatile Disc) in a format that can be installed in these devices or in a format that can be executed.

[0094] The program may also be stored on a computer connected to a network such as the Internet, and provided or distributed by being downloaded via the network.

[0095] Alternatively, instead of the division method for automatically dividing a dataset, a division model for automatically dividing a dataset may be stored on a computer connected to a network such as a server or the Internet, and another medical information processing device may download the model via the network and use it. The other medical information processing device can realize the effect of automatically dividing a dataset by calling up the division model for automatically dividing a dataset. For example, the other medical information processing device includes a processing circuit for executing a construction process of an a priori knowledge library for generating a plurality of first high-dimensional features associated with each task for a reference dataset, acquiring a priori knowledge information on the plurality of first high-dimensional features, and constructing an a priori knowledge library for each task, a selection process of a grouping feature for extracting a plurality of second high-dimensional features associated with the tasks of the division target dataset for a division target dataset, and selecting a grouping feature for dividing data from the plurality of second high-dimensional features based on the a priori knowledge information in the a priori knowledge library, and a division process of a division target dataset for automatically dividing the division target dataset into a training set and a test set based on the grouping feature.

[0096] According to at least one of the embodiments described above, the following advantages can be achieved.

[0097] 1. Reduce or even avoid the dependency on the level of experts' understanding of the attributes of the dataset and the task objectives, saving a large amount of human and time resources;

[0098] 2. Balance the allocation of data features to the training set and the test set to improve the model performance;

[0099] 3, Can be integrated into online fully automatic training system as a front-end module;

[0100] 4. Apply to multiple tasks;

[0101] 5. It can eliminate the failure or inaccuracy of extracting grouping features caused by insufficient sample size;

[0102] 6. When it is difficult to add samples, data augmentation can be performed accurately for high-dimensional features with large distribution differences;

[0103] 7. It is convenient to replenish data and is suitable for the task of adding new data.

[0104] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0105] S100 Step S100 S200 Step S200 S300 Step S300 S400 Step S400 S500 Step S500

Claims

1. generating, for each task of a first data set, a plurality of first high-dimensional features associated with the task, obtaining information about the plurality of first high-dimensional features, and constructing a library with the task as a unit by a processing circuit; extracting, for a second data set, a second plurality of high dimensional features associated with the task of the second data set, and selecting, by the processing circuitry, grouping features from the second plurality of high dimensional features for partitioning data based on the information in the library; and dividing the second data set by the processing circuitry into a training set and a test set based on the grouping features.

2. 2. The data processing method according to claim 1, wherein in constructing the library, after generating the plurality of first high-dimensional features, a distribution status and an importance weight of each of the first high-dimensional features are obtained as the information for each of the first high-dimensional features, and the library is constructed.

3. The data processing method according to claim 2 , wherein in the selection of the grouping features, a distribution status and an importance weight of each of the second high-dimensional features are obtained for each of the second high-dimensional features.

4. 4. The data processing method according to claim 3, wherein the first high-dimensional feature and the second high-dimensional feature are obtained by aggregating and fusing feature information of data extracted from each dimension for each of a predetermined number of dimensions into a multidimensional feature vector.

5. 4. The data processing method according to claim 3, wherein in the selection of the grouping features, one or more second high-dimensional features among the plurality of second high-dimensional features that overlap with the plurality of first high-dimensional features are set as candidate features, and one or more second high-dimensional features among the plurality of second high-dimensional features that do not overlap with the plurality of first high-dimensional features are set as non-candidate features, and the grouping feature is selected from the candidate features.

6. In the grouping feature selection, after extracting a plurality of second high-dimensional features for the second dataset, the second high-dimensional features being associated with a task of the second dataset, 6. The data processing method according to claim 3, further comprising: searching the library for an existing task having the highest task similarity with the second dataset and the plurality of first high dimensional features associated with the existing task when a task similarity between the task of the second dataset and an existing task in the library is equal to or greater than a first threshold; selecting, from the plurality of second high dimensional features, a second high dimensional feature that overlaps with the searched plurality of first high dimensional features as a candidate feature; and selecting, from the candidate features, a candidate feature to be used for grouping as a grouping feature by a predetermined method.

7. selecting at least a portion of second high-dimensional features from the plurality of second high-dimensional features when the task similarity is less than the first threshold; The data processing method of claim 6 , further comprising updating the at least some of the second high-dimensional features, distributions and importance weights for the second high-dimensional features, and tasks for the second dataset together with the library.

8. The data processing method according to claim 6 , further comprising comparing a task similarity between the task of the second data set and the existing task according to the subject, the organ to which the subject belongs, and a purpose of the task.

9. The predetermined method comprises:

7. The data processing method according to claim 6, further comprising: comparing distribution similarity in distribution state between the candidate features of the second dataset and the candidate features of the existing task searched from the library for each of the plurality of candidate features; and selecting the candidate feature whose distribution similarity is equal to or greater than a predetermined second threshold as the grouping feature.

10. The predetermined method comprises:

7. The data processing method according to claim 6, further comprising the steps of: selecting, from the plurality of candidate features, a top K number of candidate features (K being an integer) having the greatest importance weights based on an importance weight corresponding to each of the candidate features stored in the library; comparing, for each of the top K number of candidate features, a distribution similarity in a distribution situation between the candidate features of the second dataset and the candidate features of the existing task searched from the library; and determining, as grouping features, one or more candidate features whose distribution similarity is equal to or greater than a predetermined second threshold.

11. 10. The data processing method according to claim 9, further comprising: increasing an amount of samples relating to a candidate feature of the second dataset when, in dividing the second dataset, a distribution similarity between a distribution situation of the candidate feature in the second dataset and a candidate feature in the existing task searched from the library is less than the second threshold.

12. In the selection of the grouping feature, The data processing method of claim 5 , wherein grouping features are selected from the non-candidate features in addition to the candidate features.

13. In the selection of the grouping feature, The data processing method according to claim 12 , further comprising the step of selecting one or more of the non-candidate features as the grouping features from a plurality of the non-candidate features based on a distribution situation for the non-candidate features.

14. In the selection of the grouping feature, The data processing method according to claim 13 , further comprising the step of selecting, from the plurality of non-candidate features, one or more of the non-candidate features whose distribution similarity when distributed normally is equal to or greater than a third predetermined threshold value as the grouping feature.

15. The data processing method according to any one of claims 12 to 14, wherein in dividing the second dataset, the second dataset is divided into a training set and a test set based on the grouping features from the candidate features and / or the grouping features from the non-candidate features, or in dividing the second dataset, the second dataset is automatically divided into the training set and the test set based on manually selected grouping features from the candidate features and / or the grouping features from the non-candidate features.

16. In dividing the second data set, randomly dividing the second data set into two groups; Calculating a distribution similarity between a distribution state of each of the two grouped data sets and a distribution state of the entire data set for each of the grouping features; The data processing method according to any one of claims 12 to 14, further comprising: assigning weights to the calculated distribution similarities based on importance weights corresponding to the grouping features; repeatedly executing a process of calculating a grouping score for each group; and then leaving the grouping scheme with the highest score; one group among the grouping schemes with the highest score being a training set; and the remaining groups among the grouping schemes with the highest score being a test set.

17. if the grouping feature is derived from the candidate features, the importance weight corresponding to the grouping feature is the importance weight stored in the library corresponding to each of the candidate features; 17. The data processing method of claim 16, wherein when the grouping feature is derived from the non-candidate features, an importance weight corresponding to the grouping feature is an importance weight obtained by normalizing a distribution similarity between a distribution situation of the non-candidate features of the second data set and a normal distribution.

18. The data processing method according to any one of claims 12 to 14, wherein in dividing the second data set, the second data set is divided into the training set and the test set by a Kennard-Stone algorithm.

19. 7. The data processing method according to claim 6, wherein after the division of the second data set is completed, if there is a large difference in distribution state between the grouping feature in the divided training set and the grouping feature in the existing task searched from the library for a certain grouping feature, the number of samplings for the grouping feature in the divided training set is increased.

20. 2. The data processing method of claim 1, wherein when a new dataset is added while the splitting of the second dataset is in progress or after completion, the added dataset is split into the training set and the test set, and the newly split training set and the test set are supplemented to the existing training set and the test set, respectively.

21. generating, for each task of a first dataset, a plurality of first high-dimensional features associated with the task, and obtaining information about the plurality of first high-dimensional features to build a library with the task as a unit; extracting a second plurality of high dimensional features for a second data set associated with a task of the second data set and selecting, based on the information in the library, grouping features from the second plurality of high dimensional features for partitioning data; A processing circuit that performs a process of dividing the second data set into a training set and a test set based on the grouping features.

22. generating, for each task of a first dataset, a plurality of first high-dimensional features associated with the task, and obtaining information about the plurality of first high-dimensional features to build a library with the task as a unit; extracting a second set of high dimensional features associated with the tasks of the second set of data, and selecting, based on the information in the library, grouping features from the second set of high dimensional features for splitting the data; A program for causing a computer to execute a process of dividing the second data set into a training set and a test set based on the grouping features.

Images