Method and device for automatically dividing data set, processing circuit and storage medium

Abstract

A dividing method for automatically dividing a data set can automatically distribute data features in a training set and a test set in a balanced mode while manpower and time resources are saved, and comprises the following steps of a priori library construction step, a priori library construction step, a priori library construction step and a priori library construction step, generating a plurality of first high-dimensional features associated with the task, and obtaining prior information related to the plurality of first high-dimensional features, so as to construct a prior library by taking the task as a unit; a grouping feature selection step: for a to-be-divided data set, extracting a plurality of second high-dimensional features associated with a task of the to-be-divided data set, and based on the prior information in the prior library, selecting grouping features for dividing data from the plurality of second high-dimensional features; and a to-be-divided data set division step: automatically dividing the to-be-divided data set into a training set and a test set based on the grouping characteristics.

Description

Technical Field

[0001] The invention relates to a partitioning method, a partitioning device, a processing circuit and a storage medium for automatically partitioning a data set into a test set and a training set. Background Art

[0002] The performance of the neural network model used in deep learning is not only related to the structure of the model, but also has a lot to do with the division of the data set. These models often need to be trained with a large number of clinical samples. When training the model, it is hoped that the data is sufficient and can cover the data range of the actual application scenarios of the model.

[0003] If the data set can be accurately divided into training sets and test sets, the model training can be accelerated and the model performance can be improved, which is often the key to determining the performance of the model. In addition, the automatic grouping of the data set is also important for the automatic training framework.

[0004] In the prior art, a data set is generally divided into a training set and a test set by random division or manual division.

[0005] Random partitioning means randomly sampling all data and randomly dividing the data set into training set and test set. When using random partitioning, the result is uncertain, and it cannot guarantee that the different features of the data are reasonably distributed in the training set and test set, so the performance of the trained model is often not ideal.

[0006] Manual segmentation refers to the process of professionals performing attribute statistics, setting standards, and dividing a large amount of data into training sets and test sets based on their understanding of the data set attributes and task requirements. When using manual segmentation, it is necessary to rely on the past experience of professionals and their understanding of the data set. In addition, when performing segmentation, each different task must repeat the above process, which consumes a lot of human resources and time resources.

[0007] The prior art does not provide an ideal method for dividing data sets. It is hoped that there will be a grouping method that can save manpower and time resources and accurately and reasonably allocate data features. Summary of the invention

[0008] The present invention is made in view of the above-mentioned problems and proposes a method, processing circuit and storage medium for automatically dividing a data set, which can save manpower and time resources while automatically and evenly distributing data features in a training set and a test set, thereby accelerating the training of the model and improving the model performance.

[0009] Technical solution one is a method for automatically dividing a data set, which includes the following steps: a priori library construction step, for a reference data set, according to each of the tasks, generating multiple first high-dimensional features associated with the task, and obtaining prior information related to the multiple first high-dimensional features, so as to construct the priori library based on tasks; a grouping feature selection step, for the data set to be divided, extracting multiple second high-dimensional features associated with the tasks of the data set to be divided, based on the prior information related to the multiple first high-dimensional features in the priori library, selecting grouping features for dividing data from the multiple second high-dimensional features; a data set to be divided step, based on the grouping features, automatically dividing the data set to be divided into a training set and a test set.

[0010] Technical solution two is a processing circuit, which performs the following processing: construction processing of a priori library, for the reference data set, according to each of the tasks, generating multiple first high-dimensional features associated with the task, and obtaining prior information related to the multiple first high-dimensional features, so as to construct the priori library based on tasks; selection processing of grouping features, for the data set to be divided, extracting multiple second high-dimensional features associated with the tasks of the data set to be divided, based on the prior information related to the multiple first high-dimensional features in the priori library, selecting grouping features for dividing data from the multiple second high-dimensional features; division processing of the data set to be divided, based on the grouping features, automatically dividing the data set to be divided into a training set and a test set.

[0011] Technical solution three is a partitioning device for automatically partitioning a data set, which is equipped with a processing circuit, which performs the following processing: construction processing of a priori library, for a reference data set, according to each of the tasks, generating multiple first high-dimensional features associated with the task, and obtaining prior information related to the multiple first high-dimensional features, so as to construct the priori library based on tasks; selection processing of grouping features, for a data set to be divided, extracting multiple second high-dimensional features associated with the tasks of the data set to be divided, and based on the prior information related to the multiple first high-dimensional features in the priori library, selecting grouping features for dividing data from the multiple second high-dimensional features; partitioning processing of the data set to be divided, based on the grouping features, automatically dividing the data set to be divided into a training set and a test set.

[0012] Technical solution four is a storage medium, which stores a program that enables a computer to perform the following processing: construction processing of a priori library, for a reference data set, according to each of the tasks, generating multiple first high-dimensional features associated with the task, and obtaining prior information related to the multiple first high-dimensional features, so as to construct the priori library based on tasks; selection processing of grouping features, for a data set to be divided, extracting multiple second high-dimensional features associated with the tasks of the data set to be divided, and based on the prior information related to the multiple first high-dimensional features in the priori library, selecting grouping features for dividing data from the multiple second high-dimensional features; division processing of the data set to be divided, based on the grouping features, automatically dividing the data set to be divided into a training set and a test set.

[0013] Technical Effects

[0014] By utilizing the partitioning method, partitioning device, processing circuit and storage medium for automatically partitioning a data set of the present invention, it is possible to save manpower and time resources while automatically and evenly distributing data features between training sets and test sets, thereby accelerating model training and improving model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flowchart of the automatic data set division according to the first embodiment.

[0016] Figure 2A is a detailed flow chart showing step 200, Figure 2B Represents the synthesis from multi-dimensional feature vectors to high-dimensional features.

[0017] Figure 3 is an example of a data fingerprint.

[0018] Figure 4 is a detailed flow chart showing step 300 .

[0019] Figure 5 This is an example of a filtering method that represents grouping characteristics.

[0020] Figure 6 It is a flowchart that uses multiple random grouping methods to divide the test set and data set.

[0021] Figure 7 It is a diagram showing the comparison of model performance between the random grouping of the prior art and the automatic division of the present embodiment. DETAILED DESCRIPTION

[0022] Hereinafter, the embodiments will be described with reference to the accompanying drawings. The embodiments described below do not limit the present invention. In addition, in the following description, the same reference numerals are given to components having substantially the same functions and configurations, and repeated descriptions are performed only when necessary.

[0023] In the embodiments of the present application, the terms "first", "second", "third", etc. are used to distinguish different components in terms of title, but do not indicate the spatial arrangement or time sequence of these components, etc., and these components should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the existence of the stated features, components, and elements, but do not exclude the existence or addition of one or more other features, components, and elements.

[0024] The terms "first high-dimensional features" and "second high-dimensional features" respectively correspond to the high-dimensional features obtained when extracting features from the reference data set during the prior library construction process, and the high-dimensional features obtained when extracting features from the data set to be divided during the automatic data set division process.

[0025] The term “first threshold” is a threshold used to represent the task similarity between the existing tasks in the prior library and the tasks to which the data set to be divided belongs.

[0026] The term "second threshold" is a threshold used to represent, for a certain candidate feature, the distribution similarity between the distribution of the candidate feature in the priori library and the distribution of the candidate feature in the data set to be divided.

[0027] The term "third threshold" is a threshold used to indicate the distribution similarity between a non-candidate feature and a normal distribution in a data set to be divided. The above-mentioned "first threshold", "second threshold" and "third threshold" are pre-defined and stored by the user before the data set is automatically divided.

[0028] The term “reference dataset” refers to the dataset used when building the prior library, and the term “dataset to be divided” refers to the dataset that you hope to automatically divide into training set and test set.

[0029] (First Embodiment)

[0030] The method for automatically dividing a data set in the implementation method includes the following steps: a priori library construction step, for a reference data set, according to each of the tasks, generating multiple first high-dimensional features associated with the task, and obtaining prior information related to the multiple first high-dimensional features, so as to construct the priori library based on tasks; a grouping feature selection step, for the data set to be divided, extracting multiple second high-dimensional features associated with the tasks of the data set to be divided, based on the prior information related to the multiple first high-dimensional features in the priori library, selecting grouping features for dividing data from the multiple second high-dimensional features; a data set to be divided division step, based on the grouping features, automatically dividing the data set to be divided into a training set and a test set.

[0031] Figure 1 : is a flowchart of the first embodiment of the automatic division of data sets. Figure 1 As shown, in order to automatically divide the data set, before dividing the data set, it is necessary to first complete the construction of the priori library.

[0032] exist Figure 1 In the process, firstly, a reference data set is obtained in step S100, and each data in the reference data set includes a true value (GT).

[0033] Next, in step S200, for the reference data set, multiple first high-dimensional features of the data are generated in units of tasks, and based on the generated multiple first high-dimensional features, prior information including distribution and importance weights related to the multiple first high-dimensional features is obtained, thereby constructing a priori library. At this point, the construction of the priori library is completed.

[0034] Next, in step S300, the data set to be divided is obtained, and multiple second high-dimensional features of the data are generated for the obtained data set to be divided. Then, based on the prior information related to the multiple first high-dimensional features obtained in step S200, grouping features for dividing the data are selected from the multiple second high-dimensional features.

[0035] Next, in step S400, the data set to be divided is automatically divided into a training set and a test set according to the selected grouping features, and in step S500, the divided training set and test set are used for training and testing respectively. At this point, the automatic division of the data set is completed.

[0036] Below, refer to Figure 2A to Figure 6 Describe each step in detail.

[0037] (A priori library construction)

[0038] Figure 2A This is a detailed flowchart showing step 200 (constructing a priori library). Step S200 further includes a step of extracting a data fingerprint (step S210), a step of generating a plurality of first high-dimensional features based on the data fingerprint (step S220), and a step of constructing a priori library (step S230).

[0039] It is known that the original data in the data set generally contains a large amount of information. Different data contains different information, and the information extracted by similar tasks is often similar. Therefore, the data set is taken as a task unit to extract the features (data fingerprint) of each task. Figure 3As shown, based on prior knowledge, the raw data in the data set is divided into four main dimensions: inside the target GT, between target GTs, between target GTs and surrounding non-image information. Each dimension contains different types of data fingerprints. Taking the dimension between target GTs as an example, it can be imagined that this dimension also contains different types of data fingerprints such as overlap, distance, and connectivity. With the expansion of data volume and the enrichment of task types, these data fingerprints will be continuously updated. Therefore, by extracting data fingerprints of different dimensions and types from the data set, it can be foreseen that rich data features can be collected (step S201).

[0040] For image information (inside the target GT, between the target GT, and the target GT and its surroundings), features can be extracted through convolutional neural networks (CNNs) and the like. For non-image information such as text, it can be converted into features using embedding and the like.

[0041] In addition, in this embodiment, the features themselves in each dimension are not used directly, but high-dimensional features with richer information are used.

[0042] As a method for generating high-dimensional features, for example, in m dimensions such as within the target GT, between the target GT, and the target GT and the surrounding and non-image information, the respective data fingerprints are summarized into a multidimensional feature vector X by weighting, averaging, maximum and minimum, median or splicing methods. i (X i =[x i1 ,x i2 ,x i3 …x ij ], i = 1, 2, 3, ... m, j is the dimension of the feature vector), and then in order to improve the characterization ability of the data set, such as Figure 2B As shown, for the set of multidimensional feature vectors X = {x 1 , x 2 , ..., x m}, for example, by using methods such as Cartesian product, the feature vectors in X are fused to generate the above multi-dimensional feature vectors into high-dimensional features F (step S202).

[0043] The high-dimensional feature F generated above corresponds to the first high-dimensional feature in the feature extraction of the reference data set, and corresponds to the second high-dimensional feature in the feature extraction of the data set to be divided.

[0044] The above describes the generation method of high-dimensional features. For ease of understanding, the high-dimensional features F generated in the process of building the prior library are called the first high-dimensional features, and the high-dimensional features F generated in the process of automatically dividing the data set to be divided are called the second high-dimensional features. However, there are also cases where the first high-dimensional features and the second high-dimensional features are not distinguished in order to express commonality and are simply referred to as high-dimensional features.

[0045] In this implementation, the prior library is constructed in units of tasks, each task has its own data set, and after a plurality of first high-dimensional features are generated according to the tasks, the distribution of GTs in each first high-dimensional feature is counted.

[0046] In addition, for different types of tasks, one or more features may have different importance in explaining the difference, and thus may be assigned different weights. When constructing the prior library, the importance weight of the first high-dimensional feature may be defined in advance based on experience or automatically obtained based on other models.

[0047] Therefore, when constructing the prior library, as shown in Table 1 below, according to different types of tasks, the corresponding high-dimensional features (first high-dimensional features) and the distribution of each high-dimensional feature (first high-dimensional feature), the prior information of importance weight, etc. can be saved for subsequent comparison with the data set to be divided.

[0048] Table 1

[0049]

[0050] Table 1 is only an example, and the high-dimensional features between tasks may overlap with each other, and the number of high-dimensional features varies according to the task.

[0051] (Selection of grouping features)

[0052] Figure 4 It is a detailed flowchart showing the selection of grouping features, and is a detailed flowchart of the aforementioned step 300.

[0053] like Figure 4 As shown, in the step of selecting the grouping feature, firstly, the data set to be divided is obtained, and in step S310, multiple second high-dimensional features of the data are generated for the obtained data set to be divided according to the same method as the generation of the first high-dimensional feature, and for each second high-dimensional feature, the distribution related to the second high-dimensional feature is obtained. As mentioned above, the second high-dimensional feature is the same as the first high-dimensional feature, which is obtained by aggregating the feature information of the data extracted from each dimension into a multi-dimensional feature vector according to the specified multiple dimensions and fusing the feature vectors in X through methods such as Cartesian product.

[0054] Next, in step S320, the task similarity of the task of the data set to be divided is compared with each task in the prior library. For example, the task similarity of the task of the data set to be divided is compared with the existing tasks in the prior library according to the target body, the organ to which the target body belongs and the task purpose.

[0055] Specifically,

[0056] Organs: According to the location of the organs, they are divided into head and neck, chest, abdomen, lower limbs, etc. When they belong to the same organ, the similarity is considered high, and the greater the distance between the organs, the lower the similarity.

[0057] In terms of target bodies: according to the target bodies of the tasks, they are divided into blood vessels, organ segments and lobes, tumors, etc. When they belong to the same target body, the similarity is considered high. The greater the difference in the morphology of the target body, the lower the similarity is considered.

[0058] In terms of task purpose: according to the task purpose, it is divided into segmentation, positioning, detection, classification, etc. When they belong to the same task purpose, the similarity is considered high, if they belong to different task purposes, the similarity is considered low.

[0059] When the similarities of the above aspects are to be considered comprehensively, the above similarities can be weighted and summed to obtain a total task similarity. In this embodiment, when comparing the tasks of the data set to be divided with the existing tasks in the priori database, the comparison is made based on the total task similarity.

[0060] Next, in step S330, it is determined whether there is an existing task in the prior library whose similarity with the task of the data set to be divided is above the first threshold. If it is determined that there is no existing task, it is considered that the data set to be divided is not similar enough to all the previously saved existing tasks, the data set to be divided is typical, and the data set to be divided needs to be updated to the prior library.

[0061] At this time, proceed to step S340.

[0062] In step S340, all the second high-dimensional features in the data set to be divided are used as non-candidate features F 1 ~F n Using the distribution of the multiple second high-dimensional features generated in step S310, the distribution of each second high-dimensional feature in the data set to be divided is compared with the distribution similarity D of the normal distribution through methods such as chi-square test and KL divergence. i , (i=1,2,3...n), by taking the similarity D corresponding to each second high-dimensional feature i As shown in Formula 1, the importance weight W corresponding to each second high-dimensional feature is obtained. i .

[0063]

[0064] Next, at least a portion of the second high-dimensional features is selected from the plurality of second high-dimensional features in step S350. The at least a portion of the second high-dimensional features here may be all of the second high-dimensional features, or may be second high-dimensional features further selected from all of the second high-dimensional features. When it is desired to select high-dimensional features representative of the task during the update, the second high-dimensional features whose distribution similarity to the distribution during the normal distribution is greater than a third threshold value specified in advance may be selected.

[0065] Then, in step S360, the tasks of the data set to be divided, at least a portion of the selected second high-dimensional features, the distribution information corresponding to the at least a portion of the second high-dimensional features, and the importance weights corresponding to the at least a portion of the second high-dimensional features are updated to the prior library.

[0066] If it is determined in step S330 that there are existing tasks in the prior library that make the task similarity above the first threshold value specified in advance, then proceed to step S370. In step S370, one or more existing tasks with the highest task similarity are selected from the existing tasks that meet the first threshold value found in the prior library, and then multiple first high-dimensional features associated with the one or more existing tasks and prior information related to the multiple first high-dimensional features (distribution and importance weights) are further obtained.

[0067] Next, in step S380, based on the found multiple first high-dimensional features and the generated multiple second high-dimensional features, candidate features and non-candidate features are obtained. Figure 5 As shown, the first high-dimensional feature is a high-dimensional feature saved in the prior library in units of tasks, and the second high-dimensional feature is a high-dimensional feature obtained from the data set to be segmented and in units of tasks. The second high-dimensional feature can be divided into candidate features and non-candidate features, and the candidate feature is a portion of multiple second high-dimensional features that is repeated with the multiple first high-dimensional features found, and the non-candidate feature is a portion of multiple second high-dimensional features that is not repeated with the multiple first high-dimensional features found. In this embodiment, the feature used to automatically divide the data set into a test set and a training set is called a grouping feature, which is selected from the second high-dimensional feature, but the grouping feature can be selected only from the candidate feature, or can be selected from the candidate feature and non-candidate feature separately.

[0068] The following are two ways to select grouping features.

[0069] (Select only from candidate features)

[0070] Through the screening of steps S310 to S330 and S370, it can be considered that the existing tasks in the prior library found at this time are tasks with high similarity to the tasks of the data set to be divided. If the task similarity is high, the possibility of the same features existing in different tasks is also high.

[0071] As mentioned above, the candidate feature is a feature common to the first high-dimensional feature and the second high-dimensional feature, that is, the candidate feature is a feature common to the existing task and the task of the data set to be divided. Features that are common in similar tasks are considered to be key features for the automatic division of the data set to be divided. It can be predicted that using this key feature as a grouping feature can achieve good division results.

[0072] Therefore, when selecting grouping features from the candidate features, as in step S391, at least a portion of the candidate features is selected as the grouping features. The so-called at least a portion here can be all of the multiple candidate features or a portion of the multiple candidate features.

[0073] As a selection method, when selecting a grouping feature from the candidate features, for each of the multiple candidate features, the distribution similarity of the candidate feature of the data set to be divided and the candidate feature of the existing task found from the prior library is compared, and the candidate feature with a distribution similarity above a predetermined second threshold is selected as the grouping feature. The distribution similarity can be obtained by methods such as the chi-square test or the KL divergence.

[0074] In addition, the a priori library stores the importance weight corresponding to each high-dimensional feature. By referring to the importance weights stored in the a priori library, we can know which features are important for the existing tasks. Since the existing tasks and the tasks of the data to be divided are highly similar, we can roughly infer that the features that are important for the existing tasks are also important for the tasks of the data set to be divided.

[0075] Therefore, as another selection method, it can be that according to the importance weights corresponding to each candidate feature stored in the prior library, the top K candidate features with the largest importance weights are selected from multiple candidate features. For each of the top K candidate features, the distribution similarity of the candidate feature in the data set to be divided and the candidate feature of the existing task found from the prior library is compared in terms of distribution. One or more candidate features whose distribution similarity is above a second threshold specified in advance are used as grouping features, and K is an integer.

[0076] (Selected from candidate features and non-candidate features respectively)

[0077] In addition to selecting from the candidate features, grouping features can also be selected from non-candidate features. Although non-candidate features are not stored in the prior library, they are also important for some tasks. Therefore, as in step S390, in addition to candidate features, grouping features are also selected from non-candidate features. By selecting grouping features from candidate features and non-candidate features respectively, the accuracy of data segmentation can be further improved.

[0078] As a method for selecting grouping features from non-candidate features, one or more non-candidate features may be selected as grouping features from multiple non-candidate features based on the distribution of the non-candidate features. Specifically, using the distribution of the non-candidate features in the multiple second high-dimensional features generated in step S310, a chi-square test, KL divergence, or other methods are used to select a distribution similarity D between the distribution and the normal distribution. i , (i=1,2,3...n) one or more non-candidate features that are greater than a third threshold value specified in advance are used as grouping features.

[0079] In addition, after obtaining the above distribution similarity D i , (i=1,2,3...n), and then the similarity D corresponding to each non-candidate feature is calculated. i As in the above formula 1, the importance weight W corresponding to each non-candidate feature is obtained. i The importance weight Wi can be used for subsequent multiple random partitions.

[0080] In addition, in this embodiment, the data set to be divided can be automatically divided into a training set and a test set based on the grouping features from the candidate features and / or the grouping features from the non-candidate features, or the data set to be divided into a training set and a test set based on the grouping features obtained after manual screening of the above grouping features.

[0081] Below, refer to Figure 6 To illustrate the steps of automatic partitioning of the data set to be partitioned. Figure 6 yes Figure 1 The detailed flowchart of step S400 in FIG. 4 shows the process of dividing the test set and the training set by using multiple random grouping methods.

[0082] The prior art uses random grouping to divide the training set and the test set, but the distribution of features in each data set after random grouping is not necessarily the same as the distribution of features in the overall data set, so there is a problem of unreasonable data distribution. The purpose of step S400 is to select a grouping method in which the distribution of grouping features in each data set is closest to the distribution of grouping features in the overall data set.

[0083] In this embodiment, multiple random groupings are performed, and the score of each random grouping is calculated. The random grouping with the highest score is selected, and one of the two groups obtained by the random grouping is used as the training set, and the remaining group of the two groups obtained by the random grouping is used as the test set.

[0084] Specifically, first in step S410 , the data set to be divided is randomly divided into two groups (group A and group B) according to a certain ratio (for example, a ratio of 20%:80%).

[0085] Next, in step S420, for each grouping feature selected in step S300, the distribution similarity of the grouping feature of each group (Group A and Group B) after grouping is compared with the distribution similarity of the grouping feature of the entire data set to be divided, and a score is assigned based on the distribution similarity.

[0086] In addition, by using the importance weight corresponding to each grouping feature, it can be known which grouping feature is more important for the current task, and better segmentation results can be obtained by increasing the weight of the grouping feature. When the grouping feature is selected from the candidate feature, the corresponding importance weight is the importance weight stored in the prior library. When the grouping feature is selected from the non-candidate feature, the corresponding importance weight is calculated using the above formula 1.

[0087] Therefore, in step S430, the scores obtained in step S420 are weighted according to the importance weight of each grouping feature, and the weighted score of this grouping is calculated.

[0088] Next, in step S440, it is determined whether the process of step S410 to step S430 has been executed a predetermined number of times (for example, N times). If not, the process returns to step S410 and continues to execute step S410 to step S430. If it has been executed N times, the process proceeds to step S450.

[0089] In step S450, the weighted scores of each random grouping in the N random groupings are compared, the grouping method with the highest score is retained, and group B in the grouping method is set as a training set, and group A in the grouping method is set as a test set.

[0090] Next, in step S500 , training is performed using the training set and the test set obtained in step S400 .

[0091] The above describes a method of dividing the test set and the data set by using multiple random grouping methods, but the Kennard-Stone algorithm can also be used to automatically divide the data set to be divided into the training set and the test set based on the grouping features selected in step S300. The above Kennard-Stone algorithm is a prior art, so detailed description is omitted.

[0092] By adopting the method for automatically dividing a data set of the present embodiment, different high-dimensional features and the distribution and importance weights corresponding to each high-dimensional feature can be obtained according to different tasks. By comparing the prior library constructed in advance and the data set to be divided based on this information, the grouping features that have a greater impact on the grouping effect can be selected from the data set to be divided. Then, by grouping based on the selected grouping features, the data distribution can be made more reasonable, thereby improving the model performance.

[0093] Compared with manual data grouping, this implementation reduces or even avoids the reliance on professionals' understanding of data set attributes and task objectives, saving a lot of human and time resources.

[0094] Compared with the original random grouping method, such as Figure 7 As shown, this implementation can make data distribution more reasonable and improve model performance.

[0095] In addition, since the method for automatically dividing a data set in this embodiment does not require human intervention, the technology can also be integrated into an online fully automatic training system as a front-end module.

[0096] (Variation Example)

[0097] As described in the first embodiment, different types of tasks correspond to different high-dimensional features. After the high-dimensional features are generated, information such as the distribution and importance weight corresponding to the high-dimensional features can be obtained. By comparing the distribution of multiple high-dimensional features (first high-dimensional features) stored in the prior library with the distribution of multiple high-dimensional features (second high-dimensional features) in the data set to be divided, the differences in certain high-dimensional features between the data in the data set to be divided and the data of the existing tasks can be found. The so-called certain high-dimensional features here refer to the high-dimensional features that are present in both the multiple first high-dimensional features and the multiple second high-dimensional features, that is, Figure 5 The intersection of the first high-dimensional feature and the second high-dimensional feature shown is the candidate feature.

[0098] In addition, as described in the first embodiment, when selecting grouping features from candidate features, it is necessary to consider whether the similarity of the distribution of the first high-dimensional feature and the second high-dimensional feature is greater than a specified threshold, and only select candidate features whose similarity of distribution is greater than the specified threshold as grouping features. However, if the difference in the distribution of all candidate features is less than the specified threshold, it is impossible to select a grouping feature from the candidate features. In addition, there is also a situation where the grouping feature that should have been extracted from the candidate features is not accurately extracted due to insufficient sample size.

[0099] The variation involves a data supplementation strategy that can increase the sample size in a targeted manner after the candidate features with excessive differences are found.

[0100] For example, when comparing the distribution of each candidate feature, when the difference in distribution between the candidate feature of the data set to be divided and the candidate feature of the existing task found from the prior library is greater than a specified threshold, the candidate feature is not directly abandoned, but the sample size related to the candidate feature is first increased, and then the distribution comparison of the data set to be divided and the existing task is continued for the candidate feature with the supplemented sample size.

[0101] Therefore, according to this variation, failure or inaccurate extraction of grouping features due to insufficient sample size can be eliminated.

[0102] (Second Embodiment)

[0103] The above modification example describes a countermeasure for excessively large differences (in distribution) when the sample is easy to replenish. However, there are also cases where the sample amount is difficult to replenish.

[0104] In view of the above situation, the second implementation proposes a data enhancement strategy, that is, after the automatic division of the data set to be divided is completed, data enhancement is performed on the grouping features in the divided training set that are greatly different from the prior library.

[0105] For example, after the automatic division of the data set to be divided is completed, for each grouping feature, the difference in distribution between the grouping feature in the training set and the grouping feature found from the prior library is compared. If the difference is large, for example, exceeding a predetermined threshold, the sampling times for the grouping feature with large distribution difference are increased, thereby expanding the amount of data related to the grouping feature.

[0106] Here, since what needs to be compared is the difference between the distribution of the grouping features in the training set and the distribution of the grouping features in the existing tasks (with high task similarity) found from the prior library, the grouping features actually refer to the features common to the first high-dimensional features and the second high-dimensional features, i.e., the candidate features. In addition, although the training set is exemplified here, data enhancement can also be performed on the grouping features in the test set that are greatly different from the existing tasks in the prior library.

[0107] According to the second embodiment, when it is difficult to increase the sample size, data enhancement can be performed on grouping features with large differences in distribution.

[0108] (Third Embodiment)

[0109] In addition, for tasks that have completed automatic division or are in the process of automatic division, data sets are sometimes supplemented due to new goals or needs. The third embodiment involves the situation where new data needs to be supplemented while automatic division is in progress or after completion. The third embodiment can be seamlessly applied to the first embodiment.

[0110] For example, when the automatic division of the data set to be divided is in progress or after completion, when you want to add a new data set, you only need to automatically divide the newly added data set into a training set and a test set according to the division method of the automatic division of the data set in the first embodiment, and add the newly divided training set and test set to the existing training set and test set respectively.

[0111] According to the third implementation, whether the task is in progress or completed, when you want to add new data, you do not need to repeat the entire automatic division process. You only need to use the method of the first implementation to divide the new data into training sets and test sets and add them to the existing training sets and test sets. This makes it convenient to supplement data and improves the efficiency of the model.

[0112] (Other embodiments)

[0113] In the above implementations, in order to obtain more comprehensive information, the automatic partitioning method of the data set is described by taking high-dimensional features as an example, but it should be understood that the automatic partitioning of the data set using low-dimensional features is also included in the present application.

[0114] In the above implementation, the method for automatically dividing a data set is described, but it can be understood that as long as each function in the method for automatically dividing a data set can be realized, the above function can also be realized by hardware, or by a combination of hardware and software.

[0115] For example, when the method of automatically dividing a data set is implemented by a processing circuit, i.e., a processor, the various processing functions of the processor are stored in the storage circuit in the form of a program that can be executed by a computer. The processor implements the functions corresponding to each program by reading out and executing each program from the storage circuit. In addition, instead of storing the program in the storage circuit, it can be configured to load the program directly into the circuit of the processor. In this case, the processor implements the function by reading out and executing the program loaded into the circuit. In addition, the processor of the above-mentioned embodiment is not limited to being configured as a single circuit, and can also be configured to combine multiple independent circuits to form a processor to implement its function.

[0116] The term "processor" used in the above description means, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (for example, a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA).

[0117] The program executed by the processor may be provided by pre-loading it into a ROM (Read Only Memory) or a storage circuit, etc. Alternatively, the program may be provided by recording it in a computer-readable storage medium such as a CD (Compact Disc)-ROM, FD (Flexible Disk), CD-R (Recordable), DVD (Digital Versatile Disc), etc., in a form that can be installed in these devices or in a form that can be executed.

[0118] Alternatively, the program may be stored in a computer connected to a network such as the Internet, and provided or distributed by being downloaded via the network.

[0119] In addition, instead of the automatic data set division method, the automatic data set division model can be saved on a server or a computer connected to a network such as the Internet, and downloaded and used by other medical information processing devices via the network. Other medical information processing devices can achieve the effect of automatic data set division by calling the automatic data set division model. For example, other medical information processing devices are equipped with a processing circuit, and the processing circuit performs the following processing: a priori library construction process, for the reference data set, according to each of the tasks, a plurality of first high-dimensional features associated with the task are generated, and prior information related to the plurality of first high-dimensional features is obtained, so as to construct the priori library in units of tasks; a grouping feature selection process, for the data set to be divided, a plurality of second high-dimensional features associated with the task of the data set to be divided are extracted, and based on the prior information in the priori library, a grouping feature for dividing data is selected from the plurality of second high-dimensional features; a data set to be divided is divided, based on the grouping feature, the data set to be divided is automatically divided into a training set and a test set.

[0120] According to at least one of the embodiments described above, the following advantages can be achieved:

[0121] 1. Reduce or even avoid the dependence of professionals on the degree of understanding of data set attributes and task objectives, saving a lot of human resources and time resources;

[0122] 2. Evenly distribute data features in the training set and test set, improving model performance;

[0123] 3. Can be integrated into the online fully automatic training system as a front-end module;

[0124] 4. Suitable for multiple tasks;

[0125] 5. It can eliminate the failure or inaccurate extraction of grouping features caused by insufficient sample size;

[0126] 6. When it is difficult to increase the sample size, data enhancement can be performed on high-dimensional features with large distribution differences;

[0127] 7. It can easily supplement data and is suitable for tasks of appending new data.

[0128] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the scope of the subject matter of the invention. These embodiments and their variations are included in the scope and subject matter of the invention, and are also included in the invention described in the claims and their equivalents.

Claims

1. A method for automatically partitioning a data set, wherein: The steps include: A priori library construction step, for the reference data set, generates a plurality of first high-dimensional features associated with the task according to each of the tasks, and obtains prior information related to the plurality of first high-dimensional features, thereby constructing the priori library in units of tasks; A step of selecting grouping features, for the data set to be divided, extracting a plurality of second high-dimensional features associated with the tasks of the data set to be divided, and selecting a grouping feature for dividing the data from the plurality of second high-dimensional features based on the prior information in the prior library; The step of dividing the data set to be divided automatically divides the data set to be divided into a training set and a test set based on the grouping characteristics.

2. The method for automatically dividing a data set according to claim 1, In the step of constructing the priori library, after a plurality of first high-dimensional features are generated, for each first high-dimensional feature, the distribution and importance weight of each first high-dimensional feature are obtained as the priori information to construct the priori library.

3. The method for automatically dividing a data set according to claim 2, In the step of selecting the grouping features, for each of the second high-dimensional features, the distribution and importance weight of each of the second high-dimensional features are obtained.

4. The method for automatically dividing a data set according to claim 3, The first high-dimensional feature and the second high-dimensional feature are obtained by aggregating feature information of data extracted from various dimensions into a multi-dimensional feature vector and fusing the vectors according to the specified multiple dimensions.

5. The method for automatically dividing a data set according to claim 3, In the step of selecting the grouping features, one or more second high-dimensional features among the plurality of second high-dimensional features that are repeated with the plurality of first high-dimensional features are used as candidate features, and one or more second high-dimensional features among the plurality of second high-dimensional features that are not repeated with the plurality of first high-dimensional features are used as non-candidate features, The grouping feature is selected from the candidate features.

6. The method for automatically dividing a data set according to any one of claims 3 to 5, In the step of selecting the grouping features, after extracting a plurality of second high-dimensional features associated with the tasks of the data set to be divided from the data set to be divided, When the task similarity between the task of the to-be-divided data set and the existing tasks in the priori database is greater than a first threshold, An existing task with the highest similarity to the task of the data set to be divided and the multiple first high-dimensional features associated with the existing task are found from the prior library, and the second high-dimensional features among the multiple second high-dimensional features that are repeated with the multiple first high-dimensional features found are used as candidate features, and candidate features for grouping are selected from the candidate features as grouping features through a predetermined method.

7. The method for automatically dividing a data set according to claim 6, When the task similarity is less than the first threshold, selecting at least a portion of the second high-dimensional features from the plurality of second high-dimensional features, The at least a portion of the second high-dimensional features, the distribution conditions and importance weights associated therewith, and the tasks of the data set to be divided are updated together into the priori library.

8. The method for automatically dividing a data set according to claim 6, The task similarity between the task of the to-be-divided data set and the existing task is compared according to the target body, the organ to which the target body belongs, and the task purpose.

9. The method for automatically dividing a data set according to claim 6, The predetermined method is, For each of the multiple candidate features, compare the distribution similarity of the candidate feature of the data set to be divided with the candidate feature of the existing task found from the prior library, and select the candidate feature whose distribution similarity is above a second threshold specified in advance as the grouping feature.

10. The method for automatically dividing a data set according to claim 6, The predetermined method is, According to the importance weights corresponding to the candidate features stored in the priori database, the first K candidate features with the largest importance weights are selected from the multiple candidate features. For each of the first K candidate features, the distribution similarity of the candidate feature in the data set to be divided and the candidate feature of the existing task found from the prior library is compared, and one or more candidate features whose distribution similarity is above a second threshold specified in advance are used as grouping features, where K is an integer.

11. The method for automatically dividing a data set according to claim 9, In the step of dividing the data set to be divided, for a certain candidate feature, when the distribution similarity between the candidate feature in the data set to be divided and the candidate feature in the existing task found from the prior library is less than the second threshold, the amount of samples related to the candidate feature in the data set to be divided is increased.

12. The method for automatically dividing a data set according to claim 5, In the step of selecting the grouping features, In addition to the candidate features, grouping features are also selected from the non-candidate features.

13. The method for automatically dividing a data set according to claim 12, In the step of selecting the grouping features, Based on the distribution conditions related to the non-candidate features, one or more non-candidate features are selected from a plurality of the non-candidate features as the grouping features.

14. The method for automatically dividing a data set according to claim 13, In the step of selecting the grouping features, From the plurality of non-candidate features, one or more non-candidate features whose distribution similarity with the normal distribution is greater than a third threshold value that is predetermined in advance are selected as the grouping features.

15. The method for automatically dividing a data set according to any one of claims 12 to 14, In the step of dividing the data set to be divided, the data set to be divided is automatically divided into a training set and a test set based on the grouping features from the candidate features and / or the grouping features from the non-candidate features; Alternatively, in the step of dividing the data set to be divided, the data set to be divided is automatically divided into a training set and a test set based on the grouping features obtained after manually screening the grouping features from the candidate features and / or the grouping features from the non-candidate features.

16. The method for automatically dividing a data set according to any one of claims 12 to 14, In the step of dividing the data set to be divided, after repeatedly performing the following processing, the grouping method with the highest score is retained, and one group in the grouping method is used as a training set, and the remaining groups in the grouping method are used as a test set. The processing is: Randomly divide the data set to be divided into two groups; According to each of the grouping features, calculating the distribution similarity between the distribution of each of the two grouped data sets and the distribution of the overall data set; Based on the importance weight corresponding to each of the grouping features, weights are assigned to the calculated distribution similarities respectively, and the grouping score of each group is calculated.

17. The method for automatically dividing a data set according to claim 16, When the grouping feature comes from the candidate feature, the importance weight corresponding to the grouping feature is the importance weight corresponding to each of the candidate features stored in the priori database. When the grouping feature comes from the non-candidate feature, the importance weight corresponding to the grouping feature is the importance weight obtained by unitizing the distribution similarity between the non-candidate feature in the data set to be divided and the normal distribution.

18. The method for automatically dividing a data set according to any one of claims 12 to 14, In the step of dividing the data set to be divided, the data set to be divided is automatically divided into a training set and a test set by using the Kennard-Stone algorithm.

19. The method for automatically dividing a data set according to claim 6, After the automatic division of the data set to be divided is completed, for a certain grouping feature, when the distribution of the grouping feature in the divided training set is greatly different from the grouping feature in the existing task found from the prior library, the sampling times of the grouping feature in the divided training set are increased.

20. The method for automatically dividing a data set according to claim 1, When the automatic division of the data set to be divided is in progress or after completion, when a new data set is added, the newly added data set is automatically divided into a training set and a test set according to the division method for automatically dividing a data set according to claims 1 to 17, and the newly divided training set and test set are added to the existing training set and test set respectively.

21. A processing circuit, the processing circuit performing the following processing: The prior library is constructed by generating, for each of the tasks, a plurality of first high-dimensional features associated with the tasks for the reference data set, and obtaining prior information related to the plurality of first high-dimensional features, thereby constructing the prior library in units of tasks; A grouping feature selection process is performed, wherein, for a data set to be divided, a plurality of second high-dimensional features associated with the tasks of the data set to be divided are extracted, and based on the prior information in the prior library, a grouping feature for dividing data is selected from the plurality of second high-dimensional features; The data set to be divided is divided into a training set and a test set based on the grouping characteristics.

22. A partitioning device for automatically partitioning a data set, comprising a processing circuit, wherein the processing circuit performs the following processing: The prior library is constructed by generating, for each of the tasks, a plurality of first high-dimensional features associated with the tasks for the reference data set, and obtaining prior information related to the plurality of first high-dimensional features, thereby constructing the prior library in units of tasks; A grouping feature selection process is performed, wherein, for a data set to be divided, a plurality of second high-dimensional features associated with the tasks of the data set to be divided are extracted, and based on the prior information in the prior library, a grouping feature for dividing data is selected from the plurality of second high-dimensional features; The data set to be divided is divided into a training set and a test set based on the grouping characteristics.

23. A storage medium storing a program for causing a computer to execute the following processing: The prior library is constructed by generating, for each of the tasks, a plurality of first high-dimensional features associated with the tasks for the reference data set, and obtaining prior information related to the plurality of first high-dimensional features, thereby constructing the prior library in units of tasks; A grouping feature selection process is performed, wherein, for a data set to be divided, a plurality of second high-dimensional features associated with the tasks of the data set to be divided are extracted, and based on the prior information in the prior library, a grouping feature for dividing data is selected from the plurality of second high-dimensional features; The data set to be divided is divided into a training set and a test set based on the grouping characteristics.

Images