Artificial intelligence model training data set construction method and system

Abstract

The invention discloses an artificial intelligence model training data set construction method and system, and relates to the technical field of artificial intelligence model training, and the method comprises the steps: presetting a plurality of data sources for a construction system, and carrying out the distributed storage of the data obtained by different data sources; creating a balanced extraction model cluster for the distributed storage architecture, and performing cooperative training on the created balanced extraction model cluster; using the trained model cluster to extract storage data with balanced distribution, and arranging the storage data into an original data set; sampling data in the original data set to obtain a high-quality training sample; and arranging the obtained high-quality training samples into a training data set, and persisting the training data set for use during artificial intelligence model training. The training data set constructed by the method can fully reflect the data distribution of the real world, so that the model learns correct features and rules, and the accuracy of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence model training technology, and in particular to a method and system for constructing an artificial intelligence model training data set. Background Art

[0002] With the rapid development of artificial intelligence technology, artificial intelligence models are increasingly used in various fields, from image recognition, speech recognition to natural language processing, artificial intelligence is gradually changing the way people live and work. However, the performance of artificial intelligence models depends to a large extent on the quality and quantity of their training data sets. Therefore, how to efficiently and accurately construct artificial intelligence model training data sets has become a hot topic and difficulty in current research. Summary of the invention

[0003] The present invention provides a method for constructing an artificial intelligence model training data set, comprising: Step 1: Preset multiple data sources for building the system, and store the data obtained from different data sources in a distributed manner; Step 2: Create a balanced extraction model cluster for the distributed storage architecture and perform collaborative training on the created balanced extraction model cluster; Step 3: Use the trained model cluster to extract the evenly distributed storage data and organize it into the original data set; Step 4: Sample the data in the original data set to obtain high-quality training samples; Step 5: Organize the obtained high-quality training samples into a training data set and persist it for use during artificial intelligence model training.

[0004] A method for constructing an artificial intelligence model training data set as described above, wherein a balanced extraction model cluster is created for a distributed storage architecture, and the created balanced extraction model cluster is collaboratively trained, which is specifically divided into the following sub-steps: The central server sends the initialized balanced extraction model to each storage node locally; Each storage node uses its own local data to train its local balanced extraction model, and uploads the extracted data samples and local model parameters to the central server after training is completed; The central server performs global verification on the distribution characteristics of all data samples. If the verification passes, training is stopped. If not, all local model parameters are integrated to generate a new set of model parameters, which are then sent to each storage node. Each storage node uses the received new model parameters to update the local balanced extraction model and conduct a new round of training.

[0005] As described above, a method for constructing an artificial intelligence model training data set, in which data in the original data set is sampled to obtain high-quality training samples, is specifically divided into the following sub-steps: Perform cluster analysis on the data in the original data set to obtain multiple data clusters; Continuous sampling is performed based on the distance between the data in each data cluster and the cluster center; The number of samples is increased during continuous sampling to obtain more valuable training samples.

[0006] The method for constructing a training data set for an artificial intelligence model as described above, wherein the obtained high-quality training samples are organized into a training data set and then persisted, is specifically divided into the following sub-steps: Standardize the data format of training samples and store them as training datasets; Perform data enhancement on training samples to further enrich the training data set; The processed training dataset is transferred to the cloud for persistence.

[0007] The present invention also provides an artificial intelligence model training data set construction system, including: a data acquisition module, an extraction model cluster creation module, a data extraction module, a data sampling module, and a data persistence module; The data acquisition module is used to preset multiple data sources for the system and store the data acquired from different data sources in a distributed manner; The extraction model cluster creation module is used to create a balanced extraction model cluster for the distributed storage architecture and to perform collaborative training on the created balanced extraction model cluster; The data extraction module is used to extract evenly distributed storage data using the trained model cluster and organize it into the original data set; The data sampling module is used to sample the data in the original data set to obtain high-quality training samples; The data persistence module is used to organize the obtained high-quality training samples into training data sets and persist them for use during artificial intelligence model training.

[0008] The beneficial effects achieved by the present invention are as follows: it can fully reflect the data distribution in the real world, so that the model can learn the correct features and rules, thereby improving the accuracy of the model; by including diverse data, the training set can help the model maintain stable performance when facing different inputs, thereby enhancing the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0010] Figure 1 This is a flow chart of a method for constructing an artificial intelligence model training data set provided in Example 1 of the present application; Figure 2 This is a schematic diagram of an artificial intelligence model training data set construction system provided in Example 2 of the present application. DETAILED DESCRIPTION

[0011] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0012] Embodiment 1 like Figure 1 As shown, the first embodiment of the present application provides a method for constructing an artificial intelligence model training data set, including: Step S10: Preset multiple data sources for building the system, and store data obtained from different data sources in a distributed manner; The preset data source can be a local database of a company, institution or individual, or it can be data of multiple modalities publicly available on the Internet. This multi-source heterogeneous data feature can make the constructed training set more diverse and more realistic. The extracted multi-source heterogeneous data must be uniformly quantified and then stored to ensure the consistency of the training data; on the other hand, in order to improve the read and write performance of the construction system, the acquired data will be stored in a distributed manner, and the storage node will also provide some data processing services locally to reduce the pressure on the central server and improve the response rate.

[0013] Step S20: creating a balanced extraction model cluster for the distributed storage architecture, and performing collaborative training on the created balanced extraction model cluster; Balanced data distribution is crucial for an efficient and accurate training data set, but the data distribution characteristics of different data sources are different, and the amount of data obtained from multiple data sources is also very large. It is difficult to quickly and accurately extract data samples under the expected distribution state. Therefore, the concept of clustering is proposed here to overcome the above difficulties. Specifically: Step S21: The central server sends the initialized balanced extraction model to each storage node locally; The balanced extraction model is used to extract data samples that meet the expected distribution state in each storage node. Its mathematical expression is: , where Y is the output of the balanced extraction model, S is the set of data samples extracted by the balanced extraction model. In the initial state, the S set has only one data, that is, any data selected from the storage data set X. The subsequent balanced extraction model will update the S set every time it outputs. It is any data remaining after removing all elements from set S from set X. is any data in the set S, is the size of the S set (i.e. the number of samples currently extracted), It is any data in the expected data distribution set P (a set of data samples describing the user's expected distribution state, for a certain data source), is the size of set P (the number of samples in set P), yes right The contribution parameter, express right is the contribution parameter, and h is the bandwidth parameter, which is used to control the smoothness of the model.

[0014] Initializing the model means setting the bandwidth parameter and each contribution parameter in the model to a default value. The default value set in this embodiment is 0.5.

[0015] Step S22: Each storage node uses its own local data to train its local balanced extraction model, and uploads the extracted data samples and local model parameters to the central server after the training is completed; During the training process, the chi-square test method is continuously used to verify whether the extracted sample set S is consistent with the data distribution status in the expected data distribution set P, and the various parameters in the model are continuously iterated according to the verification results until the verification is passed. The training is stopped and the last extracted sample set S and the model parameters at this time are uploaded to the central server.

[0016] Step S23: The central server performs global verification on the distribution characteristics of all data samples. If the verification passes, the training is stopped. If not, all local model parameters are integrated to generate a new set of model parameters, which are then sent to each storage node. The global verification also uses the chi-square test method to verify whether all data sample sets uploaded by the storage node are consistent with the data distribution state in the expected global data distribution set C (a set of global data samples under the user's expected distribution state, which is common to all data sources). If they are consistent, the verification passes and a stop training instruction is issued to each storage node. Otherwise, the verification fails and the formula is used: , calculate the fused values ​​of each model parameter in turn, and send the fused new model parameters to each storage node, where are the model parameters after fusion, It represents the data distribution distance between the data sample set uploaded by the j-th storage node and the global data distribution set (obtained using KL divergence or Wasserstein distance algorithm). j ranges from 1 to m, where m is the total number of storage nodes. It represents the data distribution distance between the data sample set uploaded by the k-th storage node and the global data distribution set. k ranges from 1 to m. Represents the local model parameters uploaded by the j-th storage node.

[0017] Step S24: Each storage node uses the received new model parameters to update the local balanced extraction model and conduct a new round of training; After each storage node uses the new model parameters to update the local balanced extraction model, it uses the local data again to perform a new round of training on the updated model. After the training is completed, the sample set and model parameters are uploaded to the central server for verification. This is repeated until the instruction to stop training is received.

[0018] Step S30: extracting evenly distributed storage data using the trained model cluster and arranging them into original data sets; The trained model cluster can extract data sets in the user's expected distribution state from each storage node. After receiving these data sets, the central server organizes them into a large collection and names them as the original data set. Subsequent training data sets are generated from the original data set.

[0019] Step S40: sampling the data in the original data set to obtain high-quality training samples; Sampling the original data set can reduce data noise and improve sample value. A good sampling method can directly affect the quality of the final training sample. The sampling method provided in this application is specifically divided into the following sub-steps: Step S41: performing cluster analysis on the data in the original data set to obtain multiple data clusters; Cluster analysis can quickly divide the original data set into multiple data clusters with high internal similarity. Sampling based on these data clusters can ensure balanced and diversified sampling.

[0020] Step S42: performing continuous sampling according to the distance between the data in each data cluster and the cluster center; Continuous sampling means taking the cluster center as the starting point and collecting data samples every L distance. The initial number of samples collected is 200. The initial collection distance L is given by the formula: Calculated, among which is the influence coefficient of data density on sampling frequency, is the influence coefficient of data distribution on sampling frequency, and It indicates the tth and t+1th data from the cluster center outwards, T is the total amount of data in the current data cluster, express and The distance between is the mean of the current data cluster, express and The distance between.

[0021] Step S43: increasing the number of samples during continuous sampling to obtain more valuable training samples; The farther away from the cluster center, the lower the similarity of the data, and the higher the training value for the model. Therefore, the further outward, the larger the sample size should be, so as to obtain more valuable training samples. The incremental coefficient set in this embodiment is 0.05, which means that after the first sample collection is completed, the amount of collected data should increase by 5% for every L distance until the sampling is completed. During the sampling process, all calculation results that have decimals are rounded down.

[0022] Step S50: organizing the obtained high-quality training samples into a training data set and persisting it for use in artificial intelligence model training; Persistent datasets can be easily loaded quickly during model training, while ensuring data consistency and repeatability. The organization of training samples specifically includes the following sub-steps: Step S51: standardize the data format of the training samples and store them as a training data set; Organize the training samples into a table format (such as CSV, Parquet), ensure that each column has a clear field name, and for unstructured data such as images, text, audio, etc., ensure that the file naming is standardized and metadata (such as annotation information) is stored in a separate table or JSON file. After the organization is completed, store all training samples in the training dataset.

[0023] Step S52: Perform data enhancement on the training samples to further enrich the training data set; There are many existing technologies for data enhancement, which can be selected as needed and will not be described here.

[0024] Step S53: transmitting the processed training data set to the cloud for persistence; Use cloud storage technologies such as AWS S3 and Google Cloud Storage to persist the processed training data sets. Its advantages of high availability, distributed access, and privacy protection can meet all the needs of large-scale training data storage.

[0025] Embodiment 2 like Figure 2 As shown, the second embodiment of the present application provides an artificial intelligence model training data set construction system, including: a data acquisition module 21, an extraction model cluster creation module 22, a data extraction module 23, a data sampling module 24, and a data persistence module 25; The data acquisition module 21 is used to preset multiple data sources for the system and store the data acquired from different data sources in a distributed manner; The preset data source can be a local database of a company, institution or individual, or it can be data of multiple modalities publicly available on the Internet. This multi-source heterogeneous data feature can make the constructed training set more diverse and more realistic. The extracted multi-source heterogeneous data must be uniformly quantified and then stored to ensure the consistency of the training data; on the other hand, in order to improve the read and write performance of the construction system, the acquired data will be stored in a distributed manner, and the storage node will also provide some data processing services locally to reduce the pressure on the central server and improve the response rate.

[0026] The extraction model cluster creation module 22 is used to create a balanced extraction model cluster for the distributed storage architecture and perform collaborative training on the created balanced extraction model cluster; specifically, it includes the following submodules: 1. The balanced extraction model distribution submodule is deployed on the central server and is used to distribute the initialized balanced extraction model to each storage node locally; The balanced extraction model is used to extract data samples that meet the expected distribution state in each storage node. Its mathematical expression is: , where Y is the output of the balanced extraction model, S is the set of data samples extracted by the balanced extraction model. In the initial state, the S set has only one data, that is, any data selected from the storage data set X. The subsequent balanced extraction model will update the S set every time it outputs. It is any data remaining after removing all elements from set S from set X. is any data in the set S, is the size of the S set (i.e. the number of samples currently extracted), It is any data in the expected data distribution set P (a set of data samples describing the user's expected distribution state, for a certain data source), is the size of set P (the number of samples in set P), yes right The contribution parameter, express right is the contribution parameter, and h is the bandwidth parameter, which is used to control the smoothness of the model.

[0027] Initializing the model means setting the bandwidth parameter and each contribution parameter in the model to a default value. The default value set in this embodiment is 0.5.

[0028] 2. The balanced extraction model training submodule is deployed locally on each storage node and is used to train its local balanced extraction model using local data. After the training is completed, the extracted data samples and local model parameters are uploaded to the central server. During the training process, the chi-square test method is continuously used to verify whether the extracted sample set S is consistent with the data distribution status in the expected data distribution set P, and the various parameters in the model are continuously iterated according to the verification results until the verification is passed. The training is stopped and the last extracted sample set S and the model parameters at this time are uploaded to the central server.

[0029] 3. The global verification submodule is deployed on the central server and is used to globally verify the distribution characteristics of all data samples. If the verification passes, the training is stopped. If not, all local model parameters are integrated to generate a new set of model parameters and sent to each storage node. The global verification also uses the chi-square test method to verify whether all data sample sets uploaded by the storage node are consistent with the data distribution state in the expected global data distribution set C (a set of global data samples under the user's expected distribution state, which is common to all data sources). If they are consistent, the verification passes and a stop training instruction is issued to each storage node. Otherwise, the verification fails and the formula is used: , calculate the fused values ​​of each model parameter in turn, and send the fused new model parameters to each storage node, where are the model parameters after fusion, It represents the data distribution distance between the data sample set uploaded by the j-th storage node and the global data distribution set (obtained using KL divergence or Wasserstein distance algorithm). j ranges from 1 to m, where m is the total number of storage nodes. It represents the data distribution distance between the data sample set uploaded by the k-th storage node and the global data distribution set. k ranges from 1 to m. Represents the local model parameters uploaded by the j-th storage node.

[0030] 4. The balanced extraction model update submodule is deployed locally on each storage node and is used to update the local balanced extraction model using the received new model parameters and conduct a new round of training; After each storage node uses the new model parameters to update the local balanced extraction model, it uses the local data again to perform a new round of training on the updated model. After the training is completed, the sample set and model parameters are uploaded to the central server for verification. This is repeated until the instruction to stop training is received.

[0031] The data extraction module 23 is used to extract the evenly distributed stored data using the trained model cluster and organize it into an original data set; The trained model cluster can extract data sets in the user's expected distribution state from each storage node. After receiving these data sets, the central server organizes them into a large collection and names them as the original data set. Subsequent training data sets are generated from the original data set.

[0032] The data sampling module 24 is used to sample the data in the original data set to obtain high-quality training samples; it specifically includes the following submodules: 1. Cluster analysis submodule, used to perform cluster analysis on the data in the original data set to obtain multiple data clusters; Cluster analysis can quickly divide the original data set into multiple data clusters with high internal similarity. Sampling based on these data clusters can ensure balanced and diversified sampling.

[0033] 2. The continuous sampling submodule is used to perform continuous sampling according to the distance between the data in each data cluster and the cluster center, and increase the number of samples during continuous sampling to obtain more valuable training samples; Continuous sampling means taking the cluster center as the starting point and collecting data samples every L distance. The initial number of samples collected is 200. The initial collection distance L is given by the formula: Calculated, among which is the influence coefficient of data density on sampling frequency, is the influence coefficient of data distribution on sampling frequency, and It indicates the tth and t+1th data from the cluster center outwards, T is the total amount of data in the current data cluster, express and The distance between is the mean of the current data cluster, express and The distance between.

[0034] During continuous sampling, the number of samples is increased to obtain more valuable training samples. The farther away from the cluster center, the lower the similarity of the data, and the higher the training value of the model. Therefore, the further outward, the larger the sample size should be, so as to obtain more valuable training samples. The increment coefficient set in this embodiment is 0.05, that is, starting from the completion of the first sample collection, the number of collected data should be increased by 5% for every L distance until the sampling is completed. During the sampling process, all calculation results are rounded down when decimals appear.

[0035] The data persistence module 25 is used to organize the obtained high-quality training samples into a training data set and persist it for use during the training of the artificial intelligence model; it specifically includes the following submodules: 1. Data format standardization submodule, used to standardize the data format of training samples and store them as training data sets; Organize the training samples into a table format (such as CSV, Parquet), ensure that each column has a clear field name, and for unstructured data such as images, text, audio, etc., ensure that the file naming is standardized and metadata (such as annotation information) is stored in a separate table or JSON file. After the organization is completed, store all training samples in the training dataset.

[0036] 2. Data enhancement submodule, used to enhance the training samples and further enrich the training data set; There are many existing technologies for data enhancement, which can be selected as needed and will not be described here.

[0037] 3. Data storage submodule, used to transfer the processed training data set to the cloud for persistence; Use cloud storage technologies such as AWS S3 and Google Cloud Storage to persist the processed training data sets. Its advantages of high availability, distributed access, and privacy protection can meet all the needs of large-scale training data storage.

[0038] Corresponding to the above embodiment, an embodiment of the present invention provides a computer storage medium, including: at least one memory and at least one processor; The memory is used to store one or more program instructions; A processor is used to run one or more program instructions to execute a method for constructing an artificial intelligence model training data set.

[0039] Corresponding to the above embodiments, an embodiment of the present invention provides a computer-readable storage medium, which contains one or more program instructions, and the one or more program instructions are used by a processor to execute a method for constructing an artificial intelligence model training data set.

[0040] The embodiments disclosed in the present invention provide a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed on a computer, the computer executes the above-mentioned method for constructing an artificial intelligence model training data set.

[0041] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0042] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0043] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0044] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0045] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus DRAM (DRRAM).

[0046] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0047] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0048] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing an artificial intelligence model training data set, characterized in that: include: Step 1: Preset multiple data sources for building the system, and store the data obtained from different data sources in a distributed manner; Step 2: Create a balanced extraction model cluster for the distributed storage architecture and perform collaborative training on the created balanced extraction model cluster; Step 3: Use the trained model cluster to extract the evenly distributed storage data and organize it into the original data set; Step 4: Sample the data in the original data set to obtain high-quality training samples; Step 5: Organize the obtained high-quality training samples into a training data set and persist it for use during artificial intelligence model training.

2. The method for constructing an artificial intelligence model training data set according to claim 1, characterized in that: Create a balanced extraction model cluster for the distributed storage architecture and perform collaborative training on the created balanced extraction model cluster. The specific steps are as follows: The central server sends the initialized balanced extraction model to each storage node locally; Each storage node uses its own local data to train its local balanced extraction model, and uploads the extracted data samples and local model parameters to the central server after training is completed; The central server performs global verification on the distribution characteristics of all data samples. If the verification passes, training is stopped. If not, all local model parameters are integrated to generate a new set of model parameters, which are then sent to each storage node. Each storage node uses the received new model parameters to update the local balanced extraction model and conduct a new round of training.

3. The method for constructing an artificial intelligence model training data set according to claim 1, characterized in that: Sampling the data in the original data set to obtain high-quality training samples is divided into the following sub-steps: Perform cluster analysis on the data in the original data set to obtain multiple data clusters; Continuous sampling is performed based on the distance between the data in each data cluster and the cluster center; The number of samples is increased during continuous sampling to obtain more valuable training samples.

4. The method for constructing an artificial intelligence model training data set according to claim 1, characterized in that: The obtained high-quality training samples are organized into training data sets and persisted, which is specifically divided into the following sub-steps: Standardize the data format of training samples and store them as training datasets; Perform data enhancement on training samples to further enrich the training data set; The processed training dataset is transferred to the cloud for persistence.

5. A system for constructing an artificial intelligence model training data set, characterized in that: include: Data acquisition module, extraction model cluster creation module, data extraction module, data sampling module, data persistence module; The data acquisition module is used to preset multiple data sources for the system and store the data acquired from different data sources in a distributed manner; The extraction model cluster creation module is used to create a balanced extraction model cluster for the distributed storage architecture and to perform collaborative training on the created balanced extraction model cluster; The data extraction module is used to extract evenly distributed storage data using the trained model cluster and organize it into the original data set; The data sampling module is used to sample the data in the original data set to obtain high-quality training samples; The data persistence module is used to organize the obtained high-quality training samples into training data sets and persist them for use during artificial intelligence model training.

6. The artificial intelligence model training data set construction system according to claim 5, characterized in that: The extraction model cluster creation module specifically includes: The balanced extraction model delivery submodule is deployed on the central server and is used to deliver the initialized balanced extraction model to each storage node locally; The balanced extraction model training submodule is deployed locally on each storage node and is used to train its local balanced extraction model using local data. After the training is completed, the extracted data samples and local model parameters are uploaded to the central server. The global verification submodule is deployed on the central server and is used to globally verify the distribution characteristics of all data samples. If the verification passes, the training is stopped. If not, all local model parameters are integrated to generate a new set of model parameters, which are then sent to each storage node. The balanced extraction model update submodule is deployed locally on each storage node and is used to update the local balanced extraction model using the received new model parameters and perform a new round of training.

7. The artificial intelligence model training data set construction system according to claim 5, characterized in that: The data sampling module specifically includes: The cluster analysis submodule is used to perform cluster analysis on the data in the original data set to obtain multiple data clusters; The continuous sampling submodule is used to perform continuous sampling according to the distance between the data in each data cluster and the cluster center, and to increase the number of samples during continuous sampling to obtain more valuable training samples.

8. The artificial intelligence model training data set construction system according to claim 5, characterized in that: The data persistence module specifically includes: The data format standardization submodule is used to standardize the data format of the training samples and store them as training data sets; The data enhancement submodule is used to enhance the training samples and further enrich the training data set; The data storage submodule is used to transfer the processed training data set to the cloud for persistence.

Images