Model production method, device, electronic device, and storage medium

Abstract

The present disclosure provides a model production method, system, device, and storage medium. It relates to the field of computer application technologies, especially to artificial intelligence fields such as model production, model management, and model testing. The specific implementation solution is as follows: receive a model production request; based on the model production request, determine a dataset to be trained and first resource configuration information for the dataset to be trained; based on the first resources indicated by the first resource configuration information, use the dataset to be trained to train a preset model to obtain a target model. According to the technical solution of the present disclosure, the usage threshold of the model production line can be reduced, and the efficiency of model generation can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer application technologies, and particularly to the field of artificial intelligence such as model production, model management, model testing, etc. Specifically, it relates to a model production method, apparatus, electronic device, and storage medium. Background Art

[0002] With the development of artificial intelligence, artificial intelligence technologies have been applied to all walks of life. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications cover all fields of artificial intelligence. In a practical sense, machine learning is a method of using data to train a model and then using the model for prediction. The model production line can solidify the steps of model training and model deployment to achieve the purpose of training a new model and deploying the model online. In the traditional model production line, users need to understand a lot of basic knowledge and write code, which has a relatively high threshold for users without programming skills or those who do not understand model parameters, and the efficiency of model generation is low. Summary of the Invention

[0003] The present disclosure provides a model production method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of the present disclosure, there is provided a model production method, including:

[0005] Receiving a model production request;

[0006] Based on the model production request, determining a to-be-trained data set and first resource configuration information for the to-be-trained data set;

[0007] Based on the first resource indicated by the first resource configuration information, training a pre-set model with the to-be-trained data set to obtain a target model.

[0008] According to a second aspect of the present disclosure, there is provided a model production apparatus, including:

[0009] A first receiving module, configured to receive a model production request;

[0010] A determining module, configured to determine a to-be-trained data set and first resource configuration information for the to-be-trained data set based on the model production request;

[0011] A generating module, configured to train a pre-set model with the to-be-trained data set based on the first resource indicated by the first resource configuration information to obtain a target model.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method provided in the first aspect above.

[0016] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are for causing the computer to execute the method provided in the first aspect above.

[0017] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program which, when executed by a processor, implements the method provided in the first aspect above.

[0018] According to the technical solution of the present disclosure, the usage threshold of the model production line can be reduced and the efficiency of model generation can be improved.

[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0021] Figure 1 is a flowchart of a model production method according to an embodiment of the present disclosure;

[0022] Figure 2 is a flowchart of data cleaning according to an embodiment of the present disclosure;

[0023] Figure 3 is a flowchart of detection based on a target model according to an embodiment of the present disclosure;

[0024] Figure 4 is a schematic diagram of a model production architecture according to an embodiment of the present disclosure;

[0025] Figure 5 is a schematic diagram of the composition structure of a model production device according to an embodiment of the present disclosure;

[0026] Figure 6 is a schematic diagram of the composition structure of a model production system according to an embodiment of the present disclosure;

[0027] Figure 7 is a schematic diagram of a model generation scenario according to an embodiment of the present disclosure;

[0028] Figure 8 It is a block diagram of an electronic device for implementing the model production method of the embodiments of the present disclosure. Detailed implementation manners

[0029] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0030] The terms "first", "second", "third", etc. in the embodiments of the specification, claims and the above-mentioned drawings of the present disclosure are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] The embodiments of the present disclosure provide a model production method, which can be applied to an electronic device. The electronic device is applied in a model production line. Specifically, the electronic device can be a component of the model production line or can be independent of the model production line but can be connected to the model production line. The electronic device includes but is not limited to a fixed device and / or a mobile device. For example, the fixed device includes but is not limited to a server, and the server can be a cloud server or a general server. For example, the mobile device includes but is not limited to one or more terminals such as a mobile phone or a tablet computer. As Figure 1 shown, the model production method includes:

[0032] S101: Receive a model production request;

[0033] S102: Based on the model production request, determine a to-be-trained data set and first resource configuration information for the to-be-trained data set;

[0034] S103: Based on the first resources indicated by the first resource configuration information, use the to-be-trained data set to train a pre-set model to obtain a target model.

[0035] In the embodiments of the present disclosure, a model production request may be a request input by a user through a user interface. In practical applications, an electronic device displays multiple data sets to the user through the user interface, so that the user can specify one data set from the multiple data sets as the data set to be trained. Further, the model production request may further include first resource configuration information, which is used to indicate information about the resources required for model training. The first resource configuration information includes configuration information of at least one of the following resources: Central Processing Unit (CPU), memory, and Graphics Processing Unit (GPU). It can be understood that in some embodiments, when the model production request does not carry the first resource configuration information, the electronic device automatically determines the first resource configuration information for the model production request.

[0036] In the embodiments of the present disclosure, a pre-set model is an initial model used to train a target model. For example, the pre-set model may be a Region Convolutional Neural Network (RCNN) model. For another example, the pre-set model may be a Fully Convolutional Network (FCN) model. For still another example, the pre-set model may be a model based on the YOLOV3 (You Only Look Once Version3) algorithm.

[0037] In the embodiments of the present disclosure, a target model is a model produced based on a model production request. For example, the target model may be an object detection model. For another example, the target model may be a text matching model. For still another example, the target model may be an image classification model. The above are only exemplary descriptions and do not limit all possible types of the target model. Here, they are not listed exhaustively.

[0038] In the embodiments of the present disclosure, after obtaining the target model, store the target model for subsequent use by the user.

[0039] The technical solution of the embodiments of the present disclosure determines a data set to be trained and first resource configuration information based on the received model production request; trains the pre-set model with the data set to be trained based on the first resources indicated by the first resource configuration information to obtain a target model; in this way, the user does not need to care about code writing and model tuning, and only needs to input a model production request, and the system can automatically complete model generation, which not only reduces the usage threshold of the model production line, but also improves the efficiency of model generation.

[0040] In some embodiments, based on a model production request, a dataset to be trained is determined, including: receiving a first dataset uploaded through a user terminal; and determining the dataset to be trained from the first dataset based on the model production request.

[0041] Here, the number of data in the dataset to be trained is less than or equal to the number of data in the first dataset. In the embodiments of the present disclosure, the number of the first datasets is not limited. When multiple first datasets are received through the user terminal, preprocessing is performed on the multiple first datasets to obtain one or more datasets that can be used as the dataset to be trained for the user terminal.

[0042] In this way, determining the dataset to be trained from the first dataset uploaded by the user terminal based on the model production request can make the produced target model associated with the first dataset and improve the matching degree of the target model.

[0043] In some embodiments, determining the dataset to be trained from the first dataset includes: preprocessing the first dataset and determining the dataset to be trained from the preprocessed first dataset; or, when the first dataset has been preprocessed, determining the dataset to be trained from the first dataset.

[0044] In some embodiments, in response to detecting that preprocessing is required for a first dataset uploaded by the user terminal, all or part of the data in the preprocessed first dataset are formed into the dataset to be trained. In some embodiments, in response to detecting that no preprocessing is required for a first dataset uploaded by the user terminal, all or part of the data in the first dataset are formed into the dataset to be trained.

[0045] In some embodiments, in response to detecting that preprocessing is required for multiple first datasets uploaded by the user terminal, the datasets specified in the model production request are selected from the preprocessed multiple first datasets, and all or part of the data in the specified datasets are formed into the dataset to be trained. In some embodiments, in response to detecting that no preprocessing is required for multiple first datasets uploaded by the user terminal, the datasets specified in the model production request are selected from the multiple first datasets, and all or part of the data in the specified datasets are formed into the dataset to be trained.

[0046] In this way, the diversity of the dataset to be trained can be improved, which helps to enhance the diversity of the target model.

[0047] In some embodiments, preprocessing the first dataset includes cleaning the data in the first dataset.

[0048] Here, the cleaning process includes, but is not limited to, various basic cleaning operations such as automatically deblurring, de-approximating, rotating, and mirroring the image dataset. For example, the data in the first dataset can be cleaned using the technology of the intelligent data service platform (EasyData). The embodiments of the present disclosure do not limit the technology used for the cleaning process.

[0049] In some embodiments, preprocessing the first dataset includes performing annotation processing on the data in the first dataset.

[0050] Here, the annotation processing includes, but is not limited to, manual annotation and automatic annotation. After receiving the first dataset imported by the user through the user interface, when detecting the annotation information input by the user through the user interface, save the annotation information, which can support the user to perform manual annotation on the data in the first dataset. If no annotation information input by the user through the user interface is detected within the preset time, perform intelligent annotation on the data in the first dataset and save the intelligent annotation result. The intelligent annotation can use two methods: active learning and a specified model, and can automatically screen and annotate difficult examples. After manual confirmation and reaching the standard, the annotation is completed.

[0051] In this way, by preprocessing the received dataset, the accuracy of the finally saved dataset is improved, and thus accurate data can be provided for subsequent rapid completion of model training, which not only helps to improve the efficiency of model generation but also helps to improve the accuracy of the generated model.

[0052] In some embodiments, before training the pre-set model using the dataset to be trained, it may further include: removing the mislabeled data and unlabeled data in the dataset to be trained.

[0053] To ensure the accuracy of the data in the dataset to be trained, it is necessary to perform removal processing on the interfering data in the dataset to be trained. Figure 2 The flow chart of data cleaning is shown as Figure 2 As shown, the process includes:

[0054] S201: Receive the start cleaning instruction;

[0055] S202: Detect the image information in the dataset to be trained, and then execute S203;

[0056] S203: Whether there is an image path. If yes, execute S204; if no, execute S206;

[0057] S204: Whether the annotation information is correct. If yes, execute S205; if no, execute S207;

[0058] S205: Whether the number of annotation information is greater than 0. If yes, execute S209; if no, execute S208;

[0059] Here, the annotation information being equal to 0 means there is no annotation information.

[0060] S206: Delete the image information;

[0061] S207: Delete the annotation information;

[0062] S208: Delete the image;

[0063] S209: Retain the image corresponding to the image information and its annotation information.

[0064] In this way, performing interference removal processing before training can provide an accurate data basis for the subsequent generation of the target model, thereby helping to improve the accuracy of the generated target model.

[0065] In some embodiments, training a pre-set model with a dataset to be trained based on the first resource indicated by the first resource configuration information to obtain a target model includes: determining the currently available resources; determining pre-set parameters for the pre-set model based on the currently available resources; training the pre-set model with parameters being the pre-set parameters using the dataset to be trained based on the first resource indicated by the first resource configuration information to obtain a target model.

[0066] Among them, the currently available resources include at least one of the following: the number of currently available GPU resources; the amount of currently available video memory.

[0067] Here, different pre-set models correspond to different pre-set parameters.

[0068] Here, the pre-set parameters include but are not limited to: the number of iterative training times (which can be denoted as epoch), the number of training samples in each batch (which can be denoted as batch_size), the number of batches required to complete one epoch (which can be denoted as iterations), and the learning rate (which can be denoted as learning_rate).

[0069] Among them, one epoch refers to the process of all data being sent into the network to complete one forward calculation and backpropagation process. Due to the large amount of data, it is impossible to send all data into the pre-set model at once. Therefore, the method of sending data into the pre-set model in batches is adopted. During training, it is not enough to train all data once, and it needs to be repeated multiple times to fit and converge.

[0070] Among them, each time a part of the data is sent into the network for training, and batch_size is the number of training samples in each batch.

[0071] Among them, iterations is the number of batches required to complete one epoch.

[0072] Suppose there are 2000 data points, divided into 4 batches, then batch_size = 500. To run all the data for training and complete 1 epoch, 4 iterations are required.

[0073] For different pre-trained models, relatively optimal pre-set parameters are provided respectively. For example, when the pre-trained model is RCNN, the pre-set parameters used include: epoch = 15, batch_size = 2, lr = 0.00025, etc.

[0074] In some embodiments, the process of automatically adjusting batch_size is as follows:

[0075] Traverse all GPUs, find the GPU with the smallest video memory, and record the smallest video memory as MIN_GPU_MEM;

[0076] According to the pre-trained model used, determine the memory consumed by each data point and record it as MEM_PER_BATCH; for example, for Fast RCNN, it can be 2800, and for YOLOV3, it can be 1400;

[0077] Finally, batch_size = MAX((MIN_GPU_MEM / MEM_PER_BATCH), 1).

[0078] In some embodiments, the process of automatically adjusting learning_rate is as follows:

[0079] Record the number of GPUs as GPU_NUM, and use the value calculated in the previous step for BATCH_SIZE;

[0080] According to the pre-trained model used, determine BASE_LR; for example, for Fast RCNN, it can be 0.000125, and for YOLOV3, it can be 0.000125;

[0081] Finally, learning_rate = BASE_LR * BATCH_SIZE * GPU_NUM.

[0082] In this way, the user does not need to manually adjust the parameters of the model. Only by inputting the first data set and the model generation request, the system can automatically complete the generation of the model, which not only reduces the usage threshold of model generation but also improves the efficiency of model generation.

[0083] In some embodiments, determining pre-set parameters for a pre-trained model includes: determining pre-set parameters for the pre-trained model according to the number of currently available GPU resources and video memory.

[0084] For example, automatically adjust the batch_size and learning_rate according to the number of GPUs and the amount of video memory to ensure the training effect and avoid video memory overflow.

[0085] In this way, the preset parameters of the preset model can be made to conform to the currently available resources, thereby avoiding problems such as training interruption or errors caused by the preset parameter settings not conforming to the resource configuration.

[0086] In some embodiments, training a preset model with preset parameters using a dataset to be trained to obtain a target model includes: obtaining a training subset and a validation subset from the dataset to be trained; training the preset model with preset parameters using the training subset, and obtaining multiple models during the training process; validating the multiple models using the validation subset respectively to obtain first accuracies corresponding to the multiple models; and determining the target model according to the model with the highest first accuracy among the multiple models.

[0087] Here, obtaining a training subset and a validation subset from the dataset to be trained includes: obtaining a training subset and a validation subset from the dataset to be trained according to a preset ratio.

[0088] Here, the preset ratio can be 8:1. Among them, the training subset accounts for 80% of the data in the dataset to be trained, the validation subset accounts for 10% of the data in the dataset to be trained, and the remaining 10% can be used as a test subset of the dataset to be trained. It can be understood that the preset ratio can be set or adjusted according to design requirements such as model generation speed or generated model accuracy.

[0089] Here, the multiple models are a general reference, and the specific number will vary according to the number of iteration times. For the same model production line, its training iteration times are preset by the system and do not require user setting and adjustment.

[0090] Here, the first accuracy is the accuracy obtained by validating the target model using the validation subset.

[0091] For example, training the preset model using the training subset, and obtaining n models during the training iteration process, denoted as N1, N2,..., Nn; validating these n models using the validation subset respectively to obtain the first accuracy r1 of model N1, the first accuracy r2 of model N2,..., the first accuracy rn of model Nn; if r1 > r2 >... > rn, then the test subset can be used to test the model N1 with the highest first accuracy to obtain the second accuracy r1' of model N1; finally, determining model N1 as the target model.

[0092] Here, taking model N1 as an example, the first accuracy rate r1 = the number of samples in the validation subset accurately identified by model N1 / the total number of samples in the validation subset. The second accuracy rate r1' = the number of samples in the test subset accurately identified by model N1 / the total number of samples in the test subset. It can be seen that the value of the first accuracy rate can reflect the performance of the target model, and the value of the second accuracy rate can also reflect the performance of the target model.

[0093] In this way, the optimal target model can be selected for the dataset to be trained, improving the performance of the finally generated target model.

[0094] In some embodiments, determining the target model according to the model with the highest first accuracy rate among multiple models includes: obtaining a test subset from the dataset to be trained; testing the model with the highest first accuracy rate among multiple models using the test subset to obtain the second accuracy rate; determining the target model according to the second accuracy rate.

[0095] Here, the second accuracy rate is the accuracy rate obtained by validating the target model using the test subset.

[0096] In some embodiments, if the second accuracy rate meets a certain threshold, the model with the highest first accuracy rate is used as the target model; if the second accuracy rate does not meet a certain threshold, a training subset, a validation subset, and a test subset are re-obtained from the dataset to be trained until the second accuracy rate meets a certain threshold, and the model with the highest first accuracy rate is used as the target model.

[0097] In some embodiments, determining the target model according to the model with the highest first accuracy rate among multiple models includes: obtaining a test subset from the dataset to be trained; testing the model with the highest first accuracy rate among multiple models using the test subset to obtain the second accuracy rate; outputting the second accuracy rate to the user terminal, and determining the target model based on the instruction information fed back by the user terminal for the second accuracy rate.

[0098] In some embodiments, when the electronic device receives a display operation of the model list sent by the user terminal, it outputs the performance metrics of each target model in the model list, including the first accuracy rate and the second accuracy rate, to indicate the performance strength of each target model, thereby helping the user terminal select a target model that meets the requirements from the model list.

[0099] In this way, it is not only convenient to search for each generated target model later, but also convenient to output and display the performance of each target model later.

[0100] In some embodiments, after obtaining the target model, the target model can be applied for detection.

[0101] As Figure 3 shown, the detection process includes:

[0102] S301: Receive a detection request, which includes data to be detected and second resource configuration information;

[0103] S302: Detect the data to be detected based on the second resource indicated by the second resource configuration information and the target model, and obtain a detection result of the data to be detected.

[0104] Here, the data to be detected includes but is not limited to an image to be detected, a text to be detected, etc.

[0105] In some embodiments, the above S301 and S302 may be executed after S103.

[0106] In the embodiments of the present disclosure, the detection request may be input by a user through a user interface.

[0107] In the embodiments of the present disclosure, the second resource configuration information is the resource used when running the target model. The second resource configuration information includes at least one of the following resource configuration information: CPU, memory, GPU.

[0108] It should be noted that the second resource configuration information may be the same as the first resource configuration information or different from the first resource configuration information.

[0109] In this way, the generated target model can be tested online, improving the efficiency of model testing.

[0110] In some embodiments, when multiple target models are obtained in S103, the detection request may further include a target specified model specified from the multiple target models. For example, when n target models are obtained in S103, denoted as N1, N2,..., Nn respectively, if the detection request specifies N1 as the model for detecting data, then the model N1 is used as the target specified model. Further, detecting the data to be detected based on the second resource indicated by the second resource configuration information and the target model, and obtaining a detection result of the data to be detected includes: detecting the data to be detected based on the second resource indicated by the second resource configuration information and the target specified model, and obtaining a detection result of the data to be detected. For example, if the model N1 is used as the target specified model, the second resource is allocated to the model N1 so that the model N1 runs based on the second resource and detects the data to be detected.

[0111] In this way, the generated target model can be tested online, improving the efficiency of model testing.

[0112] Figure 4 Shows a schematic diagram of a model production architecture, as Figure 4 shown, the model production architecture includes three main parts: dataset preparation, model training, and online testing.

[0113] Among them, dataset preparation includes:

[0114] Step 1: Data Import. The user imports the image dataset into the system. The system provides dataset management functions, which can manage multiple datasets and multiple groups of datasets. With the help of Baidu EasyData technology, various basic cleaning operations such as automatic deblurring, de-approximation, rotation, and mirroring can be performed on the image dataset. If the imported dataset carries annotation information, step 2 can be omitted.

[0115] Step 2: Data Annotation. The user can annotate the imported data. With the help of Baidu EasyData technology, it supports manual annotation and automatic annotation. Automatic annotation can use two methods: active learning and specified model, and can automatically screen and annotate difficult examples. After manual confirmation and reaching the standard, the annotation is completed.

[0116] Among them, for model training, the user only needs to specify a complete dataset with annotation information and configure the resources used (such as CPU, memory, GPU). After the entire training cycle, the target model can be output.

[0117] Here, model training includes:

[0118] Step 1: Load Data. The system displays all currently available datasets, as well as basic information such as specific images and labels in the datasets. The user can select a group of datasets for the system to load.

[0119] Step 2: Data Cleaning. There may be some incorrect data or data without annotation information in the input image data, which interferes with normal training. Therefore, data cleaning is required before training.

[0120] Step 3: Data Splitting. The dataset to be trained is split into a training subset, a validation subset, and a test subset. Among them, the training subset accounts for 80%, and the validation subset and the test subset each account for 10%. The training subset is used for model training, the validation subset is used for iterative validation, and the test subset is used for testing the model effect.

[0121] Step 4: Model Training. Use the preset model and preset parameters for training. For different preset models, relatively optimal preset parameters are provided respectively. For example, for Fast RCNN, the following preset parameters are used (epoch = 15, batch_size = 2, lr = 0.00025, etc.); at the same time, batch_size and learning_rate are automatically adjusted according to the number of GPUs and the video memory size to ensure the training effect and avoid video memory overflow.

[0122] Step 5: Model Evaluation. Multiple models generated during the iterative process are predicted and evaluated using the test subset, and the accuracy rates of the test subset and the validation subset are statistically calculated.

[0123] Step 6: Model Screening. According to the metrics of the validation subset, select the optimal model and enter it into the model management system, and record the metrics.

[0124] After training is completed, the user can use the trained model to start the prediction service and conduct service testing.

[0125] Here, the online testing includes:

[0126] Step 1: Select a model. The user selects the model to be tested from the model list.

[0127] Step 2: Start the prediction service. The user configures the information such as CPU, memory, GPU, etc. required for the prediction service, and can set the service auto-release time to save resources.

[0128] Step 3: Online testing. The system pre-packages the interfaces required to call the prediction service. The user can upload the data to be tested in the interface for testing, and the system will display the prediction results, including information such as the annotation location and confidence level.

[0129] In addition, the user can start the prediction service for multiple models to compare the training effects between models.

[0130] In addition, the model parameters can be exposed instead of being built-in or automatically adjusted, so that the user can adjust the parameters by himself to optimize the model effect.

[0131] The model production architecture described in the embodiments of the present disclosure abstracts and encapsulates the entire life cycle of model production, provides a complete and out-of-the-box target model production pipeline. The user does not need to care about code writing and model parameter tuning. Only by inputting the dataset, the system can automatically complete the entire process of data cleaning, data splitting, model training, model evaluation, and model screening, and can manage and conduct online testing on the generated models. In the form of a pipeline, the target model can be trained in a very short time, and the dataset and the produced models can be uniformly managed. In addition, the user does not need to have professional capabilities, which greatly reduces the usage cost.

[0132] It should be understood that Figure 4 the shown architecture diagram is merely illustrative and not restrictive. Those skilled in the art can make various obvious changes and / or substitutions based on Figure 4 the examples, and the obtained technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0133] The embodiments of the present disclosure provide a model production device, which is applied to a model production line. As Figure 5As shown in the figure, the model production device may include: a first receiving module 501, configured to receive a model production request; a determination module 502, configured to determine a to-be-trained data set and first resource configuration information for the to-be-trained data set based on the model production request; and a generation module 503, configured to train a pre-set model using the to-be-trained data set based on the first resources indicated by the first resource configuration information to obtain a target model.

[0134] In some embodiments, the first receiving module 501 is further configured to receive a first data set uploaded by a user terminal; and the determination module 502 is further configured to determine the to-be-trained data set from the first data set based on the model production request.

[0135] In some embodiments, the determination module 502 is further configured to: preprocess the first data set and determine the to-be-trained data set from the preprocessed first data set; or, in the case where the first data set has been preprocessed, determine the to-be-trained data set from the first data set.

[0136] Among them, the preprocessing includes at least one of the following:

[0137] Cleaning the data in the first data set;

[0138] Labeling the data in the first data set.

[0139] In some embodiments, the generation module 503 includes: a first determination sub-module, configured to determine currently available resources; a second determination sub-module, configured to determine pre-set parameters for the pre-set model based on the currently available resources; and a generation sub-module, configured to train the pre-set model with the pre-set parameters using the to-be-trained data set based on the first resources indicated by the first resource configuration information to obtain a target model.

[0140] Among them, the currently available resources include: the number of currently available GPU resources and / or the currently available video memory.

[0141] In some embodiments, the generation sub-module is configured to obtain a training subset and a validation subset from the to-be-trained data set; train the pre-set model with the pre-set parameters using the training subset, and obtain multiple models during the training process; verify the multiple models respectively using the validation subset to obtain first accuracies respectively corresponding to the multiple models; and determine the target model according to the model with the highest first accuracy among the multiple models.

[0142] In some embodiments, the generation sub-module is further configured to obtain a test subset from the dataset to be trained; use the test subset to test the model with the highest first accuracy among multiple models to obtain a second accuracy; determine a target model according to the second accuracy, or output the second accuracy to a user terminal, and determine the target model based on the instruction information fed back by the user terminal for the second accuracy.

[0143] In some embodiments, the model production device may further include: a second receiving module 504 (not shown in the figure), configured to receive a detection request, where the detection request includes data to be detected and second resource configuration information; a testing module 505 (not shown in the figure), configured to detect the data to be detected based on the second resources indicated by the second resource configuration information and the target model to obtain a detection result of the data to be detected.

[0144] In some embodiments, in the case where there are multiple target models, the detection request further includes a target designated model designated from the multiple target models, and the testing module 505 (not shown in the figure) is further configured to detect the data to be detected based on the second resources indicated by the second resource configuration information and the target designated model to obtain a detection result of the data to be detected.

[0145] Those skilled in the art should understand that the functions of the various processing modules in the model production device of the embodiments of the present disclosure can be understood with reference to the relevant descriptions of the foregoing model production method. The various processing modules in the model production device of the embodiments of the present disclosure can be implemented by a simulation circuit that implements the functions described in the embodiments of the present disclosure, or can be implemented by software that executes the functions described in the embodiments of the present disclosure running on an electronic device.

[0146] The model production device of the embodiments of the present disclosure can automatically complete model generation, which not only reduces the usage threshold of the model production line but also improves the efficiency of model generation.

[0147] The embodiments of the present disclosure provide a model production system applied to a model production line. As Figure 6As shown, the model production system may include: a user interface interface 601 for receiving a plurality of first data sets and receiving a model production request; a data management module 602 for preprocessing the plurality of first data sets to obtain a plurality of second data sets; a model training module 603 for determining a data set to be trained and first resource configuration information based on the model production request, where the data set to be trained is a data set in the second data sets; training a preset model using the data set to be trained based on the first resource indicated by the first resource configuration information to obtain a target model; an online testing module 604 for receiving a detection request through the user interface interface, where the detection request includes data to be tested, a specified target model, and second resource configuration information; and inputting the data to be tested into the specified target model based on the second resource indicated by the second resource configuration information to obtain a detection result of the data to be tested.

[0148] In some embodiments, the data management module 602 is configured to preprocess the plurality of first data sets by at least one of the following methods: cleaning the data in the plurality of first data sets; and annotating the data in the plurality of first data sets.

[0149] In some embodiments, the data management module 602 is further configured to remove mislabeled data and unlabeled data in the data set to be trained before the model training module 603 trains the preset model using the data set to be trained.

[0150] In some embodiments, the model training module 603 is further configured to determine currently available resources; determine preset parameters for the preset model based on the currently available resources; and train the preset model with the preset parameters using the data set to be trained based on the first resource indicated by the first resource configuration information to obtain a target model.

[0151] In some embodiments, the model training module 603 is specifically configured to determine preset parameters for the preset model according to the currently available number of GPU resources and video memory.

[0152] In some embodiments, the model training module 603 is further configured to obtain a training subset and a validation subset from the data set to be trained; train the preset model with the preset parameters using the training subset, and obtain a plurality of models during the training process; verify the plurality of models respectively using the validation subset to obtain first accuracies corresponding to the plurality of models respectively; and determine the target model according to the model with the highest first accuracy among the plurality of models.

[0153] In some embodiments, the model training module 603 is further configured to obtain a test subset from the dataset to be trained; test the model with the highest first accuracy among multiple models by using the test subset to obtain a second accuracy; determine the target model according to the second accuracy, or output the second accuracy to the user terminal, and determine the target model based on the instruction information fed back by the user terminal for the second accuracy.

[0154] In some embodiments, the data management module 602 is further configured to save each target model and record the performance metrics of each target model, where the performance metrics include the first accuracy and the second accuracy.

[0155] Those skilled in the art should understand that the functions of the processing modules in the model production system according to the embodiments of the present disclosure can be understood with reference to the relevant descriptions of the foregoing model production method. Each processing module in the model production device according to the embodiments of the present disclosure can be implemented by an analog circuit that implements the functions described in the embodiments of the present disclosure, or can be implemented by the operation of software that executes the functions described in the embodiments of the present disclosure on an electronic device.

[0156] The model production system according to the embodiments of the present disclosure can automatically complete model generation, which not only reduces the usage threshold of the model production line but also improves the efficiency of model generation.

[0157] Figure 7 shows a schematic diagram of the model production scenario. From Figure 7It can be seen that an electronic device such as a cloud server receives multiple first data sets imported from various terminals; preprocesses the received multiple first data sets to obtain multiple second data sets; determines a data set to be trained and first resource configuration information based on model production requests received from various terminals; and trains a preset model using the data set to be trained based on the first resource indicated by the first resource configuration information to obtain a target model. The electronic device receives detection requests sent from various terminals, where the detection requests include data to be detected, a specified target model, and second resource configuration information; and inputs the data to be detected into the specified target model based on the second resource indicated by the second resource configuration information to obtain a detection result of the data to be detected. In this way, the user does not need to care about code writing and model parameter tuning. The user only needs to input the first data set, and the system can automatically complete the entire process of data cleaning, data splitting, model training, model evaluation, and model screening, and can manage and online test the generated model. For example, if a user wants to detect target object 1 in a batch of images, the user first imports some data sets with annotation information into the model production system, and the model production system automatically generates a target model, and then uses the target model to detect target object 1 in this batch of images. Another example is that if a user wants to detect target text in a batch of texts, the user first imports some data sets with annotation information into the model production system, and the model production system automatically generates a target model, and then uses the target model to detect target text in this batch of texts.

[0158] The present disclosure does not limit the number of terminals and electronic devices. In practical applications, there may be multiple terminals and multiple electronic devices.

[0159] It should be understood that Figure 7 the illustrated scenario diagrams are merely illustrative and not restrictive. Those skilled in the art can make various obvious changes and / or substitutions based on Figure 7 the examples, and the obtained technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0160] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0161] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0162] Figure 8FIG. 0 shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0163] As Figure 8 shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0164] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0165] The computing unit 801 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the model production method. For example, in some embodiments, the model production method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the model production method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the model production method in any other suitable way (e.g., by means of firmware).

[0166] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application-specific standard products (ASSPs), system on chip (SOC) systems, complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or a controller of a general purpose computer, a special purpose computer, or other programmable model production devices, such that when the program codes are executed by the processor or the controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0168] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0170] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0171] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client and server relationship is generated by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating a blockchain.

[0172] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0173] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A model production method, comprising: Receiving a model production request; Based on the model production request, determining a to-be-trained data set and first resource configuration information for the to-be-trained data set; wherein, based on the model production request, determining the to-be-trained data set includes: receiving a first data set uploaded by a user terminal; based on the model production request, determining the to-be-trained data set from the first data set; Training a pre-set model with the to-be-trained data set based on the first resource indicated by the first resource configuration information to obtain a target model; Wherein, the training the pre-set model with the to-be-trained data set based on the first resource indicated by the first resource configuration information to obtain a target model includes: Determining currently available resources; Based on the currently available resources, determining pre-set parameters for the pre-set model; Training the pre-set model with the parameter of the pre-set parameters with the to-be-trained data set based on the first resource indicated by the first resource configuration information to obtain the target model; Wherein, the training the pre-set model with the parameter of the pre-set parameters with the to-be-trained data set to obtain the target model includes: Obtaining a training subset and a validation subset from the to-be-trained data set; Training the pre-set model with the parameter of the pre-set parameters with the training subset, and obtaining multiple models during the training process; Validating the multiple models respectively with the validation subset to obtain first accuracy rates respectively corresponding to the multiple models; Determining the target model according to the model with the highest first accuracy rate among the multiple models.

2. The method according to claim 1, wherein The determining the to-be-trained data set from the first data set includes: Preprocessing the first data set, and determining the to-be-trained data set from the preprocessed first data set; or, In the case where the first data set is preprocessed, determining the to-be-trained data set from the first data set.

3. The method according to claim 2, wherein the preprocessing includes at least one of the following: Cleaning the data in the first data set; Labeling the data in the first data set.

4. The method according to claim 1, wherein, The currently available resources include: the number of currently available graphics processing unit (GPU) resources and / or the currently available video memory.

5. The method according to claim 1, wherein, The determining the target model according to the model with the highest first accuracy rate among the multiple models includes: Obtaining a test subset from the to-be-trained data set; Testing the model with the highest first accuracy rate among the multiple models with the test subset to obtain a second accuracy rate; Determining the target model according to the second accuracy rate, or outputting the second accuracy rate to the user terminal, and determining the target model based on instruction information fed back by the user terminal for the second accuracy rate.

6. The method according to any one of claims 1-5, further comprising: Receiving a detection request, where the detection request includes to-be-detected data and second resource configuration information; Detecting the to-be-detected data based on the second resource indicated by the second resource configuration information and the target model to obtain a detection result of the to-be-detected data.

7. According to the method described in claim 6, the target model is a plurality of target models, and the detection request further includes a target designated model specified from the plurality of target models. Detecting the data to be detected based on the second resource indicated by the second resource configuration information and the target model to obtain a detection result of the data to be detected includes: Detecting the data to be detected based on the second resource indicated by the second resource configuration information and the target designated model to obtain a detection result of the data to be detected.

8. A model production device, comprising: A first receiving module, configured to receive a model production request; A determining module, configured to determine a training data set to be trained and first resource configuration information for the training data set to be trained based on the model production request; wherein, the first receiving module is further configured to receive a first data set uploaded by a user terminal; the determining module is further configured to determine the training data set to be trained from the first data set based on the model production request; A generating module, configured to train a pre-set model using the training data set to be trained based on the first resource indicated by the first resource configuration information to obtain a target model; Wherein, the generating module includes: A first determining sub-module, configured to determine currently available resources; A second determining sub-module, configured to determine pre-set parameters for the pre-set model based on the currently available resources; A generating sub-module, configured to train the pre-set model with the pre-set parameters using the training data set to be trained based on the first resource indicated by the first resource configuration information to obtain the target model; Wherein, the generating sub-module is configured to: Obtain a training subset and a validation subset from the training data set to be trained; Train the pre-set model with the pre-set parameters using the training subset, and obtain a plurality of models during the training process; Validate the plurality of models respectively using the validation subset to obtain first accuracies respectively corresponding to the plurality of models; Determine the target model according to the model with the highest first accuracy among the plurality of models.

9. The device according to claim 8, wherein The determining module is further configured to: Pre-process the first data set, and determine the training data set to be trained from the pre-processed first data set; Or, In the case where the first data set has been pre-processed, determine the training data set to be trained from the first data set.

10. According to the device described in claim 9, the pre-processing includes at least one of the following: Performing a cleaning process on the data in the first data set; Performing an annotation process on the data in the first data set.

11. The apparatus according to claim 8, wherein, The currently available resources include: the number of currently available Graphics Processing Unit (GPU) resources and / or the currently available video memory.

12. The apparatus according to claim 8, wherein, The generating sub-module is further configured to: Obtain a test subset from the training data set to be trained; Test the model with the highest first accuracy among the plurality of models using the test subset to obtain a second accuracy; Determine the target model according to the second accuracy rate, or output the second accuracy rate to a user terminal, and determine the target model based on the instruction information feedback by the user terminal for the second accuracy rate.

13. The device according to any one of claims 8-12, further comprising: A second receiving module, configured to receive a detection request, where the detection request includes data to be detected and second resource configuration information; A testing module, configured to detect the data to be detected based on the second resource indicated by the second resource configuration information and the target model, and obtain a detection result of the data to be detected.

14. The device according to claim 13, wherein the target model is a plurality of target models, and the detection request further includes a target specified model specified from the plurality of target models, and the testing module is further configured to: Detect the data to be detected based on the second resource indicated by the second resource configuration information and the target specified model, and obtain a detection result of the data to be detected.

15. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product, comprising a computer program, where the computer program, when executed by a processor, implements the method according to any one of claims 1-7.

Images