A remote model training system
Through the remote model training system combining local neural network processing units and remote cloud, real equipment verification data is used to adjust the training files, solving the problems of long and low efficiency of existing AI models, and achieving more efficient model training and accuracy.
Patent Information
- Application Number
- CN202410172303.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-06
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2044-02-06
AI Technical Summary
The existing AI model training methods are carried out in the cloud, and lack verification on real devices, resulting in long training time and low efficiency, unable to meet the accuracy requirements, and need to be retrained.
Design a remote model training system, combine local neural network processing units and remote cloud, and adjust the training file through real devices to verify data feedback until the model meets the accuracy requirements on the real device, saving training time and resources.
Through verification data-oriented training of local neural network processing units, the AI model is verified using real device data, which improves the efficiency and accuracy of model training and reduces the training time.
Smart Images
Figure CN118036693B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to but is not limited to system-on-chip technology, and particularly to a remote model training system. Background Art
[0002] In the related art, the conventional model training method of artificial intelligence (AI) is to remotely train the model in the cloud; in the process of training the model, part of the data set is used as test data to test the model, and according to the test results, the model is adjusted to continue the simulation training. After repeated simulation training and testing, the model is calibrated with actual data to obtain a model that meets the requirements. When the above method is used for model training, although the model is calibrated with actual data as test data, it is not inferred and verified in the real device of AI. If the obtained model does not meet the accuracy requirements after verification on the real device, the model needs to be retrained, which is time-consuming and inefficient. Summary of the invention
[0003] The following is a summary of the subject matter described in detail in this application. This summary is not intended to limit the scope of the claims.
[0004] The disclosed embodiments provide a remote model training system that can reduce the training time of an AI model and improve the training efficiency of an AI model.
[0005] The embodiment of the present disclosure provides a remote model training system, including: a model training device, and a local neural network processing unit and a remote cloud connected to the model training device; wherein:
[0006] A model training device, configured to: output a training file including an initial AI model, a training data set, and predetermined model training steps to a remote cloud; receive verification data from a local neural network processing unit; determine whether to continue the AI model training based on the received verification data; when it is determined to continue the AI model training, redetermine the training file based on the verification data, and send the redetermined training file to the remote cloud;
[0007] The remote cloud is configured to: receive the training file, train the AI model in the training file through the training data set according to the model training steps in the training file; output the trained AI model to the local neural network processing unit according to a preset period;
[0008] The local neural network processing unit is connected to the real device and is configured to: obtain actual data from the real device; verify the trained AI model through the obtained actual data to obtain verification data; and send the obtained verification data to the model training device.
[0009] In an exemplary embodiment, the preset period includes: a time period determined according to a percentage value of the training progress of the AI model.
[0010] In an exemplary embodiment, the model training device is configured to redetermine the training file based on the verification data, including: redetermining the training data set based on the verification data and adjusting the composition of the AI model.
[0011] In an exemplary embodiment, the remote cloud is configured as:
[0012] When a training file is received for the first time, the initial AI model in the training file is trained using a training data set according to the model training steps in the training file; the trained AI model is output to the local neural network processing unit according to a preset period;
[0013] Starting from the second time, each time a re-determined training file is received, the AI model with the adjusted composition contained in the training file is trained according to the re-determined training data set in the training file; and the trained AI model is output to the local neural network processing unit according to a preset period.
[0014] In an exemplary embodiment, the model training device is configured to determine whether to continue the training of the AI model according to the received verification data, including:
[0015] Based on the verification data, determine whether the accuracy of the AI model is greater than or equal to a preset accuracy threshold; when the accuracy of the AI model is greater than or equal to the preset accuracy threshold, determine that the AI model meets the preset accuracy requirements, and notify the remote cloud to stop AI model training; when it is determined that the AI model does not meet the preset accuracy requirements, continue training the AI model.
[0016] In an exemplary embodiment, the local neural network processing unit includes: a system-on-chip SoC based on FPGA and MCU;
[0017] Among them, the MCU is used as the main processor for data processing of the AI model; the FPGA logic implements the convolution operation accelerator as a coprocessor for parallel acceleration of convolution operations when running the AI model.
[0018] In an exemplary embodiment, the model training device is configured to: adjust the training data set of the AI model and the composition of the AI model according to the verification data, including:
[0019] In one exemplary embodiment, the AI model is used to perform object recognition.
[0020] Compared with the related art, the present application includes: a model training device, and a local neural network processing unit and a remote cloud connected to the model training device; wherein the model training device is configured to: output a training file containing an initial AI model, a training data set and predetermined model training steps to the remote cloud; receive verification data from the local neural network processing unit; determine whether to continue the training of the AI model based on the received verification data; when it is determined to continue the AI model training, redetermine the training file based on the verification data, and send the redetermined training file to the remote cloud; the remote cloud is configured to: receive the training file, train the AI model in the training file through the training data set according to the model training steps in the training file; output the trained AI model to the local neural network processing unit according to a preset period; the local neural network processing unit is connected to a real device, and is configured to: obtain actual data from the real device; verify the trained AI model through the obtained actual data to obtain verification data; and send the obtained verification data to the model training device. The disclosed embodiment trains the AI model guided by the verification data of the local neural network processing unit, uses the actual data of the real device to verify the local AI model, feeds the verification data back to the remote AI model training, and adjusts the training file of the AI model according to the feedback verification results. The model training is completed until the trained AI model passes the actual data verification of the real device and meets the pre-set accuracy requirements, thereby saving time and resources for model training and improving training efficiency.
[0021] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.
[0023] Figure 1 It is a structural block diagram of the remote model training system according to an embodiment of the present disclosure;
[0024] Figure 2 A schematic diagram of an AI model interpreter according to an embodiment of the present disclosure;
[0025] Figure 3 It is a structural block diagram of the remote model training system according to an embodiment of the present disclosure;
[0026] Figure 4 A schematic diagram of the process of parsing the AI model according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.
[0028] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.
[0029] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.
[0030] Although actual data correction is used in related technologies, the correction process is always simulated on a graphics processing unit (GPU) terminal and has not been inferred or calculated on a real device. It only focuses on the training side. Even if the trained model meets the preset requirements, if there are problems when the model is applied to a real device for calculation, the training quality needs to be improved.
[0031] Figure 1 is a structural block diagram of the remote model training system according to an embodiment of the present disclosure, such as Figure 1As shown, it includes: a model training device, and a local neural network processing unit and a remote cloud connected to the model training device; wherein,
[0032] A model training device, configured to: output a training file including an initial AI model, a training data set, and predetermined model training steps to a remote cloud; receive verification data from a local neural network processing unit; determine whether to continue the AI model training based on the received verification data; when it is determined to continue the AI model training, redetermine the training file based on the verification data, and send the redetermined training file to the remote cloud;
[0033] The remote cloud is configured to: receive the training file, train the AI model in the training file through the training data set according to the model training steps in the training file; output the trained AI model to the local neural network processing unit according to a preset period;
[0034] The local neural network processing unit is connected to the real device and is configured to: obtain actual data from the real device; verify the trained AI model through the obtained actual data to obtain verification data; and send the obtained verification data to the model training device.
[0035] The disclosed embodiment trains the AI model guided by the verification data of the local neural network processing unit, uses the actual data of the real device to verify the local AI model, feeds the verification data back to the remote AI model training, and adjusts the training file of the AI model according to the feedback verification results. The model training is completed until the trained AI model passes the actual data verification of the real device and meets the pre-set accuracy requirements, thereby saving time and resources for model training and improving training efficiency.
[0036] In an exemplary instance, the preset period of the embodiment of the present disclosure may be a percentage value of the training progress of the AI model, and the specific percentage value may be set by a person skilled in the art based on experience; for example, greater than or equal to 60 percent of the training progress.
[0037] In an exemplary instance, an embodiment of the present disclosure redefines the training file based on the verification data, including: redetermining the training data set based on the verification data and adjusting the composition of the AI model.
[0038] In an exemplary instance, the remote cloud of the disclosed embodiment is configured as follows: when a training file is received for the first time, the initial AI model in the training file is trained using a training data set according to the model training steps in the training file; the trained AI model is output to a local neural network processing unit according to a preset period; starting from the second time, each time a re-determined training file is received, the AI model with an adjusted composition contained in the training file is trained according to the re-determined training data set in the training file; the trained AI model is output to the local neural network processing unit according to a preset period.
[0039] In an exemplary embodiment, the remote cloud in the embodiment of the present disclosure may be a server in a remote data center.
[0040] In an exemplary embodiment, the AI model in the embodiment of the present disclosure is the front-end component of the AI application, and one AI application may include more than one AI model.
[0041] In an exemplary embodiment, the model training device of the embodiment of the present disclosure is configured to determine whether to continue the training of the AI model according to the received verification data, including:
[0042] Determine whether the accuracy of the AI model is greater than or equal to a preset accuracy threshold based on the verification data; when the accuracy of the AI model is greater than or equal to the preset accuracy threshold, determine that the AI model meets the preset accuracy requirements, and notify the remote cloud to stop AI model training; when it is determined that the AI model does not meet the preset accuracy requirements, continue training the AI model;
[0043] In an exemplary embodiment, the local neural network processing unit of the embodiment of the present disclosure is implemented based on a system on chip (SoC) of FPGA and MCU, wherein the MCU serves as a main processor for data processing, and the FPGA logic implements a convolution operation accelerator as a coprocessor for parallel accelerated execution of convolution operations.
[0044] The local neural network processing unit of the disclosed embodiment receives the AI model obtained by remote cloud training, runs the inference of the AI model, obtains verification data based on the inference result, and determines whether the AI model meets the preset accuracy requirements based on the verification data. If it is determined that the AI model meets the accuracy requirements based on the verification data of the AI model, feedback is given to the model training device, and the AI model training on the remote cloud is stopped. If it is determined that the AI model does not meet the accuracy requirements based on the verification data of the AI model, the verification data is fed back to the model training device, and the model training device adjusts the training data set and the composition of the AI model based on the verification data, and the remote cloud uses the re-determined training data set to train the AI model with the adjusted composition until the accuracy of the model meets the requirements.
[0045] In an exemplary embodiment, the model training device of the embodiment of the present disclosure is configured to: adjust the training data set of the AI model and the composition of the AI model according to the verification data, including:
[0046] Add or delete the training data set of the AI model based on the verification data, add or delete the layers of the AI model, and / or adjust the operator types of the layers.
[0047] The disclosed embodiment takes character recognition as an example. Various factors such as the character's posture (standing, sitting, squatting, etc.), the character's position (front, back, side), and the character's clothing may affect the character's recognition. According to the verification results, it may be necessary to increase the data set to fully display the physical characteristics of a person. The level and operator of the model in the disclosed embodiment directly determine the complexity of the model, and thus determine the accuracy of the model operation; according to the verification results, such as the result deviation of a certain level of model is large, not convergent enough, etc., it can be determined by referring to the relevant principles whether it is necessary to add a layer of convolution, pooling, etc.
[0048] In one exemplary embodiment, the AI model of the disclosed embodiment is used to perform object recognition, such as car recognition.
[0049] The following briefly describes the embodiments of the present disclosure through application examples. The application examples are only used to illustrate the embodiments of the present disclosure and are not used to limit the protection scope of the embodiments of the present disclosure.
[0050] Application Examples
[0051] The present disclosure implements car recognition as an example, in which the car image captured by the camera is used to infer and identify the car by running the AI model.
[0052] During the AI model training process, the training data set is a large number of various types of cars. When the AI model is trained to a certain percentage (preset period), on the one hand, it is tested with actual data in the remote cloud, and on the other hand, the trained AI model is transmitted back to the local neural network processing unit, and the local neural network processing unit is loaded on the real device to run the inference of the AI model and obtain the inference result of the AI model on the real device; based on this inference result, refer to relevant technologies and combine the test in the remote cloud to obtain verification data; modify the composition of the trained AI model and the training data set according to the obtained verification data, and then correct the AI model. When it is determined based on the verification data that the accuracy of the test in the remote cloud and the accuracy of the local real device test meet the requirements, the training can be ended.
[0053] The AI model trained remotely in the cloud in the disclosed embodiment will be continuously transmitted back to the local neural network processing unit for real-time verification according to a preset period.
[0054] The following is a brief description of the embodiments of the present disclosure by taking the AI model training and verification of car detection as an example.
[0055] Application Examples
[0056] Figure 2 Schematic diagram of the AI model interpreter of the present disclosure embodiment, as shown in Figure 2 As shown, the AI model interpreter includes a model parsing module, a model data analysis module and a model data output module; wherein,
[0057] The model parsing module reads the car_detection.tflite file of the AI model, uses the json file format, parses the tflite format into the json format, and outputs the json format file car_detection.json to the model data analysis module.
[0058] The model data analysis module parses and extracts the data in the json file; the json format file structure of the embodiment of the present disclosure is simple and vivid, each layer of the model is clear at a glance, and it is very easy to extract data. Through the json format file, the operator type (for example, CONV_2D) of each layer of the AI model, the shape of the model, input and output, various parameter data, weight data and bias data, etc. can be vividly parsed.
[0059] In an exemplary instance, the AI model interpreter of the embodiment of the present disclosure may be implemented in an FPGA using a hardware description language; for example, the AI model interpreter may be a module implemented using Verilog or VHDL language.
[0060] Figure 3 : is a structural block diagram of the remote model training system according to an embodiment of the present disclosure, such as Figure 3 As shown, the AI model of car detection in the embodiment of the present disclosure enters the AI model interpreter, and the AI model is parsed by the AI model interpreter to decompose the parameter data, weight data and bias data of the AI model. The decomposed parameter data, weight data and bias data respectively enter the MCU, convolution operation accelerator and Flash unit of the local neural network processing unit to realize the operation of the AI model.
[0061] Figure 4 Schematic diagram of the process of parsing the AI model in the embodiment of the present disclosure, such as Figure 4As shown, the operator type (e.g., CONV_2D) and calculation data of each layer of the AI model are parsed through a json format file. The calculation data may include: shape, input and output, various parameter data, weight data, and bias data. Taking the operator type as Conv2D, the calculation data may include various types of parameter data such as padding, stride, dilation_factor, in_offset, out_offset, wt_offset, in_ch, in_dim, out_ch, out_dim, ker_ch, and ker_dim, as well as weight data and bias data, which are well known to those skilled in the art.
[0062] In an exemplary embodiment, the embodiment of the present disclosure opens a file in json format; three matrices can be defined to store various parameter data, weight data, and bias data of each layer of the AI model, specifically:
[0063] Loop through each layer of AI model and determine the operator type of each layer of AI model;
[0064] If the operator type is Conv2D, various parameter data, weight data and bias data are extracted according to the model form of Conv2D, and saved to the corresponding positions of the corresponding matrix; if it is DepthwiseConv2D, various parameter data, weight data and bias data are extracted according to the model form of DepthwiseConv2D, and saved to the corresponding positions of the corresponding matrix; if it is AveragePool2D, various parameter data, weight data and bias data are extracted according to the model form of AveragePool2D, and saved to the corresponding positions of the corresponding matrix; if it is Softmax, various parameter data are extracted according to the model form of Softmax, and saved to the corresponding positions of the corresponding matrix; the model data output module outputs the parameter data matrix, weight data matrix and bias data matrix saved in each layer model to various docking modules in the next step.
[0065] The disclosed embodiment parses and obtains the parameter data, weight data, and bias data of each layer of the AI model, and gives them to the MCU unit and the Flash unit respectively.
[0066] The front-end camera collects video data of the car, and after simple image processing, it is used as image data input of the convolution operation accelerator. The MCU unit controls the parameter data output to the accelerator unit. The MCU unit controls the weight data and bias data of the Flash unit to be output to the accelerator unit to run the accelerator.
[0067] Starting from the first layer of AI models, process each layer of AI models according to this process.
[0068] After the AI model is processed, the verification result is obtained. The verification result is output to the model training device, and the model training device compares the accuracy of the verification result with golden. The comparison items will be set in advance. For car detection, such as vehicle model, shape, color and other features, and the threshold is also set. For example, if the accuracy reaches 95%, the disclosed embodiment believes that the AI model training can be completed, then the remote cloud is notified to end the training, and the current AI model can be used as the final state. If the accuracy does not meet expectations, analyze which items do not meet expectations, adjust the training method and training data set, and upload them to the remote cloud. The remote cloud continues to train the adjusted AI model according to the new training data set.
[0069] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
Claims
1. A remote model training system, characterized in that: include: A model training device, and a local neural network processing unit and a remote cloud connected to the model training device; wherein, A model training device, configured to: output a training file including an initial artificial intelligence (AI) model, a training data set, and predetermined model training steps to a remote cloud; receive verification data from a local neural network processing unit; determine whether to continue the AI model training based on the received verification data; when it is determined to continue the AI model training, redetermine the training file based on the verification data, and send the redetermined training file to the remote cloud; The remote cloud is configured to: receive the training file, train the AI model in the training file through the training data set according to the model training steps in the training file; output the trained AI model to the local neural network processing unit according to a preset period; The local neural network processing unit is connected to the real device and is configured to: obtain actual data from the real device; verify the trained AI model through the obtained actual data to obtain verification data; and send the obtained verification data to the model training device.
2. The remote model training system according to claim 1, characterized in that: The preset period includes: a time period determined according to the percentage value of the training progress of the AI model.
3. The remote model training system according to claim 1, characterized in that: The model training device is configured to re-determine the training file based on the verification data, including: Redefine the training data set and adjust the composition of the AI model based on the verification data.
4. The remote model training system according to claim 3, characterized in that: The remote cloud is configured as: When the training file is received for the first time, the initial AI model in the training file is trained using the training data set according to the model training steps in the training file; and the AI model obtained through training is output to the local neural network processing unit according to the preset period; Starting from the second time, each time the re-determined training file is received, the AI model with the adjusted composition contained in the training file is trained according to the re-determined training data set in the training file; and the AI model obtained through training is output to the local neural network processing unit according to the preset period.
5. The remote model training system according to claim 3, characterized in that: The step of adjusting the training data set of the AI model and the composition of the AI model according to the verification data includes: According to the verification data, the training data set of the AI model is added or deleted, and the levels of the AI model and / or the operator types of the levels are adjusted.
6. The remote model training system according to any one of claims 1 to 5, characterized in that: The model training device is configured to determine whether to continue the training of the AI model according to the received verification data, including: Based on the verification data, determine whether the accuracy of the AI model is greater than or equal to a preset accuracy threshold; when the accuracy of the AI model is greater than or equal to the accuracy threshold, determine that the AI model meets the preset accuracy requirements, and notify the remote cloud to stop AI model training; when it is determined that the AI model does not meet the accuracy requirements, continue training the AI model.
7. The remote model training system according to any one of claims 1 to 5, characterized in that: The local neural network processing unit includes: a system on chip SoC based on a field programmable gate array FPGA and a microcontroller unit MCU; Among them, the MCU is used as the main processor for data processing of the AI model; the FPGA logic implements the convolution operation accelerator as a coprocessor for parallel acceleration of convolution operations when running the AI model.
8. The remote model training system according to any one of claims 1 to 5, characterized in that: The AI model is used to perform object recognition.