A training method, device and equipment for machine learning program

By identifying and labeling training data within the confidence range in machine learning programs, and automatically training new programs, the problem of time-consuming data set collection and calibration during machine learning training is solved, and efficiency and accuracy are improved.

CN112884158BActive Publication Date: 2024-11-26HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201911207495.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-11-29
Publication Date
2024-11-26
Estimated Expiration
2039-11-29

AI Technical Summary

Technical Problem

During the training process of machine learning programs, data set collection and calibration work is large and time-consuming, resulting in inefficiency.

Method used

By obtaining the first type of training data in the training data set, adding labeling information using the confidence range identified by the machine learning program, and using this data to train new machine learning programs to reduce the user's manual calibration and data collection work.

Benefits of technology

It reduces the time for user manual calibration and data collection, improves the performance and accuracy of machine learning programs, and realizes automatic iterative update of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112884158B_ABST
    Figure CN112884158B_ABST
Patent Text Reader

Abstract

The present application provides a training method, device and equipment for a machine learning program, the method comprising: obtaining a training data set, the training data set comprising a first type of training data, the confidence of the first type of training data being within a specified confidence range; the confidence being the confidence that the target in the first type of training data identified by the machine learning program belongs to a specified label; adding annotation information to the first type of training data according to the specified label; training a new machine learning program according to the feature information and annotation information of the first type of training data in the training data set, the new machine learning program being used to identify the confidence that the target in the input data belongs to the specified label. Through the technical solution of the present application, the performance of the machine learning program is continuously improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of machine learning, and in particular to a training method, apparatus and device for a machine learning program. Background Art

[0002] Machine learning is a way to achieve artificial intelligence. It is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Machine learning is used to study how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning pays more attention to algorithm design, enabling computers to automatically learn patterns from data and use the patterns to predict unknown data.

[0003] Machine learning has been widely used, such as deep learning, data mining, computer vision, natural language processing, biometric recognition, search engines, medical diagnosis, detection of credit card fraud, securities market analysis, DNA sequencing, speech and handwriting recognition, strategic games and robotics applications, etc.

[0004] A machine learning program is a program trained based on machine learning technology. During the training process of a machine learning program, it is necessary to manually collect data sets and manually calibrate the data in the data sets.

[0005] However, since the training process requires a large amount of data, the workload of the data set collection process is large and time-consuming. In addition, the workload of the data calibration process is large and time-consuming. Summary of the invention

[0006] In view of this, the present application provides a training method for a machine learning program, the method comprising:

[0007] Acquire a training data set, the training data set including first-category training data, the confidence of the first-category training data being within a specified confidence range; wherein the confidence is the confidence that a target in the first-category training data identified by a machine learning program belongs to a specified label;

[0008] Adding annotation information to the first type of training data according to the specified label;

[0009] A new machine learning program is trained based on the feature information and annotation information of the first type of training data in the training data set, and the new machine learning program is used to identify the confidence that the target in the input data belongs to the specified label.

[0010] The present application provides a training method for a machine learning program, the method comprising:

[0011] Using the deployed machine learning program to identify the confidence that the object in the input data belongs to the specified label;

[0012] If the confidence level is within a specified confidence level range, the input data is determined as first-category training data, and the first-category training data is added to a training data set;

[0013] Among them, the first type of training data in the training data set is used to train a new machine learning program.

[0014] The present application provides a training device for a machine learning program, the device comprising:

[0015] An acquisition module, configured to acquire a training data set, wherein the training data set includes first-category training data, and the confidence of the first-category training data is within a specified confidence range; wherein the confidence is the confidence that the target in the first-category training data identified by a machine learning program belongs to a specified label;

[0016] An adding module, used to add annotation information to the first type of training data according to the specified label;

[0017] A training module is used to train a new machine learning program based on the feature information and annotation information of the first type of training data in the training data set, and the new machine learning program is used to identify the confidence that the target in the input data belongs to the specified label.

[0018] The present application provides a training device for a machine learning program, the device comprising:

[0019] A recognition module for using the deployed machine learning program to identify the confidence that the object in the input data belongs to the specified label;

[0020] a processing module, configured to determine the input data as first-category training data if the confidence level is within a specified confidence level range, and add the first-category training data to a training data set;

[0021] Among them, the first type of training data in the training data set is used to train a new machine learning program.

[0022] The present application provides a management device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the following steps:

[0023] Acquire a training data set, the training data set including first-category training data, the confidence of the first-category training data being within a specified confidence range; wherein the confidence is the confidence that a target in the first-category training data identified by a machine learning program belongs to a specified label;

[0024] Adding annotation information to the first type of training data according to the specified label;

[0025] A new machine learning program is trained based on the feature information and annotation information of the first type of training data in the training data set, and the new machine learning program is used to identify the confidence that the target in the input data belongs to the specified label.

[0026] The present application provides an intelligent device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the following steps:

[0027] Using the deployed machine learning program to identify the confidence that the object in the input data belongs to the specified label;

[0028] If the confidence level is within a specified confidence level range, the input data is determined as first-category training data, and the first-category training data is added to a training data set;

[0029] Among them, the first type of training data in the training data set is used to train a new machine learning program.

[0030] It can be seen from the above technical solutions that in the embodiment of the present application, since the confidence of the first type of training data in the training data set is within the specified confidence range, it is possible to add annotation information to the first type of training data according to the specified label, thereby reducing the user's manual calibration operation on the data, reducing the user's workload, and saving the user's manual calibration time. Since the first type of training data is obtained using a machine learning program, it reduces the user's manual data collection operations, reduces the user's workload, and saves the user's manual data collection time. Moreover, the machine learning program can be continuously iterated and updated to continuously improve the performance of the machine learning program, so that the performance of the machine learning program is better and the machine learning results of the machine learning program are more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0032] Figure 1 is a flow chart of a training method for a machine learning program in one embodiment of the present application;

[0033] Figure 2 is a flow chart of a training method for a machine learning program in another embodiment of the present application;

[0034] Figure 3 This is a schematic diagram of an application scenario in one embodiment of the present application;

[0035] Figure 4 is a flow chart of a training method for a machine learning program in another embodiment of the present application;

[0036] Figure 5 This is a schematic diagram of an application scenario in one embodiment of the present application;

[0037] Figure 6 is a flow chart of a training method for a machine learning program in another embodiment of the present application;

[0038] Fig. 7A is a structural diagram of a training device for a machine learning program in one embodiment of the present application;

[0039] Figure 7B is a structural diagram of a management device in one embodiment of the present application;

[0040] Fig. 8A is a structural diagram of a training device for a machine learning program in one embodiment of the present application;

[0041] Figure 8B It is a structural diagram of a smart device in one implementation of the present application. DETAILED DESCRIPTION

[0042] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than limiting the present application. The singular forms of "a", "said" and "the" used in the present application and claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.

[0043] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, in addition, the word "if" used may be interpreted as "at..." or "when..." or "in response to determination".

[0044] Machine learning is a way to achieve artificial intelligence. It is used to study how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Deep learning is a subcategory of machine learning, and neural networks are a way to implement deep learning. For the convenience of description, neural networks are used as an example to briefly introduce the structure and function of neural networks. For other subcategories of machine learning, the structure and function are similar to those of neural networks and will not be repeated here.

[0045] Exemplarily, the neural network may include but is not limited to: convolutional neural network (CNN), recurrent neural network (RNN), fully connected network, etc. The structural units of the neural network may include but are not limited to: convolutional layer (Conv), pooling layer (Pool), excitation layer, fully connected layer (FC), etc., without limitation.

[0046] In the convolution layer, the data is convolved by using a convolution kernel to enhance the data features. The convolution layer uses a convolution kernel to perform convolution operations within a spatial range. The convolution kernel can be a matrix of size m*n. The input of the convolution layer is convolved with the convolution kernel to obtain the output of the convolution layer. The convolution operation is actually a filtering process. In the convolution operation, the pixel value f(x, y) of the point (x, y) on the data is convolved with the convolution kernel w(x, y). For example, a 4*4 convolution kernel is provided. The 4*4 convolution kernel contains 16 values, and the size of these 16 values ​​can be configured as needed. Slide on the data in sequence according to the size of 4*4 to obtain multiple 4*4 sliding windows. The 4*4 convolution kernel is convolved with each sliding window to obtain multiple convolution features. These convolution features are the output of the convolution layer and are provided to the pooling layer.

[0047] In the pooling layer, it is actually a downsampling process. By taking the maximum value, minimum value, and average value of multiple convolutional features (i.e., the output of the convolutional layer), the amount of calculation can be reduced and the feature invariance can be maintained. In the pooling layer, the principle of local correlation of data can be used to subsample the data, thereby reducing the amount of data processing and retaining useful information in the data.

[0048] In the excitation layer after the pooling layer, an activation function (such as a nonlinear function) can be used to map the features output by the pooling layer, thereby introducing nonlinear factors, so that the neural network can enhance its expression ability through nonlinear combination. Among them, the activation function of the excitation layer can include but is not limited to the ReLU (Rectified Linear Units) function. Taking the ReLU function as an example, the ReLU function can set all the features of the output of the pooling layer that are less than 0 to 0, while the features that are greater than 0 remain unchanged.

[0049] In the fully connected layer, the fully connected layer is used to fully connect all the features input to the fully connected layer to obtain a feature vector, and the feature vector can include multiple features. Furthermore, the fully connected layer can also use a 1*1 convolution layer to form a fully convolutional network.

[0050] In practical applications, one or more convolutional layers, one or more pooling layers, one or more excitation layers, and one or more fully connected layers can be combined to construct a neural network according to different needs.

[0051] Of course, the above is just an example of a neural network, and there is no limitation on the structure of the neural network.

[0052] In order to use neural networks to achieve target recognition (such as face detection, human body detection, vehicle detection, and there is no restriction on this target recognition scenario), the training process of the neural network and the use process of the neural network are involved.

[0053] In the training process of the neural network, the training data can be used to train the neural network parameters in the neural network, such as convolution layer parameters (such as convolution kernel parameters), pooling layer parameters, excitation layer parameters, full connection layer parameters, etc. There is no restriction on this, and all neural network parameters in the neural network can be trained. By training the neural network parameters in the neural network, the neural network can fit the mapping relationship between input and output.

[0054] In the process of using a neural network, input data can be provided to the neural network, and the neural network processes the input data, such as using all neural network parameters to process the input data to obtain output data, and the input data and the output data satisfy the input and output mapping relationship fitted by the neural network. Finally, the neural network is used to achieve target recognition, such as face detection, human body detection, vehicle detection, etc.

[0055] In the above process, the structure and function of the neural network are used as an example to introduce the structure and function of machine learning. That is to say, in the training process of machine learning, the training data is used to train the various machine learning parameters in the machine learning, and finally the machine learning program is obtained. In the use of machine learning, the input data is provided to the machine learning program, and the machine learning program processes the input data, such as using all machine learning parameters to process the input data to obtain output data, and finally realize target recognition.

[0056] Exemplarily, the machine learning program can be a machine learning model, a machine learning algorithm, or a combination of a machine learning model and a machine learning algorithm. There is no limitation on this machine learning program as long as it can implement the machine learning function. This article takes the machine learning program as an example for illustration.

[0057] For example, the training data can be used to train various machine learning parameters in the machine learning program to obtain a machine learning program (such as a machine learning type APP), and the machine learning program can be used to achieve target recognition. For example, the input data is input into the machine learning program, and the machine learning program performs target recognition on the input data, for example, the machine learning program realizes face detection, human body detection, vehicle detection, etc.

[0058] In the related art, in order to train various machine learning parameters in a machine learning program, it is necessary to manually collect data sets and manually calibrate the data in the data sets. The data set collection process is labor-intensive and time-consuming. Moreover, the data calibration process is labor-intensive and time-consuming.

[0059] In view of the above findings, a training method for a machine learning program is proposed in the embodiments of the present application, which can reduce the user's manual calibration operation on data and reduce the user's workload. It can reduce the user's manual data collection operation and reduce the user's workload. Moreover, the machine learning program can be continuously iterated and updated, so that the performance of the machine learning program is better and the machine learning results of the machine learning program are more accurate.

[0060] The technical solution of the embodiments of the present application is described below in conjunction with specific embodiments.

[0061] See also Figure 1 FIG. 1 is a flow chart of a training method for a machine learning program. The method can be applied to any type of device without limitation. The method may include:

[0062] Step 101, obtaining a training data set, the training data set including a first type of training data, the confidence of the first type of training data is within a specified confidence range, and illustratively, the confidence is the confidence that the target in the first type of training data identified by a machine learning program belongs to a specified label.

[0063] Exemplarily, the training data set may include multiple training data. For the convenience of distinction, the training data in the training data set may be divided into first-category training data and second-category training data. The first-category training data is training data whose confidence is within a specified confidence range. The training data set may also include second-category training data, which is training data to which annotation information has been added by the user. Of course, the training data set may also include other types of training data, which is not limited thereto.

[0064] Exemplarily, for input data, the confidence that the target in the input data belongs to a specified label can be identified by using a deployed machine learning program. If the confidence is within a specified confidence range, the input data can be determined as first-class training data, and the first-class training data is added to a training data set, and the training data in the training data set is used to train a new machine learning program. If the confidence is not within the specified confidence range, the input data may not be first-class training data.

[0065] Exemplarily, obtaining a training data set may include, but is not limited to: obtaining a first category of training data and a program identifier of a machine learning program; determining a data set identifier corresponding to the program identifier; and then, adding the first category of training data to the training data set corresponding to the data set identifier.

[0066] In this embodiment, it is necessary to train the machine learning program using the training data set, and use the machine learning program to identify the target of the input image. Based on this, the functional module for training the machine learning program using the training data set can be recorded as module A, and the functional module for using the machine learning program to identify the target of the input image can be recorded as module B. Module A and module B can be deployed on any device.

[0067] For example, module A and module B can be deployed on the same device as two functional modules of the device. Alternatively, module A and module B can be deployed on different devices as two independent devices. For the convenience of description, the following description takes module A and module B deployed on different devices as an example.

[0068] Exemplarily, the device where module A is located is called a management device, that is, the management device is used to train the machine learning program using a training data set. The management device can be a PC (Personal Computer), a server, a training platform (such as a machine learning training platform, a deep learning training platform, etc.), an application platform, a terminal device, a laptop computer, etc. There is no restriction on the type of this management device.

[0069] Exemplarily, the device where module B is located is called a smart device, that is, the smart device is used to use a machine learning program to perform target recognition on the input image. The smart device can be a smart server at the center, or an IPC (IPCamera) at the edge, or an NVR (Network Video Recorder) device at the edge, etc. There is no restriction on the type of this smart device.

[0070] Of course, the implementation method of the above-mentioned management device and intelligent device is only an example of the present application and is not limited thereto, as long as the functions of module A and module B can be realized.

[0071] In one possible implementation, the smart device can deploy a machine learning program, and the machine learning program is continuously updated. That is, the management device continuously generates a new machine learning program, the smart device obtains the new machine learning program, and deploys the new machine learning program. For example, after the management device generates machine learning program A1, the smart device obtains machine learning program A1 and uses machine learning program A1 to identify the target. After the management device generates machine learning program A2, the smart device obtains machine learning program A2, uses machine learning program A2 to replace machine learning program A1, uses machine learning program A2 to identify the target, and so on.

[0072] The smart device can use the deployed machine learning program to identify the confidence that the target in the input data belongs to the specified label. If the confidence is within the specified confidence range, the smart device can determine the input data as the first type of training data and send the first type of training data to the management device. If the confidence is not within the specified confidence range, the input data is not the first type of training data.

[0073] For example, the smart device can obtain input data 1 and provide the input data 1 to the machine learning program. Assuming that the machine learning program is used to implement face detection, the machine learning program can select a target area from the input data 1, and the target area is the target in the input data 1. Then, the machine learning program determines the confidence that the target area belongs to a face (when the machine learning program is used to implement face detection, the specified label can be a face). If the confidence is 80%, it means that the probability that the target area belongs to a face is relatively high, and when the confidence is 20%, it means that the probability that the target area belongs to a face is relatively low.

[0074] For another example, assuming that a machine learning program is used to implement data classification (such as distinguishing data containing human bodies from data containing animals), the machine learning program can select a target area from input data 1, and this target area is the target in input data 1. Then, the machine learning program determines the confidence that the target area contains animals (when the machine learning program is used to implement data classification, the specified label can be "contains animals" or "contains humans"). If the confidence is 80%, it means that the probability that the target area contains animals is relatively high, and if the confidence is 20%, it means that the probability that the target area contains animals is relatively low.

[0075] Of course, the above is just an example of "using a machine learning program to identify the confidence that a target in the input data belongs to a specified label", and there is no limitation to this. The specified label is related to the type of machine learning program.

[0076] Exemplarily, the specified confidence range can be configured based on experience, such as the specified confidence range is [a%, b%], the value of a can be greater than 50, the value of b can be less than 100, and b is greater than a. Of course, a greater than 50 and b less than 100 are just examples and are not limited to this. For example, the specified confidence range can be [70%, 95%].

[0077] For example, when the confidence level of identifying that the target in input data 1 belongs to the specified label is 100%, it means that the target in input data 1 definitely belongs to the specified label, and the machine learning program has the best recognition effect on input data 1, and it is no longer necessary to use input data 1 for training, that is, the value of b can be less than 100. Of course, for other confidence levels such as 96%-99%, it can also mean that the machine learning program has a better recognition effect on input data 1, so the value of b can be 95. Of course, the above is just an example.

[0078] For example, when the confidence level of identifying that the target in input data 1 belongs to the specified label is less than 50%, it means that the probability that the target in input data 1 belongs to the specified label is relatively small, less than 50%, that is, input data 1 will not improve the performance of the machine learning program, and it is no longer necessary to use input data 1 for training. Therefore, the value of b can be greater than 50, such as taking the value of b as 70. Of course, the above is just an example.

[0079] In summary, the specified confidence range can be pre-configured as [70%, 95%]. After the smart device uses the machine learning program to identify the confidence that the target in the input data 1 belongs to the specified label, if the confidence is in [70%, 95%], the input data 1 is used as the first type of training data and the first type of training data is sent to the management device. If the confidence is not in [70%, 95%], the input data 1 is not the first type of training data.

[0080] After receiving the first type of training data (such as input data 1) sent by the smart device, the management device adds the first type of training data to the training data set, that is, the management device can continuously update the training data set.

[0081] In a possible implementation, the smart device may send the first type of training data and the program identifier of the machine learning program to the management device. After the management device obtains the first type of training data and the program identifier of the machine learning program, it may determine the data set identifier corresponding to the program identifier, and add the first type of training data to the training data set corresponding to the data set identifier. Exemplarily, the training data in the training data set is used to train the machine learning program corresponding to the program identifier.

[0082] For example, the management device is used to train multiple types of machine learning programs, and different machine learning programs are trained using different training data sets, such as using the training data in training data set A to train machine learning program A, using the training data in training data set B to train machine learning program B, using the training data in training data set C to train machine learning program C, and so on.

[0083] Assuming that the smart device uses machine learning program A to perform target recognition on input data, the smart device sends the first type of training data and the program identifier of machine learning program A to the management device. The management device determines that the data set identifier corresponding to the program identifier is the data set identifier of training data set A, adds the first type of training data to training data set A, and trains machine learning program A using training data set A.

[0084] Exemplarily, the management device can maintain a correspondence between a program identifier and a data set identifier. Based on the correspondence, after the management device receives the first type of training data and the program identifier of the machine learning program, it can determine the data set identifier corresponding to the program identifier. There is no restriction on this process.

[0085] Step 102: adding annotation information to the first type of training data according to the specified label.

[0086] Exemplarily, since the confidence of the first type of training data is within the specified confidence range, the confidence is within the specified confidence range (such as [70%, 95%]), indicating that the probability that the target in the first type of training data belongs to the specified label is relatively high, therefore, the management device adds annotation information to the first type of training data according to the specified label, and the annotation information indicates that the target in the first type of training data belongs to the specified label, such as the annotation information is 1 or 0. Of course, the above is only an example of adding annotation information to the first type of training data, and there is no limitation to this.

[0087] Step 103: train a new machine learning program based on the feature information and annotation information of the training data in the training data set (such as the first type of training data and the second type of training data). Exemplarily, the new machine learning program is used to identify the confidence that the target in the input data belongs to the specified label.

[0088] Exemplarily, the management device can train a new machine learning program based on the feature information and annotation information of each training data in the training data set, such as training each machine learning parameter in the machine learning program, thereby obtaining a new machine learning program. Among them, through each machine learning parameter in the new machine learning program, a mapping relationship between input and output can be fitted, and there is no restriction on this training process.

[0089] Exemplarily, after the management device trains a new machine learning program, the smart device can obtain the new machine learning program. For example, the management device sends the new machine learning program to the smart device, or the smart device reads the new machine learning program from the management device, or the management device stores the new machine learning program to a certain address, and the smart device downloads the new machine learning program from the address, without limitation. After the smart device obtains the new machine learning program, it updates the new machine learning program to the deployed machine learning program, and identifies the confidence that the target in the input data belongs to the specified label based on the deployed machine learning program.

[0090] Obviously, the management device can continuously generate new machine learning programs, and the smart device can obtain the new machine learning program and use the new machine learning program to update the local machine learning program.

[0091] In a possible implementation, if the amount of unused training data in the training data set reaches a preset threshold (which can be configured based on experience and is not limited to this), a new machine learning program can be trained based on the feature information and annotation information of the training data in the training data set. Alternatively, if the interval between the current time and the time when the machine learning program was last trained reaches a preset duration (which can be configured based on experience and is not limited to this), a new machine learning program can be trained based on the feature information and annotation information of the training data in the training data set. Of course, the above trigger conditions are only two examples of training a new machine learning program, and there is no limitation on the trigger conditions, which can be configured according to actual needs.

[0092] In a possible implementation, when it is necessary to train a new machine learning program based on the feature information and annotation information of the training data in the training data set, GPU (Graphics Processing Unit) resources can also be applied for the training task of the machine learning program. Based on the applied GPU resources, the new machine learning program can be trained based on the feature information and annotation information of the training data in the training data set. After the new machine learning program is trained, the GPU resources can also be released.

[0093] Exemplarily, when a machine learning program needs to be trained, a training task for the machine learning program can be created, and GPU resources can be applied for the training task for the machine learning program. If there are currently available GPU resources, such as a specified number of idle GPUs, the training task for the machine learning program is provided to these GPUs, and these GPUs train the new machine learning program based on the feature information and annotation information of the training data in the training data set. After the new machine learning program is trained, the GPU resources can also be released.

[0094] For example, if there are currently no available GPU resources, the training tasks of the machine learning program can be stored in the task queue. Until there are available GPU resources, the training tasks of the machine learning program are selected from the task queue and provided to the GPU.

[0095] Of course, the above-mentioned method of using GPU resources to train new machine learning programs is only an example of this application. The management device can also use other computing resources (such as CPU resources, etc.) to train new machine learning programs.

[0096] In a possible implementation, the machine learning program may include but is not limited to: a machine learning classification program; or a machine learning detection program; or a machine learning classification detection hybrid program. Of course, the above are just a few examples of machine learning programs, and the type of machine learning program is not limited.

[0097] Exemplarily, a machine learning classification program may refer to: based on the machine learning classification program, it is possible to distinguish the type of data, that is, it is used to implement data classification, for example, to distinguish between data containing human bodies and data containing animals. A machine learning detection program may refer to: based on the machine learning detection program, it is possible to detect the target area of ​​the data, that is, it is used to implement data detection, for example, to detect whether the target area of ​​the data is a human face. A machine learning classification and detection hybrid program may refer to: based on the machine learning classification and detection hybrid program, it is possible to distinguish the type of data and detect the target area of ​​the data, that is, it is used to implement data classification and data detection. For example, first distinguish between data containing human bodies and data containing animals, and then detect whether the target area of ​​the data is a human face for the data containing human bodies. Of course, the above are only examples of machine learning classification programs, machine learning detection programs, and machine learning classification and detection hybrid programs, and there is no limitation to this.

[0098] In a possible implementation manner, the data in the above embodiments may be image data, voice data, or text data, and the type of the data is not limited.

[0099] It can be seen from the above technical solutions that in the embodiment of the present application, since the confidence of the first type of training data in the training data set is within the specified confidence range, it is possible to add annotation information to the first type of training data according to the specified label, thereby reducing the user's manual calibration operation on the data, reducing the user's workload, and saving the user's manual calibration time. Since the first type of training data is obtained using a machine learning program, it reduces the user's manual data collection operations, reduces the user's workload, and saves the user's manual data collection time. Moreover, the machine learning program can be continuously iterated and updated to continuously improve the performance of the machine learning program, so that the performance of the machine learning program is better and the machine learning results of the machine learning program are more accurate.

[0100] Based on the same concept as the above method, another training method for a machine learning program is also proposed in the embodiment of the present application, see Figure 2 As shown, the method may include:

[0101] Step 201, using the deployed machine learning program to identify the confidence that the target in the input data belongs to the specified label, the confidence is used to represent the probability that the target in the input data belongs to the specified label.

[0102] Step 202: If the confidence level is within a specified confidence level range, the input data is determined as first-category training data, and the first-category training data is added to the training data set.

[0103] Exemplarily, the training data in the training data set is used to train a new machine learning program.

[0104] In a possible implementation, a data set identifier corresponding to a program identifier of a machine learning program may be determined, and the first type of training data may be added to a training data set corresponding to the data set identifier.

[0105] Exemplarily, if the machine learning program is deployed on an intelligent device, the intelligent device executes step 201 and step 202. After the intelligent device determines the input data as the first type of training data, the first type of training data and the program identifier of the machine learning program are sent to the management device, so that the management device determines the data set identifier corresponding to the program identifier, and adds the first type of training data to the training data set corresponding to the data set identifier. The training data in the training data set is used to train the machine learning program corresponding to the program identifier.

[0106] For the specific implementation of step 201 - step 202 , please refer to the above embodiment and will not be described again here.

[0107] It can be seen from the above technical solutions that in the embodiment of the present application, since the confidence of the first type of training data in the training data set is within the specified confidence range, it is possible to add annotation information to the first type of training data according to the specified label, thereby reducing the user's manual calibration operation on the data, reducing the user's workload, and saving the user's manual calibration time. Since the first type of training data is obtained using a machine learning program, it reduces the user's manual data collection operations, reduces the user's workload, and saves the user's manual data collection time. Moreover, the machine learning program can be continuously iterated and updated to continuously improve the performance of the machine learning program, so that the performance of the machine learning program is better and the machine learning results of the machine learning program are more accurate.

[0108] The following describes the training process of the above machine learning program in combination with specific application scenarios.

[0109] See also Figure 3 As shown, it is a schematic diagram of the application scenario of an embodiment of the present application. The training platform and the application platform are used to realize the function of the management device. The training platform and the application platform can be software systems. The training platform and the application platform can be deployed on different devices respectively, or on the same device, without limitation. The training platform and the application platform can communicate through an API (Application Programming Interface). An intelligent device is a device that uses a machine learning program to realize target recognition, and can be an intelligent server at the center end, an IPC at the edge end, or an NVR device at the edge end, without limitation. Exemplarily, the management device and the intelligent device can be the same device or different devices.

[0110] The training platform supports functions such as data management, program management, program training, equipment management, data calibration, model verification, etc. The application platform supports functions such as program management, equipment management, and data management.

[0111] In one possible implementation, the training platform is used to train multiple types of machine learning programs, and different machine learning programs are trained using different training data sets. For example, machine learning program A is trained using training data set A, machine learning program B is trained using training data set B, and so on. In summary, the training platform can maintain training data set A and training data set B, and different training data sets correspond to different data set identifiers. For example, training data set A corresponds to data set identifier A1, and training data set B corresponds to data set identifier B1. For the convenience of description, in the subsequent embodiments, the example of training machine learning program A using training data set A is used for illustration.

[0112] In one possible implementation, the training platform supports the training functions of machine learning programs, such as supporting the training functions of machine learning classification programs, supporting the training functions of machine learning detection programs, and supporting the training functions of machine learning classification and detection hybrid programs. Of course, the above are only examples and there is no limitation to this.

[0113] In the above application scenario, see Figure 4 As shown, the training method of the machine learning program may include:

[0114] Step 401, the training platform trains the machine learning program A1 based on the training data set A.

[0115] Exemplarily, when the training platform trains the machine learning program A for the first time, the training data set A may include the second type of training data, where the second type of training data is training data to which the user has added annotation information. Based on the feature information and annotation information of the training data in the training data set A, the training platform may train the machine learning program A, and record the machine learning program A trained for the first time as machine learning program A1.

[0116] For example, the training platform supports a training data upload function (such as a batch upload or non-batch upload function), and the user can upload the training data to the training data set A. The training data uploaded by the user can be training data with or without annotation information.

[0117] The training data to which the annotation information has been added can be used as the second type of training data in the training process of the machine learning program A1. For the training data to which the annotation information has not been added, the training platform uses a certain method to add annotation information to the training data, and there is no restriction on the annotation method; for example, an interface is displayed to the user, and the user adds annotation information to the training data in the interface. After the annotation information is added to the training data, the training data is used as the second type of training data in the training process of the machine learning program A1.

[0118] To sum up, the training data set A may include the second type of training data to which the user has added annotation information, and the training platform can train the machine learning program A1 based on the feature information and annotation information of the second type of training data in the training data set A, without any restrictions on this training process.

[0119] Exemplarily, the user may trigger the training platform to train the machine learning program A1. For example, the training platform starts training the machine learning program A1 after receiving a start command input by the user.

[0120] Exemplarily, a trigger condition can be configured on the training platform, and when the trigger condition is met, the training platform starts training the machine learning program A1. For example, the trigger condition can be a preset threshold value of the number of training data (which can be configured based on experience and is not limited to this). If the number of untrained training data in the training data set A reaches the preset threshold value, the training platform starts training the machine learning program A1. Of course, the above trigger condition is only an example of training the machine learning program A1, and there is no limitation on this trigger condition.

[0121] Exemplarily, the training platform can provide multiple GPUs, and the training of the machine learning program A1 is implemented through these GPUs. Of course, the training platform can also implement the training of the machine learning program A1 through other types of computing resources, and there is no limitation to this. The training process implemented through GPU is taken as an example below.

[0122] When the machine learning program A1 needs to be trained, the training platform can create a training task for the machine learning program A1 and apply for GPU resources, such as a specified number of GPUs, for the training task for the machine learning program A1.

[0123] If there are currently available GPU resources, such as a specified number of GPUs, the training platform provides the training tasks of the machine learning program A1 to these GPUs, and these GPUs train the machine learning program A1 according to the training data set A. After the machine learning program A1 is trained, the GPU resources can also be released.

[0124] If there are currently no available GPU resources, the training task of the machine learning program A1 can be stored in the task queue. Until there are available GPU resources, the training task of the machine learning program A1 will be selected from the task queue and provided to the GPU.

[0125] In summary, when it is necessary to train the machine learning program A1, the training platform supports the dynamic application and release of GPU resources, supports the task queuing mechanism, and improves the utilization rate of GPU resources.

[0126] Exemplarily, the GPU may be any type of GPU, without limitation.

[0127] Step 402: The application platform obtains the machine learning program A1 from the training platform.

[0128] Exemplarily, after obtaining the machine learning program A1, the training platform can send the machine learning program A1 to the application platform. Alternatively, after obtaining the machine learning program A1, the training platform can send a notification message to the application platform, which notification message indicates that the training platform has obtained the machine learning program A1. After the application platform receives the notification message, it obtains the machine learning program A1 from the training platform. Alternatively, the application platform can periodically query the training platform whether there is a new machine learning program. If so, it obtains the machine learning program A1 from the training platform. Of course, the above are just a few examples and there is no limitation to this.

[0129] Exemplarily, the training platform supports an API interface for program export, and the application platform obtains the machine learning program A1 from the training platform through the API interface. Of course, the implementation of the API interface is just an example and is not limited to this, as long as the application platform can obtain the machine learning program A1 from the training platform.

[0130] Exemplarily, the application platform may also establish a mapping relationship between the program identifier of the machine learning program A1 and the data set identifier A1 of the training data set A. For example, the mapping relationship between the program identifier of the machine learning program A1 and the data set identifier A1 is pre-configured on the application platform. Alternatively, when the application platform obtains the machine learning program A1 from the training platform, it may also obtain the data set identifier A1 from the training platform and establish a mapping relationship between the program identifier of the machine learning program A1 and the data set identifier A1.

[0131] Step 403: The application platform sends the machine learning program A1 to the smart device.

[0132] Exemplarily, the smart device supports an API interface for program import. After the application platform obtains the machine learning program A1, it can send the machine learning program A1 to the smart device (there can be multiple smart devices managed by the application platform) through the API interface. Of course, the implementation of the API interface is just an example and is not limited to this, as long as the application platform can send the machine learning program A1 to the smart device.

[0133] In step 404, the smart device deploys the machine learning program A1, and uses the machine learning program A1 to identify the confidence that the target in the input data belongs to the specified label, and the confidence is used to represent the probability that the target in the input data belongs to the specified label. For example, the smart device can provide the input data to the machine learning program A1, and the machine learning program A1 identifies the confidence that the target in the input data belongs to the specified label.

[0134] Step 405: If the confidence level is within the specified confidence level range, the smart device sends the input data (i.e., the input data is used as the first type of training data) and the program identifier of the machine learning program A1 to the application platform.

[0135] Exemplarily, the specified confidence range may be configured based on experience, such as specifying a confidence range of [a%, b%], where the value of a may be greater than 50, the value of b may be less than 100, and b is greater than a.

[0136] Exemplarily, if the confidence level is not within the specified confidence level range, the smart device does not need to send the input data to the application platform, that is, the input data is not used as the first type of training data.

[0137] Step 406, after receiving the first type of training data and the program identifier of the machine learning program A1 sent by the smart device, the application platform determines the data set identifier A1 corresponding to the program identifier, and adds the first type of training data to the training data set A corresponding to the data set identifier A1.

[0138] Exemplarily, since the application platform establishes a mapping relationship between the program identifier of the machine learning program A1 and the data set identifier A1 of the training data set A, based on the mapping relationship, the application platform can determine that the data set identifier A1 corresponds to the program identifier of the machine learning program A1.

[0139] Exemplarily, the training platform supports a model validation API interface, based on which the application platform can add the first type of training data to the training data set A through the model validation API interface.

[0140] Of course, the implementation method of the model validation API interface is just an example and there is no restriction on it, as long as the application platform can add the first type of training data to the training data set A.

[0141] In step 407, the training platform obtains a training data set A, which may include a first type of training data (i.e., training data whose confidence is within a specified confidence range and has not yet been annotated) and a second type of training data (training data to which annotated information has been added by the user).

[0142] Exemplarily, the training platform supports an API interface for importing uncalibrated training data, an API interface for importing calibrated training data, and a model verification API interface. The API interface for importing uncalibrated training data is used to import uncalibrated training data, that is, the user uploads the training data to the training data set A, and the training data is training data without adding annotation information, see step 401, the training platform can add annotation information to the training data, and the training data after adding the annotation information is used as the second type of training data. The API interface for importing calibrated training data is used to import calibrated training data, that is, the user uploads the training data to the training data set A, and the training data is training data with added annotation information, that is, the second type of training data. The model verification API interface is used to import the first type of training data, and the import process is shown in step 406.

[0143] Of course, the above API interfaces are only examples and are not limited thereto, as long as the first type of training data and the second type of training data can be added to the training data set A.

[0144] In step 408, the training platform adds annotation information to the first type of training data according to the specified label.

[0145] Exemplarily, since the confidence of the first category of training data is within a specified confidence range, such as [70%, 95%]), the confidence can indicate that the probability that the target in the first category of training data belongs to the specified label is relatively high. Therefore, the training platform can add annotation information to the first category of training data. The annotation information is used to indicate that the target in the first category of training data belongs to the specified label. For example, the annotation information can be 1 or 0.

[0146] Exemplarily, the training platform supports manual verification of the accuracy of the first type of training data. For example, for each first type of training data, the user verifies whether the confidence of the first type of training data is within a specified confidence range. If yes, the first type of training data is retained in the training data set A, and the training platform adds annotation information to the first type of training data. If no, the first type of training data is deleted from the training data set A.

[0147] Step 409: The training platform trains the machine learning program A2 according to the feature information and annotation information of the training data in the training data set A (such as the first category of training data and the second category of training data).

[0148] Exemplarily, when the training platform trains machine learning program A for the second time, the training data set A may include first-category training data and second-category training data. Based on the feature information and annotation information of the training data in the training data set A, the training platform can train machine learning program A. There is no restriction on this training method, and the machine learning program A trained for the second time is recorded as machine learning program A2.

[0149] Exemplarily, a trigger condition can be configured on the training platform, and when the trigger condition is met, the training platform starts training the machine learning program A1. For example, the trigger condition can be a preset threshold value of the number of training data (which can be configured based on experience and is not limited to this). If the number of untrained training data in the training data set A reaches the preset threshold value, the training platform starts training the machine learning program A2. Alternatively, the trigger condition can be a preset duration (which can be configured based on experience and is not limited to this). If the interval between the current time and the time when the machine learning program A was last trained reaches the preset duration, the training platform starts training the machine learning program A2. Of course, the above trigger conditions are only examples and are not limited to this.

[0150] Exemplarily, the training platform can train the machine learning program A2 through multiple GPUs. The specific process is shown in step 401, except that the training data in the training data set A changes, which will not be repeated here.

[0151] After the training platform trains the machine learning program A2, the application platform obtains the machine learning program A2 from the training platform and sends the machine learning program A2 to the smart device. The smart device deploys the machine learning program A2 (i.e., uses the machine learning program A2 to replace the locally used machine learning program A1), and uses the machine learning program A2 to identify the confidence that the target in the input data belongs to the specified label. If the confidence is within the specified confidence range, the smart device sends the input data (i.e., the first category of training data) to the application platform, and the application platform adds the first category of training data to the training data set A. The training platform trains the machine learning program A2 based on the feature information and annotation information of the training data in the training data set A.

[0152] By analogy, the training platform can continuously generate new machine learning program A, and the application platform can send the new machine learning program A to the smart device so that the smart device can use the machine learning program A to update the local machine learning program A, thereby achieving optimized training of the machine learning program A.

[0153] See also Figure 5 As shown, it is a schematic diagram of the application scenario of an embodiment of the present application, the management device is used to realize the functions of the training platform and the application platform, and the smart device is a device that uses a machine learning program to realize target recognition.

[0154] In one possible implementation, the management device is used to train multiple types of machine learning programs, and different machine learning programs are trained using different training data sets. In the subsequent embodiments, the management device is used to train machine learning program A using training data set A as an example.

[0155] In the above application scenario, see Figure 6 As shown, the training method of the machine learning program may include:

[0156] Step 601, the management device trains the machine learning program A1 based on the training data set A.

[0157] Exemplarily, the training data set A may include the second type of training data, which is training data to which annotation information has been added by the user. Based on the feature information and annotation information of the training data in the training data set A, the management device may train the machine learning program A1.

[0158] Step 602: The management device sends the machine learning program A1 to the smart device.

[0159] In step 603, the smart device deploys the machine learning program A1, and uses the machine learning program A1 to identify the confidence that the target in the input data belongs to the specified label, that is, the probability that the target belongs to the specified label.

[0160] Step 604: If the confidence level is within the specified confidence level range, the smart device sends the input data (i.e., the input data as the first type of training data) and the program identifier of the machine learning program A1 to the management device.

[0161] Step 605: After receiving the first type of training data and the program identifier of the machine learning program A1 sent by the smart device, the management device determines the data set identifier corresponding to the program identifier, and adds the first type of training data to the training data set corresponding to the data set identifier (such as training data set A).

[0162] Step 606: The management device obtains a training data set A, which may include a first type of training data (i.e., training data whose confidence is within a specified confidence range and has not yet been labeled) and a second type of training data (training data to which the user has added labeling information).

[0163] Step 607: The management device adds annotation information to the first type of training data according to the specified label.

[0164] Step 608: The management device trains the machine learning program A2 according to the feature information and annotation information of the training data in the training data set A (such as the first category of training data and the second category of training data).

[0165] In step 609, the management device sends the machine learning program A2 to the smart device, and the smart device uses the machine learning program A2 to identify the confidence that the target in the input data belongs to the specified label, and so on.

[0166] Illustratively, steps 601 to 609 may refer to steps 401 to 409 and are not described in detail here.

[0167] Based on the same application concept as the above method, the present application also proposes a training device for a machine learning program, such as Fig. 7A As shown, it is a structural diagram of the device, and the device includes:

[0168] An acquisition module 711 is used to acquire a training data set, wherein the training data set includes a first type of training data, and the confidence of the first type of training data is within a specified confidence range; wherein the confidence is the confidence that the target in the first type of training data identified by using a machine learning program belongs to a specified label; an adding module 712 is used to add annotation information to the first type of training data according to the specified label; and a training module 713 is used to train a new machine learning program according to the feature information and annotation information of the first type of training data in the training data set, wherein the new machine learning program is used to identify the confidence that the target in the input data belongs to the specified label.

[0169] When acquiring a training data set, the acquisition module 711 is specifically used to: acquire the first type of training data and the program identifier of the machine learning program; determine the data set identifier corresponding to the program identifier; and add the first type of training data to the training data set corresponding to the data set identifier.

[0170] The training module 713 is specifically used to train a new machine learning program based on the characteristic information and annotation information of the first category of training data in the training data set: if the number of untrained training data in the training data set reaches a preset threshold, the new machine learning program is trained based on the characteristic information and annotation information of the first category of training data in the training data set.

[0171] The training module 713 is specifically used to train a new machine learning program according to the feature information and annotation information of the first type of training data in the training data set:

[0172] Apply for GPU resources for the training task of the machine learning program; based on the GPU resources, train a new machine learning program according to the feature information and annotation information of the first type of training data in the training data set; after the new machine learning program is obtained through training, release the GPU resources.

[0173] Based on the same application concept as the above method, a management device is also proposed in the embodiment of the present application. From the hardware level, the hardware architecture diagram of the management device provided in the embodiment of the present application can be seen in Figure 7B The management device may include: a processor 721 and a machine-readable storage medium 722, wherein the machine-readable storage medium 722 stores machine-executable instructions that can be executed by the processor 721; the processor 721 is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0174] For example, the processor 721 is used to execute machine executable instructions to implement the following steps:

[0175] Acquire a training data set, the training data set including first-category training data, the confidence of the first-category training data being within a specified confidence range; wherein the confidence is the confidence that a target in the first-category training data identified by a machine learning program belongs to a specified label;

[0176] Adding labeling information to the first category of training data according to the designated label; training a new machine learning program according to the feature information and labeling information of the first category of training data in the training data set, the new machine learning program being used to identify the confidence that a target in the input data belongs to the designated label.

[0177] Based on the same application concept as the above method, the present application also proposes a training device for a machine learning program, such as Fig. 8A As shown is a structural diagram of the device, which includes: an identification module 811, used to use a deployed machine learning program to identify the confidence that a target in the input data belongs to a specified label; a processing module 812, used to determine the input data as a first type of training data if the confidence is within a specified confidence range, and add the first type of training data to a training data set; wherein the first type of training data in the training data set is used to train a new machine learning program.

[0178] When the processing module 812 adds the first type of training data to the training data set, it is specifically used to:

[0179] Determine a data set identifier corresponding to the program identifier of the machine learning program; and add the first type of training data to the training data set corresponding to the data set identifier.

[0180] Based on the same application concept as the above method, an intelligent device is also proposed in the embodiment of the present application. From the hardware level, the hardware architecture diagram of the intelligent device provided in the embodiment of the present application can be seen in Figure 8BThe smart device may include: a processor 821 and a machine-readable storage medium 822, wherein the machine-readable storage medium 822 stores machine-executable instructions that can be executed by the processor 821; the processor 821 is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0181] For example, the processor 821 is used to execute machine executable instructions to implement the following steps:

[0182] Using the deployed machine learning program to identify the confidence that the object in the input data belongs to the specified label;

[0183] If the confidence level is within a specified confidence level range, the input data is determined as first-category training data, and the first-category training data is added to a training data set;

[0184] Among them, the first type of training data in the training data set is used to train a new machine learning program.

[0185] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, wherein a number of computer instructions are stored on the machine-readable storage medium, and when the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0186] Exemplarily, the above-mentioned machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as CD, DVD, etc.), or similar storage medium, or a combination thereof.

[0187] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0188] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0189] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0190] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0191] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0193] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for training a machine learning program, characterized in that: The method comprises: Acquire a training data set, the training data set including first-category training image data, the confidence of the first-category training image data being within a specified confidence range; wherein the confidence is the confidence that the target in the first-category training image data identified by a machine learning program belongs to a specified label; wherein the specified confidence range is [a%, b%], if the confidence that the target in the image data belongs to the specified label is greater than b%, then the image data is not used as the first-category training image data, and does not participate in the training of the machine learning program; if the confidence that the target in the image data belongs to the specified label is less than a%, then the image data is not used as the first-category training image data, and does not participate in the training of the machine learning program; wherein a% is greater than 50%, and when the confidence is within the specified confidence range [a%, b%], it indicates that the probability that the target in the first-category training data belongs to the specified label is high; Adding annotation information to the first type of training image data according to the specified label; wherein the annotation information indicates that the target in the first type of training data belongs to the specified label; Training a new machine learning program according to the feature information and annotation information of the first type of training image data in the training data set, wherein the new machine learning program is used to identify the confidence that the target in the input image data belongs to the specified label; Among them, if the new machine learning program is a machine learning classification program, then the machine learning classification program is used to distinguish the types of targets in the input image data; or, if the new machine learning program is a machine learning detection program, then the machine learning detection program is used to realize the detection of target areas in the input image data; or, if the new machine learning program is a machine learning classification and detection hybrid program, then the machine learning classification and detection hybrid program is used to distinguish the types of targets in the input image data and realize the detection of target areas in the input image data.

2. The method according to claim 1, characterized in that The obtaining of the training data set comprises: Obtaining first type of training image data and a program identifier of the machine learning program; Determine a data set identifier corresponding to the program identifier; The first category of training image data is added to the training data set corresponding to the data set identifier.

3. The method according to claim 1, characterized in that The step of training a new machine learning program according to the feature information and annotation information of the first type of training image data in the training data set includes: If the amount of unused training data in the training data set reaches a preset threshold, a new machine learning program is trained according to the feature information and annotation information of the first type of training image data in the training data set.

4. The method according to claim 1, characterized in that: The step of training a new machine learning program according to the feature information and annotation information of the first type of training image data in the training data set includes: Apply for GPU resources for the training task of the machine learning program; based on the GPU resources, train a new machine learning program according to the feature information and annotation information of the first type of training image data in the training data set; after the new machine learning program is obtained through training, release the GPU resources.

5. A method for training a machine learning program, characterized in that: The method comprises: Using the deployed machine learning program to identify the confidence that the object in the input image data belongs to the specified label; If the confidence level is within a specified confidence level range, the input image data is determined as first-category training image data, and the first-category training image data is added to a training data set; Wherein, the first type of training image data in the training data set is used to train a new machine learning program; wherein, the specified confidence range is [a%, b%], if the confidence that the target in the image data belongs to the specified label is greater than b%, then the image data is not used as the first type of training image data and does not participate in the training of the machine learning program; if the confidence that the target in the image data belongs to the specified label is less than a%, then the image data is not used as the first type of training image data and does not participate in the training of the machine learning program; wherein, a% is greater than 50%, and when the confidence is within the specified confidence range [a%, b%], it indicates that the probability that the target in the first type of training data belongs to the specified label is high; wherein, the annotation information of the first type of training image data indicates that the target in the first type of training data belongs to the specified label; Among them, if the new machine learning program is a machine learning classification program, then the machine learning classification program is used to distinguish the types of targets in the input image data; or, if the new machine learning program is a machine learning detection program, then the machine learning detection program is used to realize the detection of target areas in the input image data; or, if the new machine learning program is a machine learning classification and detection hybrid program, then the machine learning classification and detection hybrid program is used to distinguish the types of targets in the input image data and realize the detection of target areas in the input image data.

6. The method according to claim 5, characterized in that The step of adding the first type of training image data to a training data set comprises: Determine a data set identifier corresponding to a program identifier of the machine learning program; The first category of training image data is added to the training data set corresponding to the data set identifier.

7. A training device for a machine learning program, characterized in that: The device comprises: An acquisition module is used to acquire a training data set, wherein the training data set includes a first type of training image data, and the confidence of the first type of training image data is within a specified confidence range; wherein the confidence is the confidence that the target in the first type of training image data identified by the machine learning program belongs to the specified label; wherein the specified confidence range is [a%, b%], and if the confidence that the target in the image data belongs to the specified label is greater than b%, then the image data is not used as the first type of training image data, and does not participate in the training of the machine learning program; if the confidence that the target in the image data belongs to the specified label is less than a%, then the image data is not used as the first type of training image data, and does not participate in the training of the machine learning program; wherein a% is greater than 50%, and when the confidence is within the specified confidence range [a%, b%], it indicates that the probability that the target in the first type of training data belongs to the specified label is high; An adding module, configured to add annotation information to the first type of training image data according to the specified label; wherein the annotation information indicates that the target in the first type of training data belongs to the specified label; A training module, used for training a new machine learning program according to the feature information and annotation information of the first type of training image data in the training data set, the new machine learning program being used for identifying the confidence that the target in the input image data belongs to the specified label; Among them, if the new machine learning program is a machine learning classification program, then the machine learning classification program is used to distinguish the types of targets in the input image data; or, if the new machine learning program is a machine learning detection program, then the machine learning detection program is used to realize the detection of target areas in the input image data; or, if the new machine learning program is a machine learning classification and detection hybrid program, then the machine learning classification and detection hybrid program is used to distinguish the types of targets in the input image data and realize the detection of target areas in the input image data.

8. A training device for a machine learning program, characterized in that: The device comprises: A recognition module for using the deployed machine learning program to identify the confidence that the object in the input image data belongs to the specified label; a processing module, configured to determine the input image data as first-category training image data if the confidence level is within a specified confidence level range, and add the first-category training image data to a training data set; Wherein, the first type of training image data in the training data set is used to train a new machine learning program; wherein, the specified confidence range is [a%, b%], if the confidence that the target in the image data belongs to the specified label is greater than b%, then the image data is not used as the first type of training image data and does not participate in the training of the machine learning program; if the confidence that the target in the image data belongs to the specified label is less than a%, then the image data is not used as the first type of training image data and does not participate in the training of the machine learning program; wherein, a% is greater than 50%, and when the confidence is within the specified confidence range [a%, b%], it indicates that the probability that the target in the first type of training data belongs to the specified label is high; wherein, the annotation information of the first type of training image data indicates that the target in the first type of training data belongs to the specified label; Among them, if the new machine learning program is a machine learning classification program, then the machine learning classification program is used to distinguish the types of targets in the input image data; or, if the new machine learning program is a machine learning detection program, then the machine learning detection program is used to realize the detection of target areas in the input image data; or, if the new machine learning program is a machine learning classification and detection hybrid program, then the machine learning classification and detection hybrid program is used to distinguish the types of targets in the input image data and realize the detection of target areas in the input image data.

9. A management device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the following steps: Acquire a training data set, the training data set including first-category training image data, the confidence of the first-category training image data being within a specified confidence range; wherein the confidence is the confidence that the target in the first-category training image data identified by a machine learning program belongs to a specified label; wherein the specified confidence range is [a%, b%], if the confidence that the target in the image data belongs to the specified label is greater than b%, then the image data is not used as the first-category training image data, and does not participate in the training of the machine learning program; if the confidence that the target in the image data belongs to the specified label is less than a%, then the image data is not used as the first-category training image data, and does not participate in the training of the machine learning program; wherein a% is greater than 50%, and when the confidence is within the specified confidence range [a%, b%], it indicates that the probability that the target in the first-category training data belongs to the specified label is high; Adding annotation information to the first type of training image data according to the specified label; wherein the annotation information indicates that the target in the first type of training data belongs to the specified label; Training a new machine learning program according to the feature information and annotation information of the first type of training image data in the training data set, the new machine learning program being used to identify the confidence that the target in the input image data belongs to the specified label; Among them, if the new machine learning program is a machine learning classification program, then the machine learning classification program is used to distinguish the types of targets in the input image data; or, if the new machine learning program is a machine learning detection program, then the machine learning detection program is used to realize the detection of target areas in the input image data; or, if the new machine learning program is a machine learning classification and detection hybrid program, then the machine learning classification and detection hybrid program is used to distinguish the types of targets in the input image data and realize the detection of target areas in the input image data.

10. A smart device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the following steps: Using the deployed machine learning program to identify the confidence that the object in the input image data belongs to the specified label; If the confidence level is within a specified confidence level range, the input image data is determined as first-category training image data, and the first-category training image data is added to a training data set; Wherein, the first type of training image data in the training data set is used to train a new machine learning program; wherein, the specified confidence range is [a%, b%], if the confidence that the target in the image data belongs to the specified label is greater than b%, then the image data is not used as the first type of training image data and does not participate in the training of the machine learning program; if the confidence that the target in the image data belongs to the specified label is less than a%, then the image data is not used as the first type of training image data and does not participate in the training of the machine learning program; wherein, a% is greater than 50%, and when the confidence is within the specified confidence range [a%, b%], it indicates that the probability that the target in the first type of training data belongs to the specified label is high; wherein, the annotation information of the first type of training image data indicates that the target in the first type of training data belongs to the specified label; Among them, if the new machine learning program is a machine learning classification program, then the machine learning classification program is used to distinguish the types of targets in the input image data; or, if the new machine learning program is a machine learning detection program, then the machine learning detection program is used to realize the detection of target areas in the input image data; or, if the new machine learning program is a machine learning classification and detection hybrid program, then the machine learning classification and detection hybrid program is used to distinguish the types of targets in the input image data and realize the detection of target areas in the input image data.