Model training method and device

Abstract

An embodiment of the present application provides a model training method and device, the method comprising: obtaining a basic visual model and a first sample data set; annotating information on each sample data in the first sample data set through a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is annotated with information of multiple first detection objects; training the basic visual model based on the first sample data set, the second sample data set and multiple graphics processors to obtain a trained first multi-target visual model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method and device. Background Art

[0002] With the rapid development of large models, the complexity of the models and the demand for computing resources have increased significantly. However, in actual application scenarios, the direct application of large models faces challenges due to the limitations of hardware resources and real-time requirements. Existing technologies usually simplify the models through model compression or pruning techniques, but these methods often lose some information when dealing with multiple target detections, resulting in performance degradation.

[0003] In order to improve performance, the existing visual model only processes a single target task, and the processing efficiency of the target task is low. In addition, in the process of training the visual model, the model is trained through a graphics processing unit (GPU) technology, which has low processing efficiency and results in a long model training time. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application provide a model training method and device that overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect, an embodiment of the present application provides a model training method, the method comprising:

[0006] Acquire a basic visual model and a first sample data set, where sample data in the first sample data set is image data;

[0007] Annotating each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is annotated with information of multiple first detection objects;

[0008] The basic visual model is trained based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model.

[0009] Optionally, the step of annotating information of each sample data in the first sample data set by using a large visual model to obtain a second sample data set includes:

[0010] Acquire multiple first detection objects;

[0011] Based on the large visual model and the multiple first detection objects, information is annotated for each sample data in the first sample data set to obtain a second sample data set.

[0012] Optionally, the step of annotating information of each sample data in the first sample data set based on the large visual model and the plurality of first detection objects to obtain a second sample data set includes:

[0013] Inputting the first sample data set into a large visual model, and annotating information on each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set, wherein each sample data in the third sample data set is annotated with information of the multiple first detection objects;

[0014] The information of the first detection object marked in the third sample data set is updated to obtain a second sample data set.

[0015] Optionally, the training of the basic visual model based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model includes:

[0016] Inputting the first sample data set into the basic visual model, and performing parallel calculations through multiple graphics processors to obtain a first labeling result;

[0017] Calculating a loss value of the basic visual model according to the first annotation result and the second sample data set;

[0018] The basic visual model is trained based on the loss value to obtain a trained first multi-objective visual model.

[0019] Optionally, after the basic visual model is trained based on the first sample data set, the second sample data set and a plurality of graphics processors to obtain a trained first multi-target visual model, the method further includes:

[0020] Acquire at least one second detection object;

[0021] The first multi-object visual model is updated based on the at least one second detection object to obtain an updated second multi-object visual model.

[0022] In a second aspect, an embodiment of the present application further provides a model training device, the device comprising:

[0023] A first acquisition module, used to acquire a basic visual model and a first sample data set, where the sample data in the first sample data set is image data;

[0024] a labeling module, configured to label each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is labeled with information of a plurality of first detection objects;

[0025] A training module is used to train the basic visual model based on the first sample data set, the second sample data set and multiple graphics processors to obtain a trained first multi-target visual model.

[0026] Optionally, the marking module is specifically used to:

[0027] Acquire multiple first detection objects;

[0028] Based on the large visual model and the multiple first detection objects, information is annotated for each sample data in the first sample data set to obtain a second sample data set.

[0029] Optionally, when the labeling module labels each sample data in the first sample data set based on the large visual model and the multiple first detection objects to obtain the second sample data set, it is specifically used to:

[0030] Inputting the first sample data set into a large visual model, and annotating information on each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set, wherein each sample data in the third sample data set is annotated with information of the multiple first detection objects;

[0031] The information of the first detection object marked in the third sample data set is updated to obtain a second sample data set.

[0032] Optionally, the training module is specifically used to:

[0033] Inputting the first sample data set into the basic visual model, and performing parallel calculations through multiple graphics processors to obtain a first labeling result;

[0034] Calculating a loss value of the basic visual model according to the first annotation result and the second sample data set;

[0035] The basic visual model is trained based on the loss value to obtain a trained first multi-objective visual model.

[0036] Optionally, the device further comprises:

[0037] A second acquisition module, used to acquire at least one second detection object;

[0038] An updating module is used to update the first multi-target visual model based on the at least one second detection object to obtain an updated second multi-target visual model.

[0039] In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a transceiver, and a processor:

[0040] A memory for storing a computer program; a transceiver for sending and receiving data under the control of a processor; and a processor for reading the computer program in the memory and executing the method as described in the first aspect above.

[0041] In a fourth aspect, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method described in the first aspect above.

[0042] In the above-mentioned embodiment of the present application, a second sample data set is obtained by obtaining a basic visual model and a first sample data set, and information is annotated on each sample data in the first sample data set through a large visual model, and each sample data in the second sample data set is annotated with information of multiple first detection objects. Then, the basic visual model is trained based on the first sample data set, the second sample data set, and multiple graphics processors to obtain a trained first multi-target visual model. Through the concurrent processing technology of multiple graphics processors, the processing efficiency and task execution speed can be greatly improved, thereby shortening the time for model training and deployment. In addition, the first multi-target visual model can simultaneously meet the detection requirements of multiple targets, thereby improving the processing efficiency of multi-target tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0045] Figure 2 A specific flow chart of the model training method provided in the embodiment of the present application;

[0046] Figure 3 A structural block diagram of a model training device provided in an embodiment of the present application;

[0047] Figure 4 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0049] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0050] In the prior art, the integration of model distillation with the annotation process and training platform is low, resulting in low efficiency in data screening and task management. The following is a simple example to illustrate the technical background of this application:

[0051] Assume that in a certain traffic scenario, it is necessary to identify scenarios where two-wheeled electric vehicles are not wearing helmets, illegally carrying passengers, or driving in the wrong direction. If a traditional small model is used, it may be necessary to output a small model for electric vehicles of different styles or colors, different types of helmets, data of carrying or not carrying passengers, whether the vehicle is moving in the wrong direction, etc., and each small model needs to be distilled and trained separately. The entire process requires the creation of model training tasks for each type of label in order to solve a single scenario, and the output is doubled during the data collection, labeling, and training processes. Therefore, the existing model distillation technology in the prior art has insufficient support for processing multi-target tasks, and it is difficult to maintain performance stability while ensuring multi-target detection accuracy; in addition, the traditional small model optimization process is time-consuming.

[0052] Therefore, the present application provides a model training method and device, which can quickly filter sample data through a large model during the data collection process, and also supports automatic labeling of the large model. During the training process, multiple models with multiple labels can also be trained into an all-in-one multi-target visual model. A high-precision small model can be used to solve problems that originally required multiple models to solve. During installation and deployment, computing power and costs can also be greatly saved.

[0053] The model training method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0054] Specifically, the present application embodiment provides a model training method, which can be applied to a standard training platform, such as Figure 1 As shown, the following steps may be specifically included:

[0055] Step 101: Acquire a basic visual model and a first sample data set, where sample data in the first sample data set is image data.

[0056] The Labeling and Training Platform refers to a comprehensive platform for data labeling and model training, which supports automated data processing and model training workflows.

[0057] First, the task is created. The user initiates a multi-target model distillation task on the standard training platform. The user first selects multiple first detection objects to be processed. Each first detection object is a label, and its corresponding label set is {I1, I2, I3, ..., In}. The user sets different distillation targets for each label. There is no upper limit on the number of labels. In a specific embodiment, the number of labels can reach thousands to meet the needs of complex scenarios, such as social security monitoring, sensitive target monitoring in the field of national security, etc.

[0058] In practical applications, the label selection process is very flexible and can be customized according to needs. For example, multiple first detection objects are: whether to wear a helmet, whether to go against traffic, the first detection object of whether to wear a helmet corresponds to label I1, and the first detection object of whether to go against traffic corresponds to label I2.

[0059] Users use the large model search engine Telesearch technology to filter out data streams related to the task or data streams corresponding to the device to ensure the quality and relevance of the training data. Assume that the first data set selected by the user is D = {d1, d2, ..., dm}, where m≥100 means that the number of sample data in the data set needs to be greater than or equal to 100. The first sample data set D can be obtained by filtering specific devices, specific time periods, and semantic labels, and the first sample data set D can also be obtained through the above-mentioned method of third-party data. The larger the number of samples m, the more beneficial it is to the distillation optimization effect. The Telesearch technology can accurately filter out data that meets specific detection requirements from a large number of data streams or devices, which not only improves the accuracy of data selection, but also enhances the pertinence and effect of model training.

[0060] Among them, Telesearch refers to a large model search engine that supports the screening and retrieval of feature data through semantics and images. It is usually used to filter and select specific data streams or devices to efficiently support the model training and optimization process.

[0061] Model distillation refers to a model compression technique that extracts knowledge from a large model (teacher model) and transfers it to a smaller model (student model) to preserve performance while reducing computing resource requirements.

[0062] Step 102: annotate each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is annotated with information of multiple first detection objects.

[0063] Each sample data in the first sample data set is input into the large visual model for information annotation. The large visual model is used to annotate each sample data with information of multiple first detection objects to obtain sample data after information annotation. The multiple sample data after information annotation constitute the second sample data set.

[0064] For example, the first detection objects are: whether a person is wearing a helmet, whether a person is walking against traffic. The sample data in the first sample data set is a first picture. The first picture is input into the large visual model, and the large visual model is used to detect whether the person in the first picture is wearing a helmet and annotate it, and detect whether the person in the first picture is walking against traffic and annotate it.

[0065] Among them, the big vision model refers to a deep learning model with large-scale parameters and high complexity, which can handle multiple tasks and has strong generalization ability. Through the generalization ability of the big vision model, the accuracy and response speed of the model can be significantly improved in multi-target detection scenarios.

[0066] Step 103: Train the basic visual model based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model.

[0067] The standard training platform uses the first sample data set as training data and the second sample data set as real data, and iteratively trains the basic visual model through parallel processing of multiple GPUs to obtain the trained first multi-target visual model. The above-mentioned deep integration of model distillation training technology with the standard training platform realizes the integrated operation of data screening, task management and model training, greatly improves the efficiency of the entire process, makes data preparation and model training in complex scenarios smoother, and reduces the need for manual intervention.

[0068] Among them, multi-target (Multi-Label) refers to processing multiple different detection targets or labels in one task. The first multi-target visual model can be applied in multiple complex scenarios. For example, in social security management and law enforcement, for some high-risk public areas, such as nighttime commercial areas, entertainment venues, or other areas with complex security, real-time monitoring of video streams is required to detect and identify behaviors that may endanger public safety. In the field of national security, for specific intelligence monitoring tasks, it can be effectively applied to the perimeter monitoring of confidential units, especially in public areas near some confidential units. Monitoring suspicious behaviors, such as taking pictures or staying for a long time, has important security significance. For example, in the streets near some confidential units, the national security department needs to monitor possible external threats, which may include taking pictures near confidential units, staying for a long time, etc. When someone is detected taking pictures near a confidential unit, the first multi-target visual model can quickly trigger an alarm signal to notify security personnel to intervene and handle it, which not only improves the national security department's ability to respond to potential threats, but also enhances the protection of sensitive areas, ensuring the effective protection of state secrets and security.

[0069] In the above-mentioned embodiment of the present application, a second sample data set is obtained by obtaining a basic visual model and a first sample data set, and information is annotated on each sample data in the first sample data set through a large visual model, and each sample data in the second sample data set is annotated with information of multiple first detection objects. Then, the basic visual model is trained based on the first sample data set, the second sample data set, and multiple graphics processors to obtain a trained first multi-target visual model. Through the concurrent processing technology of multiple graphics processors, the processing efficiency and task execution speed can be greatly improved, thereby shortening the time for model training and deployment. In addition, the first multi-target visual model can simultaneously meet the detection requirements of multiple targets, thereby improving the processing efficiency of multi-target tasks.

[0070] As an optional specific embodiment, the step 102 uses a large visual model to annotate each sample data in the first sample data set to obtain a second sample data set, which specifically includes:

[0071] Step 1021, obtaining a plurality of first detection objects;

[0072] Step 1022: Based on the large visual model and the multiple first detection objects, information is annotated for each sample data in the first sample data set to obtain a second sample data set.

[0073] First, multiple first detection objects are obtained, such as whether a helmet is worn, whether the vehicle is traveling in the wrong direction, etc. Each sample data in the first sample data set is input into the large visual model. The large visual model performs information annotation according to the multiple first detection objects to obtain a second sample data set after information annotation.

[0074] For example, the labeling result of each sample data di (i.e., any one of d1 to dm) is M(di) = {l1, l2, ..., lp}, that is, each sample data is labeled with p labels through the large visual model, where p ≤ n, and the labeling result is recorded as Bauto. The labeling result (i.e., the second sample data set) obtained after labeling m sample data is expressed by the following formula:

[0075]

[0076] Furthermore, the step 1022 performs information labeling on each sample data in the first sample data set based on the large visual model and the multiple first detection objects to obtain a second sample data set, specifically including:

[0077] Inputting the first sample data set into a large visual model, and annotating information on each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set, wherein each sample data in the third sample data set is annotated with information of the multiple first detection objects;

[0078] The information of the first detection object marked in the third sample data set is updated to obtain a second sample data set.

[0079] First, multiple first detection objects are obtained, such as whether a helmet is worn, whether the vehicle is traveling in the wrong direction, etc. Each sample data in the first sample data set is input into the large visual model. The large visual model performs information annotation according to the multiple first detection objects to obtain a third sample data set after information annotation. When all the information of the first detection objects in the third sample data set is correct, the second sample data set is the third sample data set. If the information of the first detection object annotated in the third sample data set needs to be modified, the information of the first detection object annotated in the third sample data set is updated to obtain an updated second sample data set.

[0080] For example, the labeling result of each sample data di (i.e., any one of d1 to dm) is M(di) = {l1, l2, ..., lp}, that is, each sample data is labeled with p labels through the large visual model, where p ≤ n, and the labeling result is recorded as Bauto. The labeling result (i.e., the third sample data set) obtained after labeling m sample data is expressed by the following formula:

[0081]

[0082] The user can verify and update the above annotation result Bauto. Assuming that the annotation result after user verification is Bmanual (i.e., the second sample data set), then:

[0083]

[0084] It should be noted that during the update process of the third sample data set, the annotation progress can be paused and saved at any time so as to continue to update the annotation later.

[0085] As an optional specific embodiment, the step 103 trains the basic visual model based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model, including:

[0086] Inputting the first sample data set into the basic visual model, and performing parallel calculations through multiple graphics processors to obtain a first labeling result;

[0087] Calculating a loss value of the basic visual model according to the first annotation result and the second sample data set;

[0088] The basic visual model is trained based on the loss value to obtain a trained first multi-objective visual model.

[0089] The first sample data set is used as training data and the second sample data set is used as real data. The first sample data set is input into the basic visual model for training. Parallel calculation is performed through parallel processing of multiple GPUs to obtain a first labeling result. The first labeling result is compared with the second sample data set, and the loss value of the basic visual model is calculated. The basic visual model is trained through iterative training until the model converges to obtain a trained first multi-target visual model.

[0090] The core of the model training stage is to use distributed data parallel (DDP) technology (i.e. multi-GPU concurrency technology) to obtain the first multi-target visual model M[I1, I2, I3, …, In] after training, which significantly improves the efficiency of model training.

[0091] Specifically, the value obtained by dividing the total number of target tasks by the number of GPUs available on the standard training platform is the target task that each GPU needs to process. By having multiple GPUs process multiple target tasks simultaneously, the efficiency of the overall process can be ensured. When each GPU has processed its assigned target task, the standard training platform will generate the final first multi-target visual model. When the task status changes to "Completed", the first multi-target visual model will prompt the user, indicating that the model training process has been completed.

[0092] As an optional specific embodiment, after the step 103 trains the basic visual model based on the first sample data set, the second sample data set and a plurality of graphics processors to obtain the trained first multi-target visual model, the method may further include:

[0093] Acquire at least one second detection object;

[0094] The first multi-object visual model is updated based on the at least one second detection object to obtain an updated second multi-object visual model.

[0095] After obtaining the first multi-target visual model, if the user needs to add a first detection object, there is no need to retrain the model, and it can be updated based on the first multi-target visual model to improve the model training efficiency, that is, obtain one or more second detection objects and a first sample data set, and use the first multi-target visual model to annotate each sample data in the first sample data set to obtain a fourth sample data set, and each sample data in the fourth sample data set is annotated with information of at least one second detection object; then, based on the first sample data set, the fourth sample data set and multiple graphics processors, the first multi-target visual model is updated to obtain a second multi-target visual model.

[0096] like Figure 2 As shown, the above solution is described below through a specific embodiment:

[0097] Step 201: Obtain a basic visual model and a first sample data set.

[0098] Step 202: Acquire multiple first detection objects.

[0099] Step 203: Input the first sample data set into the large visual model, and annotate each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set.

[0100] Step 204: Update the information of the first detection object marked in the third sample data set to obtain a second sample data set.

[0101] Step 205: Input the first sample data set into the basic visual model, and perform parallel calculations through multiple graphics processors to obtain a first labeling result.

[0102] Step 206: Calculate the loss value of the basic visual model according to the first labeling result and the second sample data set.

[0103] Step 207: Train the basic visual model based on the loss value to obtain a trained first multi-objective visual model.

[0104] Step 208: Acquire at least one second detection object.

[0105] Step 209: updating the first multi-target visual model based on the at least one second detection object to obtain an updated second multi-target visual model.

[0106] In summary, the above embodiment of the present application obtains a basic visual model and a first sample data set, and annotates each sample data in the first sample data set through a large visual model to obtain a second sample data set. Through the generalization ability of the large visual model, the accuracy and response speed of the model can be significantly improved in multi-target detection scenarios. Then, the basic visual model is trained based on the first sample data set, the second sample data set, and multiple graphics processors to obtain a trained first multi-target visual model. Through the concurrent processing technology of multiple graphics processors, the processing efficiency and task execution speed can be greatly improved, thereby shortening the time for model training and deployment. Moreover, the first multi-target visual model can simultaneously meet the detection requirements of multiple targets and improve the processing efficiency of multi-target tasks. In addition, the model distillation training technology is deeply integrated with the standard training platform to realize the integrated operation of data screening, task management, and model training, which greatly improves the efficiency of the entire process, making data preparation and model training in complex scenarios smoother and reducing the need for manual intervention.

[0107] The above introduces the model training method provided in the embodiment of the present application. The following will introduce the model training device provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0108] like Figure 3 As shown, the embodiment of the present application also provides a model training device 300, the device comprising:

[0109] The first acquisition module 301 is used to acquire a basic visual model and a first sample data set, where the sample data in the first sample data set is image data.

[0110] The Labeling and Training Platform refers to a comprehensive platform for data labeling and model training, which supports automated data processing and model training workflows.

[0111] First, the task is created. The user initiates a multi-target model distillation task on the standard training platform. The user first selects multiple first detection objects to be processed. Each first detection object is a label, and its corresponding label set is {I1, I2, I3, ..., In}. The user sets different distillation targets for each label. There is no upper limit on the number of labels. In a specific embodiment, the number of labels can reach thousands to meet the needs of complex scenarios, such as social security monitoring, sensitive target monitoring in the field of national security, etc.

[0112] In practical applications, the label selection process is very flexible and can be customized according to needs. For example, multiple first detection objects are: whether to wear a helmet, whether to go against traffic, the first detection object of whether to wear a helmet corresponds to label I1, and the first detection object of whether to go against traffic corresponds to label I2.

[0113] Users use the large model search engine Telesearch technology to filter out data streams related to the task or data streams corresponding to the device to ensure the quality and relevance of the training data. Assume that the first data set selected by the user is D = {d1, d2, ..., dm}, where m≥100 means that the number of sample data in the data set needs to be greater than or equal to 100. The first sample data set D can be obtained by filtering specific devices, specific time periods, and semantic labels, and the first sample data set D can also be obtained through the above-mentioned method of third-party data. The larger the number of samples m, the more beneficial it is to the distillation optimization effect. The Telesearch technology can accurately filter out data that meets specific detection requirements from a large number of data streams or devices, which not only improves the accuracy of data selection, but also enhances the pertinence and effect of model training.

[0114] Among them, Telesearch refers to a large model search engine that supports the screening and retrieval of feature data through semantics and images. It is usually used to filter and select specific data streams or devices to efficiently support the model training and optimization process.

[0115] Model distillation refers to a model compression technique that extracts knowledge from a large model (teacher model) and transfers it to a smaller model (student model) to preserve performance while reducing computing resource requirements.

[0116] The labeling module 302 is used to label each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is labeled with information of multiple first detection objects.

[0117] Each sample data in the first sample data set is input into the large visual model for information annotation. The large visual model is used to annotate each sample data with information of multiple first detection objects to obtain sample data after information annotation. The multiple sample data after information annotation constitute the second sample data set.

[0118] For example, the first detection objects are: whether a person is wearing a helmet, whether a person is walking against traffic. The sample data in the first sample data set is a first picture. The first picture is input into the large visual model, and the large visual model is used to detect whether the person in the first picture is wearing a helmet and annotate it, and detect whether the person in the first picture is walking against traffic and annotate it.

[0119] Among them, the big vision model refers to a deep learning model with large-scale parameters and high complexity, which can handle multiple tasks and has strong generalization ability. Through the generalization ability of the big vision model, the accuracy and response speed of the model can be significantly improved in multi-target detection scenarios.

[0120] The training module 303 is used to train the basic visual model based on the first sample data set, the second sample data set and multiple graphics processors to obtain a trained first multi-target visual model.

[0121] The standard training platform uses the first sample data set as training data and the second sample data set as real data, and iteratively trains the basic visual model through parallel processing of multiple GPUs to obtain the trained first multi-target visual model. The above-mentioned deep integration of model distillation training technology with the standard training platform realizes the integrated operation of data screening, task management and model training, greatly improves the efficiency of the entire process, makes data preparation and model training in complex scenarios smoother, and reduces the need for manual intervention.

[0122] Among them, multi-target (Multi-Label) refers to processing multiple different detection targets or labels in one task. The first multi-target visual model can be applied in multiple complex scenarios. For example, in social security management and law enforcement, for some high-risk public areas, such as nighttime commercial areas, entertainment venues, or other areas with complex security, real-time monitoring of video streams is required to detect and identify behaviors that may endanger public safety. In the field of national security, for specific intelligence monitoring tasks, it can be effectively applied to the perimeter monitoring of confidential units, especially in public areas near some confidential units. Monitoring suspicious behaviors, such as taking pictures or staying for a long time, has important security significance. For example, in the streets near some confidential units, the national security department needs to monitor possible external threats, which may include taking pictures near confidential units, staying for a long time, etc. When someone is detected taking pictures near a confidential unit, the first multi-target visual model can quickly trigger an alarm signal to notify security personnel to intervene and handle it, which not only improves the national security department's ability to respond to potential threats, but also enhances the protection of sensitive areas, ensuring the effective protection of state secrets and security.

[0123] In the above-mentioned embodiment of the present application, a second sample data set is obtained by obtaining a basic visual model and a first sample data set, and information is annotated on each sample data in the first sample data set through a large visual model, and each sample data in the second sample data set is annotated with information of multiple first detection objects. Then, the basic visual model is trained based on the first sample data set, the second sample data set, and multiple graphics processors to obtain a trained first multi-target visual model. Through the concurrent processing technology of multiple graphics processors, the processing efficiency and task execution speed can be greatly improved, thereby shortening the time for model training and deployment. In addition, the first multi-target visual model can simultaneously meet the detection requirements of multiple targets, thereby improving the processing efficiency of multi-target tasks.

[0124] Optionally, the marking module 302 is specifically configured to:

[0125] Acquire multiple first detection objects;

[0126] Based on the large visual model and the multiple first detection objects, information is annotated for each sample data in the first sample data set to obtain a second sample data set.

[0127] First, multiple first detection objects are obtained, such as whether a helmet is worn, whether the vehicle is traveling in the wrong direction, etc. Each sample data in the first sample data set is input into the large visual model. The large visual model performs information annotation according to the multiple first detection objects to obtain a second sample data set after information annotation.

[0128] For example, the labeling result of each sample data di (i.e., any one of d1 to dm) is M(di) = {l1, l2, ..., lp}, that is, each sample data is labeled with p labels through the large visual model, where p ≤ n, and the labeling result is recorded as Bauto. The labeling result (i.e., the second sample data set) obtained after labeling m sample data is expressed by the following formula:

[0129]

[0130] Optionally, when the labeling module 302 labels each sample data in the first sample data set based on the large visual model and the multiple first detection objects to obtain the second sample data set, it is specifically used to:

[0131] Inputting the first sample data set into a large visual model, and annotating information on each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set, wherein each sample data in the third sample data set is annotated with information of the multiple first detection objects;

[0132] The information of the first detection object marked in the third sample data set is updated to obtain a second sample data set.

[0133] First, multiple first detection objects are obtained, such as whether a helmet is worn, whether the vehicle is traveling in the wrong direction, etc. Each sample data in the first sample data set is input into the large visual model. The large visual model performs information annotation according to the multiple first detection objects to obtain a third sample data set after information annotation. When all the information of the first detection objects in the third sample data set is correct, the second sample data set is the third sample data set. If the information of the first detection object annotated in the third sample data set needs to be modified, the information of the first detection object annotated in the third sample data set is updated to obtain an updated second sample data set.

[0134] For example, the labeling result of each sample data di (i.e., any one of d1 to dm) is M(di) = {l1, l2, ..., lp}, that is, each sample data is labeled with p labels through the large visual model, where p ≤ n, and the labeling result is recorded as Bauto. The labeling result (i.e., the third sample data set) obtained after labeling m sample data is expressed by the following formula:

[0135]

[0136] The user can verify and update the above annotation result Bauto. Assuming that the annotation result after user verification is Bmanual (i.e., the second sample data set), then:

[0137]

[0138] It should be noted that during the update process of the third sample data set, the annotation progress can be paused and saved at any time so as to continue to update the annotation later.

[0139] Optionally, the training module 303 is specifically used to:

[0140] Inputting the first sample data set into the basic visual model, and performing parallel calculations through multiple graphics processors to obtain a first labeling result;

[0141] Calculating a loss value of the basic visual model according to the first annotation result and the second sample data set;

[0142] The basic visual model is trained based on the loss value to obtain a trained first multi-objective visual model.

[0143] The first sample data set is used as training data and the second sample data set is used as real data. The first sample data set is input into the basic visual model for training. Parallel calculation is performed through parallel processing of multiple GPUs to obtain a first labeling result. The first labeling result is compared with the second sample data set, and the loss value of the basic visual model is calculated. The basic visual model is trained through iterative training until the model converges to obtain a trained first multi-target visual model.

[0144] The core of the model training stage is to use distributed data parallel (DDP) technology (i.e. multi-GPU concurrency technology) to obtain the first multi-target visual model M[I1, I2, I3, …, In] after training, which significantly improves the efficiency of model training.

[0145] Specifically, the value obtained by dividing the total number of target tasks by the number of GPUs available on the standard training platform is the target task that each GPU needs to process. By having multiple GPUs process multiple target tasks simultaneously, the efficiency of the overall process can be ensured. When each GPU has processed its assigned target task, the standard training platform will generate the final first multi-target visual model. When the task status changes to "Completed", the first multi-target visual model will prompt the user, indicating that the model training process has been completed.

[0146] Optionally, the device further comprises:

[0147] A second acquisition module, used to acquire at least one second detection object;

[0148] An updating module is used to update the first multi-target visual model based on the at least one second detection object to obtain an updated second multi-target visual model.

[0149] After obtaining the first multi-target visual model, if the user needs to add a first detection object, there is no need to retrain the model, and it can be updated based on the first multi-target visual model to improve the model training efficiency, that is, obtain one or more second detection objects and a first sample data set, and use the first multi-target visual model to annotate each sample data in the first sample data set to obtain a fourth sample data set, and each sample data in the fourth sample data set is annotated with information of at least one second detection object; then, based on the first sample data set, the fourth sample data set and multiple graphics processors, the first multi-target visual model is updated to obtain a second multi-target visual model.

[0150] In summary, the above embodiment of the present application obtains a basic visual model and a first sample data set, and annotates each sample data in the first sample data set through a large visual model to obtain a second sample data set. Through the generalization ability of the large visual model, the accuracy and response speed of the model can be significantly improved in multi-target detection scenarios. Then, the basic visual model is trained based on the first sample data set, the second sample data set, and multiple graphics processors to obtain a trained first multi-target visual model. Through the concurrent processing technology of multiple graphics processors, the processing efficiency and task execution speed can be greatly improved, thereby shortening the time for model training and deployment. Moreover, the first multi-target visual model can simultaneously meet the detection requirements of multiple targets and improve the processing efficiency of multi-target tasks. In addition, the model distillation training technology is deeply integrated with the standard training platform to realize the integrated operation of data screening, task management, and model training, which greatly improves the efficiency of the entire process, making data preparation and model training in complex scenarios smoother and reducing the need for manual intervention.

[0151] It should be noted here that the above-mentioned model training device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned model training method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0152] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0154] like Figure 4 As shown, an embodiment of the present application further provides an electronic device, including a memory 420, a transceiver 410, and a processor 400:

[0155] Memory 420, used for storing computer programs;

[0156] A transceiver 410, configured to transmit and receive data under the control of the processor;

[0157] The processor 400 is configured to obtain a basic visual model and a first sample data set, where the sample data in the first sample data set is image data;

[0158] Annotating each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is annotated with information of multiple first detection objects;

[0159] The basic visual model is trained based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model.

[0160] Optionally, when the processor 400 performs information labeling on each sample data in the first sample data set by using a large visual model to obtain a second sample data set, the processor 400 is specifically configured to:

[0161] Acquire multiple first detection objects;

[0162] Based on the large visual model and the multiple first detection objects, information is annotated for each sample data in the first sample data set to obtain a second sample data set.

[0163] Optionally, when the processor 400 performs information labeling on each sample data in the first sample data set based on the large visual model and the multiple first detection objects to obtain the second sample data set, it is specifically configured to:

[0164] Inputting the first sample data set into a large visual model, and annotating information on each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set, wherein each sample data in the third sample data set is annotated with information of the multiple first detection objects;

[0165] The information of the first detection object marked in the third sample data set is updated to obtain a second sample data set.

[0166] Optionally, when the processor 400 trains the basic visual model based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model, the processor 400 is specifically configured to:

[0167] Inputting the first sample data set into the basic visual model, and performing parallel calculations through multiple graphics processors to obtain a first labeling result;

[0168] Calculating a loss value of the basic visual model according to the first annotation result and the second sample data set;

[0169] The basic visual model is trained based on the loss value to obtain a trained first multi-objective visual model.

[0170] Optionally, after training the basic visual model based on the first sample data set, the second sample data set and a plurality of graphics processors to obtain a trained first multi-target visual model, the processor 400 is further configured to:

[0171] Acquire at least one second detection object;

[0172] The first multi-object visual model is updated based on the at least one second detection object to obtain an updated second multi-object visual model.

[0173] Among them, Figure 4In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 400 and memory represented by memory 420. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 410 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which transmission medium may include a wireless channel, a wired channel, an optical cable, and the like. The processor 400 is responsible for managing the bus architecture and general processing, and the memory 420 may store data used by the processor 400 when performing operations.

[0174] The processor 400 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0175] The processor calls the computer program stored in the memory to execute the model training method provided in the embodiment of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0176] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned model training method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0177] An embodiment of the present application also provides a processor-readable storage medium, which stores a computer program, and the computer program is used to enable the processor to execute the above-mentioned model training method.

[0178] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0179] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0180] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0181] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0182] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0183] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A model training method, characterized in that: The method comprises: Acquire a basic visual model and a first sample data set, where sample data in the first sample data set is image data; Annotating each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is annotated with information of multiple first detection objects; The basic visual model is trained based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model.

2. The method according to claim 1, characterized in that The step of labeling each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set includes: Acquire multiple first detection objects; Based on the large visual model and the multiple first detection objects, information is annotated for each sample data in the first sample data set to obtain a second sample data set.

3. The method according to claim 2, characterized in that The step of annotating information of each sample data in the first sample data set based on the large visual model and the plurality of first detection objects to obtain a second sample data set includes: Inputting the first sample data set into a large visual model, and annotating information on each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set, wherein each sample data in the third sample data set is annotated with information of the multiple first detection objects; The information of the first detection object marked in the third sample data set is updated to obtain a second sample data set.

4. The method according to claim 1, characterized in that: The step of training the basic visual model based on the first sample data set, the second sample data set, and a plurality of graphics processors to obtain a trained first multi-target visual model includes: Inputting the first sample data set into the basic visual model, and performing parallel calculations through multiple graphics processors to obtain a first labeling result; Calculating a loss value of the basic visual model according to the first annotation result and the second sample data set; The basic visual model is trained based on the loss value to obtain a trained first multi-objective visual model.

5. The method according to claim 1, characterized in that After training the basic visual model based on the first sample data set, the second sample data set and a plurality of graphics processors to obtain a trained first multi-target visual model, the method further includes: Acquire at least one second detection object; The first multi-object visual model is updated based on the at least one second detection object to obtain an updated second multi-object visual model.

6. A model training device, characterized in that: The device comprises: A first acquisition module, used to acquire a basic visual model and a first sample data set, where the sample data in the first sample data set is image data; a labeling module, configured to label each sample data in the first sample data set with information by using a large visual model to obtain a second sample data set, wherein each sample data in the second sample data set is labeled with information of a plurality of first detection objects; A training module is used to train the basic visual model based on the first sample data set, the second sample data set and multiple graphics processors to obtain a trained first multi-target visual model.

7. The device according to claim 6, characterized in that The annotation module is specifically used for: Acquire multiple first detection objects; Based on the large visual model and the multiple first detection objects, information is annotated for each sample data in the first sample data set to obtain a second sample data set.

8. The device according to claim 7, characterized in that When the labeling module labels each sample data in the first sample data set based on the large visual model and the plurality of first detection objects to obtain the second sample data set, the labeling module is specifically used to: Inputting the first sample data set into a large visual model, and annotating information on each sample data in the first sample data set based on the multiple first detection objects to obtain a third sample data set, wherein each sample data in the third sample data set is annotated with information of the multiple first detection objects; The information of the first detection object marked in the third sample data set is updated to obtain a second sample data set.

9. The device according to claim 6, characterized in that The training module is specifically used for: Inputting the first sample data set into the basic visual model, and performing parallel calculations through multiple graphics processors to obtain a first labeling result; Calculating a loss value of the basic visual model according to the first annotation result and the second sample data set; The basic visual model is trained based on the loss value to obtain a trained first multi-objective visual model.

10. The device according to claim 6, characterized in that The device also includes: A second acquisition module, used to acquire at least one second detection object; An updating module is used to update the first multi-target visual model based on the at least one second detection object to obtain an updated second multi-target visual model.

11. An electronic device, characterized in that: Including memory, transceiver, processor: Memory for storing computer programs; a transceiver, for transmitting and receiving data under the control of the processor; A processor, configured to read the computer program in the memory and execute the model training method as described in any one of claims 1 to 5.

12. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the model training method as described in any one of claims 1 to 5.

Images